I've always wondered how to present and market Sider, because I sometimes have a problem defining it to myself. Is it a tool to build tools, or a platform to work with information, or both, or something else? I'm having this problem because in a way it's idea is new and unique, but I also don't have a lot of experience in software development and design. My partner, Igor, who came up with the idea, understands the project much better, but I'm the one who's doing most of the work and marketing.

Taking a fresh look at the design I see it as an engine, similar to a game engine. Instead of dealing with graphics and animation, it deals with information, logic to process it and views to show it. Through modules and packages it can support new types of information and new ways to deal with it. For example a module can allow users to manage a software project using Agile practices, or store and organize notes, and so on.

What's the difference between a platform and an engine anyway? It's the scope they deal with. A software platform is a set of technologies used for building a large variety of applications. An engine focuses on a specific type of problems and many common activities associated with it. In both cases software runs on top of the platform or an engine. A platform is just a broader and more abstract engine.

What's the best way to present this idea to the user? Without extensions Sider is useless to most people, just like game engine is useless to players without a game on top of it. Game developers usually market games to regular people and engines to other developers. Many players are not even aware of the engine and that it can run other games.

This creates a problem with Sider, especially since it's fresh and hasn't been proved yet. Many users won't care that it's a flexible engine. They're interested in concrete solutions, not in potential. At the same time I always wanted to charge for the use of the engine, not extensions (at least initially). I hope this will help Sider to stand out as new extensions will become available for free. I will need to make people aware that it is an engine and that's what they are paying for.

In addition anybody can extend it. History shows that some users (power users I call them) will be interested in customizing it and adding new features. Most people will just want to use them, but extensions created by power users are important to Sider success.

I think initially, it's best to focus on promoting extensions and scenarios they support, just mentioning that more will be available. When they are available, expand marketing to include them. In time, after several iterations and adding new extensions I can try to expand and capitalize on the fact that there will be support for a large number of things, because in the past we have added many.

I'm back to blogging. I'm gonna start by changing how I write posts, making them more un-official, like a journal. Not sure yet how that will work out.

I'm also back to working on Sider. I decided to make its development process more transparent. I want to be open and honest about what I'm working on, what mistakes I make and how I fix them.

I'm in the process of putting all project documentation in one place on the wiki where it's available to anybody. I started to review and publish Sider design documents. That shouldn't take too long, at least for the initial phase where I create an outline of all features. I'm not sure yet if I will publish everything, but I'm sure that most of it will be online. The wiki is opened for registration, so anybody can contribute. I have also opened read-only access to Sider issues.

In addition I recently learned about Scrum and want to try it as a project management methodology. I'll have more user stories (which I seem to like, unlike the use cases that are too detailed for my taste) and I'm going to estimate all remaining work using abstract points, then use velocity to approximate when the next release will be available. I will probably write here about my experience and thoughts.

Whew, finally this version is finished and available for download. It took a lot of effort and much longer than I ever imagined. I've made many mistakes, but also learned a lot.

This is a technology preview release. It demonstrates the platform with most important ideas implemented. It's possible to define different types and views, but there are only few extensions available. Read release notes for details.

A little bit about us

Sider is the work of two people. My name is Slava Ivanyuk and I do most of the development. I write all code, documentation, organize design documents, etc. My partner, Igor, mostly helps by giving ideas and testing Sider. It was his idea to develop a flexible platform with potential to support any type of information.

What's next?

My next steps are to take a short break, do a post-mortem (maybe I'll blog about it) and start moving toward next release. We plan to improve Sider to the point where it has wider practical use. That means adding some features, but mostly focusing on creating extensions. Here's a list of few major things we will be working on. We'll finish adding more details to wiki roadmap sometime soon.

• Move to .NET 2 - At least evaluate the move, but most likely we will do it. It offers many improvements over .NET 1.1, not only in general but also to work with XML. Eventually Sider will move to it and the sooner the better. My only concern is possible delay it might introduce. Most likely, we will move to it in several stages.
• Persistent user options - Right now, most customization is done through extensions. We want to add application-wide settings.
• Support packages - A package is a collection of different extensions. Each package would focus on providing support for specific scenarios, like note management, project management, collections, etc.

Upcoming packages

These are just a few ideas of popular uses we want to support with Sider.

• Note management - There're many programs to work with notes, but none of them get it quite right. We want to have a flexible system that would support graphs (like wikis) of notes, tagging, filtering, etc.
• Journal - People often use notes to track progress, to capture thoughts on the topic over time, instead of creating a system for referencing information. Journal will feature special notes, with views heavily oriented on dates and calendar.
• Project management - Extensions to allow you put project plans, notes, documents in a single place. There will be support for several methodoliges, including agile (XP, Scrum, combinations), waterfall, etc. But main focus will be on making it possible to mix notes and other documents with tasks, issues, stories. There are often many ways to view project documentation: tasks and related notes, list of features and related tasks, etc. I believe some software (such as Trac) represent moves in the right direction, but they aren't quite getting everything right. Of course, they offer many options that won't be possible with Sider in the near future (online collaboration, source management integration, etc).

I also want to go back to Xelog. I like the idea behind it, but I don't like how heavyweight it ended up. I will need to fix that before moving on.

With the next post, I'll go back to discussing information management tools. I had some ideas and suggestions on how to improve my classification for them.

I started working on Sider roadmap. It's helpful to have a good roadmap to maintain focus and work on projects. It's something I should have done it much earlier.

I have ideas about Sider 4 versions ahead. At some point, I want to put them all in writing for everyone to see, but with upcoming release I'm short on time and might not be able to finish it. If not I will complete it shortly after release.

My goal is to have full design doc online (preferably in wiki). I want for people interested in Sider to have an understanding of the platform and maybe share ideas on what can be improved.

I have only a portion of it done and it's not in the form I would to put it online yet. I'm working on improving it and will start moving it to wiki, but there's so much to do, that I'm not sure when I will put it online yet.

I came to realize importance of design documents through my recent experience. I used to think I could get away without one. No, not really. Writing design document forced me to think about different aspects of Sider and understand the whole picture much better. I had hard time focusing on my work (jumping between different things) without it and my overall design is not nearly as good as I want it to be. After I created a large portion of the document, development became much easier and more enjoyable. I started to get things done quicker and understand the whole project scope better.

My main argument against writing document was bureaucracy. I was afraid I would end up with a large, hard to read or navigate document. I didn't want to spend all this time creating something useless. Maybe I just wasn't ready for it. Anyway, when writing it I used [wikipedia:MindManager] instead of word processor. [wikipedia:Mindmap] helped to put all my ideas down in writing and quickly organize them, but I think wiki is actually more suited for reading the document with its support for free linking between different pages.

I'm going to release Sider on 12/15/2006. This is going to be a demo of the platform ideas. I want to show that it's not only possible, but also easy to develop new tools to work with information using Sider. It won't be very useful from practical point of view and will feature a single package to store notes in a tree-like structure. There're some things I want to fix before release, as well as restructure wiki documentation a little and create website for Sider.

I wrote about Sider before, what it is and why I'm working on it. I also maintain a wiki with Sider documentation. You can find more details about the platform there. Today I want to discuss how it works.

Sider describes and stores all information using XML. Each Sider document is an XML document. This use of XML technologies not only saves time and simplifies development, but also makes it easier to extend Sider. A lot of people are familiar with XML and there is a lot of documentation describing every aspect of the technology.

Types

Sider types use XML Schema (XSD) to describe information structure and validate documents. XSD does have some limitations, but for now it will suffice. Type inheritance works through schema inheritance. Each new type must provide a new schema, but it's possible to extend, restrict, substitute or combine schemas from other types.

In addition to the schema, each type provides two more features: constructors and script objects. Constructor is an XSL transformation that creates a valid document of the current type. When user creates a new document through File->New type XSL constructor is used to create that document.

Type script object is a JavaScript object to provide view with document operations. When view is loaded all types (related to the current document) are loaded as well. The view doesn't have a direct access to the document, but instead relies on the type objects to manage it. This is similar to MVC pattern and promotes separation of concerns.

Views

Sider views show the document and allow the user to interact with it. Each view is displayed using hosted web browser control (IE currently, but in the future we will add support for Mozilla) and interacts with the user, with the document (through type objects) and with Sider itself (through window.external).

When a document is loaded it is transformed using XSLT provided by the view. Currently, all views generate HTML files, but it is possible to generate SVG, for example, to show graphics in the browser.

This use of web technologies has certain limitations (web browser can be slow if a page is complex, it can leak memory and so on), but at the same time it promises the widest support of different ways to show data (especially right now, when interest in the web technologies is on the rise and we see a lot of different libraries appear, such as DojoToolkit). Another big advantage is that we will be able to add support for document editing online.

In addition to HTML views can use other web technologies, like Flash, SVG, Java, ActiveX. It will even be possible to use not browse-based technologies, such as .NET window forms or C++.

View components

View components are used by the views and exist to promote code reuse. They are horizontal view plug-ins (any view can use any view component). Example of view component would be a JavaScript library (such as Dojo), spell-checking support, and so on.

Conclusion

Sider platform binds everything together. It finds and loads a document, finds the types that describe and validate it, and views to show it. Sider generates HTML document using view transformation and displays it to the user. When the user wants to save the document the platform extracts updated document from the view, validates and saves it.

This is why Sider is unique. It binds all these standard technologies (XML, HTML, JavaScript, etc.) to provide a simple, uniform information management experience for both users and developers.

In my earlier post I wrote about Sider, an information management platform. Read it if you want to learn more about what it is. Today I want to explain why I think it's important and why I'm spending time on it. Actually, I'm not working alone. I'm doing most of the stuff, writing code, documentation, etc. My partner, Igor, helps by providing feedback, helping me analyze problems and supplying new ideas.

Igor thought of the Sider and I liked it, so I decided to dedicate my time to it. We both store a lot of information on the computer, notes, e-mails, contacts, project plans, etc. By the way, information is a collection of data chunks with relationships between those chunks. A text document or a table is data, but a collection of text documents organized in such a way that connection between each document is visible, is information.

Unfortunately, current state of managing data on computers is rather weak compared to what is possible. Most programs can handle only a few documents types, and almost all of them are not very extendable. For example, Microsoft Word is focused at working with text, Excel mostly works with tabular data, OneNote stores notes in a tree-like structure, Outlook stores tasks and e-mail in lists a tree (but each task node cannot be a tree node).

As a result I need to split information I want to store on the computer between different programs. My to-do list is in one program - ToDoList, but for recurring tasks that I need reminder on I use Outlook. To store my notes I use Wikipad. To write this blog post I use Word, but to post it - w.blogger. To work with tables (and I do every day, to track time I spend on different projects) I use Excel. For mind maps I might use FreeMind. To work with my information I need to have 5-6 programs to be running and constantly switch between them.

In most cases these programs work separately and I can't reference Wikipad note in a ToDoList task (I can reference wiki file, but usually that's not enough). This need to split up between programs happens because each program was created to operate a single domain and solve only a few problems. In addition many of these programs are inflexible and it's hard to extend them if I want to add a missing feature.

Even when I can create custom extensions for the application, I need to learn it's language and API (Wikipad uses python, Microsoft Office - VBA or .NET, ToDoList is in C++). Moreover, extensions supported are limited in scope. I might be able to add a new view or report, but adding support for a new document type is usually impossible. At best I can try to adopt existing data types, but this is complicated, error-prone and time-consuming.

Sider started off as a more theoretical project, but evolved into an application to address all these problems. Sider stores information in a most-suitable, more intelligent, innter-connected way (a graph is stored as a graph, a tree as a tree, a table as a table, etc.) Working with any information on the computer should be easier and more natural, and that's what Sider is intended to do. Sider makes it easier to reuse information by referencinf it, importing and exporting, and so on.

Sider is flexible and easy to extend. There's no need to try and force existing data type to be suitable for your needs. You can either extend existing type or create new one from scratch, or combine both methods.

Another goal for Sider is to present data in the best way for the user. What is best will change depending on what the user wants to do so it is important to support a lot of different views.

Sider is about helping people to use computers for what they were invented for, to work with data. Of course, it will not solve all problems. When released it will be quite limited compared to most of the existing programs. It probably will never be as good as Word at working with text, or as good as Excel at working with tables. However, it will offer a new and unique approach at managing information and helping people to solve different problems.

In the next post I will talk more about Sider architecture and technologies used. Hopefully it will be an interesting post.

For a couple of years I have been working on a program, Sider (stands for Structured Information Document Editor). I probably should have written about it some time ago, but I think I wasn't confident enough to do it. I am now. It's a project I'm working on full time and I hope it can make many people's life easier (as well as provide some income for me. I need to eat too you know :) )

Sider is a data management platform. It aims to store information in a data structure that fits it the best, and show it in a manner that fits the user.It provides the tools to describe the data (types), how to show it (views) and takes care of storing the actual information in the documents. Sider makes it easy to relate different data to each other through inheritance, referencing and composition. For example, one type can describe an abstract tree and another describes an HTML document. Both of the types can be combined into a 3rd type to create a tree with an HTML in each node. This could be used to store notes.

This support for any combination of data types is Sider's main and unique benefit. It makes it possible to use a single program to store all information together, even if different parts of it have very different structure. There are a lot of potential benefits because of that. Searching, cataloging and managing all of the information becomes much easier.

Document

All data in Sider is stored in the documents. Each document usually consists of a number of smaller documents. In above example of a tree with HTML nodes the tree is a document and each node is a document. This is a natural way of thinking about any information. It consists of smaller parts.

Such implementation with each document consisting of smaller number of documents promotes reuse and actually makes it easier to create complex combinations. The rules on how to describe data are simple, unique and easy to understand, but the can be combined to create a large number of complex combinations.

Type

Each document is of some type. The type describes the structure of the data in a document, how to create a new document and how it can be modified.

A type is similar to a class and a document is an object. A type can consist of other types, just like a class can have fields of other classes. A type can also inherit from another type to add support for a new type or extend existing structure. This mechanism is similar to the OO inheritance but more flexibl . This makes it easier to create new types by combining and reusing existing code. In many instances it's possible to slightly modify the base type to achieve the desired result. Inheritance also ensures data structure can be maintained easily.

There are abstract types to provide common structures (a tree, a list, a table, and so on) and atomic types describing smallest unit of the data (phone number, person name, and so on. Any information that is hard to logically subdivide into anything useful). In most cases the user will see documents of concrete type, consisting of the abstract and the atomic type combinations.

View

Type just describes the data, but doesn't show it. Views are used for that. Through a view user gets to see the document and interact with it. There can be any number of views for any type to present the document in any suitable way. A common example is a table of records. It can be shown just as a data grid, as a chart or a report.

Conclusion

All of that is possible with some of the existing programs, but Sider takes it to the next level. It's a foundation on top of which new data management applications can be built. It takes care of finding types and views, loading them and initializing. It provides inheritance for the types, persistance and shows the document with the correct view. It hides all common code that developers have to write over and over again (save, load, etc) and helps him focus on solving the problem at hand and adding important, unique functionality.

Documents can reference each other (and inside of each other) so all the information is connected. A PIM module for example, contacts can link to personal notes, notes can reference plans, everything is connected.

Sider supports a system of plug-ins to provide horizontal functionality, such as spell checking, for different types and views.

In the next post I'll talk more about why I'm creating this platform and what technologies I use to make it work.