Wow, people are still tuned to my blog, even though more than a year has passed since last time I blogged. Well, I'm back, but I don't know how often I'll write new posts.

I moved to a new blogging software (BlogEngine.net) and renamed blog just to Liquid Maze (to keep it simple). The old feed will continue to function, but a new feed is available now.

I'll probably write when I feel like it instead of trying to adhere to some kind of schedule. That didn't work out too well for me. Nonetheless I'll try to keep the quality of content high.

So, stay tuned...

I just read Getting Real book and I really liked it. It was written by guys at 37 Signals, creators of Ruby on Rails and a series of successful online services. The book is about how to be more practical by staying simple and focusing only on currently problems. Among other things it shows that by doing less you can be more productive.

In other words most of the crap we are doing, or I'm doing, doesn't really matter and has no real effect on my life. I agree with that. I knew about the concept for a long time, but was never able to do much about it. After reading the book and thinking about it I'm starting to get it (maybe).

Doing less is doing just enough, not everything. There's no perfect or complete state.

People often talk about Pareto principle. In the context of this post it means 20% of all actions produce 80% of the value. According to it I just need to focus on that valuable 20% of the actions. I always understood this principle, but could never apply it practically. It's an interesting concept, but I must be missing something.

Solving only problems at hand seems more practical and easier to do. When I first saw the idea I thought, "Don't I always try to solve problems at hand?" As I read and learned about it, I realized that, no, not really. I make several mistakes.

• Try to optimize my time by solving problems early. Cost of most problems grows over time, but most of the problems I think of never happen or are completely different in reality than I thought they are.
• I act on what's urgent. In many cases urgent isn't important, but I'm afraid to lose an opportunity. What I came to realize is that many things that appear urgent, are not urgent at all and can be done at any time.
• Do something without understanding of what I'm trying to do. I try not to do that anymore :)

I don't fully agree with the book on solving problems as they appear. We can anticipate for them to happen and do something early to make it easier to solve them in the future. In a way it's prevention instead of solving something that isn't real yet.

With software development I try to follow good design practices to make my code easier to understand and maintain. When I don't, I usually need to go back and change it later on, thus increasing the total cost. If I don't separate concepts and make sure my classes are independent it's hard to replace an algorithm later when I run into performance problems or switch to database from using files. Speaking about performance, premature optimization can be bad, but if I don't think about it at all until I'm having problems I can push myself into a dead-end and have to rewrite large portions of the program.

Normal life (like I have any) is the same way actually. I can't solve all my health problems, but I can help prevent them and make it easier to handle them if they appear by eating healthy and exercising.

Don't solve problems you don't have, let that which doesn't matter slide, but don't completely close your eyes. Problems will appear. Be flexible, expect the problems to happen, expect that you will have to change your code. Most importantly, be smart about it. It's possible to go too far with anything, prevention including. Be practical and stay real, and do less, just what's really important.

In the last post I talked about estimation points and why they can result in more accurate estimations. In short summary, relative estimation is simpler and more accurate. That's what estimation points are, relative size of one task compared to the other. How long it takes to complete each point needs to be approximated from several initial iterations. Today I want to introduce additional techniques to make estimation easier.

It's usually easy to see if one task is twice as big as the other. If another task is 1 point, the twice-as-big current task is 2 points. What if the current task is about 10, 20, 100 times larger? It's impossible to say if it's 100 or 101 points. It's hard to distinguish between 20 and 21 and even between 10 or 11. It's also not important because estimations are inaccurate. 10 or 11 are close enough for errors in estimation to make it possible for 11 points task to be smaller in reality than 10. By making a task 10 or 11 we are just guessing that it's somewhere in 8 to 15 points range.

To make things easier we can group different ranges to hold all items close in size. For example all tasks in 8 to 15 range will go into 10 group, 15-25 to 20 group and so on. Two series of groups are popular - based on Fibonacci sequence (0, 0.5, 1, 2, 3, 5, 8, 13, 20, 40, 100) and on 2^n sequence (0, 0.5, 1, 2, 4, 8, 16, 20, 40, 100) or something along those lines. Around 20, groups become 20, 40, etc. in size. It doesn't really matter if it's 20 or 21, but it's easier for people to think in rounded numbers.

I tried to use this system with Sider. It was easy and fun. I didn't feel anxiety to say if something is 8 hours or 9 hours. Instead I just chose appropriate group. I also didn't have to guess if my hour is really an hour. I knew I can figure it out later.

The problem for me was that I couldn't really switch to thinking in points instead of hours and days. I almost automatically decided that 8 points is 8 hours - 1 work day. I ended up comparing each task to something that I thought would take a day or an hour.

At this stage I've no idea if my estimations are accurate and when I will complete the next Sider release. In several iterations I hope to have a better picture and a better idea when the release will happen.

I also want to mention teams. Most of the time people work in teams and they need to give estimations as a team. I usually work alone so I can't really share any experience or talk about related problems. I suggest you check out Planning Poker, a procedure and a game created to help people overcome frequent problems and give accurate estimation together with their team.

To learn more about estimation I highly suggest listening to Agile Estimation podcast.

People need to estimate as part of planning. It can be time (tasks), money (budgeting) and so on. The point is to understand the value of something. We can then decide if it's worth the time or money or in what order should we do/get things. When working with tasks we focus on more valuable actions first and if there's time we complete the rest later. We also want to know when a project will be done.

So, software developers are often asked to estimate. We often miss and underestimate. Why? Here's my list of the possible reasons. There are many books written on the subject. I have hardly read any of them so I probably miss something.

• We've never done it before, so we have no idea how long it will take. We just guess.
• We lie, because we wish we would have been better developers and could develop it quicker, or because we want to tell somebody what they want to hear, not our opinion.
• We fail to fully grasp the scope. We estimate what we see, but there are a lot of hidden additional work.

In the first case it's just a matter of experience, but I don't think this is a big factor. In either case there's not much we can do about it. I'm stuck with whatever experience I have.

In the second case, we just need to learn to be honest and face the truth, even if it's not pleasant. It might be hard to acknowledge the problem and do something about it now, but later on it will be more expensive and more unpleasant. This is applicable to all aspects of life. People hide from reality, hoping that the problem will go away on its own. It might, but if it doesn't, it grows like a weed.

We can actually do something about the last case. When we make mistakes in estimation we are consistent about it and we can compensate for it. That's exactly what an estimation point system and velocity are designed to do in Agile Development.

While listening to Agile Estimation podcast I finally understood how point estimation works. Each point is just an abstract unit used to measure relative size of the task. It assumes that it's easier for people to estimate relative size of two objects or tasks when they compare them side-by-side than to estimate an object on its own using standard units (time, length, etc.) The size of 1 point is unique for each project and each team.

When estimating we don't know yet how long it will take us to complete each point, but we can say that 4 points task is twice as large and will probably take twice as long as 2 points task to complete.

Velocity is how many points are completed during each iteration (a week, 2 weeks or however long an iteration is). At first we don't know how big it is, but after a few iterations we can look back and guess how many points we will be able to complete in the future. This makes it possible to project when the project will be done, while compensating for estimation mistakes.

There's more I want to say about estimation and I will continue writing about it in my next post.

I've always wondered how to present and market Sider, because I sometimes have a problem defining it to myself. Is it a tool to build tools, or a platform to work with information, or both, or something else? I'm having this problem because in a way it's idea is new and unique, but I also don't have a lot of experience in software development and design. My partner, Igor, who came up with the idea, understands the project much better, but I'm the one who's doing most of the work and marketing.

Taking a fresh look at the design I see it as an engine, similar to a game engine. Instead of dealing with graphics and animation, it deals with information, logic to process it and views to show it. Through modules and packages it can support new types of information and new ways to deal with it. For example a module can allow users to manage a software project using Agile practices, or store and organize notes, and so on.

What's the difference between a platform and an engine anyway? It's the scope they deal with. A software platform is a set of technologies used for building a large variety of applications. An engine focuses on a specific type of problems and many common activities associated with it. In both cases software runs on top of the platform or an engine. A platform is just a broader and more abstract engine.

What's the best way to present this idea to the user? Without extensions Sider is useless to most people, just like game engine is useless to players without a game on top of it. Game developers usually market games to regular people and engines to other developers. Many players are not even aware of the engine and that it can run other games.

This creates a problem with Sider, especially since it's fresh and hasn't been proved yet. Many users won't care that it's a flexible engine. They're interested in concrete solutions, not in potential. At the same time I always wanted to charge for the use of the engine, not extensions (at least initially). I hope this will help Sider to stand out as new extensions will become available for free. I will need to make people aware that it is an engine and that's what they are paying for.

In addition anybody can extend it. History shows that some users (power users I call them) will be interested in customizing it and adding new features. Most people will just want to use them, but extensions created by power users are important to Sider success.

I think initially, it's best to focus on promoting extensions and scenarios they support, just mentioning that more will be available. When they are available, expand marketing to include them. In time, after several iterations and adding new extensions I can try to expand and capitalize on the fact that there will be support for a large number of things, because in the past we have added many.

I'm back to blogging. I'm gonna start by changing how I write posts, making them more un-official, like a journal. Not sure yet how that will work out.

I'm also back to working on Sider. I decided to make its development process more transparent. I want to be open and honest about what I'm working on, what mistakes I make and how I fix them.

I'm in the process of putting all project documentation in one place on the wiki where it's available to anybody. I started to review and publish Sider design documents. That shouldn't take too long, at least for the initial phase where I create an outline of all features. I'm not sure yet if I will publish everything, but I'm sure that most of it will be online. The wiki is opened for registration, so anybody can contribute. I have also opened read-only access to Sider issues.

In addition I recently learned about Scrum and want to try it as a project management methodology. I'll have more user stories (which I seem to like, unlike the use cases that are too detailed for my taste) and I'm going to estimate all remaining work using abstract points, then use velocity to approximate when the next release will be available. I will probably write here about my experience and thoughts.

I haven't written at the blog in a while. I realized that my interest has faded. I've been trying to push myself to create new posts, and that's just no fun.

I'm going to take a break from it. I don't feel that I'm done with the blog, so I'm expecting to get excited about it again. I hope it won't be longer than three or four weeks. Maybe more, maybe less, but I don't really know.

Feel free to stick around if you're OK with waiting for a little bit for new posts.

Thanks to all people who have being reading my posts for their time and support.

You can now use Mindjet Research Accelerator to search through EH forums and blogs. Research Accelerator is a plug-in to allow people search for information using different web services from inside of MindManager, MS Office applications and IE. You will need to download and install plug-in, and then add Community Server Research Service.

I'd like to thank Michael Scherotter from Mindjet for inviting and adding us to the list of supported websites.

It's nice that Mindjet is working on providing wider support for finding and importing data. It can make you more effective at finding and integrating additional information with your documents.

It would be really cool if imported information could be updated dynamically. However, integrating updates and doing version control in an application where you can edit data is hard. The problem is similar to what source control programs (SourceSafe, CVS, SubVersion, etc.) try to solve. Information in them can be changed in several places and it then needs to be merged in one place.

There are many applications specializing at gathering information from many sources (news aggregators, for example). Most of them are web-based and can gather and show news, blogs, etc. on a single page. Yahoo is probably the most popular service. The problem is usage. These applications can gather information, but they don't allow user to edit it, customize it (add custom notes) or connect several related articles. MindManager is now one step closer to supporting such scenarios.

I decided to change my blog format a little bit. I don't post as often as I need for the blog to be interesting, engaging and actually successful. Too often several weeks pass before I get to write a new post and I think by the time that happens, you lose interest in what I have to say.

My goal is to do 3-4 posts a week. Up to this point it was much less frequent because I turned blogging into a long and annoying experience.

It took me about 3-4 hours to complete one post. I wanted to completely cover the subject I wrote about, but in many cases I started out with an undeveloped idea. Thinking everything through and then writing a large post took a while and I made a lot of mistakes, which had to be corrected later. I guess, by making my blog topics large I lost focus, and doing anything without one is a hard, prolonged, and frustrating process.

I'm going to try and learn from my mistakes to post more often. I plan to limit time I dedicate to each post to about an hour. Idea is to make sure I don't feel like I'm spending too much time on it, and force me to write smaller, but more focused posts. This means I will often have to split larger topics into smaller parts, sometimes taking several posts to cover a single topic.

In the last post I divided existing information management tools (mentioning some in examples) into either open or strict-type categories. Today I continue by describing what I think is done right and what I would want to see improved.

Done right

Open tools

I'm happy with open tools. Most of them have been in development for over a decade and are helpful with general problems. Microsoft has done a good job with its Office and there are several alternatives, such as OpenOffice. You can use Microsoft Word (or something similar) if you need to write anything (a report, an article, or a note). Other examples are Adobe, Macromedia (which is part of Adobe now) and other companies, all providing good tools to work with pictures and graphics.

These tools are useful in many information management scenarios, but in most instances, they can handle only cases with simple data structure. When amount or complexity of information goes up open tools become harder and harder to use and strict-type tool designed for specific problem becomes more and more appropriate.

Simpler design

Another thing I like is a general shift towards less complex design. Applications like this with less features are becoming more popular, especially web applications. This is a good change from old feature racing (where every program would try to have more features than competition), although I don't agree that cutting features out is always appropriate. I think Einstein's, "As simple as possible, but no simpler" is a good guideline in design. Eventually we might end up with applications that are too simple to be practical.

What can be improved

Strict-type tools

There exist a lot of strict-types tools that try to address specific problems. Many of them are good, but I think most need to be better. When the tool fits the problem, great, however it often doesn't. Designing and developing a good tool is difficult. It's not easy to understand the problem (because usually there's a set of problems to address) and to come up with the best solution (there can be many solutions, each with its plusses and minuses, and on top of that different people might have different preferences on how to do something). Finally it's hard to balance resources (both development and of the user PC). As a result, a lot of strict-type tools have good potential, but can see a lot of improvements.

All tools have common problems

Almost all tools have very similar problems. Few tool supports substantial source control (saving history of changes to see and compare document modifications through its lifetime). Usually there's limited support for collaboration and security. The list goes on, but the whole situation is improving.

Strict-type vs. open

Strict-type tools are often not strict enough on one hand, but on the other are too strict, making them harder to use. For example, wiki supports flexible structure to store notes in a graph, with one note being able to reference any other including its parent. They can be strict and often don't allow notes with the same names, their wiki-text is often too simple (creating tables is too hard), etc. At the same time, note (page) doesn't have any text structure, so it's impossible to enforce certain structure, where user has to fill out some fields for a new note.

What can be done

I don't really have a good answer. Designers need to always keep in mind user needs and make interface simple and intuitive. Things are moving forward. Every new generation of tools gets better. We are starting to see more flexible, context-based interface and support for larger, more complex problems. With Internet emergence demand for online tools is growing, and developers have to address that as well.

Our solution, so to speak, is Sider. We're trying a different approach, creating an environment to help with applications development. We believe in strong-type tools are the best solution for many information management problems, but at the same time developing specific tool from scratch requires a lot of effort. Sider addresses this by supporting higher-level languages to define logic, structure and interface, and to bind them together (through XML and browser technologies). It also promotes reuse of parts of existing tools through inheritance, composition, referencing and other mechanisms.

We also want to address a set of common problems I have mentioned previously. Since all tools will run on a single platform, it's possible to share horizontal features (like security or capturing document history) with all Sider-based tools.