In the last post I divided existing information management tools (mentioning some in examples) into either open or strict-type categories. Today I continue by describing what I think is done right and what I would want to see improved.

Done right

Open tools

I'm happy with open tools. Most of them have been in development for over a decade and are helpful with general problems. Microsoft has done a good job with its Office and there are several alternatives, such as OpenOffice. You can use Microsoft Word (or something similar) if you need to write anything (a report, an article, or a note). Other examples are Adobe, Macromedia (which is part of Adobe now) and other companies, all providing good tools to work with pictures and graphics.

These tools are useful in many information management scenarios, but in most instances, they can handle only cases with simple data structure. When amount or complexity of information goes up open tools become harder and harder to use and strict-type tool designed for specific problem becomes more and more appropriate.

Simpler design

Another thing I like is a general shift towards less complex design. Applications like this with less features are becoming more popular, especially web applications. This is a good change from old feature racing (where every program would try to have more features than competition), although I don't agree that cutting features out is always appropriate. I think Einstein's, "As simple as possible, but no simpler" is a good guideline in design. Eventually we might end up with applications that are too simple to be practical.

What can be improved

Strict-type tools

There exist a lot of strict-types tools that try to address specific problems. Many of them are good, but I think most need to be better. When the tool fits the problem, great, however it often doesn't. Designing and developing a good tool is difficult. It's not easy to understand the problem (because usually there's a set of problems to address) and to come up with the best solution (there can be many solutions, each with its plusses and minuses, and on top of that different people might have different preferences on how to do something). Finally it's hard to balance resources (both development and of the user PC). As a result, a lot of strict-type tools have good potential, but can see a lot of improvements.

All tools have common problems

Almost all tools have very similar problems. Few tool supports substantial source control (saving history of changes to see and compare document modifications through its lifetime). Usually there's limited support for collaboration and security. The list goes on, but the whole situation is improving.

Strict-type vs. open

Strict-type tools are often not strict enough on one hand, but on the other are too strict, making them harder to use. For example, wiki supports flexible structure to store notes in a graph, with one note being able to reference any other including its parent. They can be strict and often don't allow notes with the same names, their wiki-text is often too simple (creating tables is too hard), etc. At the same time, note (page) doesn't have any text structure, so it's impossible to enforce certain structure, where user has to fill out some fields for a new note.

What can be done

I don't really have a good answer. Designers need to always keep in mind user needs and make interface simple and intuitive. Things are moving forward. Every new generation of tools gets better. We are starting to see more flexible, context-based interface and support for larger, more complex problems. With Internet emergence demand for online tools is growing, and developers have to address that as well.

Our solution, so to speak, is Sider. We're trying a different approach, creating an environment to help with applications development. We believe in strong-type tools are the best solution for many information management problems, but at the same time developing specific tool from scratch requires a lot of effort. Sider addresses this by supporting higher-level languages to define logic, structure and interface, and to bind them together (through XML and browser technologies). It also promotes reuse of parts of existing tools through inheritance, composition, referencing and other mechanisms.

We also want to address a set of common problems I have mentioned previously. Since all tools will run on a single platform, it's possible to share horizontal features (like security or capturing document history) with all Sider-based tools.

In our lives we constantly deal with information. I've talked about what people try to do with it, but there are limits to both our physical and mental activities. To make everything easier we invent tools. There are many different kinds of tools to work with information, yet there're a lot of common aspects to their usage and design.

All tools help us reach our goals, help us with our pain, etc. Many people have different definition of what tools (programs) should do, but in the end it's all the same. A tool must solve more problems than it creates or there's no point in using it (although, sometimes it's hard to see all problems that are going to be created, before some of the existing ones are solved). As you can see from my previous post I assume people have two major goals when managing information, to understand it and to share it.

Every information management tool does only one thing. It stores information and shows it back to us. There can perform a lot of different operations, but they simply extend our abilities, allowing us to work with more information simultaneously, more accurately and to visualize it differently.

I'm mostly focusing on computer-based tools, but there are other kinds, such as writing/drawing on a physical medium (i.e. paper and pen). Paper is very flexible, but not as scalable as computers and doesn't support data manipulation well. It's easy to put just about any type of information on paper: text, pictures, diagrams, numbers, graphs, etc. and more importantly it's easy to combine different information. However, it's hard to change it once it's written down. In addition capturing, organizing (combining sheets of paper) or finding what you need becomes harder and harder as the ammount of information increases.

Before we can use any tool, we need to transfer information into it. We often understand it better in the process because in many cases we organize information as we input it. For example, when we talk to somebody about a problem we are trying to solve, we actively analyze it. As a result we can find solution without other person saying anything because we finally organized information in a pattern we came to recognize.

I want to classify information management tools by how general or strong-typed they are. On one end of the spectrum there's pen and paper (allowing us to enter information in a free form without limitations). Less extreme examples of open tools are Word and Excel. They work with specific structure of the information (linear text or tables), but the actual structure is quite open. It's possible to store a lot of different information in Word, ranging from books and articles to notes and to-do list. Excel support for tables is suited for a lot of uses.

On the opposite end of the spectrum there are tools work strictly with specific information (strong-typed tools). For example Outlook can be use to manage tasks. Each task has a strict structure and tasks are organized in a list. Most of the tools fall somewhere in-between, and there are good and bad aspects to being more open or more strong-typed.

Open tools are easier to adopt for different information scenarios. It's possible to analyze mathematical data, to work with finances and to manage a small project with Excel. Even though the tool is open, it's usually easier to work with information organized in a strict structure. The beauty of open tools is that they allow the user to invent a simple system to simulate strict-type tool (often without realizing they are doing it) and so well-designed open tools are very popular. Good tool will actually provide features to help simulate such strict-typeness (through templates, custom attributes, and so on).

There are two problems however: scalability and potential lack of features. It becomes harder to implement features for tools that support larger variety of information. For example operations to work with plain text will be simpler than operations to work with rich formated text.

Scalability problems appear because users try to use open tools not designed to handle such scenarios. Excel might be suitable when managing small project, but it becomes harder to use when the scope of the project grows. Big chunks of information need to be organized and grouped for the human to be able to comprehend it. We can work with 10-20 items in a list, but working with 200, or a 1000 items in a list would be very hard. Designing and implementing support to organize information of arbitrary type and structure is hard. If it's not done right (assuming it's even possible) user will be forced to adjust to how the program works, and it loses its openness.

Strict-type tools work much better than open tools with information that matches their design. For example, a project management application would have more potential than Excel or Word when used to manage projects, from capturing information (tasks) to analyzing and tracking progress. It could also support different information structures (such as arranging tasks in a tree), users, and so on, and provide more operations, such as grouping progress by iterations, tracking time, filtering. When information fits well with the tool it can be very easy to use, but even a small deviation makes it harder. The tool imposes on the user what information it can work with and if its design doesn't exactly fit the need there's a problem. We try to adjust it for what we need, but it's most likely not as flexible as an open tool (and it wasn't designed to be as flexible). For example if project management tool doesn't support time estimation, and we need it, we can add our estimation in parenthesis after the task title.

Strong type tool is easier to use and provides more options, when information fits it and when the tool is designed for the tasks you want to perform. Open tools are more flexible. They allow you to invent your own way to simulate strict types and structure information. When available I prefer strong-type tools, they are easier to use and don't force me to invent something. In the best cases it fits me, my information and my problems like a glove. If it doesn't, I fall back to open tools (like Word and Excel).

I was going to talk about design, what is done right and what can be improved in the existing tools, but I felt it was important to think about what kind of tools are out there first. Wow, this post is probably one of the longest I've ever written and it took me many hours to think about all this and write it out. I really hope some good will come out of this series on information management. Anyway, I'm planning to address tools design in the next post.

In my last post I talked about information and what it is. Today's post is about how people use it and work with it.

We use information to decide what to do and to communicate. Let's try to see what we can learn from this description, even though it's pretty simple and abstract.

All our actions are based on our understanding of the world around us. Most of the time we process information to improve that understanding to make better decisions based on it. We work with information to understand it better, to find meaning in it. We simplify it, arrange it in familiar patterns and relate to existing information (effectively incorporating what we learn with what we already know).

Communicating information is sharing it with other people (or ourselves when we forget something). Effective communication can be hard, even if you are just leaving notes for yourself. We need to understand information involved (like we do when deciding what to do) or remember and reproduce exact data (but people are usually pretty bad at that). In addition we need some way to store and present information. Quite often we present it in the same form as it is stored. When we write in the notebook we see everything in the form it is written (stored).

It seems that how we work with information is similar whether we want to communicate it or do something. In either case we need to understand it well. Having a good method to store and present it is also very helpful. This is a list of goals we usually have when working with information:

• Simplify - Like computers, people have limited memory and can process only so much information effectively. We want to reduce amount of data we work with to make it easier for us. To do that we can remove any duplication and irrelevant data, and enumerate similar data by assuming it is the same. The downside is that we can get less accurate picture as we eliminate data, so balance of how much information is simplified is important
• Capture information - Store it on some media. This is important both to share it and to analyze it. It's important for analysis because we, again, have a limited memory problem and can often process only a portion of data. We need to store the rest of the data while we are working on a part of it to be able to come back to it later.
• Capture relationships - Describe how smaller pieces of data relate to each other and how new information is related to existing information (where it fits in the larger picture).
• Display - How we see the information determines what it means to us and how we work with it. We often choose the most suitable way to show it depending on what we want to do. For example people are much better at understanding pictures then numbers. Yet numbers are often much more accurate then pictures.

Correct tools can greatly help us with any of the above operations. We have been inventing them since the beginning of mankind. Actually they are so common now that we use a lot of them without much thought, writing in a notebook for example. Yet I think tools and how we use them can be greatly improved to make our lives easier. In the next post I want to explore that to see what's done right, what's done wrong and can be improved.

Today I want to start a series of articles on information management. I want to look at what information is, how it relates to our lives, why it's important to think about it, how people work with it, etc. My goal with this and the following posts is to better understand the nature of the information to design better information management tools.

To understand what it is we need to take a look at data. Data is a description of an object, relationship between objects or a process. I couldn't come up with a better definition. Numbers, letters, pictures, sounds, smells are all data and they represent something to us.

Most of what we perceive in the world is gradual. There are no distinct borders between colors. On the spectrum one color changes into the other. Time is continuous as well. We created minutes and hours to measure it. We enumerate and simplify data naturally (often without thinking about it) to make it easier for us to work with it. Different pieces of data describe a small characteristic of an object, relationship, process. Looking at a brick we can figure out its size, material it is made from, weight, etc. All of these describe different aspects of the brick. We enumerated information on the brick. Without doing it it would be hard to calculate how many bricks we will need to build a house or how tall it can be before it will collapse. Use of correct tools can help us capture this information and use it more effectively.

Information is a collection of data organized into a pattern. Humans are very good at recognizing patterns and finding associated meaning. Keep in mind that data and information are objective, but their meaning is very subjective. A brick is a brick no matter who looks at it. Associated meaning (how we see it) can differ. One person might decide to use it for construction, another to use it as a weapon, while third uses it as a paperweight.

When looking at the information we might recognize it. In that case we automatically recall associated meaning. When we look at a red, rectangular stone we recognize it as a brick. Actually we like to recognize patterns and understand the information we are presented with.

Patterns we fail to recognize are noise and are very, very boring because we think they are useless. TV snow is a random pattern of black and white dots. We might recognize that something is wrong with TV, but we probably will not be able to understand all signals that force dots to be arranged in one pattern and not the other.

In the next post I want to talk more about how people work with information. This is directly related what kind of tools can help them do their job better.