A while ago I blogged about the problems with desktop Linux, and hopefully by now you've taken my advice and visited the Better Desktop project. Now, you may well be thinking, "but that's Novell! I can't possibly afford such usability testing." You're right: getting people into a testing area, watching what they do with an array of video camera, etc - that all takes money. But you can still do a whole lot to see how people are using your software and where you can make improvements.
A good place to start is at Microsoft. Surprisingly, it's a company that goes to a lot of effort to see what customers say about its products. If you've ever had the joy to use Internet Explorer, you might find all this hard to believe. But have you seen the new interface planned for Microsoft Office 12? MS is well aware that such a drastic change may push people in the direction of OpenOffice.org, but what it is banking on is that its extensive research into usability will pay off by placing a sizeable distance between MS Office and OpenOffice.org, thus re-enforcing their core market.
So, what can you learn from them? Well, let me show you this picture:
Crashes being as common as they are on Windows, most Microsofties will have seen this screen sooner or later. Oops, something has crashed! But despite the dire situation your software is in, Windows offers you the chance to send a bug report to Microsoft with one click of your mouse, along with a vague promise that they might actually read it. So far, so good.
How about this one:
At the bottom there you can see the invite to join the Customer Experience Improvement Program, which sends quality and reliability data back to Microsoft. Nothing has crashed; they just want to know how many windows you keep open, how many emoticons you like to use, how many people are in your address book, etc. They don't care about what you're saying or who you're talking to - they want empirical data that can be used to improve the product. Now we're getting somewhere.
Try this one:
That's the Customer Experience Improvement Program system for Office 2003. This time they say they will collect hardware information along with how you use the product. It even provides a special advantage: it tracks what error messages you receive, and offers to download new documentation for those errors when the help files are updated. Again, note that MS goes to a lot of bother (the whole last paragraph, in fact) to assure anonymity and ease of use.
Finally, have a look at this:
That's the installer for the new Visual Studio. Look on the left, and you'll see that they have added usability tracking to their installer too. This sends back your installation options, error messages, etc, so that MS has a better idea of how people are installing their software and what functionality they are choosing.
So installing, working with Office, chatting with your friends, and crash reports - all contribute data back to Microsoft. After at least three years of collecting this data, is it any wonder Microsoft feels confident in radically changing its Office user interface?
Now back to the point here: we can do all this in open source on a shoestring budget. It doesn't take any video cameras or surveys just to watch how people use your program, and it doesn't take that many people agreeing to take part to really make a difference. If Gnome, Gimp, etc had an option to report usage back to the developers, I know for a fact I would turn them on because it would be one more way I could give back to the open source community. What's more, I think many people feel that anything we can do to help, we'll gladly do.
Developers: take advantage of this goodwill, and let users sign up to provide passive feedback with your programs. You never know - what you learn might surprise you...