WEBlog -- Wouter's Eclectic Blog

Fri, 03 Mar 2006

Scientific studies on Open Source

It's strange, but ever since I have been a Debian Developer, people start assuming I'll be interested in any random study. They'll put up a web page (which often doesn't even validate), and then start asking strange questions that have no link with reality whatsoever.

On the one hand, I have better things to do than to cooperate with random studies from people I've never heard of before and probably never will hear of again. On the other hand, only by understanding will people see the light, and only by cooperating with such studies will one help people understand. But seeing the same mistakes over and over again is annoying me. So, here's Wouter's HOWTO perform a study on Open Source or Free Software (and get away with it without annoying people)

• Make sure you tell people up-front how much time your questionnaire will take. It's annoying as hell to see that after pressing 'submit', there's a second page which is at least as long as the previous one. And a third. And a fourth. Especially if you didn't plan for that, and have to leave soonishly.

The best way to do this is to either have all questions on one giant page, or (if that makes it too long) test-drive your questionnaire with a chronometer nearby.

• If your questionnaire contains multiple pages, don't break the back button. One questionnaire I participated in did this, resulting in me going back to change the answer to a question, and then suddenly finding that I couldn't go to the final page anymore, because the system thought I was submitting that page.
• If you have multiple choice questions, make sure the answers make sense. Having a question like Where do you use Linux? with possible answers at home, at work and at school is nice, but not if you can not choose more than one of them at the same time.
• Additionally for those of you doing multiple choice: make sure you have at least one open question, so that people can leave feedback about things they consider you left out. You're free to ignore that feedback in the final results of your study, but it may help you in better understanding the FLOSS movement.
• If English is not your native language, find someone who is very fluent in English and have them check it out. This is important because if your question is unclear, I might give you the wrong answer—which will make the results of your study rather worthless.
• Test your questionnaire on someone else before you ask hordes and hordes of FLOSS developers to fill it out, especially if you're not too familiar with the FLOSS movement yourself. FLOSS is very popular these days, it must be possible to find someone in your vicinity who's working with it, or who at least has a feel for how the community works. Their feedback may help in avoiding questions that are based on incorrect assumptions, and/or get out little things you didn't see yourself first.
• Do not mail people in private, but instead ask people in public about filling out the questionnaire. Remember the "Open" in "Open Source".
• Finally, be sure to explain what the study is trying to discover, what is going to happen with its findings, and where I can read the results when they're there. Not that I want to see the results of any possible study I participate in, but one never knows—I might be interested.

That's the most important bits, I guess. I'll probably add more in the future, if I see more things that annoy me. But for the time being, if you break three or more of the above rules, don't expect me to participate—I have better things to do with my time than to help every undergraduate on this planet with his or her study

/en/life/debian

PermaLink