Back when I was just starting graduate school, I remember already feeling as if I understood the components needed for great scientific research: knowledge of a domain, the ability to implement a system or execute an experiment, and a creative insight about a phenomenon worth studying. While the domain knowledge and ability to execute seemed like pre-requisites for doing science at all, the capacity for creativity seemed to the element that separated a great scientist from the good. Since I felt like I was good at identifying creative research, I hoped that once I immersed myself in academia and started gaining domain knowledge and engineering skill, the creative ideas would come to me. Now, almost a year into my PhD program, I feel like I have learned a great deal, but I am left with the question: Where are all those good ideas?
I decided to try a little Mechanical Turk study to see if I could spot some differences between tags generated by experts and those generated by novices. I had each Turker read 1 of 5 web pages (on the topic of “enterprise 2.0 mashups”) and enter 5 tags which they thought would be useful for bookmarking the page (either for themselves or others). I also asked them to rate how familiar they were with the subject matter (“Not at All”, “Slightly Familiar”, “Somewhat Familiar”, and “I am an Expert”)…