Posts Tagged ‘mturk ’
These slides are from a presentation I gave in Sep Kamvar’s Computational Methods in Data Mining (old website link here). In the presentation, I presented TurKit, a programming framework created by Greg Little and others at MIT that allows for programmatic iteration over tasks in Mechanical Turk. Essentially, that means that instead of the familiar paradigm of sending out a bunch of HITs and waiting for the responses, TurKit will ping AMT for answers and these answers can be used in future HITs. This allows for the use of an “improve and vote” loop, where Turkers continually improve on and validate the work of other Turkers. They had some impressive results in the paper, getting fairly high quality responses to a wide range of tasks (including image labeling, handwriting recognition, and brainstorming) for under $0.50.
I decided to try a little Mechanical Turk study to see if I could spot some differences between tags generated by experts and those generated by novices. I had each Turker read 1 of 5 web pages (on the topic of “enterprise 2.0 mashups”) and enter 5 tags which they thought would be useful for bookmarking the page (either for themselves or others). I also asked them to rate how familiar they were with the subject matter (“Not at All”, “Slightly Familiar”, “Somewhat Familiar”, and “I am an Expert”)…
Last month, I read a study by B. J. Fogg and others from the Persuasive Technology Lab at Stanford, entitled “What Makes Web Sites Credible? A Report on a Large Quantitative Study”. The paper described an early effort to systematically determine how different elements of web sites affect people’s perceptions of credibility (defined roughly as the intersection of trustworthiness and expertise). The original study design had 1400 participants completing a survey which presented them with 51 web site elements and asked them to rate how much more or less each element would affect the believability of a web site. The two questions I hope to answer, roughly, are “What has changed in the past 10 years about how people assess web site credibility?” and “Is there a cheaper, yet effective, way to do a study like this?”. The results have implications for website design.
These are some slides from a presentation I gave on some Mechanical Turk data I collected about how people are using Activity Streams. Specifically, I was interested in what tools people were using, what they were using them for, how these tools might be improved, and how people had been using these tools to collaborate/coordinate. Here’s what I found…
