A Brief Overview of TurKit

These slides are from a presentation I gave in Sep Kamvar‘s Computational Methods in Data Mining (old website link here). In the presentation, I presented TurKit, a programming framework created by Greg Little and others at MIT that allows for programmatic iteration over tasks in Mechanical Turk. Essentially, that means that instead of the familiar paradigm of sending out a bunch of HITs and waiting for the responses, TurKit will ping AMT for answers and these answers can be used in future HITs. This allows for the use of an “improve and vote” loop, where Turkers continually improve on and validate the work of other Turkers. They had some impressive results in the paper, getting fairly high quality responses to a wide range of tasks (including image labeling, handwriting recognition, and brainstorming) for under $0.50.

The presentation ends with a quick intro in the JavaScript code (from the Iterative Text Improvement example on the TurKit website) and some hopefully helpful information to know when using the Java application that you can download to try out TurKit. If I survive the end of the quarter, I hope to get a post up with some TurKit tutorial tips and lessons learned. If you have questions about TurKit, let me know, and I’ll try to get them answered in the next post!

Related posts:

1. Smartsheet Automating Data Collection Through Mechanical Turk
2. A MTurk Exploration of Activity Stream Usage