Hundreds of academic papers are published each year using data collected through Mechanical Turk. Researchers have gravitated to Mechanical Turk primarily because it provides high quality data quickly and affordably. However, Mechanical Turk has strengths and weaknesses as a platform for data collection. While Mechanical Turk has revolutionized data collection, it is by no means a perfect platform. Some of the major strengths and limitations of MTurk are summarized below.
0 Comments Click here to read/write comments
Topics: amazon mechanical turk, demographics, exclude workers, google form mechanical turk, HIT, mechanical turk, mturk, mturk api, panels, qualification, study, turkprime panels, unique worker, worker groups, workers
Problem:
Requesters may observe that some workers, even those with high Approval ratings, may not perform to their expectations on a study. Sometimes this may result in rejecting their work which affects the Worker approval rating. But, often the work is not acceptable for research but is not worthy of rejection, or, it may simply be the policy of the research lab to approve all assignments for IRB or some ethical standard they may follow.
0 Comments Click here to read/write comments
Topics: amazon mechanical turk, block worker, exclude, exclude workers, mturk, mturk api, qualification, turkprime
HyperBatch Feature
You can now run your Amazon Mechanical Turk studies at Hyper speeds. We worked hard to make the experience identical to our classic MicroBatch option so that you simply launch your study and TurkPrime does all the heavy lifting for you.
1 Comment Click here to read/write comments
Topics: amazon mechanical turk, HIT, hyperbatch, mechanical turk, microbatch, mturk, mturk api, mturkgate, turkprime
Problem:
How can a Requester who rejected an assignment in error undo his mistake? A rejected assignment affects the Worker negatively and will often impact the Requester with negative feedback which can damage the Requester's online reputation which lowers Worker participation in future HITs. What can a Requester do to reverse the rejection?
0 Comments Click here to read/write comments
Topics: amazon mechanical turk, increase participation, mturk, mturk api, requester, reverse rejection, turkprime
Problem:
Suppose you need to run a group of HITs open only to participants who are women under 50. You previously ran a HIT and know the Worker IDs that you want to reach, but have no way to email them and limit your survey to only them. How can you proceed?
0 Comments Click here to read/write comments
Topics: amazon mechanical turk, mechanical turk, mturk api, panels, qualification, worker groups
Amazon just announced that their Worker Qualification now supports US State locations. It is currently available through their API and is also available through their Web Interface.
0 Comments Click here to read/write comments
Topics: amazon mechanical turk, mturk, mturk api, qualification