Hundreds of academic papers are published each year using data collected through Mechanical Turk. Researchers have gravitated to Mechanical Turk primarily because it provides high quality data quickly and affordably. However, Mechanical Turk has strengths and weaknesses as a platform for data collection. While Mechanical Turk has revolutionized data collection, it is by no means a perfect platform. Some of the major strengths and limitations of MTurk are summarized below.
0 Comments Click here to read/write comments
Topics: amazon mechanical turk, demographics, exclude workers, google form mechanical turk, HIT, mechanical turk, mturk, mturk api, panels, qualification, study, turkprime panels, unique worker, worker groups, workers
New Safety Feature: Assignment Rejections are not Automated and Require Manual User Action
Posted by
Jonathan Robinson on Feb 19, 2016 10:39:00 AM
Many researchers set up their studies to use the TurkPrime AutoApprove feature so that they do not need to manually approve worker assignments based on the secret code that workers enter. On occasion, a researcher may set up his study incorrectly which results in many worker assignments getting automatically rejected. This was a significant cause for distress among the Mechanical Turk workers who received rejections for their work which then went unpaid and also lowered their MTurk approval rating. This was also a sore point for TurkPrimeresearcherswho had to deal with upset workers and correspond with them, and often reverse their rejections.
0 Comments Click here to read/write comments
Topics: amazon mechanical turk, autoapproval, reverse rejection, study, survey, turkprime