Some workers on MTurk are extremely active, and take the majority of posted HITs. This can lead to many issues, some of which are outlined in our previous post. Although MTurk has over 100,000 workers who take surveys each year, and around 25,000 who take surveys each month, you are much more likely to recruit highly active workers who take a majority of HITs. About 1,000 workers (1% of workers) take 21% of the HITs. About 10,000 workers (10% of workers) take 74% of all HITs.
1 Comment Click here to read/write comments
Topics: active workers, amazon mechanical turk, exclude, exclude workers, HIT, mechanical turk, mturk, online research, turkprime, workers, active
How to Create a Universal Exclude Worker List
Posted by
Jonathan Robinson on Sep 9, 2016 10:41:00 AM
Problem:
Requesters may observe that some workers, even those with high Approval ratings, may not perform to their expectations on a study. Sometimes this may result in rejecting their work which affects the Worker approval rating. But, often the work is not acceptable for research but is not worthy of rejection, or, it may simply be the policy of the research lab to approve all assignments for IRB or some ethical standard they may follow.
0 Comments Click here to read/write comments
Topics: amazon mechanical turk, block worker, exclude, exclude workers, mturk, mturk api, qualification, turkprime
Problem:
You are running a longitudinal study and have identified 1000 workers who you want to allow to take your second phase studies. How do you easily group those workers for easy access.
Or you want to exclude certain workers from taking a number of your studies and wish to group them for easy exclusion in future studies. How can you do that?
0 Comments Click here to read/write comments
Topics: amazon mechanical turk, exclude, exclude workers, include workers, Longitudinal, qualification, turkprime, worker groups
Survey Groups Ensure Unique Workers Across Multiple Concurrent Studies
Posted by
Jonathan Robinson on Aug 23, 2016 2:41:00 PM
Problem
Have you ever wanted to run multiple studies simultaneously and make sure that each worker only takes a single study? On MTurk, there is no simple way to block workers who completed one study from accepting and completing another study being run at the same time....until now.
(Note: TurkPrime's exclude feature excludes workers who have completed one study from taking another subsequent study, but not if both studies are being run at the same time.)
0 Comments Click here to read/write comments
Topics: amazon mechanical turk, exclude, exclude workers, follow up, qualification
Problem:
Suppose you're running a Mechanical Turk survey and need to exclude workers who took a previous survey. How can you quickly set this up.
Some of the currently used solutions require following multiple steps to set things up and are not turnkey solutions and others require Workers to enter their Worker ID, which may self-filter workers and limit the number of workers taking your survey.
0 Comments Click here to read/write comments
Topics: amazon mechanical turk, exclude, exclude workers, mechanical turk, mturk, turkprime, workerid