In this blog, we explain everything you could ever want to know about including and excluding participants from studies while using TurkPrime. In last week’s blog on longitudinal studies, we described our Include Workers feature, but this blog digs into the nitty-gritty and explains what our features are, when you might want to use them, and how they work.
Topics: exclude workers, include workers, prime panels, mturk toolkit, TurkPrime Features
Some workers on MTurk are extremely active, and take the majority of posted HITs. This can lead to many issues, some of which are outlined in our previous post. Although MTurk has over 100,000 workers who take surveys each year, and around 25,000 who take surveys each month, you are much more likely to recruit highly active workers who take a majority of HITs. About 1,000 workers (1% of workers) take 21% of the HITs. About 10,000 workers (10% of workers) take 74% of all HITs.
Topics: active workers, amazon mechanical turk, exclude, exclude workers, HIT, mechanical turk, mturk, online research, turkprime, workers, active
Hundreds of academic papers are published each year using data collected through Mechanical Turk. Researchers have gravitated to Mechanical Turk primarily because it provides high quality data quickly and affordably. However, Mechanical Turk has strengths and weaknesses as a platform for data collection. While Mechanical Turk has revolutionized data collection, it is by no means a perfect platform. Some of the major strengths and limitations of MTurk are summarized below.
Topics: amazon mechanical turk, demographics, exclude workers, google form mechanical turk, HIT, mechanical turk, mturk, mturk api, panels, qualification, study, turkprime panels, unique worker, worker groups, workers
Many researchers wish to target participants from specific states or regions of the United States like the Northeast or the West. The problem they often encounter is that using MTurk's Geographic Qualification to specify a particular state is often not adequate to ensure participants actually reside in the specified state.
Topics: amazon mechanical turk, exclude workers, IP address, IP block, location, region, survey, turkprime
Requesters may observe that some workers, even those with high Approval ratings, may not perform to their expectations on a study. Sometimes this may result in rejecting their work which affects the Worker approval rating. But, often the work is not acceptable for research but is not worthy of rejection, or, it may simply be the policy of the research lab to approve all assignments for IRB or some ethical standard they may follow.
Topics: amazon mechanical turk, block worker, exclude, exclude workers, mturk, mturk api, qualification, turkprime
You are running a longitudinal study and have identified 1000 workers who you want to allow to take your second phase studies. How do you easily group those workers for easy access.
Or you want to exclude certain workers from taking a number of your studies and wish to group them for easy exclusion in future studies. How can you do that?
Topics: amazon mechanical turk, exclude, exclude workers, include workers, Longitudinal, qualification, turkprime, worker groups
Have you ever wanted to run multiple studies simultaneously and make sure that each worker only takes a single study? On MTurk, there is no simple way to block workers who completed one study from accepting and completing another study being run at the same time....until now.
(Note: TurkPrime's exclude feature excludes workers who have completed one study from taking another subsequent study, but not if both studies are being run at the same time.)
Topics: amazon mechanical turk, exclude, exclude workers, follow up, qualification
We recently launched a ground-breaking feature that helps protect Mechanical Turk worker identities. It has been reported in the literature that Mechanical Turk Worker IDs can be used to identify the worker. This is because Amazon uses the same value for both the Worker ID on Mechanical Turk and elsewhere on Amazon properties like Amazon.com product reviews.
Topics: amazon mechanical turk, anonymize, anonymous, exclude workers, HIT, include workers, mechanical turk, mturk, turkprime
If you're looking for a video tutorial that explains how to setup your account on TurkPrime, how to run a longitudinal study, or how to limit who takes your HIT on Mechanical Turk, these resources are for you.
Topics: amazon mechanical turk, exclude workers, HIT, include workers, mturk, turkprime
Suppose you're running a Mechanical Turk survey and need to exclude workers who took a previous survey. How can you quickly set this up.
Some of the currently used solutions require following multiple steps to set things up and are not turnkey solutions and others require Workers to enter their Worker ID, which may self-filter workers and limit the number of workers taking your survey.
Topics: amazon mechanical turk, exclude, exclude workers, mechanical turk, mturk, turkprime, workerid