Search and Social

4 Tips: How to Use mTurk For Marketing Research at $1.21/Hour

Amazon Mechanical Turk - mTurk

by Ian Orekondy on February 21, 2011

I’ve been using Amazon’s Mechanical Turk to get vast quantities of work done quickly and cheaply over the past several months.  The tool has evolved from it’s earlier days, and many people I’ve spoken with have outdated ideas about what the tool can help you achieve.  It’s not the easiest tool in the world to use (like Fiverr.com), but if you know how to start out the right way, your chances of success are greatly improved. Here are my tips on how to use mTurk for the first time to easily get quality results.

Set Reasonable Expectations for Yourself on the First Pass

Expect that your first mTurk HIT request probably won’t be completed perfectly the first time. You really have to test your ability to write clear, concise directions; see what you get back and then refine.  Don’t spend too much $ on the first test, because you may have to do it a couple times.

Avoid combining too many tasks in a single HIT request.

For example, if your job requires someone to go to Google, search for a keyword, and then enter data into a spreadsheet AND then cross-reference that data against another resource, that combination of multiple tasks might be enough to slow down the mTurk worker (aka “turk”), confuse them, or just generally lead to lower quality.

Specifically, here’s what you should do:

  1. BREAK OUT jobs into specific tasks and SEPARATE the tasks as unique HITs.
  2. CREATE numbered bullets for each micro-task within each HIT.
  3. USE all-caps on the action words.
  4. PROVIDE examples of each step successfully executed.

This is the basic approach I’ve used for hundreds of successful mTurk HITs, and the benefits are:

  • More accurate / better quality work
  • Faster results
  • Happier Turks that are eager to work with you again (and at lower rates)

Eric W. Kratzer, founder and CEO of Open Run Studios, an internet agency specializing in Online Marketing, Branding, Strategy and Production for the Performing Arts, approached me to help his team accelerate their new business prospecting process and reduce their marketing research costs. After implementing my suggestions regarding their initial mTurk HIT, Eric said:

Ian…I took your advice and broke our original [HIT] into two steps. I ran the first HIT last night and it was ready for me by this morning. It looks like it came out perfectly on the first go! The average wage was $1.21/hour on our first project and $.52/hour on our second.. Thank you so much for inspiring us with your own mTurk experience and advice!

What Tips Would You Suggest on How to Set-Up Effective mTurk HITs?

Related Posts with Thumbnails
  • Share/Bookmark

{ 5 comments… read them below or add one }

Eric Wayne Kratzer February 21, 2011 at 3:22 pm

It amazes me that this rate is considered a desirable living wage in much of the developing world! The result is a win-win for all parties involved – we save money with our start-up and a developing world technology worker puts food on the table for his/her family. I would gauge the value created from our tiny experiment to be about a thousand dollars. Projects on a larger scale would have no limits on savings.
The reliability of results increased when we increased the Hit Approval rate (%) to greater than or equal to 98% (during the design phase of hit creation). At first we tried 99%, but there weren’t enough workers participating.

We haven’t tried it yet, but it seems that if you name the description for your hit with a letter at the beginning of the alphabet that you’re more likely for your hit to have greater visibility by workers because it will appear at the top of the list when they sort hits alphabetically.

Reply

Ian Orekondy February 21, 2011 at 5:37 pm

Hey Eric – great point about using the Criteria setting to filter in only high quality turkers to work on your HITs. I’ve found that allowing only turks with 95% approval ratings or better to work on my HITs to be a good balance between quality and quantity.

You could also consider geo-targeting turks for certain projects. I’ve tested geotargeting to only US turks and layering on 95% approval rating, but that seems to slow down the hit acceptance timing. Leaving the geo criteria as-is (global) seems to get me Indian turks almost without exception, which is fine with me, since the quality of their work is still very good, and fast.

Regarding the low wages, I’ve found that the India turks are mostly college students, usually at the Indian Institutes, and they’ve devised some ingenius ways to get various Internet research and Microsoft Excel work done very quickly. I also keep in mind that $5 can last someone a week in India, and that most of those turks are making money while they’re in school – not using mturk as their full-time job.

Plus, you know your turks are happy when they request that they send future projects to them and only them, which I’ve had happen several times. I believe it’s against the Terms of Service, but if the turks contact you directly somehow (e.g. they find your contact info on your website and then ping you on gChat), it’s actually a nice experience to interact with your workers.

Glad you’ve found success based on my mTurk advice – wish you the best of luck with future projects!
Ian Orekondy recently posted..How to Use mTurk to Get Quality Results the FIRST TimeMy ComLuv Profile

Reply

Jack October 15, 2011 at 2:06 pm

You tips regarding marketing research are really fantastic. They are very easy to implement and get brilliant results. Thanks for sharing your experience, keep up the good work.
Jack recently posted..ForexMy ComLuv Profile

Reply

gethsi October 16, 2011 at 5:19 am

What does it mean by turks? anyway, outsourcing is not only for Indian nationals but also with other Asian countries who are very much active on doing works. But please don’t exploit them much, though your 1 dollar can get one student to and from from school on our place but it seems to very low for a job well done. Thanks!
gethsi recently posted..first date outfitMy ComLuv Profile

Reply

Dan Toombs November 6, 2011 at 6:57 pm

Good, concise post. Hadn’t used Mechanical Turk for awhile. Jumped back in and was surprised how much better it was. Cheers
Dan Toombs recently posted..Got an Online Marketing Question?My ComLuv Profile

Reply

Leave a Comment

CommentLuv Enabled

Previous post:

Next post: