Journal Articles using MTurk as a Method

Hello Everyone!

I'm trying to get a grasp of journal articles that use MTurk as a part of their studies. However, I'm having a difficult time searching for papers. Every time I google "MTurk research paper" for example, I just get a bunch of research papers on the evaluation of MTurk as a tool. I want to see how MTurk is being used in different research fields.

Does anyone have journal links that they can possibly share with me? I would love to read people's work using MTurk!

Thanks

 

Do you have a particular field? MTurk is huge in the social/behavioral psych fields, so anything in this area: http://www.spsp.org/ probably has a lot of mturk cited stuff if you peruse through old programs/posters/etc.. legal has a few papers I can probably go find but IIRC the major paper had more to do with the meta approaches to using mturk (payment, task design, etc) within that field and I'm not sure how much it delved into their actual use of the platform.

Same for HCI research.. a lot of it is metaresearch about the platform itself, but there is a ton of stuff in NLP/AI that's run on MTurk and generally goes into approaches in their papers.

Almost everything here used MTurk IIRC: https://www.humancomputation.com/2019/program.html

"A Large-Scale Study of the "Wisdom of Crowds'" is especially interesting, they both utilized MTurk in their project and kind of had to delve into their own methodology/approach to using it to get from A => B so it might be sort've on target for what you're asking?

C-SATs is a health org that uses MTurk and has lightly published on it.. this might be a decent jumping off point (sorry I'm trying to pull years of stuff out of my head right now haha): https://link.springer.com/article/10.1007/s11606-017-4246-0 https://www.researchgate.net/public...r_Discriminating_Basic_Robotic_Surgery_Skills

Have not had time to look at this, but it's been repeatedly shared on SM from a lot of the academics I follow there: https://towardsdatascience.com/crow...ource=rss----7f60cf5620c9---4&gi=890d0c61f900

Honestly there is SO MUCH research done on MTurk.. just throw a dart at any social science research and you'll find stuff haha. SPSP is probably a good starting point there for purely "yea we used MTurk p's and this is what we found" kind of stuff. The AI/ML field has more interesting projects, but it's almost always going to be muddled with the more "meta" discussions of the platform itself.

 

This is a great starting point. Thanks, Chris!!

 

No problem, sorry it's 100% scatterbrained haha.

Also one jumping off point for the NLP work:

The researchers tagged have put an immense amount of work on the platform and do some neat things in that field, probably good for actual names to look into.

 

Also, I would not use MTurk as my main keyword when searching in Google Scholar or any academic databases from university libraries. I'm fairly certain that in any academic publications, authors will have to introduce MTurk in its full name. More importantly, unless the lead or main authors are the actual 'hands-on' researcher getting things done with MTurk, they won't know enough about the culture, norms, and lingos. So, using "Mturk" as a reference in their writing would be less likely. Therefore, you are less likely to get the papers that may only mention MTurk in the research methodology sections for once or twice, but they actually used MTurk as part of their studies in many different ways. If you share more about what you are looking for or the field you are working on currently, then I may be able to give you specific pointers for search or publications etc.

 

+1 for this. Chris's advice is pretty solid. Academics tend to refer to "online participant pools" and "Amazon's Mechancial Turk" (the latter sometimes followed by (MTurk)).