We are a new requester and getting ready to launch a new product which will have HITs for following use cases: 1. Usability Surveys for landing pages, charts & dashboards etc 2. Market Research Surveys ( book cover, product packaging, design mockups, domain/product names) There will be a short-survey (average of two questions, min-1, max-5) with at least one text-based answer which requires you type a sentence or two. All surveys will be hosted on our website and integrated using Mturk requester API. We have carefully designed the user interface to help save your time. However, we would like to do more to help HIT workers and help improve quality of responses. After the initial onboarding, most subsequent HITs can be completed within a minute (min) or up to 5 minutes (max). We will love to hear your feedback on: 1. Will a 30-second intro video help workers better understand what they are supposed to do? The goal behind this is to understand if turkers appreciate anything of this nature or consider this a waste of their time. 2. What's the rate of pay (per minute) that you would expect for these kinds of HITs. Since we are new, we may have to tweak these but would love to hear a good baseline to start with from you.
usually it is a time vs pay issue. workers here also are not fans of writing unless it is minimal and most expect higher pay for that. will the video be a one time thing or it can be rewatched in other HITs?
Videos will be a one-time thing. Of course, it is designed primarily to help turkers. Writing is minimal, usually a sentence or two. With that, most HITs can be completed anywhere between 1 to 5 minutes.
As far as would workers appreciate a video or see it as a wate of time, I for one appreciate anything that gives me confidence I am providing what the requestor wants. I would rather have to spend a little extra time on the first hit or even the fisrt 2 or 3 to be sure I understand what I should be doing then to be second guessing myself in the future. That being said it would be a good idea to make sure it is clearly stated in the directions that the first hit will take longer than the norm and give an average for a regular hit so people do not judge the pay based on the time spent for the first hit
Its better to have it available than not. Link to it in your HIT description as well so when your HITs get shared among workers the video can be easily found. This is a Requester who posts similar work to what you're seemingly describing and is generally well regarded in the worker community: https://turkerview.com/requesters/A1G1PATDFBA7E5-userbob He has various "tiers" or workers and from what I've seen the lowest tier tends to average ~$10/hr for usability/web testing & the higher tiers move up to around the $20/hr mark. He's used the platform for years and AFAIK gets very good data quality with his worker tiering method. In order to get into the lowest tier (qualify to work for him at all) you have to complete his training HIT, which is generally not well paid but workers understand that as an onboarding technique its a pre-req to qualify to do the work & folks don't tend to mind. Just so you know this is a HUGE time range for HITs. A 1 minute HIT at a quarter is well paid, but if that HIT somehow balloons up to 5 minutes to complete the wage is downright awful. It'd be helpful to understand why your expected completion time is such a large range in advising on payrates tbh.
if it is possible it might be helpful to have a the video either in each hit (but not required after the first) or have a link to it so that if a worker has a question later or it's been awhile since they did one of your hits they can refer back to the video. 1. not everyone uses the pplatform on a daily or weekly bases 2. sometimes questions come up after you have done a few hits refering to number 2 above it is always helpful if you or someone responds to messages from workers either through Amazon or on a thread like this. Workers like to now that if they have an issue their question is not just being thrown out into some void
Instructions should be clear and comprehensive. If a worker has any questions, they should be answered by the instructions. As a worker, I should know exactly what is expected of me from the task you've put in front of me. Either written down or in the video you mentioned. This includes the time commitment. The pay should match with the amount of time the hit takes. If you have hits that take longer, you can user the tiered system CT mentioned regarding UserBob to promote higher quality workers to do those hits for better pay because you can ensure their data quality. As somebody who does the highest tiered work from UserBob: I know how long the task will take. I get explicit instructions for what to do during the task. If I have questions, he is very responsive. But I never have questions because all my questions have already been answered by the instructions.
