Inviting workers to review HITs on TV?

Unfortunately, I can't give you insight specific to your tasks since they are US only (btw, if you don't specifically need US workers, maybe open your HITs to other English speaking countries (mainly Canada)), but I usually appreciate requesters who are responsive to workers and have had multiple long email chains with requesters on how to improve their tasks and it's appreciated when they want to improve the work they post.

Concerning asking for reviews:
Like others have said, a simple message welcoming reviews to help you improve your tasks and help other workers would be perfectly fine. While I would normally agree to the idea of giving an incentive to workers for their work, in the case of reviews, I agree with others that it could be seen as "buying reviews" which isn't a good look, so a simple message is more appropriate. I would however try to make it a bit apparent or maybe on a page before the survey code, because once I get the survey code, if I don't see an arrow at the bottom of the page, I rarely bother reading the info on that page (as it's very often a debrief or links to websites in case I was triggered by something in the study, so I don't usually read it).

Based on the current reviews on TV, I'd say I strongly recommend increasing the pay to minimum >$15/hr (based on the most recent reviews, you are oscillating between ~$7 and ~$21/hr, with most people falling under the $15 mark). I can only speak for myself here, but I usually don't even bother trying to catch tasks that are averaged at <$15/hr and if I accidentally catch one, I will usually return it as soon as I see the reviews. It usually takes a requester being at an average of >$20/hr for me to want to do the work. Work between $15-20/hr, I will do on slow days only. Also, if a survey requires writing (other than a short 1-5 word answer), I normally return it as soon as I hit that writing section unless the reviews shows it pays well above the $30/hr mark (my process will usually be to get frustrated (if the writing sections is not announced at the beginning), tab out of the survey, hover over the return button, think I'll look at the reviews, see it's at like $40/hr and begrudgingly tab back to the survey and put some thought in the answer).

I know I don't really care about giving my email address (or more specifically, using mturk to message a requester), but I know multiple workers who really don't like having to use the feature because of privacy concerns (some don't even want to contact a requester asking to overturn an unfair rejection because they don't want to send an email from their personal address), so another way of getting some feedback would be to include a feedback box in the survey. This way, you can get some feedback, it's easy to bonus workers for their feedback (without it seeming like you are buying reviews) and it gives you the opportunity to reply to that feedback in the bonus message.

Hope this helps a bit.

 

First of all, I am so sorry for the very slow response to this comment. I was away from work, and have just gotten back today. I didn't mean to receive such a valuable response and not be here to thank you for it and respond.

At this point my main concern is that we can't talk about responses being "from x population" and/or "different to the national average for [whatever variable]" if our participants live in many different multiple countries. I understand, though, that HITs largely only being open to US workers is frustrating, and I'll try to spend some more time seeing what other researchers have done about this issue.

Thankyou for your thoughts about where to put the message. I really like the idea of a feedback box (and I certainly wouldn't think of it on my own), and it's good to know that it would probably be better to be placed before the survey code. Do you think workers fear contacting requesters/placing feedback on TV, too? That would be a worry.

And thank you also for your feedback regarding pay rates. I agree that they need to be higher, and we're working on doing so for our new HITs. I also really appreciate that you've highlighted which types of HITs need higher pay (e.g. writing tasks). I learned through the forum recently that it's really frustrating for workers to effectively be penalised for spending more time on a HIT by writing more than their colleagues who may write less - which makes total sense, and is something we need to keep in mind for future studies.

Thankyou, again,
Ruth

 

I doubt people would be afraid to give feedback on TV for different reasons.

1. It would be difficult for a requester to link a TV username to a worker ID, so the possibly of retaliation from a requester is extremely low. A requester could theoretically try to screw a worker by hard blocking their worker ID and making a complaint to amazon, so people may hesitate to send an unhappy (but polite) email since it's linked to their ID, but using the forum is much safer (anonymous) in that regard.

2. Contrary to sending a message through mturk, sending a PM on TV doesn't reveal your email address, and when signing up for mturk, if you already have an amazon account, it uses that account, meaning it uses the email you used at the time, which is probably a real email address. If you could and it didn't affect your amazon account, I'm sure many worker would change the email address associated with their mturk account (I would use the one I created for mturk emails which is [workerID]@gmail for instance).

All of that to say, I'm pretty a lot of people here are more comfortable giving feedback on an anonymous forum vs from their own (possibly identifying) email address.

Perfectly understandable and that's why I always phrase it as "if you don't specifically need US workers", since many projects do.
Although you could then start discussing how different Canada is to some US states compared to those same US states vs other states, as the opinions and values of US citizens varies greatly from one state to another (as you would probably find some great variance between Canadians from Alberta vs Quebec for example).