New group of HITs (feedback welcome!)

Hi, we just posted a group of 50 HITs involving reading and rating sentences. Each HIT should take 3-5 minutes, and the reward is $0.50. We'd appreciate any feedback you have!

 

Would you mind providing a link to your hits?

 

Of course, here it is: https://www.mturk.com/mturk/preview?groupId=3PS3UFWQYMQPZ0PQ5NB5495LA3KYRA

 

Ah, that's why I couldn't find them... they're gone!

 

Title: Rate text for positive or negative attitude (WARNING: This HIT may contain adult content. Worker discretion is advised.) | PANDA
Requester: SAFE-OAI [A11FAOM86AASOH]
TurkerView: [ N/A ]
Description: You will see a number of text snippets. Each snippet can be completed in one of two ways. For each completion, indicate whether it expresses a negative or positive attitude with a number between 1 and 10. The first time you do a HIT from this group, it will be a tutorial; you'll get paid for completing the tutorial just like an ordinary HIT. The tutorial should take 5-10 minutes, and subsequent HITs from this group should take 3-5 minutes each.
Duration: 60 Min
Available: 1
Reward: $0.50
Qualifications:

• Total approved HITs GreaterThanOrEqualTo 95
• Total approved HITs GreaterThanOrEqualTo 500
• Adult Content Qualification EqualTo 1

[33CY53T7ZHCQR46P2ZHQ3NLAJ65HDU]​

 

Sorry about that! I'll post here again when we release more.

 

.

 

I was able to do one and my only feedback is that I couldn't tell what the green box was supposed to be.

Also is this a batch or can we only do one of these? (It looks like a batch to me)

 

oh, @SAFE-OAI

add a US qualification (along with other english speaking countries as well as something optional)

I know I had to re-read one one of the tutorial questions that referenced an empty wine glass being used as a metaphor, so native English proficiency appears like it would be important to you.

 

Randomacts: I'm not sure about the terminology, but you can do up to 13 of these HITs. After that, it will say "No questions now, come back later."

Randomacts: The green box indicates the "correct" answer in the tutorial; you're supposed to change your answers to the green ones before you can submit. We'll try to clarify that in the instructions, thanks!

A6_Foul_Out: Thanks for the suggestion! We're going to experiment with different qualifications.

We're going to post 100 more HITs in a minute: https://worker.mturk.com/projects/3LOAKXD6JDCRTV8Z5MZBX6D5SSGERE/tasks?ref=w_pl_prvw

 

you can reply to individual posts by hitting reply, it will put what they said into a box so you can clearly reference the point they were making

you can reply to multiple people in one post, and type your response to them beneath the quote of them you added by hitting reply.

 

@SAFE-OAI

I noticed you're increasing the hit approval rating needed for the hits, if this was intentional, an approval rating really isn't much of an indicator of anything.

all serious workers on the platform have 99% approval "greater than or equal to 99% approval"

It's pretty safe to say that the best way to ensure data quality is to identify workers who are providing you with good answers, assigning those workers a qualification and ensuring those are the only workers who can gain access to the work.

This can be as wide as you'd like 100's of workers, or as narrow as you'd want 5-10 workers.

 

Thanks for the advice. After today, we'll review the data quality and create a qualification like you described.

 

https://docs.aws.amazon.com/AWSMechTurk/latest/RequesterUI/AssigningaQualificationTypetoaWorker.html

 

To be honest if you start mass rejecting people will freak out. Just grant quals to whomever is doing the work well and you should be good going forward.

 

Understood, thanks.

 

@SAFE-OAI Just to let you know, I stopped at 13 of your hits like you said, but, it was going to let me do that 14th one. It didn't say there were no more questions.

 

Thanks for doing all those! I think it won't give you that message until you accept the 14th HIT. But if it lets you do more, that's absolutely fine.

 

I did not see the message to stop at 13 (it's not mentioned on the actual hit) and ended up doing way more. The questions never repeated. Hope it doesn't mess up my chance to get the qual.

 

I had accepted like...16 before I read what you wrote LOL I returned them because I didn't want to do more than you said. I just wanted to make sure you knew in case you were wanting to cap them.