New group of HITs (feedback welcome!)

Discussion in 'Requesters' started by SAFE-OAI, Oct 12, 2018.

  1. SAFE-OAI

    SAFE-OAI A11FAOM86AASOH

    Messages:
    8
    Ratings:
    +10
    Hi, we just posted a group of 50 HITs involving reading and rating sentences. Each HIT should take 3-5 minutes, and the reward is $0.50. We'd appreciate any feedback you have!
     
    • Love Love x 1
  2. Wilma

    Wilma Survey Slinger

    Messages:
    1,427
    Gender:
    Female
    Ratings:
    +3,019
    Would you mind providing a link to your hits?
     
  3. SAFE-OAI

    SAFE-OAI A11FAOM86AASOH

    Messages:
    8
    Ratings:
    +10
  4. Wilma

    Wilma Survey Slinger

    Messages:
    1,427
    Gender:
    Female
    Ratings:
    +3,019
    Ah, that's why I couldn't find them... they're gone!
     
    • LOL LOL x 2
  5. Randomacts

    Randomacts Survey Slinger

    Messages:
    94,622
    Gender:
    Male
    Ratings:
    +124,073
    Title: Rate text for positive or negative attitude (WARNING: This HIT may contain adult content. Worker discretion is advised.) | PANDA
    Requester: SAFE-OAI [A11FAOM86AASOH]
    TurkerView: [ N/A ]
    Description: You will see a number of text snippets. Each snippet can be completed in one of two ways. For each completion, indicate whether it expresses a negative or positive attitude with a number between 1 and 10. The first time you do a HIT from this group, it will be a tutorial; you'll get paid for completing the tutorial just like an ordinary HIT. The tutorial should take 5-10 minutes, and subsequent HITs from this group should take 3-5 minutes each.
    Duration: 60 Min
    Available: 1
    Reward: $0.50
    Qualifications:
    [33CY53T7ZHCQR46P2ZHQ3NLAJ65HDU]
     
    • Like Like x 1
  6. SAFE-OAI

    SAFE-OAI A11FAOM86AASOH

    Messages:
    8
    Ratings:
    +10
    Sorry about that! I'll post here again when we release more.
     
    • Like Like x 1
  7. A6_Foul_Out

    A6_Foul_Out Organic Meme Panda TurkerView Masters

    Messages:
    17,777
    Gender:
    Male
    Ratings:
    +23,954
  8. Randomacts

    Randomacts Survey Slinger

    Messages:
    94,622
    Gender:
    Male
    Ratings:
    +124,073
    I was able to do one and my only feedback is that I couldn't tell what the green box was supposed to be.

    Also is this a batch or can we only do one of these? (It looks like a batch to me)
     
  9. A6_Foul_Out

    A6_Foul_Out Organic Meme Panda TurkerView Masters

    Messages:
    17,777
    Gender:
    Male
    Ratings:
    +23,954
    oh, @SAFE-OAI

    add a US qualification (along with other english speaking countries as well as something optional)

    I know I had to re-read one one of the tutorial questions that referenced an empty wine glass being used as a metaphor, so native English proficiency appears like it would be important to you.
     
    • Like Like x 1
    • Today I Learned Today I Learned x 1
  10. SAFE-OAI

    SAFE-OAI A11FAOM86AASOH

    Messages:
    8
    Ratings:
    +10
    Randomacts: I'm not sure about the terminology, but you can do up to 13 of these HITs. After that, it will say "No questions now, come back later."

    Randomacts: The green box indicates the "correct" answer in the tutorial; you're supposed to change your answers to the green ones before you can submit. We'll try to clarify that in the instructions, thanks!

    A6_Foul_Out: Thanks for the suggestion! We're going to experiment with different qualifications.

    We're going to post 100 more HITs in a minute: https://worker.mturk.com/projects/3LOAKXD6JDCRTV8Z5MZBX6D5SSGERE/tasks?ref=w_pl_prvw
     
    • Today I Learned Today I Learned x 1
  11. A6_Foul_Out

    A6_Foul_Out Organic Meme Panda TurkerView Masters

    Messages:
    17,777
    Gender:
    Male
    Ratings:
    +23,954
    you can reply to individual posts by hitting reply, it will put what they said into a box so you can clearly reference the point they were making

    you can reply to multiple people in one post, and type your response to them beneath the quote of them you added by hitting reply.
     
    • Today I Learned Today I Learned x 1
  12. A6_Foul_Out

    A6_Foul_Out Organic Meme Panda TurkerView Masters

    Messages:
    17,777
    Gender:
    Male
    Ratings:
    +23,954
    @SAFE-OAI

    I noticed you're increasing the hit approval rating needed for the hits, if this was intentional, an approval rating really isn't much of an indicator of anything.

    all serious workers on the platform have 99% approval "greater than or equal to 99% approval"

    It's pretty safe to say that the best way to ensure data quality is to identify workers who are providing you with good answers, assigning those workers a qualification and ensuring those are the only workers who can gain access to the work.

    This can be as wide as you'd like 100's of workers, or as narrow as you'd want 5-10 workers.
     
    • Like Like x 1
  13. SAFE-OAI

    SAFE-OAI A11FAOM86AASOH

    Messages:
    8
    Ratings:
    +10
    Thanks for the advice. After today, we'll review the data quality and create a qualification like you described.
     
    • Like Like x 2
  14. A6_Foul_Out

    A6_Foul_Out Organic Meme Panda TurkerView Masters

    Messages:
    17,777
    Gender:
    Male
    Ratings:
    +23,954
    • Today I Learned Today I Learned x 1
  15. Randomacts

    Randomacts Survey Slinger

    Messages:
    94,622
    Gender:
    Male
    Ratings:
    +124,073
    To be honest if you start mass rejecting people will freak out. Just grant quals to whomever is doing the work well and you should be good going forward.
     
  16. SAFE-OAI

    SAFE-OAI A11FAOM86AASOH

    Messages:
    8
    Ratings:
    +10
    Understood, thanks.
     
    • Like Like x 1
  17. Tracy Wethington

    Tracy Wethington The Bone Crusher

    Messages:
    13,955
    Gender:
    Female
    Ratings:
    +22,085
    @SAFE-OAI Just to let you know, I stopped at 13 of your hits like you said, but, it was going to let me do that 14th one. It didn't say there were no more questions.
     
  18. SAFE-OAI

    SAFE-OAI A11FAOM86AASOH

    Messages:
    8
    Ratings:
    +10
    Thanks for doing all those! I think it won't give you that message until you accept the 14th HIT. But if it lets you do more, that's absolutely fine.
     
    • Like Like x 1
  19. MechaC

    MechaC Turker

    Messages:
    51
    Ratings:
    +76
    I did not see the message to stop at 13 (it's not mentioned on the actual hit) and ended up doing way more. The questions never repeated. Hope it doesn't mess up my chance to get the qual.
     
  20. Tracy Wethington

    Tracy Wethington The Bone Crusher

    Messages:
    13,955
    Gender:
    Female
    Ratings:
    +22,085
    I had accepted like...16 before I read what you wrote LOL I returned them because I didn't want to do more than you said. I just wanted to make sure you knew in case you were wanting to cap them.