Workernator

Discussion in 'mTurk Scripts & Resources' started by Kadauchi, Jan 22, 2023.

  1. Kadauchi

    Kadauchi Administrator Former MTG MotM

    Messages:
    4,367
    Ratings:
    +8,589
    This will end up being the successor to Queuebicle + Turkernator.

    There will be something concrete to use sometime in February. There will be a few testing builds between now and then, current users will have access to use these builds. You won’t have to change your subscription to use this either.
     
    • Love Love x 38
    • WOW WOW x 1
  2. lapetitebaker

    lapetitebaker Survey Slinger TurkerView Masters

    Messages:
    21,157
    Gender:
    Female
    Ratings:
    +7,895
    Thank you for the update! I'm sure you know we all truly appreciate you keeping the legacy of QBC alive.
     
    • Like Like x 9
  3. tricker

    tricker Survey Slinger

    Messages:
    9,330
    Gender:
    Female
    Ratings:
    +14,571
    Thank you so much for doing this @Kadauchi! QBC is still functioning for me but I fear it will become unstable soon. I really appreciate your efforts!
     
    • Like Like x 3
  4. mangopengo

    mangopengo New Turker

    Messages:
    13
    Gender:
    Male
    Ratings:
    +3
    Thanks for making this @Kadauchi! Looking forward to replacing QBC with this!
     
    • Like Like x 2
  5. Katlady

    Katlady Well-Known Turker

    Messages:
    2,487
    Ratings:
    +2,203
    @Kadauchi Will we be able to import our QBC data into the new tool?
    @tricker Same. I'm afraid to bonk anything sideways since it's still working.
     
    • Like Like x 3
  6. Abscheulich

    Abscheulich Active Turker

    Messages:
    287
    Ratings:
    +541
    Will this be taking over for the QBC thing? I didn't even know it wasn't working, it works for me, but I guess I do get way less hits than previously.
     
    • Like Like x 1
  7. Kadauchi

    Kadauchi Administrator Former MTG MotM

    Messages:
    4,367
    Ratings:
    +8,589
    I will try and make as much data importable as possible.


    If QBC is working for you then there is no reason to switch over.
     
    • Love Love x 4
    • Like Like x 1
  8. dtalksall

    dtalksall New Turker

    Messages:
    27
    Gender:
    Female
    Ratings:
    +2
    delete
     
    Last edited: Jan 30, 2023
  9. Jane Doe

    Jane Doe Survey Slinger

    Messages:
    2,337
    Ratings:
    +2,116
    I will probably keep my QBC unless yours is cheaper, not in a hurry to decide.
     
  10. Kadauchi

    Kadauchi Administrator Former MTG MotM

    Messages:
    4,367
    Ratings:
    +8,589
    Your QBC subscription will activate this too, so no need to change anything if you're already subscribed.
     
    • Like Like x 7
    • Love Love x 1
  11. NorthoftheBlueMts

    NorthoftheBlueMts Turker

    Messages:
    69
    Gender:
    Female
    Ratings:
    +89
    fyi all,
    Just published 12 January 2023
    "The Unintended Consequences of Automated Scripts in Crowdwork Platforms: A Simulation Study in MTurk"
    The abstract is available https://link.springer.com/article/10.1007/s10796-023-10373-x

    This whole paper is rather requester-centric, of course, but I thought it interesting if only for some of the broad brush, and quite possibly mistaken, conclusions it draws. There's no mention of Masters but "Super Turkers" is bandied about.

    "In this study, we use Discrete Event Simulation to identify and quantify the unintended consequences of the excessive use of automated scripts. Our findings show that, while the use of scripts allows some crowdworkers to identify and accept far more tasks than others, in the long run, this behaviour results in their competence persistence and reputational persistence and progressively to detrimental impacts for those workers who do not use scripts, and who may ultimately be forced to exit the platform. As a result, automated scripts have negative consequences, whereby their excessive use leads to a tragedy of the commons for all platform stakeholders, including the crowdworkers, the job requesters and the platform itself."

    A footnote therein
    ChrisTurk. (2022). TurkerViewJS. https://turkerview.com/mturk-scripts/1-turkerviewjs
     
    • Like Like x 2
    • Yikes Yikes x 2
  12. mdeem

    mdeem Survey Slinger TurkerView Masters

    Messages:
    8,106
    Ratings:
    +7,357
    I'm out of the loop. Has Chris given up on developing QBC?
    NM, found that other thread and I see its unknown what the deal is
     
    Last edited: Jan 31, 2023
  13. Kadauchi

    Kadauchi Administrator Former MTG MotM

    Messages:
    4,367
    Ratings:
    +8,589
    afaik no and it looks like he ha pushed an update to QBC and looks like there is another on the way (not sure if it public yet or just being tested).
     
    • Like Like x 2
  14. tricker

    tricker Survey Slinger

    Messages:
    9,330
    Gender:
    Female
    Ratings:
    +14,571

    1) Their language could be more general audience accessible. There is really no need to use such pretentious language, even in academic settings. This is a huge pet peeve of mine, and so far I have not been docked in any of my academic work for using 15 cent words versus 25 cent words. I'm hopeful academia will, within my lifetime, make itself more accessible to those not in the academic world.

    2) Ok, so...this means what for requesters? Mayhaps I need to read the whole article (I'm basing this off what you posted below, I'll save article for future reading after I finish this week's homework...which requires a lot of reading and watching class presentations from other students). Are they implying that the sample is biased because it only recruits people who do, or can afford, to use the automated scripts? Anyone can use the scripts, and they become affordable if you use the scripts. I am, for sure, going to need definitions on "competence" and "reputational" persistence. And determine what detrimental impacts.
     
    • Like Like x 3
  15. NorthoftheBlueMts

    NorthoftheBlueMts Turker

    Messages:
    69
    Gender:
    Female
    Ratings:
    +89
    Great, I look forward to your take on the article. It does strike me as another requester-in-a-bubble with a limited understanding of all the forces at play and deserving of an informed response.
     
    • Like Like x 3
  16. tricker

    tricker Survey Slinger

    Messages:
    9,330
    Gender:
    Female
    Ratings:
    +14,571
    I just downloaded it (thankfully, free, through my university, yay!)

    Once I finish this massive homework assignment, and my actual paid job that keeps a roof over my head, I'll read the article. And see if I can wade through the language and pick out relevant information.
     
    • Like Like x 2
    • Today I Learned Today I Learned x 1
  17. Girl Polar Bear

    Girl Polar Bear Queen of the North

    Messages:
    29,273
    Gender:
    Female
    Ratings:
    +45,742
    MA Turkernator
     
  18. NorthoftheBlueMts

    NorthoftheBlueMts Turker

    Messages:
    69
    Gender:
    Female
    Ratings:
    +89
    When you get a chance, I noted this morning on reddit, a qual, "cloudresearch workers-1k is 1" which probably refers to this CR blog post - New Feature: Exclude Highly Active Workers though there is nothing to stop new "naive" turkers using scripts. Or using <50 <100 etc. Haven't a clue where they came up with the stated percentages.
     
    • Like Like x 2
    Last edited: Feb 2, 2023
  19. tricker

    tricker Survey Slinger

    Messages:
    9,330
    Gender:
    Female
    Ratings:
    +14,571
    I gotta get on this. Grad school has a heavy workload this week and I've been bogged down in that. But I'm interested in this as a turker so I need to check out both things now.

    And I'll compile a synopsis for y'all. :emoji_grin:
     
    • Like Like x 1
  20. Random Turker

    Random Turker Survey Slinger TurkerView Masters

    Messages:
    25,777
    Gender:
    Male
    Ratings:
    +25,033
    Lol this company I tell ya.
     
    • LOL LOL x 1