09/21 - TurkerView.com Thursday!

Discussion in 'Daily mTurk HITs Threads' started by ChrisTurk, Sep 21, 2017.

Thread Status:
Not open for further replies.
  1. ChrisTurk

    ChrisTurk Administrator

    Messages:
    56,679
    Ratings:
    +162,927
    [​IMG]
    (not really, but its pretty close)
    First of all, a thank you note.
    There are over 2 dozen people in the 'beta test' group for TurkerView. That means this was a community effort, and I can't thank all of them enough. But @NBadger & @GoldGuardian deserve an extra level of gratitude - I may open the Beta forum for public view one day just to show off the insane time and effort those two put into this project. I really can't thank them enough because without their expertise and eyeballs what you're about to see would look like a steaming pile of doodoo. They've been kind enough to save me from myself more times than I would like to count :emoji_sweat_smile:

    And in no particular order, everyone who contributed to the project: @BadPanda @bttrflyyy @ceedj @CorvusSapiens @dan @DarkChild @GreenMachine842 @gurlondrums (ty for letting me pick your expensive brain) @hvnnvr @jdzane @jessers @keysman @leafs4_cup @lefty (comedic relief FTW) @MasterNyborg @Melting Glacier @slothbear @ThisPoorGuy (again, expensive brain that he always offered up for free, ty bby SELECT COUNT(skips) FROM `my_heart` WHERE target = "ThisPoorGuy" ORDER BY `beat_id` ASC;) @Tripsa @WimpLo @WalkingEmphasis @eyeofodin @Trickydude24 @ZileWrath @EssArrBee

    And if I forgot anyone I AM SORRY (please tag me so I can get you your fancy badge). Now:

    Introducing TurkerView.com
    So, about 2 months ago I asked for some feedback on a Daily Thread about worker/requester review platforms. A few weeks after that I began working on my own version and since then I've racked up 96 hours in a text-editor and god-only-knows how many burying myself in StackOverflow / MySQL / etc docs.

    The culmination of it is TurkerView (beta). It's fully sync'd with TurkerHub, so you can log into it simply using your TurkerHub account.

    The idea isn't to necessarily replace other review platforms, its to expand on TurkerHub's ideals of connecting workers (& requesters). To that end, both have a voice and the messages should be geared toward helping each other improve the platform we're all stuck on (mTruck we love you). Reviews have a complement of ratings that allow workers to quickly communicate key data points about HITs/Requesters as well as a field where they can leave actionable feedback for a Requester to use to improve from. Requesters, in turn, are able to "claim" their profiles and use them to communicate back with the worker community.

    Efficiency Is King
    I've tried to design the platform w/ workers in mind. I want folks to be able to get in and out in under a minute for most reviews. The review form itself is fairly simple, and the TurkerView script will mostly eliminate the need to manually fill out basic info.
    upload_2017-9-21_1-46-33.png

    Visual Data
    Beyond the input improvements, consuming data from the platform should be much improved over alternatives. Everything is neatly organized, color coded, and usually accompanied by a visual representation that should be fairly intuitive to pick up on.

    Computers are great at processing data dumps, humans not so much. So where possible I've tried to simplify the data in more human-readable formats (graphs, icons, colors).

    upload_2017-9-21_2-17-47.png

    Expanding Functionality
    One of the most valuable, but hardest to report & then find, pieces of information I see shared on a day-to-day basis is the completion times of individual HITs. The forum's lackluster search engine & praying isn't the optimal way to do this, so I've expanded on the basic API functions workers are used to in order to allow folks to call for completion times on individual HITs (not just the average pay for the Requester in general). This one is courtesy of @WalkingEmphasis who kinda sparked the idea in my head the other day. Example from worker.mturk queue:
    upload_2017-9-21_1-57-2.png

    For this one to work it requires serious community cooperation, of course, since there wont be an answer to the API call if no one is submitting data. So really for it to work, people are going to be required to help each other! :emoji_smile: If you use the TurkerView script w/ Kerek's Hourly (more on these below) it should take you no more than a few seconds to simply report a completion time.

    upload_2017-9-21_2-50-37.png
    Useful sorting of worker reviews is also new. Instead of just lumping everything together and asking folks to scroll through 900 reviews to find out p9r rejects a lot, hopefully community pruning can help bring the best, most in-depth reviews up to the top of the list.

    Add-on Scripts
    The platform by itself, IMHO, is pretty nice. But of course its only 1/3rd functional if you can't use it from mturk itself so I've got a script put together to import some of TurkerView's functionality into mTurk.

    Worker:
    upload_2017-9-21_2-13-53.png

    www:
    upload_2017-9-21_2-14-17.png

    Dashboard (requires Kerek's mTurk Hourly):
    upload_2017-9-21_2-14-58.png

    You can install TurkerView.user.js by clicking this link. Again, you'll also want to add Kerek's mTurk Hourly as well.

    What now?
    Now I cry at all the bugs y'all find hahaha. But seriously, please submit data. The entire platform will have been a waste of time if y'all don't find it useful so give it a go and give me feedback. Trust me, nothing you say can be meaner than what @NBadger & @GoldGuardian have already told me LOL. My current TODO list looks something similar to this:

    • Scratch & Redo the individual review page lists
    • Fill out Requester profile claim information, reply best practices documentation
    • Better data validation (will expand as I find holes, I'm sure they exist in droves)
    • Mod panel processing
      • On this note, if anyone is interested in volunteering to help keep the platform free of ickyness, please DM me.
    • TH Integrations
      • Exports
      • Notifications for things like mass-rejections, blocks, etc
      • User profile syncing (updating avatars, etc)
    • Form Fixes
      • More elegant suggested completion time display/validation
    • E-mail triggers on Requester reply
    • Script documentation
    • Pray the server its on is strong enough.
    • Sleep. Please. I neeeeeeds it.

    Statseses
    If anyone is curious "where I've been" as I said previously I've logged ~96 hours in text-editor developing the site. I fully admit that's probably 10 hours of professional dev time, I don't even want to know how inefficient I am, but its been one hell of a learning experience. Images below:
    Total time (this is JUST actual typing in the editor.. god bless there's no SO/research included here):
    upload_2017-9-21_2-26-56.png

    This week:
    upload_2017-9-21_2-27-16.png

    My sleep schedule in a nutshell (blue = coding time):
    upload_2017-9-21_2-27-46.png

    Thank you
    For scrolling this far. I'll be up for another 30 minutes or so to answer any quick questions / critical bug fix. Sorry this isn't the best intro post, I don't think I could fit everything I'd like to say in one post, and I mostly just want to thank all the people who helped put this thing together for everyone. :ba:
    [​IMG]
    [​IMG]
     
    • Love x 43
    • Like x 5
    • 5/5 Pay x 2
    • Nom Nom Nom! x 1
    • Today I Learned x 1
    Last edited by a moderator: Sep 21, 2017
  2. Randomacts

    Randomacts Survey Slinger

    Messages:
    94,622
    Gender:
    Male
    Ratings:
    +124,073
    @ChrisTurk

    I basically didn't help at all but I was in the beta group so :dunno:
     
    • 5/5 Pay 5/5 Pay x 2
    • LOL LOL x 1
  3. ChrisTurk

    ChrisTurk Administrator

    Messages:
    56,679
    Ratings:
    +162,927
    ..I just accidentally deleted Buzby from the website.. BUZBY IM SO SORRY PLEASE RE-REGISTER AND I'LL FIX YOUR ACCOUNT :emoji_fearful::emoji_disappointed_relieved::emoji_disappointed_relieved::emoji_disappointed_relieved::emoji_disappointed_relieved::emoji_disappointed_relieved::emoji_disappointed_relieved::emoji_disappointed_relieved:
     
    • LOL LOL x 18
  4. Randomacts

    Randomacts Survey Slinger

    Messages:
    94,622
    Gender:
    Male
    Ratings:
    +124,073
    [​IMG]
     
    • LOL LOL x 1
  5. hvnnvr

    hvnnvr Survey Slinger

    Messages:
    4,588
    Gender:
    Male
    Ratings:
    +6,132
    Shit broke...
     
    • LOL LOL x 5
    • Today I Learned Today I Learned x 1
    Last edited: Sep 21, 2017
  6. WimpLo

    WimpLo Survey Slingin' Batch Masta Former MTG MotM

    Messages:
    34,781
    Gender:
    Female
    Ratings:
    +73,160
    [​IMG]
     
    • LOL LOL x 3
  7. Randomacts

    Randomacts Survey Slinger

    Messages:
    94,622
    Gender:
    Male
    Ratings:
    +124,073
    Holy shit discord has video chat now.

    (I'm opening it up so that I can send buzby a PM)
     
    • Love Love x 1
  8. LLL

    LLL Survey Slinger

    Messages:
    3,092
    Gender:
    Female
    Ratings:
    +5,994
    Title: Thoughts and Feelings3(~ 30 minutes) | PANDA
    Worker: Preview | Accept | Requester
    Requester: Maya Tamir [A3JINFEQ3J2MJ6] (Contact)
    TO 1: [Pay: 4.28] [Fast: 4.84] [Comm: 4.43] [Fair: 4.64] [Reviews: 60] [ToS: 0]
    TO 2:
    [Rate: $2.44/hr] [Pen: -- days] [Res: -- of 0] [Rec: -- of 0] [Rej: 0] [ToS: 0] [Brk: 0]
    Reward:
    $3.00
    Duration: 1 hour 30 minutes
    Available: 1
    Description: 30-minute survey on your thoughts and feelings
    Requirements: Total approved HITs GreaterThanOrEqualTo 100; Exc: [57735058-57574] DoesNotExist ; TP Panel: 256998982 N GreaterThanOrEqualTo 100; HIT approval rate (%) GreaterThanOrEqualTo 98; Location In US;
    HIT exported from Mturk Suite v1.25.7
    Approved in minutes.
     
    • Nom Nom Nom! Nom Nom Nom! x 12
  9. WimpLo

    WimpLo Survey Slingin' Batch Masta Former MTG MotM

    Messages:
    34,781
    Gender:
    Female
    Ratings:
    +73,160
    Ok, I'm going to bed. Don't delete my account! :p
     
    • LOL LOL x 9
  10. Randomacts

    Randomacts Survey Slinger

    Messages:
    94,622
    Gender:
    Male
    Ratings:
    +124,073
     
    • LOL LOL x 2
  11. Melting Glacier

    Melting Glacier PE: $30.01 - That's over $1.25/hour! ┬┴┤( ͡° ͜ʖ├┬┴

    Messages:
    6,157
    Gender:
    Male
    Ratings:
    +11,423
    Ooh good idea, I was gonna find that gamertag thread and see if she dropped a SteamID there

    ah yeah that was in patch notes a while back as a limited roll-out thing. Wonder if it's for everyone or just added more people? hmmz
     
    • Love Love x 1
  12. Randomacts

    Randomacts Survey Slinger

    Messages:
    94,622
    Gender:
    Male
    Ratings:
    +124,073
    I'm already friends with her on steam and I sent a message but she hasn't been on in 4 days according to steam... but that isn't always accurate lol
     
  13. ChrisTurk

    ChrisTurk Administrator

    Messages:
    56,679
    Ratings:
    +162,927
    @Buzby k well its kinda fixed, will eventually reparse her posts & stuff if I understand correctly. I can email her her new password so no worries. Carry on folks, pay attention to the shiny object in the first post :emoji_sweat_smile::emoji_sweat_smile::emoji_sweat_smile::emoji_sweat_smile:
     
    • LOL LOL x 4
  14. Randomacts

    Randomacts Survey Slinger

    Messages:
    94,622
    Gender:
    Male
    Ratings:
    +124,073
    Email password... as in a password reset link? I hope that the forum doesn't normally email plaintext passwords lol

    :tinfoil:
     
  15. ChrisTurk

    ChrisTurk Administrator

    Messages:
    56,679
    Ratings:
    +162,927
    No it doesn't but I had to manually set her a new password on the account (it can't pull her e-mail, pass, avatar or any of her personal details / account settings), it just takes all the deleted posts and shoves them back onto a new account with the old name LOL. Not real sure what data will be missing and what will reparse to the account but guess we'll find out shortly
     
    • Like Like x 1
    • LOL LOL x 1
  16. DigitalDenizen

    DigitalDenizen Well-Known Turker

    Messages:
    2,341
    Gender:
    Male
    Ratings:
    +2,044
    You've out done yourself again my friend, Great job!
     
    • Love Love x 1
  17. Randomacts

    Randomacts Survey Slinger

    Messages:
    94,622
    Gender:
    Male
    Ratings:
    +124,073
    [​IMG]
     
    • LOL LOL x 1
  18. hvnnvr

    hvnnvr Survey Slinger

    Messages:
    4,588
    Gender:
    Male
    Ratings:
    +6,132
    @Buzby should get an unwilling guinea pig badge...
     
    • LOL LOL x 5
    • Love Love x 1
  19. lemondrop

    lemondrop Active Turker

    Messages:
    497
    Ratings:
    +732
    What does the Federal, Requester, California graph mean? I love that $6 is not showing as an acceptable pay rate. It looks great. All I can say is wow.

    [​IMG]
     
  20. hvnnvr

    hvnnvr Survey Slinger

    Messages:
    4,588
    Gender:
    Male
    Ratings:
    +6,132
    It is a graph of the requester's pay-rate versus Federal minimum wage and California minimum wage.
     
    • Today I Learned Today I Learned x 1
Thread Status:
Not open for further replies.