Introducing TurkerView.com - An mTurk Work Review Platform

First of all, a thank you note.
There are over 2 dozen people in the 'beta test' group for TurkerView. That means this was a community effort, and I can't thank all of them enough. But @NBadger & @GoldGuardian deserve an extra level of gratitude - I may open the Beta forum for public view one day just to show off the insane time and effort those two put into this project. I really can't thank them enough because without their expertise and eyeballs what you're about to see would look like a steaming pile of doodoo. They've been kind enough to save me from myself more times than I would like to count

And in no particular order, everyone who contributed to the project: @BadPanda @bttrflyyy @ceedj @CorvusSapiens @dan @DarkChild @GreenMachine842 @gurlondrums (ty for letting me pick your expensive brain) @hvnnvr @jdzane @jessers @keysman @leafs4_cup @lefty (comedic relief FTW) @MasterNyborg @Melting Glacier @slothbear @ThisPoorGuy (again, expensive brain that he always offered up for free, ty bby SELECT COUNT(skips) FROM `my_heart` WHERE target = "ThisPoorGuy" ORDER BY `beat_id` ASC @Tripsa @WimpLo @WalkingEmphasis @eyeofodin @Trickydude24 @ZileWrath @EssArrBee

And if I forgot anyone I AM SORRY (please tag me so I can get you your fancy badge). Now:

Introducing TurkerView.com
So, about 2 months ago I asked for some feedback on a Daily Thread about worker/requester review platforms. A few weeks after that I began working on my own version and since then I've racked up 96 hours in a text-editor and god-only-knows how many burying myself in StackOverflow / MySQL / etc docs.

The culmination of it is TurkerView (beta). It's fully sync'd with TurkerHub, so you can log into it simply using your TurkerHub account.

The idea isn't to necessarily replace other review platforms, its to expand on TurkerHub's ideals of connecting workers (& requesters). To that end, both have a voice and the messages should be geared toward helping each other improve the platform we're all stuck on (mTruck we love you). Reviews have a complement of ratings that allow workers to quickly communicate key data points about HITs/Requesters as well as a field where they can leave actionable feedback for a Requester to use to improve from. Requesters, in turn, are able to "claim" their profiles and use them to communicate back with the worker community.

Efficiency Is King
I've tried to design the platform w/ workers in mind. I want folks to be able to get in and out in under a minute for most reviews. The review form itself is fairly simple, and the TurkerView script will mostly eliminate the need to manually fill out basic info.

Visual Data
Beyond the input improvements, consuming data from the platform should be much improved over alternatives. Everything is neatly organized, color coded, and usually accompanied by a visual representation that should be fairly intuitive to pick up on.

Computers are great at processing data dumps, humans not so much. So where possible I've tried to simplify the data in more human-readable formats (graphs, icons, colors).



Expanding Functionality
One of the most valuable, but hardest to report & then find, pieces of information I see shared on a day-to-day basis is the completion times of individual HITs. The forum's lackluster search engine & praying isn't the optimal way to do this, so I've expanded on the basic API functions workers are used to in order to allow folks to call for completion times on individual HITs (not just the average pay for the Requester in general). This one is courtesy of @WalkingEmphasis who kinda sparked the idea in my head the other day. Example from worker.mturk queue:


For this one to work it requires serious community cooperation, of course, since there wont be an answer to the API call if no one is submitting data. So really for it to work, people are going to be required to help each other! If you use the TurkerView script w/ Kerek's Hourly (more on these below) it should take you no more than a few seconds to simply report a completion time.


Useful sorting of worker reviews is also new. Instead of just lumping everything together and asking folks to scroll through 900 reviews to find out p9r rejects a lot, hopefully community pruning can help bring the best, most in-depth reviews up to the top of the list.

Add-on Scripts
The platform by itself, IMHO, is pretty nice. But of course its only 1/3rd functional if you can't use it from mturk itself so I've got a script put together to import some of TurkerView's functionality into mTurk.

Worker:


www:


Dashboard (requires Kerek's mTurk Hourly):


You can install TurkerView.user.js by clicking this link. Again, you'll also want to add Kerek's mTurk Hourly as well.

What now?
Now I cry at all the bugs y'all find hahaha. But seriously, please submit data. The entire platform will have been a waste of time if y'all don't find it useful so give it a go and give me feedback. Trust me, nothing you say can be meaner than what @NBadger & @GoldGuardian have already told me LOL. My current TODO list looks something similar to this:

• Scratch & Redo the individual review page lists
• Fill out Requester profile claim information, reply best practices documentation
• Better data validation (will expand as I find holes, I'm sure they exist in droves)
• Mod panel processing
  • On this note, if anyone is interested in volunteering to help keep the platform free of ickyness, please DM me.
• TH Integrations
  • Exports
  • Notifications for things like mass-rejections, blocks, etc
  • User profile syncing (updating avatars, etc)
• Form Fixes
  • More elegant suggested completion time display/validation
• E-mail triggers on Requester reply
• Script documentation
• Pray the server its on is strong enough.
• Sleep. Please. I neeeeeeds it.

Thank you
For scrolling this far. I'll be up for another 30 minutes or so to answer any quick questions / critical bug fix. Sorry this isn't the best intro post, I don't think I could fit everything I'd like to say in one post, and I mostly just want to thank all the people who helped put this thing together for everyone.