Preview: TurkerView's New "View API" Infrastructure

Hey folks
First thing first: this essentially will not affect you if...

• You submit content to or participate in TV already (submitting reviews, return reviews, etc) or start doing so at any time (Learn how to get involved here)
• You're a baby Turker (~ <1k HITs)
• You only Turk casually (2-3hrs/day)
• You use the website to access TV (these changes are strictly directed at the brand new API services)

As many have noted I've been burning the midnight oil lately. While I still have more work to do before launching this officially I wanted to throw out a little preview/announcement post since this is a huge direction change for TV.

Keep reading so you know what’s up, but just know that these changes aren't geared towards any of the above user groups and the only things most users will really see out of this are improvements to the reliability of the API services for all the scripts you use (not just TVJS).

Background - 99 Problems & DDoS is all of them.
Since the day it went up the API for TurkerView has been a bit of a problem child. It has continually outpaced the resources available to it, which is pretty much the story of the entire site, and that is nothing short of fantastic. TV has been adopted at a rate I never guessed would even be possible.

Now, months later, it’s serving thousands of Turkers across at least a dozen unique scripts/apps & responding to close to 2 million requests for information every work day.

However, monitoring suggests that during peak times many requests are dropped due to the overwhelming strain placed on the system. Worse yet much of the data is cached as a rolling backup of the main database so scripts like MTS & Forker miss out on some pieces of data. We also have been unable to push out public documentation for many of the advanced API features (mostly stuff you see in TVJS) since the load would just snowball these problems.

The Solution? More Power!
Over the past 2 weeks or so we've put up a much more scalable & robust back end infrastructure. Load balancers, live read-replicas of the main database, and a full user authentication system representing an investment of hundreds of dollars/mo worth of cloud infrastructure are now set up & ready to handle the load that is presently being placed on one server.

Technical details aside this means TV will be far more responsive & capable of serving up its live database representing half a million dollars worth of task breakdowns: 80% of which are good to well paying HITs & Requesters curated from around the platform. Alongside those we'll also be opening up the specialized datasets that allow workers access to a quickly identifiable "heads up" display right on the MTurk interface that alerts them to broken HITs, unpaid screeners, and other metrics that would otherwise waste time & drag their wages down - no need to load a website or even disrupt your queue.

That --- sounds expensive?
Yes, unfortunately, it is and really it's only the tip of the iceberg. TurkerView has also become a full time commitment (and I say that with all the love in the world because I do love it). On top of my own time invested, a few hundred people currently submit the large majority of the content that is used by thousands. It is unsustainable as a long term model since it is running as a commercial services application - providing immense value to what are essentially thousands of "small businesses".

So starting in January "power" users who don't submit content will be asked to contribute back to the platform in the form of upgraded (paid) accounts for the new API (~$8/mo for single users, $11 for a usage tier aimed at multi-Turker households). Until then anyone who opts in early will receive a substantial discount on the subscriptions (they'll be available this weekend at roughly $5/$7) for as long as their subscriptions stay active.

Anyone who cannot or prefers not to invest monetarily in the services will always have access to a free usage tier OR can also "earn" the same upgrades paid users will get to the services by contributing information (reviews) instead. I'm always available to answer questions about submitting information to the site & @NBadger has made an incredible guide to getting started with TVJS. We've made tons of changes in recent months to make the platform more inviting to our huge & varied user base and workers can expect that to continue to improve with these changes.

Is this really necessary?
In short answer: yes. We've seen our space professionalize in amazing ways over the last few years and this is the next extension of that. TV has really improved how workers identify & prioritize tasks on the platform opening up more opportunities to not only earn more money but do so more efficiently with insights from some of the platform's most prolific workers. We're delivering even more value to users through upgraded infrastructure investments & it has reached the point where we need to fund those endeavors as well as the many ways we want to grow in the future.

Not only do I believe worker funded & operated infrastructure is a good thing but those contributing to it deserve to have their submissions treated fairly & actually cared for. We've seen TO experimented on, manipulated, exploited & sadly unable to sustain caring for itself [warning: explicit language]. Considering this represents the publicly recommended face of our community it is no wonder some businesses/academics would prefer to look elsewhere for workers. That hurts all of us and TV will not become that and these changes are geared at making it a more stable piece of infrastructure workers can rely on and expect to improve for years to come.

I'm also hopeful that the continued professionalization of our space will allow us to keep TV's resources in the hands of those who will put it to good use making positive contributions not only to the worker facing ecosystem but to MTurk in general.

Free Tiering
One thing I want to make clear & expand on is that there won’t be situations where a user is actively contributing to the service and is also required to pay for it. The basic approach will be that when a user submits content to TurkerView on any given day, they will be given more access to the API for that day as if they had a paid API subscription. On top of lifting daily quota limits the more content a user submits to the website in total, the more free requests they'll have at the start of every day (even days they can't submit content). In essence, the belief is that users helping other users deserve the help of others in kind & we're committed to facilitating that

Free tiering will also apply to new/inexperienced Turkers who most need the protection & help that our platform offers but who lack the experience to contribute back to it properly. We want new users to grow with us as they always have so they wont be subject to any of this. MTurk has a steep learning curve, we don't want to add additional burdens on top of that & will do everything we can to continue helping new or inexperienced workers learn the ropes.

This free tiering also applies to users who simply don't put much strain on the infrastructure to begin with as well. We want workers to find value in the services before asking them to give some back. So if you're just joining the community or grabbing some surveys with your morning coffee before work don't worry, we've got you covered & plan to keep doing so for a long time.

As for the subscription model choice I want to take a minute to touch on that choice as well. Historically we're seen worker infrastructure be funded by one-time "donations". Those do alright when costs are very low as the ebb/flow of the worker population generally brings in enough people to keep the funding level at costs. However they tend to fizzle out as time goes on despite costs staying the same (or more often than not: rising). It has almost always been the case as well that those most often willing to give are the most involved in the community aspects, which generally meant they were also generating the content and spending their time helping out with other things besides financials. I really want to reward those users & asking for people not participating to fund the infrastructure instead allows us to do so. Subscriptions make sense for our purposes & have largely been adopted by most industries outside our space. I'm really excited at the possibilities this can open up to be able to grow the services & continue to expand and improve on what we do here.

On a personal note, I've always looked at Turking as a business and I believe in investing not only in myself but the ecosystem around me & have continually trusted it would yield better results. TV wouldn't exist if that wasn't true & I think many people here understand that process so I'm very hopeful that will continue here

OK but... When?
I wanted to make this announcement before January hit and give as much notice as possible before making actual changes so rest assured there is no reason to feel rushed or concerned changes are about to disrupt your workflow.

We've currently got a few weeks worth of cloud credit for the services graciously offered by the providers we're using & TV is perfectly capable of running well beyond that so no need to worry about losing access to the place. Users will not see quota limits put in place until at least the 2nd week of January but I intend to extend that out so this post can serve as a full month's worth (so end of Jan, 2019) of advance notice for workers who rely on us. I'm not going to pull the rug out from anybody and I'll work with everyone I can reasonably accommodate if they reach out & get in touch with issues, concerns, etc.

Moreover, as I begin pushing code to tools that already exist (including TVJS) the new API will not reject requests without access keys right away (ie, it will still respond with data so your tools will not break just because they updated). I do not want to disrupt anyone's workflow so instead users will be shown an alert to register & get their auth keys first. I'll explain that very simple process in a different post, but if you've used Pushbullet for MTS/Forker before you can expect a similar process to that -- if you haven't please do it is an amazing feature offered in those tools!

Our rough timeline looks like this:

• API Keys / subscriptions will be available this weekend & TVJS will update as well with an advanced heads up for folks so it is available & ready for the work week.
• I will push code to Forker & MTS shortly after which means users on those apps can expect a much better experience from the API (no more stale/missing data) & they'll be able to import all features that currently only exist in TVJS.
• The old API will be deprecated once code is accepted upstream
  • New developer documentation is available for those who need it, DM or e-mail me! [it will be public once everything settles so devs can just wait if they prefer, but if other users rely on your work please get in touch]
  • Quotas will not be enforced or finalized for subscription plans until mid-January so we can collect usage statistics & make sure we allocate correct quotas (currently they are what we expect to be accurate based on existing usage trends). If a user subscribes before then and we change plans early supporters will be unaffected (unless we increase limits, then they'll be upgraded for free).

As mentioned above and as is always the case, hit me with any questions, comments, feedback/suggestions, etc through the forum or email


CT

 

wait for it. your social media reps on fb will get yelled at lol

 

You honestly shouldn't. Majority of people will not go beyond the free usage tier. The population of "super turkers" is relatively small as I understand it, & I don't think many of them reside around the FB groups.

 

There's social media reps on facebook I can yell at? Where do I find these poor people?

 

Wow, I really am out of the loop, how did I just find this post two weeks later? 0_o