QualifEye - MTurk Qualification Watch System

Just copying this into it's own thread since it'll be lost to the sands of time otherwise in the Daily subforum.

QualifEye? What is this? Why do you love puns?

The long & short of it is that you can now use TurkerView itself to alert you to brand new qualification tests & potential qualification hits on MTurk. You don't need to be logged into MTurk, and this will not use any of your valuable page requests to do it's work.

This allows you to focus on the work actually available to you, while opening up new opportunities during down time and eliminating wasted time & focus. You can actually utilize the "qualified only" options available in most scrapers to filter out the massive amount of junk that you will never be able to do. That is one of the few filters that applies directly to MTurk which actually increases the coverage of HITs your scraper will actually get to scrape & see.

This will also hopefully help good requesters get their work done. In many cases requesters who go through the trouble of setting up qual tests just have their HITs languish in the void because of how difficult it is to find & complete them. So we're looking to bring good workers together with those kinds of HITs & make it easier for both parties.

I think the name is funny, I'll change it some time in the future I'm sure since it's objectively pretty dumb lol. QualiFyre, maybe? ..wait I just typed that out and it sounds amazing.. "qualifier" => qualifyre.. who is coming to the festival? Sorry y'all, you know I get delirious near feature launches by now!

Current Features:

• Tracking & auto updating of qualification tests on MTurk
• Tracking & auto updating of HITs that are likely qualification, qualifiers, demographic, or pre-qualifiers for batches/surveys
• Tab alerts
• Live desktop push notifications

Qual Tests

QualifEye has a few thousand qualification tests indexed already (though for the moment it will only display the last few days worth), and it picks up new ones to the platform (or old ones making a return to being used) automatically.

In the last day of testing both of these HIT groups came about from me using QualifEye (the first had 100 available, the other 300, and both sat around for hours):

Spoiler: Reviews for HITs found w/ QualifEye

They added up to $44 worth of HITs I didn't see on scraper but did pick up the qual test & did the batches from off QualifEye and I only caught them while testing/building it, not actively using the tool. Of course YMMV, but I think QualifEye can potentially add a lot of value to your workflow & am excited to see how it's utilized!

Qual HITs

In the past few days it picked up GeoHives, Cloud Labor, & a Unanimous AI qual along with dozens of other small screening/demographic surveys. I'll be improving it based on feedback from this pilot run over the weekend, of course, but it's already pretty good at what it does.

As with all things I expect some bugs & glitches that I'll need to work out over the weekend, but I'd like folks to be able to get a feel for it & if it has actual value to them so the plan is to knock out any user/browser specific bugs by Monday

If you've got questions, comments, feedback, & the normal gauntlet of words I use here to ask for your opinions as always just throw 'em at me. I need a nap so will be back around in the morning.

Where Is It!?
You can access it directly through this link: https://turkerview.com/qualifeye/

QualifEye Link

Aaaand as always thank you to everyone who gave suggestions & helped me test things out!

 

Hello,

I have a minor issue with QualifEye.

A pair of weeks ago I had requested a feature by email: to show the requirements for each Qual Test.
It was important for me because as a non-US worker I don't qualify for most HITS and I can't find it out immediately.
Kindly you took care of it adding the "(i)" control, specifically intended to show such requirements.

Alas, at least for me, it works just seldom.
Usually clicking the (i) the next Qual Test just slides out of view like to make space for something else but no "qualifications required" appears.
Sometimes, rarely, it works but just for the first Qual Test listed.

Using Chrome 73.0.3683.86
Same issue with Firefox 66.02

 

@ChrisTurk is there a way to filter out HITs that require masters?

 

It's been a year since I started on Mturk, and I wanted to say a huge THANK YOU for making this Qualifeye. On Mturk, I found the qualification tests hard to find, which were hidden in a small drop-down underneath each HIT listed on the original queue on mturk (right where "request qualification" might be listed). It was cumbersome and time consuming, and I barely found any. This Qualifeye has helped me immensely, not only finding those tests and consent forms, but also the possible Qualification HITS. It has really helped me in my work. Thank you so very much!!!

 

qualifeye is the the appleseye! at least so far until i fail one then my opinion is subject to change at any time lmbo

 

OMG I CANT BELIEVE IT BUT IM NAILING THEM LEFT AND RIGHT SOME ARE TOO CONFUSING BUT IM KILLIN IT TY SO MUCH FOR CREATING THIS SECTION OF TURKERVIEW

 

This will help out tremendously. Thanks!

 

thank you for this resource

 

this really has turned out to be the best thing to ever happen to my mturk "career" im improving lil by lil everytime i log in to work, getting more and better work , now if you could create a script for human issues such as energy and time management we could work on that pace average thing lol jk thank you chris, i feel we could do more to help you and if so ill give it a shot....ty

 

I only just "discovered" this after browsing the turkerview page. Great tool, thank you so much for all of the tools you put out @ChrisTurk ! You make it much easier to navigate the world that is mTurk.

 

I am new and I just find out! thanks a lot, hope it will help me to catch more Hits!

 

Depending on how you search for hits. If you use Hit Forker check "qualified for". If you are just using the Hit finder on mturk there is also a place to check "qualified for".