Qual Scores

Hello! I'm relatively new to mTurk and have been reading multiple posts on this forum & a few others [ this one seems most respectful ;-) ] and I think I have the qualifications tests understood finally and [besides the mTurk ones that might tell you] I have no idea what score I have to get on most of the qual tests I've found. I understand that the adult content one I know I passed - but the others are confusing and I can't find the answers anywhere. I would have guessed it to be ??/100 but then I saw one of my scores is a "140" so now I'm just lost... lol. So if anyone can help a sister out, that'd be greatly appreciated!

Thank you & have a lovely weekend

 

The only way to know what scores are needed on a qualification test is to see the HITs that require the qualification test be taken, and then see what the Requester chooses as the minimum score required on that test. If those HITs aren't posted on MTurk at the moment, then you cannot see this information.

As an example, the requester May ElSherif (https://worker.mturk.com/requesters/A4MS43O6ZXWEE/projects) has a Qualification test associated with her HITs, and she decided that only workers who score over 80 on her qualification test can work on her HITs. (See the attached screen capture below).

In the case of the May ElSherif, she only allows Workers one try to take her qualification test. Other requesters allow Workers multiple tries at taking their qualification tests, and the only limitation is that there's a wait-time before you are allow to re-try the test (sometimes 1 minute, 2 minutes, 1 hour, 1 day... it varies) So if a requester's qual test states that you cannot re-take it, as May ElSherif's does, then it's best to plan on giving that test your undivided time and attention.

I assume that you are finding multiple qualification tests via Qualifeye (https://turkerview.com/qualifeye/) which is a really useful utility. However, just because you see a test posted on Qualifeye, that does not mean that associated HITs are available to work on on MTurk. The unique thing about a qual tests (versus a qualification HIT) is that the URL link to the qualification test remains valid long after the associated HITs are gone. So you may be able to take a qualification test via Qualifeye, get a perfect score, and then not see the HITs ever again. Sometimes the associated HITs reappear a few days, weeks, or months after you take the test. Some requesters post daily... Qualifeye give links to the HITs posted on MTurk and info about when the batch/test was first seen.

Anyway, taking a qualification test without knowing what HITs require it can be waste of time, but occasionally its helpful. Qualifications like the Adult Content Qualification ant the MTurk Premium Qualifications are used by a lot of HITs. The BoundingBoxTestScore qual from MLDataLabeler is used by them for HITs that are posted frequently.

Finally, as you pointed out, some tests definitely do not result in a simple # out of 100 score. I took one qual test and received a score of 301375. In other cases, passing the qualification test yields a simple binary score (0 vs 1) meaning you either are not qualified (score of 0) to work on the HITs, or you are qualified (score of 1).

Hope that's clear. Feel free to ask any follow up questions.

 

That was such a detailed and amazing response! I appreciate the time you took to explain this to me, and it makes complete sense.
I'm trying so hard to teach myself all of this mTurk stuff as well as learn scripts, so this took a big load off my shoulders.

I hope you have a great day & again, thank you so much!