I recently posted an HIT linking a questionnaire I'd made with Qualtrics. I got all the responses I needed, I made sure there were safeguards in place so that every Worker could only send a finished survey, etc etc. However, the survey codes people are posting don't match up with the survey codes I got from my exported Qualtrics data. All the answers seem fine enough, no one sent in any thing that was incomplete (I took precautions against it), but when I try searching a Worker's posted survey code in my exported Qualtrics csv, I never get a match. I haven't looked through every single survey code, but still. Any idea as to what's going on here? Did Qualtrics just re-randomize all the codes? Was it some other formatting error? It can't be the survey takers themselves, as there's no way that many people would turn out to be frauds or bots. Any insight is appreciated!
The simplest / most obvious test that comes to mind is simply completing the survey yourself and comparing the code it generates and gives you (that you'd paste into MTurk if you were a worker) with what Qualtrics displays to you on the back end as the requester. Just use an incognito window of your normal browser so there's no cache complications and you should get a pretty clear view of the discrepancy (if there is one). Just curious but do you mean the MTurk results CSV here? A little hard to know if you're comparing apples:apples if this isn't what I'd be expecting
Just did a quick Preview session. Somehow, the code the survey gives me does NOT match the code that gets recorded when the responses are saved. Uh oh.
Some requesters don't bother checking to see if something's wrong before mass rejecting workers. Thank you for checking first!
Well, on the bright side now you know where the problem is It's really hard for anyone to know why without being able to see all the data/information, but I imagine Qualtrics has a rep you can reach out to for troubleshooting? I'd be interested in hearing what the result is, I've never run across this myself and most of the survey code problems that are prolific on mturk tend to be much simpler in nature.. never seen a requester talk about having this one before (though it'd be easy to miss on our end since we deal more w/ workers)
