New Requester: Discrepancies Between MTurk Survey Codes & Qualtrics Survey Codes

Discussion in 'Requesters' started by Vasil Duka, May 2, 2019.

  1. Vasil Duka

    Vasil Duka New Turker

    Messages:
    3
    Gender:
    Male
    Ratings:
    +2
    I recently posted an HIT linking a questionnaire I'd made with Qualtrics. I got all the responses I needed, I made sure there were safeguards in place so that every Worker could only send a finished survey, etc etc. However, the survey codes people are posting don't match up with the survey codes I got from my exported Qualtrics data.
    All the answers seem fine enough, no one sent in any thing that was incomplete (I took precautions against it), but when I try searching a Worker's posted survey code in my exported Qualtrics csv, I never get a match.
    I haven't looked through every single survey code, but still. Any idea as to what's going on here? Did Qualtrics just re-randomize all the codes? Was it some other formatting error? It can't be the survey takers themselves, as there's no way that many people would turn out to be frauds or bots.
    Any insight is appreciated!
     
  2. ChrisTurk

    ChrisTurk Administrator

    Messages:
    42,412
    Ratings:
    +106,494
    The simplest / most obvious test that comes to mind is simply completing the survey yourself and comparing the code it generates and gives you (that you'd paste into MTurk if you were a worker) with what Qualtrics displays to you on the back end as the requester.

    Just use an incognito window of your normal browser so there's no cache complications and you should get a pretty clear view of the discrepancy (if there is one).

    Just curious but do you mean the MTurk results CSV here? A little hard to know if you're comparing apples:apples if this isn't what I'd be expecting
     
  3. Vasil Duka

    Vasil Duka New Turker

    Messages:
    3
    Gender:
    Male
    Ratings:
    +2
    Not to worry. The exported survey data from Qualtrics came as a .csv file.
     
  4. Vasil Duka

    Vasil Duka New Turker

    Messages:
    3
    Gender:
    Male
    Ratings:
    +2
    Just did a quick Preview session. Somehow, the code the survey gives me does NOT match the code that gets recorded when the responses are saved. Uh oh.
     
    • LOL LOL x 2
  5. th6378045

    th6378045 Well-Known Turker

    Messages:
    3,548
    Gender:
    Female
    Ratings:
    +2,531
    Some requesters don't bother checking to see if something's wrong before mass rejecting workers. Thank you for checking first!
     
  6. ChrisTurk

    ChrisTurk Administrator

    Messages:
    42,412
    Ratings:
    +106,494
    Well, on the bright side now you know where the problem is :p

    It's really hard for anyone to know why without being able to see all the data/information, but I imagine Qualtrics has a rep you can reach out to for troubleshooting? I'd be interested in hearing what the result is, I've never run across this myself and most of the survey code problems that are prolific on mturk tend to be much simpler in nature.. never seen a requester talk about having this one before (though it'd be easy to miss on our end since we deal more w/ workers)