I have been running interactive economic games that require real-time interactions between workers. I noticed that sometimes in a session with 60 workers, the geolocation indicates half of them are from the same city, which is statistically impossible. And these workers provide bad quality data -- it seems that they just clicked through without thinking. What makes it more suspicious is that some of the workers seem to take turns to answer the questions, and the answers they provide are quite similar, making me believe that it is actually one person using multiple accounts. This issue occurs for almost all the sessions. I have used CAPTCHA and checked for the usage of VPN, but it does not seem to solve the problem. I just want to ask: Is there any way for one person to bypass Amazon's restriction and work on the same assignment using multiple accounts? Anyone knows how to detect that and solve the issue?
Each person has a unique worker ID, however, If they have multiple ID's through whatever means (e.g. stolen), it wouldn't be hard to run multiple incognito windows with each ID logged in. Other than what you are already doing, not sure how you could circumvent it. I am not a requester, so this is just me spitballing. I think you should at the very least reject the garbage submissions and block their ID's. This will only stop the ID's you've already encountered, but it's a start. You could also try to limit their access by adding qualifications: HIT approval rate (%) >= 99 Location EqualTo US, CA, GB A custom qual. If your target audience is native English speakers, you could throw in a few English comp checks at the start to try and route them out that way. Wouldn't use multiple choice or any thing they could copy and paste into ChatGPT.
Hi, I noticed the same issue at the last days. There are about 5-8 different users with different WorkId repeat the same data at my survey. He writes "green" word for every area text and at the comment input "good". Is it possible that there are so many accounts owned by the same person? Please assist, Diana.
Thanks for the input. I do not have hard evidence that these accounts are used by the same person so I do not have strong reasons to block these accounts. I simply use some screening to filter out potentially fraudulent accounts and reject assignments if they appear to be of low quality. The other day I had two workers who asked me to give them bonuses without even participating in the experiment (they only participated in the qualification HITs). They were from the same locations according to the geolocation information collected from the qualification HITs, and their inquiry emails used exactly the same wording. Making me more convinced about the issue, even though it is not clear to me how they are able to use multiple accounts to accept and submit HITs.
That is what I am also wondering -- I don't know. In theory they cannot, but increasing evidence shows that many accounts are used by the same person.
I think the issue stems mostly from stolen accounts, but could also just be sharing. In the physical gig economy it happens a lot. Someone who doesn't have credentials shares someone's identity in exchange for some of the profits. And all you need to do is open a couple/few incognito windows to log into multiple accounts at the same.
What Josh083 said sounds pretty accurate, I would imagine if the data is showing you that you're getting the same answers from multiple accounts and the data is also showing you that half is from the same city the only way that makes sense is if you were running a specific survey which focused on the target area? But if that were the case then surely you would of told us in the original post. You did mention that ''I do not have hard evidence that these accounts are used by the same person so I do not have strong reasons to block these accounts" I'd respectfully disagree here, because at this point as a requester your data from your work is being compromised and it sounds like it's a continuous issue at the moment, I would definitely include some extra requirements for Mturk workers accepting your HIT, are you able to see their accounts using their IDs, so for example can you see how many completed HITs these accounts have finished? Are they below 50 and relatively new account sign ups or are they all above 1000? Maybe having extra requirements will help, but I personally don't buy that you don't have strong reasons, the fact your study or research is being compromised should be enough and there isn't any logical reason where by if you poll 1500 people to a survey there isn't any logical reason that 700 of them will be from the same city. That fundamentally doesn't make sense. When I used to sell on ebay for example I could filter listings by location? if your research is targeted to a specific location only then would it make sense. You have to start somewhere, I'd also definitely look into some kind of pre-screen like Josh said that cannot be just cut & pasted.
