Yeah the software that you guys seem to be using for the marking ones is a bit buggy as far as I have been able to tell. PFA/DGIS both have a history of revoking quals and sending a threatening email when someone accepts one and nopes out of them after deciding that they are too much work for the pay or that they just didn't want to do them.
To the drone strikes, or any targeting: Absolutely not. While our customers are in the government space, our datasets are intended to make more reliable geospatial datasets. We offer our customers something very unique: the ability to accurately validate and discover points at a huge scale, rapidly. You guys as Turkers are faster than any team of analysts. Period. The data we provide is not used with nefarious intent, rather it is used to better equip our customers with more reliable data than they currently have. I honestly don't believe a public crowd would ever be in the targeting process.
I've been unable to get my confirmation email. I tried having it resent, but there is nothing coming through. It's not in my spam email either. The username I chose is the same one I'm using on this forum.
Great talking points for me here. "I have felt leery about doing some batches when the object you're asking us to label is difficult to identify from satellite images." It's my team's job to determine what is discernible, and what is unreasonable t ask a crowd to identify. We often think we know what will work best with our questions. However, I know better. This is something I think is critical in communication with the crowd. You tell me which HITs suck to do, as far as difficulty in discerning the question, and I'll tweak them accordingly. "The hits which require placing a tag vary wildly in the amount of work required. Some have been good but others have required far too much time to tag everything for the pay." Completely understood. This is where that sliding scale of balance in value/quality comes into play for me. Believe it or not, I am actually a Turker too. I actually used the money I made in a few campaigns to pay for an essential oils diffuser to stink my coworkers out with (LOL). We know that feature density is significant in the HIT pay decision. If I task the crowd with counting cars in an area that I anticipate 30% of the tiles will have more than 10 tags, I will either: raise the price of the HIT, decrease the maps/ votes per assignment (HITs done faster), or a mixture of both. I try to stay in tune to that, while staying within a constrictive budget. I like you guys. I don't want you to run away so quickly. "I'll also repeat what @ChrisTurk said about swapping the keybinds. It's much easier to work on the hits when the selections are consistent between batches." This is the bane of my existence right now. This is an active item on the developer hitlist. I hate it as much as you do. Stay tuned for an update here.
So I have a question re: getting the Qual. I've registered on the site and put my turker ID in as requested, but no qual was granted to me. Do I have to contact somebody for that? (Presumably that would be you )
What about Wes Hildebrandt? They used the same tomnod software so I assumed that were part of your team. I feel like Wes Hildebrandt was the red headed stepchild of the company for some reason. At least going by the hits that they dropped and the fact that everyone forgot about them.
^Same Also, props to you for this thread @Bobby. Your answers have been extremely informative and fascinating.
I'll cover this a bit more in some following posts/ threads, as I'm about to address a new and growing problem. New users are essential, and we understand completely that there is a learning curve, albeit minute. It is perfectly okay to have a person submit tags that are inaccurate, as long as it is within their best judgement. If over several campaigns, their CrowdRank (algorithm we used to score points and users) score is consistently low, I will reach out to them and ask if I can help. If there's no reply, I will revoke their qualification in an attempt to save the quality of my dataset. Using a script to autosubmit votes: I will find you. I will eat your lunch. I will destroy everything you know and love. But in all seriousness, I will revoke your qualification, and send you an email explaining why. One of the ideas that I'm chewing on is to create a qual campaign (another one) that every user must complete, that will include acknowledging guidelines and rules against using scripts. • Using a script to open multiple tabs- You have to understand that our accuracy relies on crowd consensus. I can't have 3 people hog 70% of the HITs. That means the data might be unreliable, and that would be terrible for business. Someone having 8-10 tabs open..okay. Someone having 40-50 tabs open... not okay. • Using a script to auto-vote- Absolutely not okay. Not okay with MTurk. Not okay with us. See above in red.
Hi Bobby, thanks for joining the hub and making this thread. My only question is also about the qual, with a few others here. I did sign up on the website as well but have no qual yet. I test it every time I see a batch posted, but no go. Thanks again!
The max amount of tabs that you will see people use is 25 anyways because that is the size of the mturk queue. But if the servers are running without any issues we don't need to mutli tab that much to make your hits worth doing.
Wes is a God among mortals in this workspace. He is currently the savior of this platform, and will lead us to the light. He is my POC for the rebuild. Wes is a good guy.
A few of us called his hits "PFA hardmode" because of how insane they were. But for the same pay >.> but meh that isn't a topic for this thread though.
If ANYONE doesn't have a qual, you need to sign up at geohive.digitalgobe.com first. If you've already done that, PM me the email and/or username that you used to sign up. I'll get you the goods. Exception: I cannot grant you access to the family crowd. This qual requirement was not dictated by me.
Sure it is! He's trying to feel out the crowd. Let's help him out. BE HONEST. I'll feed your input back to him. Communication is priority in any relationship. <3
And communication is how you end up with higher quality work on mtuk imho. People are much more likely to put their A game into a hit if they feel like they are working for a human and not a faceless organization.
For the server, is there any way to cut the interface down to only having to send or use 12 tiles? The center tile being the one of interest and then the straight and diagonal on either side? There shouldn't be a need to send the tiles beyond that square. It would save bandwidth on everyone's end and allow cleaner focus on identifying whether what's in the center is an airline terminal, interstate clover, bridge, etc. Also another vote on not messing with (change the order of) the keybinds lol!
It's feedback like this that keeps me coming back. I like it. I'm taking it into a team meeting. Howboudah?
Hello, I signed up on your website and entered my ID but I still do not have quals for HITs , can you help me
