07/10 - Wait a Minute Wednesday!

A+ response from Alexa Shopping Research regarding their broken hits yesterday

Spoiler

Hello,


First, let me apologize that for some of you, this is the first you are hearing from me after having written to tell me about issues you had with my chat annotation task. I apologize that I was not able to email you all personally, but this affected many people, and I need to be efficient to make sure each of you is properly and quickly compensated for your work.


I am so sorry to have inconvenienced each of you. My work, as a research scientist, depends on your work, and I think the best way to get great work out of people is to treat them well and fairly, and not to waste their time. In that, I failed today. But I can make it up to you. I received three major pieces of feedback today:

1. Submission was broken. It is now fixed, a bug with form validation crept in after I tested (what I thought was) the last version. My eyes are now bleary from testing it.
2. You couldn’t find any missing answers, and some highlighting would have helped. You might not have actually had any missing answers, but I have nonetheless taken this feedback and now any question with a missing answer will be highlighted by a thick red border.
3. There isn’t enough time to complete the hit. I am of mixed feelings on this, as we are actually trying to prevent multi-tasking. However, some of you have filled me in on some details of how workers manage to compete for HITs by queueing them up, and I get that. If you have feedback on how long you think is reasonable, please share it. Otherwise, I am thinking I will extend the timer 15 minutes, and use some other techniques to make sure that people are focused on the task. This is important to us as it is an intensive task, as you will have seen.

And, of course, you will be compensated. I will set that up tomorrow. Each of you will receive a special Qualification which will allow you to access a make-up HIT which will just have some place-holder question. Check the box and you’ll be done.


IF YOU DID MORE THAN ONE HIT: please let me know (you can even just write the message in the subject line to speed things up.) After I can confirm this, I will add extra assignments to the make-up HIT. Everyone is on their honor to not take more of those than they actually did of the original HITs.


Again, I apologize to each of you for this inconvenience. I am sorry I can’t write to more of you personally, and I am grateful for your feedback—even if some of it is occasionally sarcastic. I am sure the experience was frustrating, and all I can say now is that it will be better next time.


AND, we’ll try again in the morning with the repaired, improved version of the HITs. I invite you to try again.


Thank you so much for your time,


Marcus D. Collins

Alexa Shopping Research, Senior Applied Scientist

 

Was looking for a gif, but this'll have to do

 

NASA LaRC [A2VH3MP54P53N8] Answer survey questions to gauge impressions of Urban Air Mobility/VTOL Transportation - $1.81 | PANDA
[div] style="border: 1px solid transparent; padding: 10px 0; display: flex; align-items: center; border-color: #ecf0f1; box-shadow: 0 1px 1px rgba(0,0,0,0.05); text-align: center;" |
[div] style="float: left; width: 25%;" |
Fair[/div]
[div] style="float: left; width: 41%;" |[div] style="float: left; width: 50%;" |
Unrated[/div]
[div] style="float: left; width: 50%;" |
Unrated[/div][/div]
[div] style="float: left; width: 33%;" | $10.18 / hour
00:10:40 / completion time[/div]
[/div]

​

[div] style="width: 100%; display: flex; flex-items: center; margin-top: 5px;" |[div] style="width: 50%; display: inline-block; margin: 0 5px 0 0;" | Pros:
Very interesting survey.[/div]
[div] style="width: 50%; display: inline-block; margin: 0 0 0 5px;" | Cons:
[/div]
[/div]

Qualifications: Total approved HITs > 100; Masters Exists ; HIT approval rate (%) > 98; Location EqualTo US;
[div] style="margin-top: 5px;" |

To read NASA LaRC's full profile check out TurkerView!​

[/div]
[div] style="display: none;" data-type="tv-review" data-revId="399552" data-reviewerId="2324" |[/div]

 

It just turned dark as night and it's pouring rain....great.

 

The Wharton School [ABIUW8FBC3EE7] Judgment study (10-20 mins, $0.50 + $1 bonus)(~ 15 minutes) - $0.50 $1.00 bonus | PANDA
[div] style="border: 1px solid transparent; padding: 10px 0; display: flex; align-items: center; border-color: #ecf0f1; box-shadow: 0 1px 1px rgba(0,0,0,0.05); text-align: center;" |
[div] style="float: left; width: 25%;" |
Low[/div]
[div] style="float: left; width: 41%;" |[div] style="float: left; width: 50%;" |
Unrated[/div]
[div] style="float: left; width: 50%;" |
Unrated[/div][/div]
[div] style="float: left; width: 33%;" | $9.61 / hour
00:09:22 / completion time[/div]
[/div]

​

[div] style="width: 100%; display: flex; flex-items: center; margin-top: 5px;" |[div] style="width: 50%; display: inline-block; margin: 0 5px 0 0;" | Pros:
[/div]
[div] style="width: 50%; display: inline-block; margin: 0 0 0 5px;" | Cons:
slider hell[/div]
[/div]

Qualifications: Total approved HITs >= 1000; HIT approval rate (%) >= 95; Location In US;
[div] style="margin-top: 5px;" |

To read The Wharton School's full profile check out TurkerView!​

[/div]
[div] style="display: none;" data-type="tv-review" data-revId="399554" data-reviewerId="524" |[/div]

 

Sean Mahoney [AAST4GGB9B0K] Security Robot Study(~ 30 minutes) - $3.00 | PANDA
[div] style="border: 1px solid transparent; padding: 10px 0; display: flex; align-items: center; border-color: #ecf0f1; box-shadow: 0 1px 1px rgba(0,0,0,0.05); text-align: center;" |
[div] style="float: left; width: 25%;" |
Generous[/div]
[div] style="float: left; width: 41%;" |[div] style="float: left; width: 50%;" |
Unrated[/div]
[div] style="float: left; width: 50%;" |
Unrated[/div][/div]
[div] style="float: left; width: 33%;" | $27.84 / hour
00:06:28 / completion time[/div]
[/div]

​

[div] style="width: 100%; display: flex; flex-items: center; margin-top: 5px;" |[div] style="width: 50%; display: inline-block; margin: 0 5px 0 0;" | Pros:
[/div]
[div] style="width: 50%; display: inline-block; margin: 0 0 0 5px;" | Cons:
[/div]
[/div]

Qualifications: SurveyGroup [2570] DoesNotExist ; Exc: [2185672-156370] DoesNotExist ; Location In US;
[div] style="margin-top: 5px;" |

To read Sean Mahoney's full profile check out TurkerView!​

[/div]
[div] style="display: none;" data-type="tv-review" data-revId="399557" data-reviewerId="3229" |[/div]

 

Kathie26 [A1TWJ0VSBSB561] Answer a survey about a mobile app interface - $1.00 | PANDA
[div] style="border: 1px solid transparent; padding: 10px 0; display: flex; align-items: center; border-color: #ecf0f1; box-shadow: 0 1px 1px rgba(0,0,0,0.05); text-align: center;" |
[div] style="float: left; width: 25%;" |
Unrated[/div]
[div] style="float: left; width: 41%;" |[div] style="float: left; width: 50%;" |
Unrated[/div]
[div] style="float: left; width: 50%;" |
Unrated[/div][/div]
[div] style="float: left; width: 33%;" | $14.46 / hour
00:04:09 / completion time[/div]
[/div]

​

[div] style="width: 100%; display: flex; flex-items: center; margin-top: 5px;" |[div] style="width: 50%; display: inline-block; margin: 0 5px 0 0;" | Pros:
[/div]
[div] style="width: 50%; display: inline-block; margin: 0 0 0 5px;" | Cons:
[/div]
[/div]

Qualifications: completed my survey already DoesNotExist ; Location EqualTo US;
[div] style="margin-top: 5px;" |

To read Kathie26's full profile check out TurkerView!​

[/div]
[div] style="display: none;" data-type="tv-review" data-revId="399558" data-reviewerId="3229" |[/div]

 

Title: Name Evaluations (~ 5 minutes) | PANDA
Requester: Rose Mellay [A2OIDTSLJVR3JT]
TurkerView: [ $14.87 / hour ]
Description: Share your opinion of different names (it is fine to take this study even if you have taken a previous version)
Duration: 30 Min
Available: 44
Reward: $1.00
Qualifications:

• Exc: [158331-157742] DoesNotExist
• Location In CA, US,
• HIT approval rate (%) GreaterThanOrEqualTo 90,

[31TSCA2OXRYIIZY638FQCJ4NL1CCL6]​

 

PSNLab MTurk [A2DWZD2E8XGA45] Thinking about ourselves and others- day 2(~ 10 minutes) - $1.47 | PANDA
[div] style="border: 1px solid transparent; padding: 10px 0; display: flex; align-items: center; border-color: #ecf0f1; box-shadow: 0 1px 1px rgba(0,0,0,0.05); text-align: center;" |
[div] style="float: left; width: 25%;" |
Generous[/div]
[div] style="float: left; width: 41%;" |[div] style="float: left; width: 50%;" |
Unrated[/div]
[div] style="float: left; width: 50%;" |
Unrated[/div][/div]
[div] style="float: left; width: 33%;" | $20.35 / hour
00:04:20 / completion time[/div]
[/div]

​

[div] style="width: 100%; display: flex; flex-items: center; margin-top: 5px;" |[div] style="width: 50%; display: inline-block; margin: 0 5px 0 0;" | Pros:
[/div]
[div] style="width: 50%; display: inline-block; margin: 0 0 0 5px;" | Cons:
[/div]
[/div]

Qualifications: Total approved HITs >= 100; Exc: [155460277-157655] DoesNotExist ; HIT approval rate (%) >= 95; Location In US; Inc: [157610-157655] Exists ;
[div] style="margin-top: 5px;" |

To read PSNLab MTurk's full profile check out TurkerView!​

[/div]
[div] style="display: none;" data-type="tv-review" data-revId="399562" data-reviewerId="147" |[/div]

 

All these Joshua Tree listings are making me itch to go on another camping trip there

 

Kathie26 [A1TWJ0VSBSB561] Answer a survey about a mobile app interface - $1.00 | PANDA
[div] style="border: 1px solid transparent; padding: 10px 0; display: flex; align-items: center; border-color: #ecf0f1; box-shadow: 0 1px 1px rgba(0,0,0,0.05); text-align: center;" |
[div] style="float: left; width: 25%;" |
Generous[/div]
[div] style="float: left; width: 41%;" |[div] style="float: left; width: 50%;" |
Unrated[/div]
[div] style="float: left; width: 50%;" |
Unrated[/div][/div]
[div] style="float: left; width: 33%;" | $15.25 / hour
00:03:56 / completion time[/div]
[/div]

​

[div] style="width: 100%; display: flex; flex-items: center; margin-top: 5px;" |[div] style="width: 50%; display: inline-block; margin: 0 5px 0 0;" | Pros:
[/div]
[div] style="width: 50%; display: inline-block; margin: 0 0 0 5px;" | Cons:
[/div]
[/div]

Qualifications: completed my survey already DoesNotExist ; Location EqualTo US;
[div] style="margin-top: 5px;" |

To read Kathie26's full profile check out TurkerView!​

[/div]
[div] style="display: none;" data-type="tv-review" data-revId="399571" data-reviewerId="5700" |[/div]

 

Jing Li [A3KGX8S92M9LE] Answer a survey about your hotel experience - $0.50 | PANDA
[div] style="border: 1px solid transparent; padding: 10px 0; display: flex; align-items: center; border-color: #ecf0f1; box-shadow: 0 1px 1px rgba(0,0,0,0.05); text-align: center;" |
[div] style="float: left; width: 25%;" |
Fair[/div]
[div] style="float: left; width: 41%;" |[div] style="float: left; width: 50%;" |
Unrated[/div]
[div] style="float: left; width: 50%;" |
Unrated[/div][/div]
[div] style="float: left; width: 33%;" | $7.47 / hour
00:04:01 / completion time[/div]
[/div]

​

[div] style="width: 100%; display: flex; flex-items: center; margin-top: 5px;" |[div] style="width: 50%; display: inline-block; margin: 0 5px 0 0;" | Pros:
[/div]
[div] style="width: 50%; display: inline-block; margin: 0 0 0 5px;" | Cons:
[/div]
[/div]

Qualifications: Total approved HITs > 100; Masters Exists ; HIT approval rate (%) > 90; Location EqualTo US;
[div] style="margin-top: 5px;" |

To read Jing Li's full profile check out TurkerView!​

[/div]
[div] style="display: none;" data-type="tv-review" data-revId="399572" data-reviewerId="2324" |[/div]

 

Abigail [ATEVL9O9FEYTU] Curiosity Video Task Guessing 6192019 - $4.50 | PANDA
[div] style="border: 1px solid transparent; padding: 10px 0; display: flex; align-items: center; border-color: #ecf0f1; box-shadow: 0 1px 1px rgba(0,0,0,0.05); text-align: center;" |
[div] style="float: left; width: 25%;" |
Fair[/div]
[div] style="float: left; width: 41%;" |[div] style="float: left; width: 50%;" |
Unrated[/div]
[div] style="float: left; width: 50%;" |
Unrated[/div][/div]
[div] style="float: left; width: 33%;" | $9.15 / hour
00:29:30 / completion time[/div]
[/div]

​

[div] style="width: 100%; display: flex; flex-items: center; margin-top: 5px;" |[div] style="width: 50%; display: inline-block; margin: 0 5px 0 0;" | Pros:
part two tomorrow[/div]
[div] style="width: 50%; display: inline-block; margin: 0 0 0 5px;" | Cons:
[/div]
[/div]

Qualifications: Total approved HITs >= 0; HIT approval rate (%) >= 95; Location EqualTo US;
[div] style="margin-top: 5px;" |

To read Abigail's full profile check out TurkerView!​

[/div]
[div] style="display: none;" data-type="tv-review" data-revId="399573" data-reviewerId="3047" |[/div]