Requester Question to MTurkers on Alternatives to HIT Pay Reduction
Hi! My name is Annabel, and I'm a requester on MTurk (http://turkopticon.ucsd.edu/A3HB3KPJ2RZ2K8). I'm on here to ask you guys how I can make my HITs more efficient so I can keep running them. Right now, my funding is becoming a bit limited, but our project still needs a lot more information.
I've decreased the amount of pay for my HITs in the past before, but I'd like to avoid decreasing the amount of pay again, especially since I'd like to keep a good relationship with the people who've been working on my HITs.
Right now, my HITs revolve around asking people to compare two sentences with each other, and to select the sentence that is the more important one. (Title: Comparing two statements with each other) The goal of these HITs is to collect enough information to create an app or a website that can automatically read the Terms & Conditions and give people only the important parts.
I do have an idea to address this (change the task so that people compare groups of sentences with each other so we get more data, and slightly increase the base pay), but was wondering whether there were other ways out there.
Are these masters? If they are, you could save money by making them non masters and put a qualification test in the hit.
Nope. Not Masters. Thanks for the suggestion though!
Originally Posted by DireForFire
- Have you added timing information and analyzed where people are spending time?
- Have you analyzed your current data set to see if demographics has an impact? If you run an analysis and find that the demographic info is not relevant to the results, drop the demographic info.
I don't know how the hits are set up or how much information you need per topic/article you are analyzing but I think grouping more than two sentences that are related to a single topic while increasing the base pay slightly would be good for productivity and give you more data, as you already stated.
Instead of a "Which of the two sentences are more important?" perhaps a system could be set up where the most important 1 or two sentences are chosen from the set.
Obviously I suggest that regardless what is chosen pay be adjusted for actual time required.
From a workers perspective I don't think this is much more work, but everyone is different and works at different paces. Grouping sentences from the same topic is easier than resetting to a new topic with every new HIT, so even comparing 2 sentences and then 2 sentences again about the same subject in the same HIT might be ok too.
Hope this helps, not sure I came across as clearly as I wanted to.
Oh, wow. Sorry for missing both of your responses! I should have had the forum settings set to receiving e-mails for additional answers!
This is a good idea - I do have timing information, but since the task itself is rather repetitive and straight forward (just choosing between two statements), there isn't quite that much to analyze! There were issues where the site kept crashing though, so I went in and just rewrote everything, so hopefully that got fixed..
Originally Posted by tilt
Demographic data currently isn't getting collected either, but I will keep this in mind for the future!
Thank you for the ideas!
This is an interesting idea - I will consider this one for another type of task to run!
Originally Posted by smorganie
What about going back to paying decently?
Originally Posted by Annabel
Sorry for the late response! I thought I had it set up so that I would be alerted whenever I received a new message, but I guess I didn't do it correctly the first time around..
Originally Posted by db42
In response to your question, I really do apologize for decreasing pay. As mentioned earlier, limited funding made it difficult to continue paying the same type of HIT at the same amount.
I did try to implement a lot of suggestions to help increase the speed of completing the task so that it'll make the HIT more tolerable and put a cap on the pay decrease (I was suggested to decrease the pay even further actually), but ultimately, it doesn't compare to the amount previously paid.
I will try and see what I can do in the future though, but in the meantime, I can quite understand if you no longer find working on my HITs worth your time. (And thanks for taking the time to read this block of text!)