Results 1 to 8 of 8

Thread: Requester Question to MTurkers on Alternatives to HIT Pay Reduction

  1. #1
    User
    Join Date
    Oct 2016
    Posts
    5
    Thanks
    2
    Thanked 2 Times in 2 Posts

    Default Requester Question to MTurkers on Alternatives to HIT Pay Reduction

    Hi! My name is Annabel, and I'm a requester on MTurk (http://turkopticon.ucsd.edu/A3HB3KPJ2RZ2K8). I'm on here to ask you guys how I can make my HITs more efficient so I can keep running them. Right now, my funding is becoming a bit limited, but our project still needs a lot more information.

    I've decreased the amount of pay for my HITs in the past before, but I'd like to avoid decreasing the amount of pay again, especially since I'd like to keep a good relationship with the people who've been working on my HITs.

    Right now, my HITs revolve around asking people to compare two sentences with each other, and to select the sentence that is the more important one. (Title: Comparing two statements with each other) The goal of these HITs is to collect enough information to create an app or a website that can automatically read the Terms & Conditions and give people only the important parts.

    I do have an idea to address this (change the task so that people compare groups of sentences with each other so we get more data, and slightly increase the base pay), but was wondering whether there were other ways out there.

  2. Thanks: (1)
  3. #2
    User DireForFire's Avatar
    Join Date
    Jul 2015
    Posts
    179
    Thanks
    137
    Thanked 62 Times in 45 Posts

    Default

    Are these masters? If they are, you could save money by making them non masters and put a qualification test in the hit.

  4. #3
    User
    Join Date
    Oct 2016
    Posts
    5
    Thanks
    2
    Thanked 2 Times in 2 Posts

    Default

    Quote Originally Posted by DireForFire View Post
    Are these masters? If they are, you could save money by making them non masters and put a qualification test in the hit.
    Nope. Not Masters. Thanks for the suggestion though!

  5. #4
    User
    Join Date
    Oct 2016
    Posts
    1
    Thanks
    0
    Thanked 1 Time in 1 Post

    Default

    - Have you added timing information and analyzed where people are spending time?
    - Have you analyzed your current data set to see if demographics has an impact? If you run an analysis and find that the demographic info is not relevant to the results, drop the demographic info.

  6. Thanks: (1)
  7. #5
    User smorganie's Avatar
    Join Date
    Sep 2012
    Location
    Florida
    Posts
    275
    Thanks
    77
    Thanked 90 Times in 42 Posts

    Default

    I don't know how the hits are set up or how much information you need per topic/article you are analyzing but I think grouping more than two sentences that are related to a single topic while increasing the base pay slightly would be good for productivity and give you more data, as you already stated.

    Instead of a "Which of the two sentences are more important?" perhaps a system could be set up where the most important 1 or two sentences are chosen from the set.

    Obviously I suggest that regardless what is chosen pay be adjusted for actual time required.

    From a workers perspective I don't think this is much more work, but everyone is different and works at different paces. Grouping sentences from the same topic is easier than resetting to a new topic with every new HIT, so even comparing 2 sentences and then 2 sentences again about the same subject in the same HIT might be ok too.

    Hope this helps, not sure I came across as clearly as I wanted to.

  8. Thanks: (1)
  9. #6
    User
    Join Date
    Oct 2016
    Posts
    5
    Thanks
    2
    Thanked 2 Times in 2 Posts

    Default

    Oh, wow. Sorry for missing both of your responses! I should have had the forum settings set to receiving e-mails for additional answers!

    Quote Originally Posted by tilt View Post
    - Have you added timing information and analyzed where people are spending time?
    - Have you analyzed your current data set to see if demographics has an impact? If you run an analysis and find that the demographic info is not relevant to the results, drop the demographic info.
    This is a good idea - I do have timing information, but since the task itself is rather repetitive and straight forward (just choosing between two statements), there isn't quite that much to analyze! There were issues where the site kept crashing though, so I went in and just rewrote everything, so hopefully that got fixed..

    Demographic data currently isn't getting collected either, but I will keep this in mind for the future!

    Thank you for the ideas!

    Quote Originally Posted by smorganie View Post
    I don't know how the hits are set up or how much information you need per topic/article you are analyzing but I think grouping more than two sentences that are related to a single topic while increasing the base pay slightly would be good for productivity and give you more data, as you already stated.

    Instead of a "Which of the two sentences are more important?" perhaps a system could be set up where the most important 1 or two sentences are chosen from the set.

    Obviously I suggest that regardless what is chosen pay be adjusted for actual time required.

    From a workers perspective I don't think this is much more work, but everyone is different and works at different paces. Grouping sentences from the same topic is easier than resetting to a new topic with every new HIT, so even comparing 2 sentences and then 2 sentences again about the same subject in the same HIT might be ok too.

    Hope this helps, not sure I came across as clearly as I wanted to.
    This is an interesting idea - I will consider this one for another type of task to run!

  10. #7
    User db42's Avatar
    Join Date
    Nov 2014
    Posts
    6,173
    Thanks
    9,367
    Thanked 9,707 Times in 4,364 Posts

    Default

    Quote Originally Posted by Annabel View Post
    This is a good idea

    Thank you for the ideas!

    This is an interesting idea
    What about going back to paying decently?

  11. #8
    User
    Join Date
    Oct 2016
    Posts
    5
    Thanks
    2
    Thanked 2 Times in 2 Posts

    Default

    Quote Originally Posted by db42 View Post
    What about going back to paying decently?
    Sorry for the late response! I thought I had it set up so that I would be alerted whenever I received a new message, but I guess I didn't do it correctly the first time around..

    In response to your question, I really do apologize for decreasing pay. As mentioned earlier, limited funding made it difficult to continue paying the same type of HIT at the same amount.

    I did try to implement a lot of suggestions to help increase the speed of completing the task so that it'll make the HIT more tolerable and put a cap on the pay decrease (I was suggested to decrease the pay even further actually), but ultimately, it doesn't compare to the amount previously paid.

    I will try and see what I can do in the future though, but in the meantime, I can quite understand if you no longer find working on my HITs worth your time. (And thanks for taking the time to read this block of text!)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •