New Requester: Seeking Advice on How to Limit a Batch to One Worker

Discussion in 'Requesters' started by liam_w, Nov 16, 2018.

  1. liam_w

    liam_w Requester

    Joined:
    Nov 13, 2018
    Messages:
    2
    Likes Received:
    1
    Hi everyone,

    Brand new requester here. We have a large number of stores which we’d like to score on attributes that impact their performance (visibility, ingress-egress, etc.). There is some subjectivity involved in this scoring, so we would like to set up a feedback loop to help the worker align with how we would score a store.

    Because of this learning curve we’d like to keep the number of workers limited and allow them to work on a large number of stores. We would start small, with five HITs in a batch, and give the worker feedback on their scoring.

    I’m contemplating the following approach to enable a worker to address multiple HITs at a time. Can you tell me whether any of this is out of line?

    First, I thought I might ask whether anyone on this forum is interested in participating. The pilot HIT is in the MT sandbox under the Requester Dena Whitaker. Anyone who checks it out and is interested may respond here with their worker ID.

    Second, I would group the five stores for the pilot into a batch of HITs. I understand that if I keep the metadata the same for all of the HITS then they will be grouped together for the worker. I will create a custom qualification for these HITs (say, “Store Classifier 1”) and assign one of the workers who has responded in the forum to this classification. It seems like this would limit all five HITs in the batch to that one worker.

    Third, I would create a second, third, fourth and fifth batch of five HITs apiece. Each of these batches would be slightly differentiated by the description (Store Classification 2, 3, 4, 5) for the HITs within them. Then, like the first batch, I would create a custom qualification for the HITS within each (say, “Store Classifier 2, 3, 4, 5”) and assign one of the workers who has responded in the forum to each of these classifications.

    I will not reject any work which appears to have been done in good faith. In the sandbox the reward is listed as $2.50, but that’s the “long term” rate. For this first set of batches I’ll set the rate at $3.50 to help offset the time to go through the instructions. For workers who have done a good job and whom I’d like to offer further batches to do, I will give a $.01 bonus. In the message accompanying the bonus I will give feedback on how the worker’s scoring for each store aligned with our scoring.

    I hope to develop a core of workers who can devote a significant amount of time to this project. At the end of the project I will also give some bonus to those who make significant contributions.

    Thanks!
     
  2. Flora

    Flora Administrator
    Staff Member

    Joined:
    Jul 16, 2012
    Messages:
    52,267
    Likes Received:
    6,451
    Hi Welcome to the forum! Thanks for joining us.

    If workers are interested, they will be sending you a private message with their worker IDs.

    Do you have a link so we can see the HIT in the sandbox? I don't see it.
     
  3. elt

    elt User

    Joined:
    Feb 19, 2013
    Messages:
    2,243
    Likes Received:
    121
    sounds like this one
    https://workersandbox.mturk.com/requesters/A3VGBHJ6N2NQYF/projects?ref=w_pl_prvw
    https://workersandbox.mturk.com/projects/329HSLC9XFD8SUST84EEK0NT4D4GH7/tasks?ref=w_pl_prvw

    I think approved hits in sandbox >1000 makes most unqualified?
    Amazon probably doesn't make that clear in the docs?
    maybe it used to be different before the site change over last year?

    1. Dena Whitaker Append attributes to retail locations $2.50
      Description
      Score stores or attributes such as type,visibility and ingress-egress and draw polygons around them

      Time Allotted
      120 MinQualifications Required
      Your Values

      HIT approval rate (%) is greater than 98
      Total approved HITs is greater than 1000
     
  4. liam_w

    liam_w Requester

    Joined:
    Nov 13, 2018
    Messages:
    2
    Likes Received:
    1
    Flora likes this.

Share This Page