Page 1 of 2 1 2 LastLast
Results 1 to 10 of 17

Thread: Beware the Block: Part 3 of 3

  1. #1
    User ergo's Avatar
    Join Date
    Jan 2009
    Location
    Maryland
    Posts
    358
    Thanks
    0
    Thanked 1 Time in 1 Post

    Default Beware the Block: Part 3 of 3

    Continued from Part 2.

    The worst part of this

    Amazon has some special forums for developers, and one is devoted to Mechanical Turk API issues. Two months ago a developer using the name mobiciousmatt posted about a problem he was having problems with bots cheating his HITs. I don't have any tolerance for bots either, but listen to what he says (all emphasis mine):

    our only real recourse seems to be to block the worker. This is fine for us. Yeah, we're out some money, but we just want to stop the problem from occurring again. The problem is that apparently Amazon deletes worker accounts that have been blocked twice. This is a massive ommission from the Developer Guide, which reads:

    "The BlockWorker operation allows you to prevent a Worker from working on your HITs. For example, you can block a Worker who is producing poor quality work. "

    The consequence of this is that workers are choosing not to work on our HITs for fear of being ousted from Mechanical Turk altogether.

    What then are we to do? Come up with a complicated qualification system? Continue to allow poor workers to work on our HITs so the few honest ones aren't afraid of being removed? Why is this not documented?
    Yep. Apparently if you get blocked twice, Amazon may remove your account. While in most cases, the accounts being blocked are cheaters and bots, there's always a possibility you could be blocked by mistake. And if you don't get your first block removed, and get blocked again, then Amazon may ban you from Mechanical Turk completely. If you don't appeal, or they ignore you, then any money in your account will be gone.

    As mobiciousmatt learned, even honest workers are afraid to work for requesters who have a history of blocking. And, unfortunately, they should be until Amazon decides to take worker concerns more seriously.

    Wrapping up (finally!)

    If you've gotten this far, you deserve a medal for your patience and stamina. You've also shown yourself to be someone who cares about worker rights. Even if you haven't been personally affected by the blocking system (yet), I encourage you to send Feedback to Amazon and let them know you are concerned about issues like these which have the potential to hurt honest people -- not only workers, but even the requesters whose money makes MTurk possible. Feel free to send Amazon a link to this series if you'd like.

    You can also make a difference by joining in the global conversation about MTurk in fine places like this. You don't have to write long-winded treatises like I do, because even a small note here has the potential to help many others. Perhaps the words we write now will influence for the better the way Amazon chooses to operate in the future. MTurk is still a diamond in the rough, but with some refinement has the potential to be truly great.

    Whew! My fingers are out of breath now.
    Last edited by ergo; 02-21-2009 at 02:17 PM.

  2. #2
    User
    Join Date
    Feb 2009
    Posts
    176
    Thanks
    112
    Thanked 64 Times in 45 Posts

    Default

    i appreciate the time you're taking to explain things to us turkers. i really hate the paranoia around the subject and it makes me reluctant to speak up. signing up with mturk i never imagined there would be any reason for them to be suspicious of people willing to work for pennies, but there you go!

    a few months ago i got blocked from 'ask a question' with a warning that i was submitting formulaic answers, er questions. i don't know if they were implying i was submitting automated responses or what. i did reply to them, assuring them i am in no way savvy enough to do any such thing. [besides the absurdity of accusing me of 'formulaic' answers when obviously questions follow the formula of who, what, when.... well, you get the idea.]

    anyways, i can't help but think i was blocked because i did 'too many' of their hits - as many as 'many 100's' in a day's time. thus, reaching their tax threshhold. if this was the case, i would have far preferred they had told me this. meanwhile, i took my turking skills to other, more lucrative turking tasks. [i did see your other recent thread on 'maxing out' on hits with requesters...hope this isn't the case with others].

    please keep up the good work and maybe one day we'll put a face to this faceless interface.

  3. #3
    User ergo's Avatar
    Join Date
    Jan 2009
    Location
    Maryland
    Posts
    358
    Thanks
    0
    Thanked 1 Time in 1 Post

    Default

    Thanks, Mischa! I definitely don't want to come across as a Negative Nancy when talking about MTurk, but I strongly believe workers deserve to know the possible pitfalls they may face. As we see with mobiciousmatt above, Amazon not only withholds helpful information from workers, but from requesters as well. I'm absolutely certain that when most requesters hit Block, they have no idea they might be putting a good person one step away from being banned from MTurk.

    As far as the "Ask a Question" HITs go, those are run by Amazon itself, so if you contact them about it I think they definitely owe you an explanation. I think it's perfectly reasonable to ask them to explain what was too formulaic about your submissions, or what was wrong with doing a lot of HITs. After all, those HITs should only take a short time to do, so of course people who like them will do a lot.

    One time I did a whole series of photo retouching HITs for Amazon which took a lot of time, and the next day found out they'd all been rejected saying they were the wrong size. After double-checking my work, verifying that every single picture met the stated requirements of the HIT, I wrote them asking for an explanation. The person at Amazon responsible for the HITs took a look at them again and discovered she'd viewed them with the wrong program, which made them look squished. She said she'd rejected everyone else's HITs for the same reason, and apparently it never occurred to her that she might be doing something wrong. Anyway, I did finally get paid around $150 for my work, and I hope that everyone else who was wrongly rejected got their money, too.

    Thank goodness Amazon didn't have the Block function back then.

    Quote Originally Posted by mischa View Post
    anyways, i can't help but think i was blocked because i did 'too many' of their hits - as many as 'many 100's' in a day's time. thus, reaching their tax threshhold. if this was the case, i would have far preferred they had told me this.
    I'm pretty sure Amazon doesn't care about US workers reaching the tax threshold, as I've done more than $600 of work for them in a year and they didn't seem to mind that it meant some paperwork. Also, as far as I know, limiting isn't the same as a block, but now that you bring it up I realize I should confirm that. I'd hate to think that people could get blocks on their record just because some requesters don't want to be bothered about extra tax work.
    Last edited by ergo; 02-21-2009 at 06:21 PM. Reason: Automerged Doublepost

  4. #4
    User Shego's Avatar
    Join Date
    Feb 2009
    Location
    East Jesus Nowhere
    Posts
    475
    Thanks
    12
    Thanked 85 Times in 57 Posts

    Default

    What I want to know, is where anybody gets off keeping money a worker has earned, whether blocked or not? If a person worked as a secretary, and suddenly the boss decided he/she wasn't working out and needed to be laid off, the boss would still be required to pay his employee for time served, so to speak. Otherwise he'd be in court! See, these are the kind of blunders that are going to get mturk shut down for good. That would be a real tragedy for me. I'm not getting rich here, but at least I can squeek by on what I get from mturk and my child support arrears. Until I build up my own business, this is all I have.

  5. #5
    User
    Join Date
    Jan 2009
    Posts
    352
    Thanks
    0
    Thanked 1 Time in 1 Post

    Default

    Quote Originally Posted by ergo View Post
    Continued from Part 2.

    The worst part of this

    Amazon has some special forums for developers, and one is devoted to Mechanical Turk API issues. Two months ago a developer using the name mobiciousmatt posted about a problem he was having problems with bots cheating his HITs. I don't have any tolerance for bots either, but listen to what he says (all emphasis mine):



    Yep. Apparently if you get blocked twice, Amazon may remove your account. While in most cases, the accounts being blocked are cheaters and bots, there's always a possibility you could be blocked by mistake. And if you don't get your first block removed, and get blocked again, then Amazon may ban you from Mechanical Turk completely. If you don't appeal, or they ignore you, then any money in your account will be gone.

    As mobiciousmatt learned, even honest workers are afraid to work for requesters who have a history of blocking. And, unfortunately, they should be until Amazon decides to take worker concerns more seriously.

    Wrapping up (finally!)

    If you've gotten this far, you deserve a medal for your patience and stamina. You've also shown yourself to be someone who cares about worker rights. Even if you haven't been personally affected by the blocking system (yet), I encourage you to send Feedback to Amazon and let them know you are concerned about issues like these which have the potential to hurt honest people -- not only workers, but even the requesters whose money makes MTurk possible. Feel free to send Amazon a link to this series if you'd like.

    You can also make a difference by joining in the global conversation about MTurk in fine places like this. You don't have to write long-winded treatises like I do, because even a small note here has the potential to help many others. Perhaps the words we write now will influence for the better the way Amazon chooses to operate in the future. MTurk is still a diamond in the rough, but with some refinement has the potential to be truly great.

    Whew! My fingers are out of breath now.
    Just make a google search with the words mturk block. See the results you get. I feel we should ask Amazon to put a representative here, to listen to our problems or suggestions. We might see some change if our ideas and suggestions are conveyed to them.

  6. #6
    User ergo's Avatar
    Join Date
    Jan 2009
    Location
    Maryland
    Posts
    358
    Thanks
    0
    Thanked 1 Time in 1 Post

    Default

    They may be able to pass the buck on some things, but they're in far too deep with others. Besides unfair practices as described here, Amazon is setting itself up for some serious backlash through the operation of broken HITs like Are These Things Different (my pet peeve), which regularly reward cheaters while punishing the innocent with false rejections and withheld pay.
    Last edited by ergo; 02-23-2009 at 12:16 PM.

  7. #7
    User Shego's Avatar
    Join Date
    Feb 2009
    Location
    East Jesus Nowhere
    Posts
    475
    Thanks
    12
    Thanked 85 Times in 57 Posts

    Default

    AMEN!! I'd like to know a little background on the person/people who set up mturk in the first placed. Because from where I'm sitting it definately wasn't a "worker." Andy is right--it's the requestors who make the real money.

  8. #8
    User
    Join Date
    Jan 2009
    Posts
    352
    Thanks
    0
    Thanked 1 Time in 1 Post

    Default

    Andy, you should send Amazon an invitation to put their rep here. We keep on exchanging ideas and suggestions, but without conveying them to mturk, we are not likely to see any solution emerge from this mess. If they are ready to hire Ergo, that would be best, but if not, they can have someone visit this site daily and get some feedback.

  9. #9
    User Shego's Avatar
    Join Date
    Feb 2009
    Location
    East Jesus Nowhere
    Posts
    475
    Thanks
    12
    Thanked 85 Times in 57 Posts

    Default

    I agree with sonica. It would be in amazon's best interest to hire somebody to handle these issues before we all get hurt!

  10. #10
    User
    Join Date
    Jul 2009
    Posts
    11
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Hm. I didn't know that. I've been blocked by CardCow. But, I've never done any work for them. It didn't bother me really until now. So, if perchance I get blocked again I could get banned? Yikes!

    So, how do you contact a requester that doesn't have any HITs up? I'm going to have to take care of this.

    I appreciate this.

Page 1 of 2 1 2 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •