Results 1 to 1 of 1

Thread: Requesters: How To Reverse a Reject.

  1. #1
    User RWStein's Avatar
    Join Date
    Aug 2012
    Location
    North Carolina
    Posts
    1,816
    Thanks
    996
    Thanked 1,057 Times in 417 Posts

    Default Requesters: How To Reverse a Reject.

    Requester Reject Reversal Procedures


    UPDATE: As of January 7th, 2013, requesters can reverse rejected assignments from within the Mechanical Turk Requester interface. For more information, visit the official Mechanical Turk blog.

    In this article I will endeavor to explain how you, as a requester, can reverse rejected HITs for work performed on Amazon Mechanical Turk.

    Why this is a good thing for both you and for the community is beyond the scope of this article but it bears mentioning here that it is. It is also worth mentioning that you need the funds available to pay for the HIT in order to reverse it - exactly like you would an approval, since a reversed reject is an approval.

    Reject reversal is unfortunately not supported in the interface that Amazon makes available to requesters but that does not mean that it cannot be done (as even some AMT support personnel have been known to claim). It simply cannot be done through the default graphical interface that Amazon makes available.

    Below I outline four different basic methods for reversing rejections in ascending order of technical difficulty. All of these make use of the AMT API, full documentation for which can be found at: http://docs.amazonwebservices.com/AW...e/Welcome.html

    I

    Contacting Amazon directly is, by far, the least technical method. Send a message to AMT support containing the AssignmentId of the rejected HIT and the WorkerIDs, if reversing for specific workers, you wish to approve and they will do the rest. I should point out here that I do not have any direct experience either using this method or having it used on my behalf. Amazon employees on Amazon forums have recommended this method for requesters who do not have the ability to make the API calls themselves. However, if you have a large number of HITs you want to un-reject they may very well tell you to follow one of the options below. The above method should work for a small number of rejections you would like to reverse. I would welcome feedback from anyone who has had experience with this method on either side, so I can fill out this section with a little more detail.

    II
    The second easiest and possibly least secure method to use is the published form at http://techlist.com/mturk/unReject.php to perform the reversal; just plug in the relevant data and you are done. Quick and easy, right?

    Why this is potentially a bad idea:
    The site where this interface is hosted is that of a third party, and is known to be at least somewhat unreliable in terms of availability.

    The site where this interface is hosted does not support SSL (HTTPS) connections and this is a known fact. This means that there is nothing stopping any third party so inclined from extracting both your Access Key ID and your Secret Access Key from your request. Here is Amazon's cautionary statement regarding your secret access key:

    Quote Originally Posted by Amazon Mechanical Turk
    Your Secret Access Key is a shared secret between you and AWS. Keep this ID secret; we use it to bill you for the AWS services you use. Never include the ID in your requests to AWS and never email the ID to anyone even if an inquiry appears to originate from AWS or Amazon.com. No one who legitimately represents Amazon will ever ask you for your Secret Access Key.
    The site where this interface is hosted does not support SSL (HTTPS) connections. If you are a member of an organization that is beholden to any of the many data privacy and/or data security standards (PCI/CISP, SOX, GLBA, HIPAA, anything in the ISO 27000 series, you get the idea) you could be very well risking your organization's certification and possibly even breaking the law.

    The site where this interface is hosted is that of a third party and there is ultimately nothing stopping that third party (or another malicious third party who manages to gain access to their server) from logging the information you submit. In fact you can be relatively assured that this information will be logged.

    III

    The next on the chain of difficulty is to host the above form yourself. This makes the assumption that you have a web server available that you are capable of installing things on, but that you are not technically apt enough to address the AMT APIs directly. I include it here for completeness' sake as much as anything else. The page code can be acquired at http://pastebin.com/raw.php?i=M909EsEj

    IV
    The final and most technically challenging, but also most flexible and ultimately - in my opinion - the best method is to write your own code to call the API using one of the SDKs that are available. There are open and up to date SDKs for:

    C#, .NET
    http://aws.amazon.com/code/Amazon-Mechanical-Turk/923

    Java
    http://aws.amazon.com/code/Amazon-Mechanical-Turk/695

    Ruby
    http://aws.amazon.com/code/Amazon-Mechanical-Turk/793

    Perl
    http://aws.amazon.com/code/Amazon-Mechanical-Turk/922

    If it were me doing it and there were no technological constraints, I would recommend the .NET API and using Powershell to write your automation. That combination has yielded the best results for me personally - but I will freely admit to being a little bit prejudiced. There is even a little tutorial on Amazon's site at:http://aws.amazon.com/articles/3801


    - For more information or to become a part of the MTurk discussion community, please sign up as a requester and discuss any issues, questions or comments you have with the actual professional workers who are performing your tasks. Sign up by clicking here: mTurk Forum Registration And contact "Andy" to have a discussion group designed just for your HIT needs.

    -Created by SaladPope (Editor: RWStein)


    Updated 1/25/2013 by Cowfin
    Last edited by RWStein; 11-12-2012 at 11:55 PM.

  2. Thanks: (2)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •