Requester Reject Reversal Procedures
UPDATE: As of January 7th, 2013, requesters can reverse rejected assignments from within the Mechanical Turk Requester interface. For more information, visit the official Mechanical Turk blog.
In this article I will endeavor to explain how you, as a requester, can reverse rejected HITs for work performed on Amazon Mechanical Turk.
Why this is a good thing for both you and for the community is beyond the scope of this article but it bears mentioning here that it is. It is also worth mentioning that you need the funds available to pay for the HIT in order to reverse it - exactly like you would an approval, since a reversed reject is an approval.
Reject reversal is unfortunately not supported in the interface that Amazon makes available to requesters but that does not mean that it cannot be done (as even some AMT support personnel have been known to claim). It simply cannot be done through the default graphical interface that Amazon makes available.
Below I outline four different basic methods for reversing rejections in ascending order of technical difficulty. All of these make use of the AMT API, full documentation for which can be found at: http://docs.amazonwebservices.com/AW...e/Welcome.html
Contacting Amazon directly is, by far, the least technical method. Send a message to AMT support containing the AssignmentId of the rejected HIT and the WorkerIDs, if reversing for specific workers, you wish to approve and they will do the rest. I should point out here that I do not have any direct experience either using this method or having it used on my behalf. Amazon employees on Amazon forums have recommended this method for requesters who do not have the ability to make the API calls themselves. However, if you have a large number of HITs you want to un-reject they may very well tell you to follow one of the options below. The above method should work for a small number of rejections you would like to reverse. I would welcome feedback from anyone who has had experience with this method on either side, so I can fill out this section with a little more detail.
The second easiest and possibly least secure method to use is the published form at http://techlist.com/mturk/unReject.php to perform the reversal; just plug in the relevant data and you are done. Quick and easy, right?
Why this is potentially a bad idea:
The site where this interface is hosted is that of a third party, and is known to be at least somewhat unreliable in terms of availability.
The site where this interface is hosted does not support SSL (HTTPS) connections and this is a known fact. This means that there is nothing stopping any third party so inclined from extracting both your Access Key ID and your Secret Access Key from your request. Here is Amazon's cautionary statement regarding your secret access key:
The site where this interface is hosted does not support SSL (HTTPS) connections. If you are a member of an organization that is beholden to any of the many data privacy and/or data security standards (PCI/CISP, SOX, GLBA, HIPAA, anything in the ISO 27000 series, you get the idea) you could be very well risking your organization's certification and possibly even breaking the law.Originally Posted by Amazon Mechanical Turk
The site where this interface is hosted is that of a third party and there is ultimately nothing stopping that third party (or another malicious third party who manages to gain access to their server) from logging the information you submit. In fact you can be relatively assured that this information will be logged.
The next on the chain of difficulty is to host the above form yourself. This makes the assumption that you have a web server available that you are capable of installing things on, but that you are not technically apt enough to address the AMT APIs directly. I include it here for completeness' sake as much as anything else. The page code can be acquired at http://pastebin.com/raw.php?i=M909EsEj
The final and most technically challenging, but also most flexible and ultimately - in my opinion - the best method is to write your own code to call the API using one of the SDKs that are available. There are open and up to date SDKs for:
If it were me doing it and there were no technological constraints, I would recommend the .NET API and using Powershell to write your automation. That combination has yielded the best results for me personally - but I will freely admit to being a little bit prejudiced. There is even a little tutorial on Amazon's site at:http://aws.amazon.com/articles/3801
- For more information or to become a part of the MTurk discussion community, please sign up as a requester and discuss any issues, questions or comments you have with the actual professional workers who are performing your tasks. Sign up by clicking here: mTurk Forum Registration And contact "Andy" to have a discussion group designed just for your HIT needs.
-Created by SaladPope (Editor: RWStein)
Updated 1/25/2013 by Cowfin