Exclusive Assignment Offered By Mturk Forum: $20 Reward - Email mturkcowfin@gmail.com
Page 1 of 2 1 2 LastLast
Results 1 to 10 of 12

Thread: Returned Hits

  1. #1
    User
    Join Date
    May 2012
    Posts
    3
    Thanks
    0
    Thanked 1 Time in 1 Post

    Default Returned Hits

    Hi everyone!
    Will returned hits hurt my approval rating? Sometimes I click on a hit, and find it difficult to complete, so I return it. I figure returning it is better than taking a chance on doing it wrong.

  2. Thanks: (1)
  3. #2
    User
    Join Date
    May 2012
    Posts
    10
    Thanks
    0
    Thanked 1 Time in 1 Post

    Default

    I was wondering the same thing. I have only been at this about a week but I have not seen a HIT with qualifications based on return percentages or hit submission percentages, and I know its preferred to return it as opposed to doing it wrong or incorrect.

    Sometimes I have to return a hit because when I accept it says there is no more work but it keeps the job open. I actually abandoned a few by accident cause I did not realize these jobs were still considered open at first. That being said there still may be some way our return % plays into things and I would also like to know for sure.

  4. Thanks: (1)
  5. #3
    User riazthemturkhelper's Avatar
    Join Date
    Jan 2012
    Location
    Alappuzha, Kerala State, India
    Posts
    1,638
    Thanks
    23
    Thanked 26 Times in 19 Posts

    Thumbs up

    Quote Originally Posted by kenaboo View Post
    Hi everyone!
    Will returned hits hurt my approval rating? Sometimes I click on a hit, and find it difficult to complete, so I return it. I figure returning it is better than taking a chance on doing it wrong.
    Quote Originally Posted by OneEye View Post
    I was wondering the same thing. I have only been at this about a week but I have not seen a HIT with qualifications based on return percentages or hit submission percentages, and I know its preferred to return it as opposed to doing it wrong or incorrect.

    Sometimes I have to return a hit because when I accept it says there is no more work but it keeps the job open. I actually abandoned a few by accident cause I did not realize these jobs were still considered open at first. That being said there still may be some way our return % plays into things and I would also like to know for sure.
    Hi Friends the the below mails from me to Mturk Team and the mail I got reply will be a better answer to both of your questions


    Here is the mail me send to Mturk Team
    Hi Mturk Team,

    Please tell me is the Returned HIT going up will make any problem in the Mturking??

    And here is the reply got from Mturk Team
    Hello Riaz,

    I’m sorry for any concern you've had with your abandoned and returned HITs stats. The statistics recorded on your Dashboard regarding the number of HITs you have returned, or that expired before you completed them, are provided only for your own information.

    These statistics are not provided to Requesters. Having some abandoned and returned HITs is normal and will not negatively affect your reputation on Mechanical Turk.

    Once a HIT is accepted, if it's not submitted right away, it remains assigned to you and you are given the opportunity to resolve it at a later time. To submit these HITs, you will need to go to the "HITs Assigned to You" link at the top of the Mechanical Turk website. From there, you will be able to view and submit the HITs that you accepted and did not complete. You may want to make a habit of checking this before exiting Mechanical Turk to insure you resolve all of the HITs that were assigned to you.

    HITs that haven’t been submitted will remain in this tab until either you submit/return the HIT, or the allotted time to complete the HIT runs out. The HIT will then be considered abandoned.

    Thank you for participating in Mechanical Turk.
    But from my suggestion please don't return any Oscar HITs as that will affect your Quality Score(Nothing to do with Approval Rate) to become a Trusted Worker.

    Hope this post will help you as expected .
    Riaz Abdul Majeed(from Kerala, India)
    Add me in Facebook. Just Google my name Riaz Abdul Majeed and click I am Feeling Lucky you are @ my Facebook Page. So that I can provide you live help


    http://mturkforum.com/image.php?type=sigpic&userid=2722&dateline=1336373  007

  6. Thanks: (1)
  7. #4
    User
    Join Date
    Apr 2012
    Posts
    87
    Thanks
    0
    Thanked 2 Times in 2 Posts

    Default

    I just hate Crowdflower Hits,after doing 4-5 Hits,they just keep saying,"thanks for your hard work "and blah blah blah.....then I have to return their hits,I cannot see this thing before accepting the hit!once you accept hit and see this message. Awful.

  8. Thanks: (1)
  9. #5
    User riazthemturkhelper's Avatar
    Join Date
    Jan 2012
    Location
    Alappuzha, Kerala State, India
    Posts
    1,638
    Thanks
    23
    Thanked 26 Times in 19 Posts

    Default

    Quote Originally Posted by gourab View Post
    I just hate Crowdflower Hits,after doing 4-5 Hits,they just keep saying,"thanks for your hard work "and blah blah blah.....then I have to return their hits,I cannot see this thing before accepting the hit!once you accept hit and see this message. Awful.
    +1, yes you are right. So me blocked them forever.
    Riaz Abdul Majeed(from Kerala, India)
    Add me in Facebook. Just Google my name Riaz Abdul Majeed and click I am Feeling Lucky you are @ my Facebook Page. So that I can provide you live help


    http://mturkforum.com/image.php?type=sigpic&userid=2722&dateline=1336373  007

  10. #6
    User
    Join Date
    May 2012
    Posts
    10
    Thanks
    0
    Thanked 1 Time in 1 Post

    Default

    Quote Originally Posted by gourab View Post
    I just hate Crowdflower Hits,after doing 4-5 Hits,they just keep saying,"thanks for your hard work "and blah blah blah.....then I have to return their hits,I cannot see this thing before accepting the hit!once you accept hit and see this message. Awful.
    As I was saying earlier I had this issue, but if you do have this happen, I learned not to return the hit, but keep it open, at least for a bit cause if more jobs open up for that hit it will let you work on one of those. So if this happens i just check the jon every 5 mins or so to see if one has opened up. This will help me sometimes.


    And thanks for the help riazam that helps tremendously. That's exactly what i was wondering. Your the best!!!!

  11. #7
    User
    Join Date
    May 2012
    Posts
    3
    Thanks
    0
    Thanked 1 Time in 1 Post

    Default

    Thank you Riazam, that was a great help!

  12. #8
    User mralaska's Avatar
    Join Date
    Nov 2013
    Posts
    1,230
    Thanks
    1,608
    Thanked 1,797 Times in 773 Posts

    Default

    It has come to my attention that this post is being held up as "proof" that return ratios do not matter. In an effort to set the record straight and for the sake of incoming links I need to wake up this sleeping thread to post some samples that demonstrate everything Amazon said about abandonment rate and submission rate was categorically wrong. The only valid argument is the magnitude of effect but I maintain even the potential of harm is something that I want to avoid. It is a bit ironic that searching for "Submission rate" on google turned up so many people complaining about getting locked out of hits for this very reason that it made it difficult to find the actual HITs to prove it to the detractors on TO. Following are a few examples I did uncover:

    ken Reimer 3/19/2014 (today):
    Qualifications Required: HIT abandonment rate (%) is less than 5
    http://mturkforum.com/showthread.php...l=1#post714854

    Product RNR currently and has for quite some time required HIT abandonment rate (%) is less than 20
    https://www.mturk.com/mturk/searchba...A2BAP2QO7MMQI9

    TR 02-06-2014 required 95 submission rate:
    http://mturkforum.com/showthread.php...685#post673685

    OCMP 04-23-2013 required 95 submission rate:
    http://mturkforum.com/showthread.php...l=1#post226595

    Extension 02-11-2014 requires 90 submission rate:
    http://mturkforum.com/showthread.php...l=1#post681995

    Brenden Lake 01-30-2014 required 75 submission rate:
    http://mturkforum.com/showthread.php...l=1#post663249

    PackageFox 03-14-2014 required 80 submission rate:
    http://www.***************/showthrea...l=1#post105305

    There are many more examples and they span the years going back at least to 2012 when the email in question was posted.

  13. Thanks: (3)
  14. #9
    User Bigfaced_Honies's Avatar
    Join Date
    Mar 2014
    Posts
    1,502
    Thanks
    4,850
    Thanked 2,583 Times in 1,048 Posts

    Default

    The main reason it's pretty stupid for requesters to use abandonment/submitted %s as a qual is that some requesters set their HITS up in such a way that you can't even tell if a HIT is available/can be done until you accept it, necessitating returns.

    They're of course free to use it as a criteria for whether or not you can do their HIT, but it's still pretty clueless of them.

  15. Thanks: (1)
  16. #10
    User mralaska's Avatar
    Join Date
    Nov 2013
    Posts
    1,230
    Thanks
    1,608
    Thanked 1,797 Times in 773 Posts

    Default

    One can debate who is being clueless. One can even argue we are clueless for accepting that kind of abuse.

    There is never a valid reason to force a person to accept a task to see what it is and such behavior should not even be allowed. Usually they are simply ways to get free work out of people because even if you return it you are giving them data. The most common cases are demographic screeners that they do not even bother setting up a separate link to screen. The most legitimate requesters will post a paid screener. Some requesters hide their instructions just so they can track to see how many people are reviewing the HIT without taking it (by returning it). It is all free data to them.

    Other tasks that hide their intent are more devious. For instance, in the last couple days Michael Toomim (as CrowdClearinghouse) posted HITs that hide the task then surprise you with something most consider reprehensible such as writing a friendly letter to a convicted child molester. His current published project includes tracking how many people are willing to continue with such work after they find out they are doing something unpleasant so when you return the HIT you are actually giving him half his data for free! He gets a bargain on the rest via rejects and uses hard blocks also but that is another story. There are other requesters that will ask for stuff like increasingly personal questions and then track at which point you have had enough by returning the HIT. There are many ways to exploit the return ratios to extract free data. These are only a few examples.

    On the flip side, for many types of batches it makes perfect sense to utilize the submission ratio and I am surprised that it is not used more often. It is my concern is that more requesters will discover a manner to use it as a useful tool to get higher quality work done in a more efficient manner. For that reason I do everything within my power to try to recover my submission ratio to an acceptable level and that means turning down tasks if it is known they are looking to get free work via task returns.

    Obviously there are some returns that are not avoidable on our end, such as software issues and servers going down, but it would be a lot simpler and natural to maintain a high submission ratio without catering to the exploitation of cheap and illegitimate requesters.

  17. Thanks: (3)
Page 1 of 2 1 2 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •