Was TurkOpticon Intended To Be a Weapon?

Discussion in 'General' started by kjross, Nov 21, 2012.

  1. kjross

    kjross Active Member

    Joined:
    Nov 7, 2012
    Messages:
    42
    Likes Received:
    0
    As a requester, it really is frustrating to see people resort to giving you low ratings simply because they are unhappy about being rejected. But I can understand that. What I don't get is when someone takes the time to give you a really positive review, a worker actually FLAGGED it, basically saying that it showed up around the same time that I started responding to people's reviews/ratings.

    Is it wrong that I have started asking our regular workers to give reviews on TO? I don't ask them for positive reviews, I actually make sure that they understand we are looking for an HONEST review. I can't control what they say but I do realize that some of our AMT workers are quite fond of me. It seems to me that TO is basically being used as a weapon, and I am not quite sure that a requester can do anything about it.

    Should I stop asking for reviews?

    KJ
     
  2. boardwalkumpire

    Joined:
    Sep 25, 2012
    Messages:
    4,509
    Likes Received:
    0
    I don't think you should stop at all. I wrote a TO review for you just today, completely of my own accord. You're going above and beyond what you have to as a requester, and I think that MTurk users appreciate that a lot, as it is the exception, not the norm. I believe the more regular workers you get, the more those positive reviews will drown out the others.


    That being said, I just read all of your TO reviews, and it's fairly obvious that the negatives (especially the person who accused someone of basically being a shill for you) are more petty and vindictive than rational, well-thought-out criticism. I think that most TO users can and do read between the lines on the TO reviews.



    I think that with the policies you've outlined in various places on the forum and your willingness to adapt to the community, you're going to attract and keep the workers that you want. The rest are, well, irrelevant.

    My $.02 anyway :)
     
    #2 boardwalkumpire, Nov 21, 2012
    Last edited by a moderator: Nov 23, 2012
  3. Neona

    Neona User

    Joined:
    Oct 20, 2012
    Messages:
    220
    Likes Received:
    0
    I haven't had a chance to work on any of your hits, they seem to be on when its not a good time for me and gone when I am able to actually sit down and try one.

    As for TurkOpticon, I am grateful for it, being fairly new I like to read what others have said about a requester. I do see there are some who are quick to leave a negative review because they didn't like the pay offered or because they got a rejection, I try to look at how many are complaining vs how many are not complaining but actually leaving good reviews. If the good outweigh the bad, I usually just assume the ones complaining are the ones who are unhappy with the requester for whatever reason and not because the requester is bad.
     
  4. cci83

    cci83 User

    Joined:
    Aug 21, 2012
    Messages:
    99
    Likes Received:
    0
    I agree. I am not much of a fan of the site myself. As an Mturk worker people harass all of my reviews. I have only put positive reviews but each one of them someone has either flagged or harassed me for putting positive reviews. Many people skim through HITs without even actually doing the work right and then blame the requester for rejecting their work. I do not do HITs by others reviews anymore as we all have our own opinions and it seems to me many just complain to complain and are not honest with the job or the reason for rejection. I came into Mturk to make income not to fuss around with other people. Since being harassed I stopped using the review site. I have no one to complain to about it either. You cannot flag someone because they do not agree with you. If I'm rejected by a requester I take it upon the requester contact them or leave alone and learn the job better for next time or don't do it all. People are nasty. This Forum is the only one I go to for Mturk or I find work on my own. I'm sorry workers are nasty to you but you have a right to reject or accept work whether they agree or not.

    I would say keep asking for a review. Thank you for taking the time to post on these forums. :)
     
  5. TAC

    TAC User

    Joined:
    Sep 15, 2012
    Messages:
    474
    Likes Received:
    0
    I've left 15 pages of reviews and only had a couple people make incorrect comments. Almost all my reviews are positive because I avoid the bad requesters thanks to TO. Not sure why you would be getting flagged/harassed.
     
  6. Khalinov

    Khalinov User

    Joined:
    Dec 12, 2011
    Messages:
    873
    Likes Received:
    0
    I take every review I read with a grain of salt. There are some bad apples who like to vent on TO, just as there are some bad requesters who deserve the flamage they receive there. IMO, members of both camps harbor attitudes of entitlement.

    I think most of us can look at a review and read between the lines. ;)
     
  7. jTurker

    jTurker User

    Joined:
    Apr 28, 2012
    Messages:
    1,855
    Likes Received:
    0
    As long as you put up a good hit and are fair with rejections then a few bad reviews should not effect your TO score. Some people just naturally complain that's what they do. Most people who read such reviews recognize that. But honestly if a bunch of people are complaining and then all of the sudden out of the blue someone who has never given a review writes a glowing review you cannot be surprised that people will question it.
    You can blame that on your fellow requestors that have tried to game the system(see Red45 and others). As long as you put up good hits that pay a fair wage then you have nothing to worry about because the good reviews will outweigh the bad.
     
  8. cci83

    cci83 User

    Joined:
    Aug 21, 2012
    Messages:
    99
    Likes Received:
    0
    Ok you are doing exactly what Im complaining about. I am getting flagged and harassed because of ignorant people who feel the need to shove people into going against their own opinions. Thats great that you left 15 pages of reviews. Like I said Im not using the site anymore and neither should anyone else because of the ignorance that comes with leaving a review that people don't agree with.

    Are you the owner/creator?
     
  9. ewd76

    ewd76 User

    Joined:
    Oct 8, 2012
    Messages:
    1,133
    Likes Received:
    0
    If you notice, many review sites of businesses and services feature little but negative reviews because satisfied users don't post reviews of them. TO is very similar. I also think that most workers don't even know of TO's existence unless they are searching the internet for issues with the requester or MTurk in general.
     
  10. ivycreek

    ivycreek User

    Joined:
    Oct 9, 2012
    Messages:
    263
    Likes Received:
    0
    I think you have actually done quite well with it by responding to reviews, in a professional manner. I thought one of the responses may have flirted with being emotionally charged but still you taking the time to respond, and posting the link of your daily HIT stats really is what TO should be about.
     
  11. turkedup

    turkedup Member

    Joined:
    Mar 27, 2012
    Messages:
    16
    Likes Received:
    0
    There's some very well thought out and sensible comments above, but I think Khalinov sums it up very nicely with his comment about both camps harboring different and valid attitudes.

    I'd generally only avoid requests if there is some huge red flags / masses of negative comments on there. But I do feel that TO has prevented many people (inc. myself) from getting seriously burned. For this reason, I use and support TO's purpose. And I do try to drop in positive reviews at times, even if I'm swimming against the tide.
     
  12. clogan98

    clogan98 User

    Joined:
    Sep 5, 2012
    Messages:
    157
    Likes Received:
    0
    Wait, you can get flagged for leaving a review? Why? It's your personal review. I leave reviews on anyone accept for the big ones like Speechink because they have their own reputation outside of TO. I've only left a negative if it was totally called for, but mostly I leave honest positive reviews. I hope I don't get flagged for doing that?!?!
     
  13. TAC

    TAC User

    Joined:
    Sep 15, 2012
    Messages:
    474
    Likes Received:
    0
    The owner/creator of what, the TO? Nope, I've only been doing this since early Sept. and have left reviews since day 1.
     
  14. TAC

    TAC User

    Joined:
    Sep 15, 2012
    Messages:
    474
    Likes Received:
    0
    The only real reasons I can think of for getting a review flagged is something like you didn't work for the requestor and rate them or you rated him/her/it strangely like getting paid in 4 days and giving them a 2 rating for payment. Dumb ass stuff like that.

    The mods seem pretty good about catching some of the BS that goes on. I've been flagged and called a shill I think twice, but the mod will tell the idiot to look at my prior feedback so they see I've reviewed many people.
     
  15. differenceengines

    Joined:
    Aug 2, 2012
    Messages:
    11
    Likes Received:
    0
    Hi all -

    This is Lilly. I'm one of the creators of TO. TO came out of hearing from lots of workers talking about getting burned by requesters who don't pay, or pay slowly, or seem to have really high reject rates. Many years ago, there weren't ways for requesters and workers to talk to each other. Where there were requester forums, it was based on requesters' kindness. TO makes sure that requesters have a reputation whether they want to communicate with workers or not.

    The reason reviews can get flagged is sometimes because people use profanity, sometimes it seems like a requester is reviewing themselves with multiple logins, etc. The sad part is that sometimes people who write positive reviews reviews, especially if they don't have a record of writing other reviews, end up suspected of "shilling" for the requester.

    I'm not really sure how to fix this, and I regret that it stresses some folks out. If you have ideas for how to make it better, post them or send them our way at turkopticon@differenceengines.com

    ~lilly
    Team Turkopticon
     
  16. amoreno

    amoreno User

    Joined:
    Nov 23, 2012
    Messages:
    876
    Likes Received:
    0
    i only pay attention to reveiws of people i actually know (i know noob but lurked for along time before joining due to newborn). it seems as though alot of bad reveiws come from people that dont get paid in a hot second. i care about approval of pay time at the moment. i also look for TOS violations and thats it.
     
  17. TAC

    TAC User

    Joined:
    Sep 15, 2012
    Messages:
    474
    Likes Received:
    0
    Lilly,

    I'm a bit confused on if the requesters can view the TO. One mod replied to one of my commenters, that the requesters could not see it. But it does look like they can from people either posing as a requester or they are the requester.

    So can they see it?
     
  18. ewd76

    ewd76 User

    Joined:
    Oct 8, 2012
    Messages:
    1,133
    Likes Received:
    0
    It would be hard to stop them if they knew how to get there. All you need is a valid email address. Free email addresses can be found anywhere.
     
  19. naturegirl

    naturegirl User

    Joined:
    Aug 30, 2012
    Messages:
    701
    Likes Received:
    0
    I know you're talking to Lilly. :) But this thread was started by a requester who viewed his TO and commented on it, inside TO. You'll see other requesters responding as well there.

    Ewd is correct; nothing stops any requester from viewing their own TO, or anyone else's. It's just that most requesters haven't known about TO.

    So. True. Reading TO is very educational in terms of how little many Turkers know about how MTurk works, such as that requesters officially are allowed 30 days to pay you. Not getting paid within 2 days seems to be reason for downgrading requesters. I make a point of rating as 5 for promptness anyone who pays within 3 days (because many good requesters pay near-instantly), but up to a little over a week still gets a 4 from me, just because that seems realistic. (In case you're wondering, I only rate someone with a 1 on promptness if it takes the full 30 days, and even someone who rejects without good reason in a day gets a 5 on promptness---because well, they were prompt!)
     
    #19 naturegirl, Nov 23, 2012
    Last edited by a moderator: Nov 23, 2012
  20. TAC

    TAC User

    Joined:
    Sep 15, 2012
    Messages:
    474
    Likes Received:
    0
    Got caught up in other thoughts about the TO and ignored the obvious! Thanks.
     

Share This Page