I normally weigh the strength of the negativity and the strength of the positivity and rate it that way. Moderately negative with a slightly positive? Slightly negative.
It is for those who disregard hits because they think its too hard or not worth it. then spend time looking for other hits or nothing worth doing.
I only use neutral if I can't figure out if they're saying something positive or negative. I also didn't get any bonuses for the last batch so don't listen to me.
Title: Psychological survey about important personal wishes Requester: Timur Sevincer [AI2PXI60ZWGGH] (TO) Description: Write about an important personal wish and answer a series of questions about yourself Reward: $0.80 Qualifications: Total approved HITs is not less than 100, HIT approval rate (%) is not less than 95, Location is US Link: https://www.mturk.com/mturk/preview?groupId=2FH56XBAT2D509FOYOUQ9JG7U7Q040 [size=-2]Powered by non-amazonian script monkeys [/size] 10 mins
I don't know if this is the right way to do it but because we are told they come from movie reviews I look at conflicting comments in that context. I weigh a comment about a very important element of the movie (script, pacing) more heavily than a comment about something about a less important element. To use a made up example if I got "The stunning location shots provide only momentary distraction from the overwrought acting" I'd mark that as some level of negative. ETA: Or even if it said "some" instead of "only momentary"
How would you evaluate "It's Tommy's job to clean the peep booths surrounding her, and after viewing this one, you'll feel like mopping up, too?" See, to me that would be at least moderately positive.
How would you judge their rejection rate, I am reading through and doing the best I can, but I have some difficulty between moderate and very. Nothing crazy, and from the description it seems I am fine, but you have worked for them before.
I don't know if we "should" discuss specific ones since it's majority rules but I would call this neutral because I don't know how the person meant it. I don't think they reject at all unless you are wildly off, like marking something that says "This is the worst movie I've ever seen in my life" as any form of positive.
The GTRI-HSID qual tests are hard but there no minimum number of approved hits qual so if you can get the test quals you can do them. The Reading Comprehension test says it can't be retaken at the top of the test but it's actually on a 10 minute retake so if you don't pass you aren't cut off from the hits forever.
That's an awful one because it could be rated so many different way. I think I'd rate it some kind of negative. Like, you'll feel like you're in someplace so dirty you'll want to clean it.
At the top of the hit they explain how they grade. Looks pretty fair and well thought out. They seem to understand that if they throw rejections all around that they will get crap data.
Also, if the positive and negative things seem equal, I go with what they finish with like "It was uninmaginative and cliche, but the fantastic acting makes it bearable" (slightly positive, also I totally just made that up)
This is their TO page under the requester name Comp.Social http://turkopticon.differenceengines.com/A2UJIFWXZLA2MJ
Who knows? So far it feels as if a lot of people got cold feet and haven't been doing their stuff, and they're doing whatever possible to get folks to do the HITS (and leave good TO: I think the bonus message specifically mentions Turkopticon, either that or the HIT does).