Anyone seen wild swings on their scoring? I usually work on the adult hits or flag the unrelated images but I decided to try the side by side comparisons. Worked for several hours and monitored my score about every 10 minutes which stayed right at 100. I left the comp for 15 minutes, come back, and my score is 22 and obviously I lost the qualification. They aren't hard to do (most of the AC's are obvious) and even if I was guessing, I would have a 50% of getting them right. On the other hits for them, my score will drop a few points, go back up, repeat on and on. But this was an unexpected avalanche!
The only ones I have really done for them were the adult images HITs and I started to notice the reuse of certain images, so I am pretty sure that they throw them into the mix to automatically update the score of the qualification associated with the HIT. I did one or two of the others but the search terms were so broad sometimes that it looks like there can be many that are directly related with the search and not just indirectly related. So maybe what you were thinking was the proper selection was not, and too many of them were the predisposed images. This is just my guess really.
id have to agree mannafest, i did some of the flag unrelated images and some of the search terms were in french and spanish and one of them said, "something" with all different images...they were all something so i left them as directly related. does anyone know how fast they approve or how lenient they are about those types of hits.
I'm curious to know as well as I have done many hits for them. It's been two days. Haven't had anything approved. Makes me feel discouraged to keep working for them because I'm not willing to invest all that time and not get paid.
They will definitely get paid. RNR does not reject -- at least, they haven't any time recently. They are on a 3 day auto-approve, so wait until tomorrow. What will happen if they don't like your work is you will lose the qualification and won't be able to do that particular HIT ever again (for instance, if you lose the qual for the Unrelated HITs, you can still do the adult HITs.) But that happens automatically, not through manual review.
Some of you guys have some serious issues with stressing over small things. Why would you stress because a requester has 30-days to pay you. However, ProductRnR hits are set on 3-day auto approval and are paying out in three days. Every single one of them. So why would you stress over (2) days without payment? Do you know any other job that pays you every 2-days? If so please direct me towards the right direction.
Because with real jobs you generally have some kind of contract or wage/salary agreement and you know that you're going to get paid. With Mturk, especially when you're working for a requester you aren't familiar with, it's a crapshoot. It's not out of the realm of possibility you can do work and never get paid for it, and just have it rejected at 29 days. In my opinion, that 30 day window is way too generous, and I tend to stick to requesters that pay out more quickly than that. ProductRNR, in this case, is one, but don't rail on people because they simply raise questions. Yet another one of the problems with Mturk is that we as workers have to do the outside research through things like TO to determine which requesters are legitimate/trustworthy and which aren't. Someone who's completely new to Turking isn't going to have knowledge or access to the same resources that some of the "older" people on this forum do. Putting people down for raising questions just drives more people away from what could be a good community.
