This thread is for the requester SpeechTranscript, who has Shouting, Conclusion, and Example HITs up. Post any questions or comments you may have here.
I'll try posting something first from a question earlier in the forum. "Originally Posted by Irma hey did anyone else get rejected from SpeechTranscript "Help phones understand you when you shout! [label speech as shouting or not"? I just got 5 rejections and 3 of them are still pending, this has brought my ratings down real bad :/ I don't see what I was doing wrong so I say avoid maybe, as I did everything according to the training bit. But is it just me? Or did any1 else get rejected? I did a few, but they were all approved. You have to listen REALLY carefully on this one. If their voices go up even a smidge, I usually put "shouted". I didn't do a lot because I was kind of nervous about their judgments. Probably won't do many more unless there isn't much else."
I've submitted about 40ish of these so far with no rejections, but I'm now 1 away from my 3rd quality assurance question being answered incorrectly. I'm nervous to do anymore as I'm sure I will get my 3rd one at some point and be blocked from doing them.
Where does it tell you if a QA was answered incorrectly? Now I'm curious, as I've had no rejections either, but I didn't know I could see if I've gotten a QA question wrong..
SpeechTranscript here. If you have a long submission history like 40, and maintain "quality" (i.e. just keep doing whatever you're doing), it's impossible to get rejected. Also, an automatic block from getting a QA question wrong doesn't affect your relationship with Amazon. I'll copy paste some more stuff that I had sent to GertHeart earlier by e-mail If you could forward this to the correct forum/etc: Let's define the training session: this session is presented to everyone and contains 7 turns. Let's define a quality assurance (QA) question: a QA question is one of the turns presented in the training session. One QA is distributed per HIT. The rejection criteria was if you got 4 QA questions wrong and completed less than 6 HITs. Once you hit 2,3 QA questions wrong, you are sent back to training, so there is no reason to not follow directions that are this explicit. The nature of this task is subjective. However, you are given the answers and reasoning behind those answers during the training, which removes that subjectivity. Given that, it's not acceptable to deviate from our guidelines that much. We've given liberal assignment accepts to those who fail many QA's and submit many assignments (not even that many ~10). Upon the first set of rejections, everyone is in uproar, and I don't understand why. --- Edit: Everyone absolutely have a chance to redeem themselves (un-reject) by messaging me, doing some more sets, and showing proficiency. The training session is there so that we have some consistency. Workers can't label whatever they feel on examples they've already been shown. Rejects are in the really small minority; most workers don't get QA questions incorrect. Hope you guys can understand my reasoning. Thanks a lot to those who've participated!
I've gotten the comments, but they're all like "Great job! Tell your friends!" or something. Very cheerful...it feels so encouraging
I have done a few of these, have only gotten 2 rejects which isn't a big deal and I do like the HITs but I don't think I will do anymore. Honestly the pay is way too low for me to risk getting a block, I have none on my account and I don't really want one lol I know I'm fairly paranoid when it comes to Turking but some of us that Turk NEED the money, it isn't just for extra spending cash.
I did a few of them but was still a bit confused, if the person is shouting in all three, I would mark yes for the last one but since the voice didn't change in all three, should it have been a No? Once it tossed me back into the training mode, I quit and went on to something else. I figured I must not be doing them the way you are wanting them done. Also it would have been helpful when I did miss to get some sort of feedback immediately as to why that one was bad (ie like Crowdflower does) so I would be able to move forward and not repeat the mistake.
Again, a "temporary blacklist" on my side doesn't affect your relationship with Amazon. My blacklist is not the same as what is commonly referred to as a "block". No worker should feel paranoid if they're working for non-scam-y requesters. Your first question is directly explained in one of the training examples. I had introduced a feedback mechanism initially, but workers abused it by returning the HIT immediately and starting a new one. That just wholly defeated the purpose of my QA.
I personally do not complete your HITs anymore, but I have done them in the past. My recommendation is to create a qualification test, as a way to train your workers better and ensure the best work you can get. Personally, I wish that you would increase pay, possibly to the level of $0.12 cents a HIT or so, to ensure that workers get a fair wage. I understand that as a requester, that would double your fees, and may cause you to go overbudget, it is just my personal reasoning regarding my attitude towards the HITs.