I am counting myself lucky that I haven't gotten any actual rejections. Only these lowered -1 emails supposedly reflecting the quality of my work. I am disturbed on your behalf BlueMoods that you actually got a rejection. I got a rejection from one of their HIT's one time over 6 months ago after doing hundreds of their hits, and went absolutely through the roof. I shot them a scathing email. I am not saying that in the past (not now god forbid, with this system in place) I have not rushed through a transcript and not double checked it before submitting. It's .08 for gods sake. But in my mind, the only reason to ever get a rejection is when work is just out and out fraud. I know, never in any work I have ever done on mturk has my work been anything close to fraudulent. And I feel confident based on your earlier posts, that yours has not either. They emailed me apologizing and paid me for the HIT, but could not (or would not) reverse the rejection. I am wondering how things are over at SpeechInk these days. I may have to get them another look. I'll have to dig through my closet and find my transcription pedal first! Joking aside, I hope CS gets it together. I have enjoyed working their hits and would be sorely disappointed if things keep going on a downward turn.
Thanks, you two. I sent them an email and asked if I could see what I transcribed vs what the reviewer corrected. If I did make a truckload of mistakes, serious or otherwise, I definitely want to know what they were! I also don't understand why my qualification score is still at 2007, unless it takes a minute to catch up in the system. Nobody's perfect, obviously, but I take a bit longer on the HITS than it would if I just typed and hit submit. I listen to it one time all the way through(a trick a former transcription instructor taught us), then however many times I need to get the entire file transcribed. Then I listen to it again while reading along , sometimes more than once. I feel like the majority of my mistakes would be either misplaced commas, not capitalizing words that should be, capitalizing words that shouldn't be, and the dreaded end of the HIT. I'm leery or ending with a period even if I'm 99.9% sure it's the end of a sentence. It may sound that way to me but not to the reviewer. At any rate, I'll be interested to see what they have to say when/if they get back to me. I appreciate all the kind words and support, guys! I'm a bit rattled today but I'll get over it and the sun WILL come up tomorrow
I would guess that we are all quite decent transcribers who care about the quality of our work. Maybe the amount of workers is so high that Crowdsurf doesn't need to pay attention to us little guys, but I would think that quality of the work being submitted will drop with this system. Again, maybe we are just being weeded out because our work isn't optimal, but I'm pretty confident in my work. By the way, after recently reviewing the guidelines, they completely changed their mind about periods at the end of hits. So I printed out the guidelines a month or two ago and this is what their guidelines said: Always close a HIT with a period, unless you're 85% sure that it's the middle of a sentence, then leave it open, WITHOUT PUNCTUATION do not use ellipsis. . . or hyphen -. Fast forward to this week when I checked again and it's pretty much opposite!!!! It reads: Always close a HIT WITHOUT PUNCTUATION unless you are 95% sure that the HIT ends in a complete sentence. What in the world? That seems like a change that should be emailed to workers to give us a heads up on such a drastic change. After getting all those -1s I reviewed the guidelines again just to see if I was missing something and I did see that. Still, that's not my fault. I can't be expected to just know all the changes in the guidelines week by week. I have just kind of flipped from being thrilled with the company to sadly disappointed. Maybe things will come around, but I am definitely not a happy Crowdsurfer at this point in time. Glad we have a place here to vent though. Thanks for the updates guys.
Yeah, I noticed that change 2 or 3 weeks ago. That sure was a complete 180 and will chizzle away at your score fast if you don't know. Thing is, soon after the change, I wasn't sure if even all the reviewers were aware of it. So you are damned if you do and damned if you don't. This week I decided to throw caution to the wind and make some money. It doesn't seem to matter how much I strive for quality, so I might as well give it up and pay some friggin bills. Whateveeeer. This week it's all about how low can I go. Why not? Not really, but I'm not going to fester over it anymore. That was so last week!
I haven't yet heard back from Crowdsurf but like winryan said, they may be overwhelmed with emails and such. Patience is not my strong suit. This will be a long wait for me! However, in the meantime, I've discovered Reese Webmaster. He/she/they seem pretty good. Only a couple of basic guidelines and plenty of wiggle room and room for judgment calls. I have been talking to someone on Reddit since yesterday who said that s(he) has made tons of judgment calls and never gotten rejected or reprimanded for any of them. I did do one HIT last night for $2.00. It was approved and paid this afternoon. Now, this may not be for you if you don't care for southern accents...the HITs are all from the North Carolina legislature. Being a Georgia girl, that poses no hardship on me! I'm in Florida now and you don't really hear a lot of southern drawl down here...mostly northern or Latino...so it's nice to hear that "southern charm" accent again. The main thing I don't like is their interface. Not nearly as user friendly as Crowdsurf's. You have to open the audio in a separate tab and go back and forth, or have two windows open. And it's not as easy to navigate the media player. Last night I found myself dragging the slider a bit too far back and going all the way back to the beginning!
I've completed those too BlueMoods and when there are a large amount of speaker changes it really slows the transcription down for me. I always right clicked on the link for the mp3 file and clicked save to save them to my computer to use with my own media player or transcription software. Those hits aren't always there either so I'm not sure you can count on them. I never set an alert for them though so I honestly don't know how often they are posted. Maybe there are many every day. Southern drawl comments made me smile As long as what you're doing at the moment makes you happy, who cares, keep on doing it! Cheers and thanks for keeping it rolling.
I'm hoping someone will be able to answer this, as I'm kind of going through Crowdsurf withdrawals tonight. My QC score is at 1.5. My qualification score is 1007. I'm not blocked (at least not yet ) I know, because earlier I accepted a HIT out of curiosity but then returned it immediately. So: A. Why didn't my qualification score drop due to the rejection? I was getting the -1 scores when the new scoring system first rolled out and yet the HITs were still approved. I would think anything with poor enough quality to be rejected would at least get a -1, or probably a -10. Yet it hasn't changed. B. That said, how does the QC score affect my ability to do transcriptions? If it drops below a certain average will I then be disqualified? Or is based strictly off of my qualification score? Which, as I said, hasn't dropped or risen from 1007 in over a week at least. C. The stats box on the HIT page shows your stats for the current and previous week. Is the QC score for the current week averaged with previous week(s) or is it a stand-alone stat? I was thinking of waiting until next Monday when the current stats boxes will reset to 0 before trying to do any HITs. What do you all think? if I do any HITs before next week am I risking losing my qualification and/or getting blocked?
A: this is no rejection. you just returned the HIT and everything is ok. No changes made to any quality score. Rejected HITs give you a -40. -1 is for fair quality and -10 is for poor quality. Crowdsurf accepts allmost everything but fraud. B: There is no general answer to this. There might be a chance that you loose the ability to work on their HITs if your quality drops too low. What is sure is, that you wil be blocked from their work if your "qualified to work on...." falls below 900. the other quality score benefits high quality workers. If you are a high quality worker, you will be invited to the CrowdVirtualizer system which offers some other work and better pay. C: it is the average for the current/last week. Older stats do not influence this values
Thanks for your answers! It especially helps to know that the current week's stats is an average of the current and previous weeks. That explains a lot. I was referring to a rejection that I got on Monday. It dropped my quality score to 1.5 but my qualification score has not changed at all. During the initial rollout of the new system, it got to 1007 and is still at 1007. No points were deducted for the rejection apparently, at least not as of now.
I'm not sure if it's a combination. When the score was reset this Monday it said 0.0. I turned in some that had no errors, and it said the score was a 4. When I was marked down or up the score would rise and fall more dramatically at the beginning of the week. But as my numbers increase, the score fluctuates less. Yesterday was a lot better. I was able to do over a hundred and kept it above 3.7, which was a heck of a lot better compared to last week, when it plummeted down to 3.3, even though I was being so careful. I suspect this week is better because the transcripts are easier, and yeah, I might of been working off hours when there are less reviewers, so one or two bad apples can make a bigger difference in mucking up the scores. It seems to go better during the day than over night. Also, it's just much better this week in general, like maybe something that was off got sorted out . I don't really know for sure. I just hate leaving b****ing and griping, and my comments have been more about that lately, and probably justifiably, doesn't mean that I don't respect or like them, just was getting a bit frustrated. But it's only fair to come back here and balance that out to report that yesterday was an all and all much better day for whatever reason.
I completely agree. I do think I will hold off on transcribing until I hear back from the, or until Monday when the counter resets, whichever comes first. I absolutely adore Crowdsurf in spite of the frustrations. They are an amazing requester with relatively good pay, usually good amount of work flow, and a good mix of easy and difficult HITs. They are extremely supportive 99% of the time. I did get an email yesterday from them but it was in response to something I had asked a couple of weeks ago. So for now I'm writing off no response (yet) on my latest email as just a huge amount of things on their plate and they will get to it when they can. I'm glad your day and week are going well. Hopefully the worst is behind you.
I think this week has been fantastic for me in terms of scores. I never used to stay above 3.5 consistently. I would mostly hover below that. But for the first time ever, I saw my score go above 3.8 even though I try to do the HITs as fast as I can because I have a slow internet connection and the pay is just 8 cents plus bonuses.
Thanks for the helpful feedback VarunBatra! All the feedback we get will help to determine whether the problem is our individual quality of work or possibly a temporary kink in the new system. I'd like to comment on the scoring this week but I haven't completed a hit for Crowdsurf since 8/4 after taking this respite to hopefully gain some resolve before returning. I did notice a post on their hits previously that said they were working to resolve erroneous rejections and ratings given on 8/5 so maybe my reviews were linked to that but mine were from the 4th. I did receive a response from Katie today asking for my worker id so that she could send me a quality report that could shed light on the reasoning behind the negative reviews. I will say that Katie has always been great at responding to my inquiries in the past which I've always appreciated. That's really why the immediate negative ratings coming my way were such a surprise. Was just such a 180 from what I know of Crowdsurf in the past: Great customer service, fun hits, immediate pay and fair rating system. The pay is of course low, but in the pool of mturk, they get a thumbs up from me. In any case, we will see what the report shows.
I also think that something has been worked out, and am feeling much better. The last few days have been MUCH smoother. I have not received any -1 scores. 2 days ago I received 4 +10 scores in one night that propelled me to 1005. I have sat there now ever since. I am assuming that I as long as I continue to do quality transcripts it will just more or less sit there which is completely fine with me. I already have reviewer qualification, even though I never really do reviews much and I have been on Crowd Virtualizer for over a year. So, I am not sure exactly what I am reaching for beyond that. I just want to make sure that nothing gets messed up in terms of me being able to continue to do their transcriptions, and was worried about losing the qualification. I am so happy that things feel back to normal. Now that summer is over I am hoping they have more academic lecture type transcriptions. I prefer these over the computer coding and such. I am inclined to think it was a few over zealous reviewers and maybe they got weeded out? Winryan, I hope you get a satisfactory response from them. I was less than impressed with what I got from them. But as long as the system is straightened out, I guess that is what is important
Very glad to hear things are going better for you dirving. I did hear back from Katie and, although she did include one transcript that scored low and showed how it should be correctly transcribed, it should not have been designated as "poor" according to the criteria on their site. She did point out something that I had missed in the guidelines about the correct usage of all right vs alright. I don't know about you guys, but one thing I like about transcribing is the constant chance to write, even if it's someone else's words, it's good practice. I've learned some grammatical errors I've been making my whole life. I Looked it up and sure enough, alright is absolutely discouraged by professionals and most say it's not even a word although it's becoming more and more accepted. Not going to lie, it is embarrassing to realize an error like this that I've been making so long. Here I'm talking about good writing and writing run-on after run-on lol. I also had a misunderstanding about how to mark hits containing random sounds without any speaking. I was marking them blank which was incorrect. So really what I take away from all this is how useful actual feedback could be to workers if it was somehow supplied without them having to email support to ask. If I was notified about any of these things during my first day of hits, I'd never had made them again! That would have nipped it in the bud right away and upped the quality of this worker's submissions with just a couple simple correction messages. In any case, I guess I'm just jabbering on here, but I feel like this is an obvious feature lacking in the Crowdsurf system. I completed about 50 hits yesterday with success and really didn't change a thing. Didn't have any with "all right" in them or that were the blank situation I described and yet I received good scores. I'm a pretty positive person, but ya, maybe Mr. Over Zealous was let go. Or maybe he's in the basement and they "fixed the glitch." (Office Space reference)
If it makes you feel any better, winryan, I had been using "alright" from the beginning. Apparently we weren't the only ones, because the first version of the guidelines I had did not have it on there. The updated version does, so I'm assuming that enough people used "alright" to make it necessary to put it into writing. I had also, all along, being using 'em (I've been talking to 'em) and did not catch in the guidelines that, even though these are verbatim transcripts, they want us to always use "them" no matter how it is pronounced. I'm glad things are getting better for everyone. I still haven't heard back yet about my rejection and my score, but plan to try a couple of HITs next week and see how that goes before diving back into the deep end of the pool.
Me too! I was so annoyed last week my pride couldn't take admitting that I did find a constant error I was making after looking at the guidelines yet again - and that was using alright. I didn't realize it was slang either! And yeah, looked it up like you. My whole life I thought it was a proper word. At least one of the first titles to come up in the Google serach page is: " Is alright a word yet?". It makes me cringe when I think about all the times I was putting in alright, instead all right, when doing the transcripts for that - famous college that begins with an s - there was a professor on there that was saying that word almost every sentence, all right? You know, the older I get the more I agree with all the people who learn English as a second language. English IS weird. For example, why is the word already not spelled as all ready? Maybe English has been underrated for having a rebellious streak. It doesn't like to follow consistent rules! One cool thing about Crowdsurf and their clients videos is you learn a lot of new stuff.
Speaking of words that you've thought were proper all your life...I have to thank a smart @$$ I was arguing with on Facebook a few months ago for pointing out to me that alot is NOT a word. Otherwise, my scores would be even lower! And yes, I would hate to have to learn English as a second language. There are way too many conflicting rules and exceptions to those rules.
winryan, could you give more information about the feedback you got from CrowdSurf concerning your use of blank audio in your transcriptions. I sent a request the other day to support, and they didn't really answer the question. I was told the use of multiple Sound/Music Tags, etc. would be addressed in future guideline updates. For example, I hate when you have blank audio followed by music or noise. According to the guidelines, you are not to place two tags side by side. So basically, I was told by support you should use the most prominent, "salient is the word support used" tag. I definitely had to look up "salient," but got the gist of the meaning from the sentence. This leaves the use of tags really open to interpretation and of course, being marked down by a reviewer.