Castingwords Transcribers - Questions and Answers

Discussion in 'CastingWords' started by nobody, Jul 15, 2012.

  1. nobody

    nobody User

    Joined:
    May 23, 2012
    Messages:
    317
    Likes Received:
    0
    CastingWords is awesome!! ;)

    I was Startpaging (I quit using Google two years ago), and I noticed that other thread shows up on page two when you search for castingwords. o_0 Not good. hapless mentioned CW seeing that thread, but after seeing that search I thought, what about people considering using them? They might be deterred because of the title. Not good for them or anyone working for them. So I thought I'd start a new one for our discussions.

    I only did one HIT for them today. I'll keep checking back for more editing HITs though. They pop up at the oddest times. There haven't been any Express or Diff Audio ones this weekend, and those budget HITs have the worst audio EVAH. Seriously, how hard is it to speak in the general direction of the microphone? Do you have to do your interviews at airports, train stations and/or in the middle of crowded restaurants? Gah!

    Enjoy your Sunday everyone!
     
  2. Whimsy

    Whimsy User

    Joined:
    Apr 22, 2012
    Messages:
    805
    Likes Received:
    0
    Hahaha! Good call on the new thread.

    I am still slowly working my way through Improve jobs. Even grading during commercials. What I don't get is the 9 - no editing needed notation when giving a grade. Bah. I have given 9s with a note about how/where edits need to be made.

    Crazy, too, how some people will SKIP a whole section of audio. Not even 30 seconds but multiple sentences. I just hope they get weeded out quickly.
     
  3. ds_36

    ds_36 User

    Joined:
    May 25, 2012
    Messages:
    150
    Likes Received:
    0
    We've been wanting that for all of CW time. Unfortunately even if the current ones get blocked there will just be more to replace them.
     
  4. hapless

    hapless Guest

    Let's check the style guide:

    http://turkerdocs.castingwords.com/documentation/grades/guidelines.html

    > "... 9. Excellent transcript that needs no editing (or very little). ..."

    Note that last part: "... (or very little)."
    But see also:​

    http://turkerdocs.castingwords.com/documentation/grade_examples/

    > [Transcripts graded 9] "... do not need any editing, maybe something small that could go unfixed."

    "... maybe something that could go unfixed."
     
  5. Whimsy

    Whimsy User

    Joined:
    Apr 22, 2012
    Messages:
    805
    Likes Received:
    0
    For me, that's commas all the way. Long paragraphs and sentences get an 8. If there's a mess up with filler words - an 8. But the comma thing is really tricky. I've actually screwed myself out of time and money by listening to the full length of an audio...just to be not sure whether to give it an 8 or a 9 (commas, mainly) and just returned it.

    I like fixing much better than grading.
     
  6. hapless

    hapless Guest

    And good call on writing the company name without a space ...
    but bad call ;-) forgetting to capitalize the W ("CastingWords" not "Castingwords") in the thread title.
     
  7. hapless

    hapless Guest

    I seem to be on auto-8's (most of the time) no matter what I do and no matter whether a Turker gave me a 9 or not. So if you grade one of mine, flip a coin.

    (Ha ha, just joking, don't flip a coin. LOL)
     
    #7 hapless, Jul 15, 2012
    Last edited by a moderator: Jul 15, 2012
  8. Whimsy

    Whimsy User

    Joined:
    Apr 22, 2012
    Messages:
    805
    Likes Received:
    0
    Have you ever opened up one of those Improve a Transcript HITs? Seen the box with the worker's ID, PPT, consistency, expected grade?

    I am thinking (HoPiNg) that the 9s I get from Improve jobs may help my "expected" score get higher and THEN auto approve at 9 =) I have no idea if the two scores mix or if my delusion will help me in any way -- but I still try. I want to have 20 9s for every "other" score I get. My time on the computer's limited lately but it gives me something to work for. I'm certainly not making a lot of money on this, especially with over 50 of my cw bonuses still missing. Tech is "working" on it.


    hahaha. as an aside, there was a funny request for how to type up some speakers.
    one got typed like this
    TWO GOT TYPED LIKE THIS
    Three Got Typed Like This

    which would be great if we had the whole chunk and knew who each of the people were on the clip =P It was clear audio but I skipped it.
     
  9. hapless

    hapless Guest

    BTW there's an expedited edit up right now, great audio, go get and enjoy!
    (TE 90+)
     
  10. Whimsy

    Whimsy User

    Joined:
    Apr 22, 2012
    Messages:
    805
    Likes Received:
    0
    Thanks, but no expedited HITs showing for me and it's my bedtime. Maybe I'll be able to Improve it tomorrow. Now I'm just hoping I don't dream about what all the women I've been listening to today have been talking about lol
     
  11. hapless

    hapless Guest

  12. hapless

    hapless Guest

    ... and you know you have to click the "Expedited Edit" section heading to see them (same as with all sections other than "Audio Transcription").
     
  13. hapless

    hapless Guest

    Revisiting the question of what a 9 really means ...


    I quoted from the style guide:

    http://turkerdocs.castingwords.com/documentation/grades/guidelines.html

    > "... 9. Excellent transcript that needs no editing (or very little). ..."

    http://turkerdocs.castingwords.com/documentation/grade_examples/

    > [Transcripts graded 9] "... do not need any editing, maybe something small that could go unfixed."


    Consider also this:

    http://turkerdocs.castingwords.com/documentation/grades/guidelines.html

    > "A transcription should not be graded down because there are a lot of [xx]s or [inaudible]s. If they are not clearly understood then they are OK."

    "If they are not clearly understood then they are OK":
    Exactly how should that be interpreted? Does it mean e.g.: (1) The presence of one or more [xx] tags should not affect the grade if editors cannot resolve them? Or does it perhaps mean: (2) The presence of one or more [xx] tags should not affect the grade if it is reasonable to believe the original transcriber tried but couldn't resolve them and yet editors can resolve some or all of them? :confused:
     
    #13 hapless, Jul 16, 2012
    Last edited by a moderator: Jul 16, 2012
  14. Whimsy

    Whimsy User

    Joined:
    Apr 22, 2012
    Messages:
    805
    Likes Received:
    0
    I think it's the first one, for the most part. I guess it has to be a case-by-case evaluation. If someone puts in [xx] every third sentence just to skip through less than stellar audio, they should be graded down. In my opinion, if audio gets to be too difficult for me, it gets returned.

    I got some feedback on my feedback left for other transcribers. This one confused me:

    "Improved transcript 3 grades from 6 to 9Just an FYI: while book titles (and movies, TV shows, songs, etc.) s/b in quotes, only the first use of each title, after that, no quotes."

    Why's a book title only quoted the first time? I checked the style guide and it hadn't mentioned the "first time only". This is where I get confused because I don't know alllllllll the (oh em gee so many) standard rules of writing. Is this one I completely missed?

    My other feedback was:

    Improved transcript 1 grades from 8 to 9Your comment: "Try to make longer sentences short, even though people "speak" in run-ons."

    However, be sure to tell them don't indiscriminately break up run-on sentences into simple sentences (independent clauses), use commas at appropriate places to maintain flow of speaker's words."

    Commas. Yay!? lol I know how to use them but I have a very hard time telling someone else how to use them.

    I liked getting feedback on my feedback. This is all still very new to me but I enjoy doing transcription work. I don't want to be doing the same thing wrong consistently.
     
  15. nobody

    nobody User

    Joined:
    May 23, 2012
    Messages:
    317
    Likes Received:
    0
    Dammit, I would give that an 8. :p

    ETA: And I can't change the title. :p
     
    #15 nobody, Jul 16, 2012
    Last edited by a moderator: Jul 18, 2012
  16. hapless

    hapless Guest

    Surely, no matter how you use commas, some editors might possibly disagree.

    I've used too many (?) commas at seemingly 'appropriate places' to maintain flow, but someone observed that I 'broke' the flow with too many commas. Conversely I've sometimes used too few commas. Even if my comma usage were perfect by one editor's standard, another might possibly disagree.

    Granted I'm no expert on all this comma stuff. Even if I were, I'm not sure if that would help me get more 9's. (I understand that there can be some disagreement even among some respected grammarians and well-known style references.)

    No matter what I do, it's probably another 8 for me today. ;-)
     
  17. Whimsy

    Whimsy User

    Joined:
    Apr 22, 2012
    Messages:
    805
    Likes Received:
    0
    I'm pretty sure something similar to this has been posted...but IDK where and if it's in the "sucks" forum it's lost to me lol I had a question about PPT at one point and I just "found" the answer today. I hadn't explored the support site much once I saw my current/open ticket. Anyways, this is about grading:

    How Do Grades Work?

    OK... this answer is long. But since the question is asked a bit here it is, in a fairly authoritative version, at least for today.

    Grades are composed of 3 parts -- your graders, your track record, and our automated quality checks. And beyond that anyone at CastingWords can override the final grade during a certain time window. Given that the time window is short for most assignments that are done by good workers, or for work that is due soon to customers that happens only for a small minority of hits.

    Our automated checks are mainly used for rejecting bad work quickly. However we do have metrics influence grades of accepted work. Most of our checks are indicators of poor quality, however we do have a few items that correlate with high quality and can increase your grade.

    Your track record is also part of the grade -- the longer you've done work for us the less likely it is that you'll get a poor grade. But this component has a highly stabilizing effect -- if all you turn in is 7s for months getting 9s is going to be harder. (8s should be easy enough.) Also this section doesn't trust new workers much and tends to push grades downwards -- but only a little bit.

    The grades the graders assign are the most important part of your grade by far. However these grades do not say it all in the numbers. Your graders are not all trusted the same so some graders grades matter more than others, and additionally certain types of grades -- the kinds with long written descriptions matter more. Finally certain check-boxes that your graders assign can change the overall grade away from their number assigned number grade.

    So all three of those things run over your assignment -- and in the end a grade comes out. It's a lot of code - it's constantly being tweaked -- so there are probably bugs and edge cases. However our opinion is that right now the main problems that we see are coming out it are because of the graders. We're working cleaning up the pool of graders, fixing skew and training the graders more, and coming up with new grade HITs that produce better results.

    Nathan (CastingWords co-founder)
     
  18. Whimsy

    Whimsy User

    Joined:
    Apr 22, 2012
    Messages:
    805
    Likes Received:
    0
    Turker gave me a 9 today. Feedback: No errors.

    Final grade: 8

    I don't know whether to laugh, cry, or flip someone off. In the end I'm just going to catch up on my magazines.

    Anyone else take more notice of how friends, family, strangers speak? I told my cousin I'd transcribe her any day =) I'd never want to be the one transcribing my boyfriend.
     
  19. hapless

    hapless Guest

    Don't sweat it. ;-) That happens quite often. Check the final edit when it shows up, if you want to see what (if anything) was changed.

    I don't know. If indeed that is a CW rule, then IMO they ought to mention it in the guide.
     
  20. hapless

    hapless Guest

    I tried hard on an 8-minute transcript with some tricky spots ...

    Turker 1 grade: 9
    Turker 2 grade: 9

    Auto-grade: 8
     
    #20 hapless, Jul 18, 2012
    Last edited by a moderator: Jul 18, 2012

Share This Page