Please Join Me On A Crusade To Fix Bad CrowdFlower Test Questions!

Discussion in 'Requesters' started by liladybugz36, Nov 4, 2012.

  1. liladybugz36

    Joined:
    Dec 24, 2011
    Messages:
    224
    Likes Received:
    1
    I Talked to Crowdflower.....

    So, a couple of weeks back I had an hour long conversation with an executive (Brian) from Crowdflower. First off all I would like to say that he was very moved by my enthusiasm and felt compelled by my "crusade".

    The conversation was really awesome and we discussed everything I could think of that Crowdflower could do to improve for workers, and draw in the mTurk workforce. We compared different Requester's to Crowdflower, and I explained ways they could optimize their system to meet or exceed the standards of those Requester's, and also told them ways they already met those standards.

    The end result was me ending up as a beta tester on newly designed hits and giving my feedback on what I thought about them from the design, to glitches and anything I could think of that I felt would improve the flow and work process for an mTurk worker.

    I did this on a variety of hits and will continue to do this in the future. So please never say you can't make a difference because all it takes is one person and one idea to enact change! With motivation and desire, anything is possible! There's even a bonus - I even get premium pay to give my feedback! :eek:
     
    #21 liladybugz36, Dec 31, 2012
    Last edited by a moderator: Sep 18, 2013
  2. Charlie Turkelly

    Joined:
    Jul 15, 2012
    Messages:
    1,126
    Likes Received:
    0
    [​IMG]

    You rock.
     
  3. admiraljohn

    Joined:
    Oct 2, 2012
    Messages:
    2,581
    Likes Received:
    0
    On numerous occasions I've been dinged with what I thought was a bad test question. I've utilized the "This is unfair" button, typed in why I thought it was unfair, and a few minutes later (while working on the same task) I get a message that my accuracy has been adjusted due to a test question being re-scored.

    This is honestly the only option open... you can email them all day but they don't care; if you stop working a task someone else will hop right in to do it, so there's no motivation for them to change.

    **EDIT**: I just read liladybugz36's most recent post... apparently I stand corrected. Nicely done!
     
  4. TAC

    TAC User

    Joined:
    Sep 15, 2012
    Messages:
    474
    Likes Received:
    0
    That's good CF can get some first hand experience.

    I had 2 reversed today with my score adjusted up that I stated as unfair. This was from hits I did a couple days ago. Seems like it is progressing in the right direction.
     
  5. liladybugz36

    Joined:
    Dec 24, 2011
    Messages:
    224
    Likes Received:
    1
    Very cool! I'm so glad to hear that. It seems like response times from support is improving drastically! This is so awesome to hear!!!! I'm glad to hear that you put your voice in and motivated a positive response.
     
  6. Andy

    Andy Administrator
    Staff Member

    Joined:
    Jan 25, 2009
    Messages:
    2,219
    Likes Received:
    21
    Glad to hear they are listening.
     
  7. TAC

    TAC User

    Joined:
    Sep 15, 2012
    Messages:
    474
    Likes Received:
    0
    I just had two more incorrect ones and commented accordingly. Are there any certain words or a way that comments should be written that gets more attention or flagged quicker?

    Not sure if it's been answered before, but how do these incorrect things get marked wrong in the first place? Hopefully, it's not majority rules.
     
  8. liladybugz36

    Joined:
    Dec 24, 2011
    Messages:
    224
    Likes Received:
    1
    No, it is not majority rules. The test questions are mostly written by the companies that use Crowdflower to source their work through, and some are written internally by Crowdflower staff.

    To get attention more quickly, I always use full capital letters in my title and use the word URGENT in it as well. That will grab the staffs attention more easily and single your post out.

    They really do fix these questions when they are flagged so I'm glad you are letting them know... and of course for your percentages!
     
    #28 liladybugz36, Jan 5, 2013
    Last edited by a moderator: Sep 18, 2013
  9. TAC

    TAC User

    Joined:
    Sep 15, 2012
    Messages:
    474
    Likes Received:
    0
    When you say title, are you talking about emailing them? I'm talking about disagreeing, marking unfair, and leaving a comment.
     
  10. liladybugz36

    Joined:
    Dec 24, 2011
    Messages:
    224
    Likes Received:
    1
    No, not emailing them. I'm talking about the headline when you leave a comment.
     
  11. razorbacks0121

    Joined:
    Oct 1, 2012
    Messages:
    2,241
    Likes Received:
    0
    Duplicate post. Ignore this.
     
    #31 razorbacks0121, Jan 8, 2013
    Last edited by a moderator: Jan 8, 2013
  12. razorbacks0121

    Joined:
    Oct 1, 2012
    Messages:
    2,241
    Likes Received:
    0
    Question: How much does a 55 inch tv cost?

    Answer: j h g e h g j h g j h g j y h g jf h g j h



    That's not a sign of a messed up grading system. That's a sign of plain laziness. I sent an email to Crowdflower discussing this situation. I think if he just starts hard blocking everyone who is to lazy too give an effort, then it will cut out 99.9% of the problems. Some people are so lazy just over a computer, it makes me feel ashamed to think about their house hold situation. If you can't click on google, read search results, and type your own answer without cheating, I'd be highly surprised if you even give an effort keeping your house clean or brushing your teeth for that matter. I understand the grading system is complicated at times for those who do try. But realize there has to be a grading system that is so strict intact to ensure lazy people are submitting lazy work.
     
    #32 razorbacks0121, Jan 8, 2013
    Last edited by a moderator: Jan 8, 2013
  13. razorbacks0121

    Joined:
    Oct 1, 2012
    Messages:
    2,241
    Likes Received:
    0
    Question: Are aquarium heaters easy to install?

    Answer: v b v c v c t b v c b c v b c y v b v u c b


    Question: What are some different types of allergy symptoms?

    Answer: d f h d f g s d a? f g sh df h d g s d fh f d s f h s d f h s d f gh s d f g d



    One after another.
     
  14. ewd76

    ewd76 User

    Joined:
    Oct 8, 2012
    Messages:
    1,133
    Likes Received:
    0
    Sometimes my allergies do feel just about like that.
     
  15. TAC

    TAC User

    Joined:
    Sep 15, 2012
    Messages:
    474
    Likes Received:
    0
    There are correcting things quicker on the adult hits. Had two more incorrect rejections, disputed it, both reversed in a couple days with my score adjusted accordingly. Had another unfair one yesterday (regarding incorrect drug identification - nugs of marijuana WTH?) and left a strongly worded dispute in all caps per suggestion. I expect it to be reversed as well.

    It is improving.
     
  16. razorbacks0121

    Joined:
    Oct 1, 2012
    Messages:
    2,241
    Likes Received:
    0

    That's odd. I've done thousands of each CF and CS and never received a rejection.
     
  17. HappyFish

    HappyFish User

    Joined:
    Sep 8, 2012
    Messages:
    1,778
    Likes Received:
    0
    Obviously TAC is talking about getting downgraded for 'errors' and not actual rejections.
     
  18. wahoo7881

    wahoo7881 User

    Joined:
    Dec 27, 2012
    Messages:
    731
    Likes Received:
    0

    This is my face reading that.

    [​IMG]

    wtf is a nug of marijuana? Is that the new unit the kids are using these days? Is the metric system gone? It used to be grams and ounces. I must be getting old or something. Time to trade in my bike for a hoverround.
     
  19. naturegirl

    naturegirl User

    Joined:
    Aug 30, 2012
    Messages:
    701
    Likes Received:
    0
    Liladybugz, I am totally impressed. I've been away from the forums for awhile, and I come back in and here you are, making even further progress! Really nice job. :)
     
  20. HappyFish

    HappyFish User

    Joined:
    Sep 8, 2012
    Messages:
    1,778
    Likes Received:
    0
    You have a nug if you're too poor for grams or ounces.

    On topic, everybody's still getting cutting off for terrible QC.
     

Share This Page