Hello Everyone, I'm a professor located at a University in the Pacific Northwest. Recently, my colleagues and I decided to post our first survey on Mturk. We had a limited amount of money to pay workers so we decided to use a combined payment and lottery system approach. Rather than pay everyone $1 or slightly less, we thought it might be more desirable to pay each respondent a very minimal amount for their time, but then enter their Worker ID in a raffle with the change to win up to $300 USD. If your location is eligible then you can find the survey under the title: "Earn up to $300 USD by completing this Work Attitudes Survey" Description is: "Win up to $300 (U.S. Currency) by responding to a survey about your job and workplace." The keywords are: "survey, attitudes, raffle, questionnaire, lottery, psychology, experiment, business, money, easy" First, I hope you will consider participating in this important study concerning employee attitudes that we will eventually share with the scientific community. Depending on English language comprehension so far we've seen it take from 6 minutes up to 18 minutes. Second, for future surveys, what are your preferences? Do you prefer a survey with a payment of an equivalent minimum wage? (e.g., .10 per minute) or would you prefer a combination marginal payment and entry into a raffle? Finally, we would like to be as transparent as possible to ensure you that we are indeed paying out the raffle prize. The raffle ends in approximately 60 days from now and we are trying to come up with a way to post the winners to the Mturk Forum while keeping your full identity confidential. Any suggestions or preferences in that regard? Thank you for your time and participation. All the best - Matt
To be honest, I would rather receive a working wage. $0.10/min is a decent guideline for most people, but if the work is fun and enjoyable most will work for less. It's all about risk vs reward. I'm not going to do a hit for less than minimum wage and risk getting rejected when I can take a little longer to find another hit that pays more. I've never won a raffle, and I know only one or two people who have. For me at least, it's not worth the effort to do an underpaid survey for a minimal chance of getting a reward from it. I'd rather have someone pay me adequately for my time; I wouldn't pay a group of people to paint my house by raffle, for example. As far as time frames go, they are VERY difficult to judge. I personally had a requester reject me because I completed her $1 survey 5 seconds under her "Time threshold". If you're considering using time as a threshold for scammers (which you didn't indicate, but I don't feel like enough requesters know), please drop that idea. Everyone works at a different pace, and everyone has their own method for doing hits. I haven't done the hit, so I can't comment on it, but I wanted to point out one of your TO ratings. TO (turkopticon) is a way for workers to rank requesters based on several attributes. This worker rated you as a 1 on pay, and a 1 on fairness, saying Your other TO rating was near perfect, stating This shows the two different types of people on mturk. Some are willing to do work for a potentially high payoff, others are not. From what I've seen, and the people I know a little more about on this forum, they are trying to make a living wage. Most would see your hit, view the TO, see someone say it took 20 minutes, and skip right by it. Some might do it, but you're missing out on a large number of people. These are just my opinions, and despite being a mod here I'm still fairly new at this (I almost have 10k approved hits, 6 rejects, 99.9% approval rating). I'm also a "fun money" turker, meaning I turk for extra cash for fun things, I have my own career that pays me what I need to survive. To sum up, because I know this is fairly long, I love seeing requesters here! Thank you for posting, thank you for listening, and thank you for asking! Seriously, if more requesters actually reached out to their workers, mturk would be FANTASTIC! I would rather see higher payments per hit, and maybe a $5 lottery or something. The chance at $5 is enough to motivate almost everyone to do the hits, if the pay is halfway decent. $0.10/min is a decent guideline based on the work expected. For a typical survey (~40% bubbles, ~20% reading, ~10% writing, ~10% demographics, ~20% other (pictures, videos, etc)), that's standard. For more bubbles, more pay. People have a hard time with "bubble hell" surveys, and will give up halfway through if they don't feel they're making progress. On a related note, I'd MUCH rather see a page counter "completed 10/50 pages" than a "You're almost done" message ANY DAY. As for posting on the forums, you can post a list of mturk ID's and none would be the wiser honestly. Nobody knows anyone else's ID unless we tell it to people, and even then nobody really cares. It doesn't matter to the workers anyway, only the requesters, and honestly no offense but your survey will be forgotten about a week after payment anyway. Ask questions! Hop into the "greatest hits" thread for the day and talk to the active workers! Get our opinions, and help us help you become the ideal requester! Thanks again for stopping in, hopefully others will comment as well. Sorry for the book, it's a slow night and I'm feeling verbose lol! I'm going to go do your hit, I'll post again if I have anything to add.
Just finished the hit, took me almost exactly 12 minutes. Since I'm not expecting to win the raffle, I just got paid $0.50/hr to do your survey. My thoughts: I would have been happier to see each of the sections of bubbles as a separate page. It would have been more pages, but I wouldn't have been confronted with a wall of tiny text and bubbles each time I clicked next. The "almost done" messages seemed to be at the 1/2 and 3/4 mark, you could replace the text to "halfway done" and it would feel a lot quicker. I'd suggest you take a look at the data you get and consider upping your thresholds. 100 hits approved is actually a fairly low number. New turkers would be happy with you, but 1000 hits is typically the "lower bound" to rule out the bulk of the scammers. I'd still recommend upping the pay. It was discouraging seeing all the bubbles, and if it were a hit I had found on my own I would have given up early on. I would have been happy to do that hit at around $.60-.80 as a lower bound honestly, without the possibility of a bonus. Anyway, just my thoughts! Hopefully others will come in and chime in as well Best of luck, I hope to see more from you in the future, and I'll be on the lookout for that bonus
I never expect to win a raffle, and I second Tjololo's initial post. Honestly, anything under $1 (as far as surveys go) is suspect to me, these days. I'm still new, but I've determined that my process works better when I "hold out" for higher paying hits. The time it takes to do most surveys is far too long, for the pay that is offered. It is my opinion that this is the reason the lower paying surveys stay up for so long (or as long as they do), while the >$1 hits disappear in minutes. Of course, the more you pay/HIT, the more quality control you will want. As Tjololo stated, setting higher qualifications will narrow your pool down to those who are both familiar with "how" to smoothly complete your surveys, but also those that are actually interested in doing it correctly. I don't mean the "specificsurvey is 1" kind, but the abandon rate, approval, and HITs submitted kind. Make your approval and abandon rating somewhere around 95-97, and HITs submitted 1k, and you will see a much better return. Best of luck.
Welcome to this forum. I had taken your survey some 2/3 days back. As other turkers said, I had never won a raffle or lottery. Just to give a try I had completed your survey. If you plan to post survey regularly, it will be nice to pay a minimum wage atleast rather than a raffle because in a long run, if turker didn't won we turkers will hesitate to take your survey and you may not get the required number of people to complete the survey. Instead, you can restrict the number of turkers by assigning some qualification like NO OF HITS COMPLETED IS > 5000 OR HIT APPROVAL PERCENTAGE IS > 98 etc. and pay little bit higher for all those who complete your survey. Good Luck.
Tjololo, uknome, and vkn2007, Thanks for the detailed feedback! This is really helpful to get your perspectives. We spent a lot of time trying to figure out the best way to compensate workers. We wanted to make sure Turkers felt is was a task that was worth their time. And we also wanted to use our research funds effectively. Based on feedback from Amazon Mturk support and our reading of some research guides out there we were kind of guided in the direction of compensation + raffle. Our research team's thinking was that workers would prefer the chance $300 payout for about 10 minutes of work rather than a guaranteed $1. However as Tjololo pointed out, preferences apparently vary pretty widely. I'll seek some more feedback on some of the other forums too. Again, this is very helpful feedback and we'll use it when designing subsequent research projects. And thanks so much for completing the survey too! All the best - Matt
Hi Matt, you had indicated that the result will be declared after 60 days and it will put in this forum. Its almost 60 days completed now and is there any update about the raffle lottery?