5 Dollar Data Updated Feb. 2009

5 DOLLAR DATA PROJECT by: Dan Rockwell - dan@lextant.com a series of experiments using amazon mechanical turk last updated: 2/24/2009

WHAT IS THE 5 DOLLAR DATA PROJECT? How much data can ya get for 5 bucks using amazon’s mechanical turk service? what the hell is that? A web service that allows you to assign human interface tasks, or HITS to real people who will then complete these task for real money.

WHO ARE THESE PEOPLE? Stay at home moms, people working on lunch breaks, average joe’s, you really don’t know. why do they do this? It’s money. Most of the tasks on mechanical turk are basic things like: - look at an image and give keywords to help train a computer how to look at images for search engines - ﬁnd search results on the web, give a human perspective to teach a computer

How’s it Work?

How do you create a HIT? 1. put some money into your account 2. click create a hit 3. ask the question, write instructions, etc 4. deﬁne the answer type 5. deﬁne how many people or answers you want 6. specify a time frame for completion 7 set the price you’ll pay for an answer . 8. start receiving answers 9. approve/reject/message 10. download results

1 STATE OF PARENTAL CONTROL FOR PC’S AND THE WEB HOW IT SHOULD IT CHANGE

PROJECT 1 - SIMPLE QUESTION 37 responses in 4 days Paid 10 cents per answer Total cost $3.70 example We make computer time quot;family time.quot; When the kids want to go online, we allow her to do so, however, one of us has to be with her, monitoring what she is doing. She is only 9, so when she goes online it's to a gaming site or to look something up. I would like to allow her to go online without me looking over her shoulder, but I also want her to have freedom to surf safely. I would like to see some kind of summary or reporting of the sites she went to, the people she talked to and the content of any emails she may receive/ send.

PROJECT 1 - LEARNINGS pretty cool fast fun to see the data arrive how do i get at demographic info how can i ﬁgure out who these folks are? not supposed to ask who they are bt w

2 CHOOSE 5 WEBSITES THAT DEFINE YOUR STYLE & WHY

PROJECT 2 - DEFINE YOUR STYLE IN URL’S 12 responses in 3 days Paid 10 cents per answer Total cost $3.80 example Taylormade.com - the reason this site represents my style is that I love to golf and the only clubs I hit are Taylormade. Scrapbook.com - I like this page because I TelevisionWithoutPity - I love TV, love to scrapbook. It gives me ideas for pages but not so much that I am blind to that I can incorporate into my scrapbooks. the bad parts. I'm smart as well as a smart-ass. This site isn't too cheerful, or too happy. just like me.

PROJECT 2 - LEARNINGS very cool people seemed to enjoy this task loads of meaty content! how can push for something emotional?

3 DETAIL 5 SITUATIONS / EMOTIONAL EVENTS THAT OCCURRED TO YOU TODAY FIND A WEB SITE THAT EMBODIES THAT FEELING TELL ME WHY

PROJECT 3 - EMOTIONAL OUTPUT 14 responses in 3 days Paid 25 cents per answer Total cost $2.75 example Anger - Angry365 site- I was angry at my computer today. Shock - postsecret.com - I am shocked and depressed about people today. Tired - Bedding.com site - I felt really sleepy today, worn down. Frisky - FREDRICKS.com - I felt attractive today, and what's sexier than lingerie?

PROJECT 3 - LEARNINGS very cool people seemed to enjoy this task a bit tricky with the whole emotional embodiment thing what are the limits to what people will do?

4 CREATE A YOUTUBE VIDEO ANSWERING THE QUESTION WHAT IS DESIGN TO YOU, WHATS GOOD DESIGN FEEL LIKE GIVE AN EXAMPLE OF GOOD DESIGN

PROJECT 4 - GO MAKE A VIDEO 1 response in 3 days Paid 2 DOLLARS per answer Total cost $2 example

PROJECT 4 - LEARNINGS not so good $2 wasn’t enough maybe be hard to do people may not have web cams let’s test prime next!

5 CREATE A COLLAGE OF YOU WERE, ARE, AND WILL BE + DO THE “FEARS” COLLAGE FROM DELL PROJECT + ANSWER SOME BASIC QUESTIONS

PROJECT 5 - PRIME TEST! 25 responses in 2 HOURS! Paid 1 DOLLAR per completed work Total cost $25 example

PROJECT 5 - LEARNINGS GREAT stress test for PRIME! 62 people hit PRIME in 2 HOURS! swimming in data people appeared to enjoy the task people ﬁgured out how to group concepts easily this is very cool

6 FIND AND UPLOAD 10 PHOTOS THAT REPRESENT AN IDEAL EXPERIENCE AND TELL WHY ( test our photo journal tool )

A tarrot card PROJECT 6 - PRIME PHOTO JOURNAL reading is fun. 11 responses in 2 days Paid 50 cents per completed work Total cost $5.50 I love drinking wine. example My family. I want to be in the spotlight. Outdoor concert.

PROJECT 6 - LEARNINGS another GREAT stress test for PRIME! tested our photo journal process tested our server load capability we asked people to rate the task itself: people thought it was easy, simple, fast but boring, lame and they should of been paid a buck

7 transcribe an audio ﬁle access the web, listen to a 45min ﬁle, transcribe

Rich:Yeah. I mean, I don’t have an answer for what the solution PROJECT 7 - transcription to that is, but I would say that when it comes to the message boards that I have experienced I can almost classify them into two groups. There’s the philanthropic kind of message board where people are just talking about non concrete things and it 1 responses in 1 days is all a view, it is just opinions. Which is great for some people, but I am not interested in that. If I want someone’s opinion I will talk to them. When I have a concrete problem and I need an answer for something speciﬁc and I don’t have anyone I can ask. I have Paid $8.00 for transcription tapped out everyone that I can ask. Then it is great to be able to go to that and say, “Hey, how do I do this?” I have answered quite a few people over the years in that respect too.You know, I try to pay back where I get that information from. Total cost $8.00 Female: Correct me if I am wrong. So you see that as a supplemental resource to what you have ofﬂine. Rich: Right. Right.Yeah. Female: Ok. example Rich: I mean if I think about an organization. I don’t know who you guys are doing this for. But say it is IBM or whoever. First and foremost I think it is about the people. Then yo have go all these other supplemental things like job boards, or message https://www.box.net/shared/static/25l2q8fsws.mov boards, or technical help, or discounts on things or whatever they have. But those are all kind of like secondary parts of those organizations and communities Male 1: So in your ideal community the center would be around people. And what would you get out of this ideal community of yours? Rich: Well like I said, this is going to make me seem like a lush, but I think one of the best where they just get a bar and they say if you are a designer show up here and you can shoot the shit with some other people. I think for me that works out great because it is after hour and I can turn off the work Rich for a little while and I can walk in there and still have a pretty open discussion about…

PROJECT 7 - LEARNINGS at ﬁrst it felt hit or miss if someone would do this 1 person took on the task and did an amazing job reads like a movie script turks can access audio on the web, listen to it, and document/transcribe transcription appears to be one of the bigger uses of mechanical turk (based on what i’ve seen)

8 analyze some data

PROJECT 8 - analysis DUDE THIS IS TOO MUCH WORK FOR $5.00!! 2 responses in 3 days each turk only did one participant of Paid $5.00 for analysis data each explained how they did their analysis Total cost $10.00 each commented on how much this work was, and while interesting couldn’t example complete the work due to time/money

PROJECT 8 - LEARNINGS too much work for the money involved analysis many never be viable for turkers without context turks were unsure what they were looking for in the data turks do have excel and can edit ofﬁce documents

9help me brainstorm ( can turks help you brainstorm while you do something else)

• Contact staffing and temporary work agencies (that PROJECT 9 - brainstorm specialize in placement for industrial, construction or warehouse related activities) 27 responses in 4 hours • Contact the unemployment office • On site / Near site intercepts (try to time it for the morning or afternoon when the “roach coach” is Paid $.05 per answer there) • Contact facilities: Distribution centers, warehouses, bottling companies, drywall suppliers. Total cost $1.35 • Contact organizations: Teamsters, Longshoremans, Warehousemans, United Transportation, Association of machinists, Aeorspace Workers our question: • Contact companies: Costco, Sam’s clu, Fed Ex, UPS, Supermarkets • Contact other: DMV (to find out who has a reach how would you go about truck license), local fork lift suppliers (to see who they sell their trucks to): http://www.chicagolift.com/ recruiting forklift home.html , http://www.yeslifts.com/ drivers in chicago for a • Search / post on job boards like Monster.com: http:// www.forklift.driver.jobs.com/ , http:// research study? www.nationalcontractors.com/ • Print attractive, colorful posters and place them at service stations, bars, public places • Drop off and fax flyers to appropriate agencies and companies

PROJECT 9 - LEARNINGS yeeehaw people love fun open ended questions many responses very fast simple, fun task turks seemed to enjoy the advisory/helping like action of this task time & energy to do worth the 5 cent reward

10 paid ya, can ya give me a bit more?

PROJECT 10 - the drill down 10 responses in 2 hours Paid $.25 per answer Total cost $2.50 task: Complete a collage. After • The goal of this project was to test initial completion, ask the whether or not participants after having completed an initial assignment would participant to come back return (by our request) to give further detail on the assignment without giving and drill down further on additional payment. an aspect of the collage.

PROJECT 10 - LEARNINGS yes it is possible for turks to complete a task and then return to give additional information - after payment this suggests that turks are honorable to a degree of the work they do, payment accepted and are willing to go the extra “mile” to give additional clariﬁcation on an assignment turks are potentially perfect participants for studies because of the environment they are on, and expected interaction they often have turks could be potentially too eager to express an opinion, but is that a bad thing?

11 brand ransom

PROJECT 11 - brand ransom 23 responses in 2 hours Paid $1.00 per ransom note Total cost $25.00 task: create a ransom note Using a ransom note generator on the web, recall a recent frustration with a brand/experience and articulate it via the online tool.

PROJECT 11 - LEARNINGS awesome task, turks had a lot of fun with this one unique & fun task for participants, engaging and expressive had to lead by example but this assured a good range of success compared to typical quant data, this task essentially asked the same question but gave the participant a richer language to answer, resulting in an artifact that is both meaningful + art like

mechanical turk take aways FAST & easy to use perfect for testing web apps in development excellent discovery tool great for independent projects, pilot homework test instructions, wild methods, explore ideas extremely cost effective way to get a taste of data on topic you desire excellent to pair up with fast, fun, and unique engagement apps online

mechanical turk issues still not believable despite having some demographical data, its hard for people to believe that anyone would do these tasks exploitive how much credibility can you give the results? real back to back testing is required where does it belong in your tool box

NEXT STEPS continue to experiment and think beyond what you feel is silly or doesn’t make sense to do use it to test homework assignments, methods, rapid protos conduct more experiments FASTER do more demographic / personality proﬁling of turks integrate it into PRIME and other tool sets develop a methodology/offer around it? fast trending?

http://crowdbuzzing.com	782
http://www.slideshare.net	13
http://feeds.feedburner.com	9
http://doodleporn.tumblr.com	7
http://translate.googleusercontent.com	3
http://www.toobla.com	1
http://www.linkedin.com	1

5 Dollar Data Updated Feb. 2009

by Dan Rockwell on Feb 24, 2009

Accessibility

Categories

Upload Details

Usage Rights

7 Embeds 816

Statistics

5 Dollar Data Updated Feb. 2009 Presentation Transcript