Neha gupta why turk_brightonseo


Published on

Published in: Business, Technology
1 Comment
1 Like
  • Can you enable download function?
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Neha gupta why turk_brightonseo

  1. 1. Why people „Turk‟? Neha Gupta Ethnographer & PhD Researcher University of Nottingham & Xerox Research Centre Europe (XRCE)
  2. 2. David Martin Jacki O‟Neill (Now MSR India) Ben Hanrahan
  3. 3. Image: Neha Gupta Andy Crabtree Tom Rodden The University of Nottingham
  4. 4. What is Mechanical Turk? Co-ordinate a crowd (a large group of people), to do micro-work (mostly small contributions) that solves problems (that software and one user cannot do) Image:
  5. 5. Amazon Mechanical Turk (AMT): • mturk launched November 2, 2005 • Artificial artificial intelligence • Work : Requesters Providers / Workforce: Turkers • Human Intelligence Tasks (HITs): payment often in cents/seconds • Other crowdsourcing platforms, other types of work e.g. freelancer, odesk, elance
  6. 6. Types of task on AMT • Micro-tasking: image tagging, duplicate recognition, SEO, data entry, translation, transcription, content generation, obj. classification Image:
  7. 7. Why research on AMT ? • Little in-depth qualitative studies of Turkers • Survey of demographics – Ross et. al. • Turkers Bill of Rights – Irani, Silberman • Law in crowdsourcing industry – Felstiner • AMT design – Khanna et. al. • Concerns: Are workers perfectly happy with conditions and wages? • Information deficit/opacity • Lack of choice and opportunities • Imbalance / Lack of power of redress
  8. 8. How did we conduct research ? • Virtual ethnography of online forums like mturkforum and Turker Nation by Dr. Dave Martin, XRCE • Study involved deep reading of threads, collection of key threads/ posts, recurrent themes, exemplars • Ethnographic Studies of Turkers in their respective homes and places of work in India • Study involved Interviews (skype, phone and in-person), walkthroughs, observations, data capture through audio / video / screen recordings
  9. 9. Why Do Turkers Turk? • Motivated by earning money • Learn new things, “earn money while having fun”, but the aim still is to earn money • Compare themselves to others • Set themselves goals and targets • For some AMT is a compromise: Problems accessing the regular job market, Need to supplement their income, Some housebound.
  10. 10. Process of Turking for a Turker Look for HIT Accept HIT Work on HIT Submit HIT and wait for approval If approved, get paid, if not, get rejected. Wait for feedback Wait for money to arrive Go back for more work Worry about blocks, ratings, qualifications, suspensions, future work.
  11. 11. Turking = flexible working? Example 1: Is turking flexible? Mansoor You cannot find much work on AMT during the day… in the morning I go to institute for BBA for taught and practical classes.. Class starts at 11.30 am, I leave home at 10.30am, we have 1 hour theory, 1 hour practical and then we practice for 1 – 2 hrs.. and then I come back at 4 pm and rest. Then do work on AMT. I do the most work at night time after 7.30 sometimes till 1, 2 or 3 am at night… I also have to do house work in the morning […] but if I got more work in the mornings I am willing to sit and work all day.
  12. 12. Turkers Turking for Fun? Example 2, 3: Turkers Turking for Fun? Danturker This attitude would be requesters dream come true. The workers come here to have fun and play and the lousy pay for work is not an issue. This attitude helps create low pay for the AMT work force that does care about fair pay. Larak56 I agree with most everyone here. While I do find some of the HITS fun and actually learn an incredible amount by doing HITS, I do it for the cash.
  13. 13. Varying Expectations Example 4: Varying Expectations Defectturk I would rather pull weeds, do laundry, wash the car, clean the toilet than work for this shit paying requester. Clunky interface, too many tags/ratings/whatever per hit, works out to about a buck an hour. They should be banned from mturk, or even better forced to work on their own shit paying hits.
  14. 14. Varying Expectations Example 5: Varying Expectations Nagen I only do tasks that I have done before so that I am familiar with the instructions. […] But I think one of past few times, something I did went wrong as I got some warning emails from Amazon and finally got blocked. I was pretty sure I had done the task correctly and had seen the instructions before but perhaps I misunderstood something and now I might get suspended. We keep trying to contact them (AMT) via emails. I still have around $50 in my account. They are an American company, they are pukka (genuine/honest/honourable) in their procedures, I trust they‟ll be fair to me and give me my money back..
  15. 15. Reputation of Turkers Example 6: Reputation based on qualifications, ratings Mansoor
  16. 16. Reputation of Turkers Example 6: Reputation based on qualifications, ratings Mansoor Now these HITs are available 24X7 for these guys (who have score of a 100) and I see them everyday but can‟t do them, which hurts a little. I was careless when the Requester was testing Turkers with qualifications, so my score is less. I should have worked hard [...] but I was in a rush.”
  17. 17. How Much Do Turkers Earn? • Considerable variation but Turking is low wage work, wages off-set against time taken to find work, amount of concentration required etc. • Highest earners on Turker Nation ~$15-16k per year High earners in India ~$8k-$10k per year Low earnings can be as low as $260 - $520 per year ($5-10 per week) • U.S Turkers generally aspire to earning $7-10 per hour. Indian Turkers aspire to earn according to benchmarks set e.g. Call centre work ~ Rs 10,000 ~ $160 per month • Lower paid easy work to increase reputation and ranking e.g. Oscar Smith
  18. 18. How Much Do Turkers Earn? • Lower paid easy work to increase reputation and ranking e.g. Oscar Smith
  19. 19. Invisible Turkers & their hidden work • AMT as a labour market: • Easy for requesters to devalue/ignore workers • Invisibility accentuated by anonymity, distance, digital interaction The work to make the Turking work: • Work of searching, learning, optimising, organising. hidden, no monetary compensation • Discussing best practices with fellow-turkers, supporters & requesters
  20. 20. Hidden work for Turkers
  21. 21. Good and Bad Requesters Example 7: A Good Requester Explodeman All hits I have done for Project Tatooine in the past week have approved and are fair pay. Good Requester.
  22. 22. Good and Bad Requesters Example 8: A Bad Requester Iambob I have done a number of his hits. Then one day I get this email: “You received a bonus from Top Page Solutions. The value of your bonus is $0.01 USD yes ONE CENT. The requester included this note: If you spent half the amount of time looking for a job as you spent scamming your way through Mechanical Turk HITs maybe you’d be able to find a real job” The next day I get a block message from him. WTF?! … My hit total is under 2000 so I do not see how I am a scammer in his mind.
  23. 23. Turker Ethics? Both U.S. and Indian Turkers take a strong ethical position • Duty to be honest about requesters and admit mistakes. • Good behaviour is practically and ethically the best way to help fellow Turkers and influence the market • Ripping requesters off is not endorsed • Turkers would like their voice heard and help from Amazon
  24. 24. Turker-Requester Relationships • U.S. Turkers like anonymity and flexibility, Indian Turkers don‟t mind anonymity but prize relationship with American requesters • Turkers want productive relationships with requesters • They want fair pay for fair work: decent wages, fairness in judging work, timely payment • Value courteous and fair communication • Want to communicate with requesters about work, design of tasks etc. • Respect works both ways: regular work from good requesters is highly prized
  25. 25. Key Takeaways for Requesters • Think about Turkers as people, colleagues • Communicate your thoughts • Pay Turkers fairly • Provide feedback (positive / negative) when possible • Try to be involved / engaged
  26. 26. Thank You ! If you use / would like to use AMT to get some work done, I would love to hear your experiences, questions or anything you would like to discuss. Please feel free to talk to me. @Neha_Gi_Ji
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.