Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

WorkerRep: Building Trust on Crowdsourcing Platform Using Blockchain

215 views

Published on

Crowdsourcing is a process where an individual or an organization utilize the talent pool present over the Internet to accomplish their task(s). These platforms offer numerous advantages such as reduced cost, better quality, and lower task completion time. To execute tasks efficiently, with the worker pool available on the platform, task posters rely on the reputation managed and maintained by the platform. Usually, reputation management system works on ratings provided by the task posters. Such reputation systems are susceptible to several attacks as users or the platform owners, with malicious intents, can jeopardize the reputation system with fake reputations. A blockchain based approach for managing various crowdsourcing steps provides a promising direction to manage reputation system. We propose a crowdsourcing platform where each step of crowdsourcing process is managed as transactions in Blockchain. This helps in establishing better trust in the platform users and addresses various attacks which are possible on a centralized crowdsourcing platform. We have built the proposed platform on the Ethereum framework. Our system utilizes IOTA's consensus mechanism which reduces the cost for task evaluation to almost zero.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

WorkerRep: Building Trust on Crowdsourcing Platform Using Blockchain

  1. 1. WorkerRep: Building Trust on Crowdsourcing Platform Using Blockchain Gurpriya Kaur Bhatia (MT16021) Advisors: Dr. Ponnurangam Kumaraguru(IIITD) Dr. Alpana Dubey(Accenture Technology Labs) LinkedIn/in/gurpriya-kaur -562ab89a/ @gurpriya06 gurpriya.bhatia.9
  2. 2. Thesis Committee ➢ Dr. Arun Balaji Buduru, IIITD ➢ Mr. Vikrant Kaulgud, Accenture Technology Labs ➢ Dr. Alpana Dubey, Accenture Technology Labs (Co - Chair) ➢ Dr. Ponnurangam Kumaraguru, IIITD (Chair) 2
  3. 3. Crowdsourcing Platform Overview ENTITIES WORKFLOW 3
  4. 4. Crowdsourcing Platform Overview ENTITIES WORKFLOW Task 4
  5. 5. Crowdsourcing Platform Overview ENTITIES WORKFLOW Worker Task 5
  6. 6. Crowdsourcing Platform Overview ENTITIES WORKFLOW Task Poster Worker Task 6
  7. 7. Crowdsourcing Platform Overview ENTITIES WORKFLOW Task Poster Worker Platform Task 7
  8. 8. Crowdsourcing Platform Overview ENTITIES WORKFLOW Platform Task Poster Worker Task Post Task 8
  9. 9. Crowdsourcing Platform Overview ENTITIES WORKFLOW Platform Task Poster Worker Task Post Task Task Search and Apply 9
  10. 10. Crowdsourcing Platform Overview ENTITIES WORKFLOW Platform Task Poster Worker Task Post Task Task Search and Apply Select worker 10
  11. 11. Crowdsourcing Platform Overview ENTITIES WORKFLOW Platform Task Poster Worker Task Post Task Task Search and Apply Select worker Perform task 11
  12. 12. Crowdsourcing Platform Overview ENTITIES WORKFLOW Platform Task Poster Worker Task Post Task Task Search and Apply Select worker Submit solution 12 Perform task
  13. 13. Crowdsourcing Platform Overview ENTITIES WORKFLOW Platform Task Poster Worker Task Post task Task search and apply Select worker Submit solutionPerform review 13 Perform task
  14. 14. Crowdsourcing Platform Overview ENTITIES WORKFLOW Platform Task Poster Worker Task Post Task Task Search and Apply Select worker Submit solutionPerform review and send feedback 14 Perform task
  15. 15. Blockchain 15 BankUsers Transaction Ledger Decentralized Distributed Miner Type
  16. 16. Ethereum 16 Blockchain - based distributed computing platform Ether Smart Contract Ethereum
  17. 17. Ethereum 17 User Ethereum Wallet Smart Contract 9f4195924cb79195ef9b07e 04e641a86e2eaa98340b40 15e8884a9f4ba4a6ec0 d59b04abcbeaf11aeebb6325e0 393bfdd1a016a5ae6b58c31a67 f1aa95030be6b02ae8577e8cbb e17c4577272fa3055ea2d53917 c07e43d9105e4d7e5f65318e 0x0ca7B9c0E24c Ac746F29BA984e C8Ba79856c3a83 Private Key Public Key Public Address
  18. 18. DISTRIBUTED STORAGE 18 InterPlanetary File System Storage cost 1. Data 2. Hash QmfJgmCHkame9yxbNG7VwrzYJf8uZr dbHHcor6oNPBYQou User
  19. 19. Motivation 19
  20. 20. Motivation 20
  21. 21. Why Blockchain? 21
  22. 22. Research Aim ➢ Modeling crowdsourcing platform with Blockchain based design principles.. ➢ Propose submission evaluation strategy that is insusceptible to malicious users so as to build a robust and immutable reputation system for our platform. 22
  23. 23. Outline ➢ Proposed Architecture ➢ Steps in Crowdsourcing Cycle ➢ Evaluating Submission and Updating Reputation ➢ Experiments and Robustness Against Attacks ➢ Conclusion 23
  24. 24. Proposed Crowdsourcing Platform Overview 24 Post Task Task Search Select workerApply for task Perform task Submit solutionPerform review and send feedback
  25. 25. Application Layer Storage Layer Ethereum Layer IPFS Front End-Framework Web3.js User Contract Task Contract Submission Contract Agreement Contract Evaluation Contract Meta Mask Proposed Architecture Off Blockchain On Blockchain 25
  26. 26. User Registration 1. Profile Info 2. Hash 3. Create Worker/ Task poster INPUT : Hash , Public key User User Contract 4. User Registration transaction mined 5. Registration Successful OUTPUT: Worker / Task Poster 26
  27. 27. Post Task 2. Hash 1. Task Info Task Poster 3. Post Task ( Hash, Reward, Skills, Title) 4. Task transaction mined Task Contract Task 27
  28. 28. Task Search and Task Registration 28 Tasks ListWorker 1. Task search Task Contract 2. Task register (taskID)
  29. 29. Task Poster Agreement 1. Create 2. Informs worker 3. Accept 6. Submit hash Worker Task Assignment and Submission Off - Blockchain hashing 4. Submission 5. Hash Evaluation Contract 7. Perform evaluations 29
  30. 30. Agreement 4. Hash 8. Update Reputation Worker Evaluator N Evaluator 1 Task Assignment, Evaluation and Updating Reputation 1. Submission 2. Encryption Encryption algorithm 3. Store encrypted submission 5. Hash 6. Assign evaluators 7. Review 30
  31. 31. Outline ➢ Research Motivation ➢ Research Aim ➢ Proposed Architecture ➢ Steps in Crowdsourcing Cycle ➢ Evaluating Submission and Updating Reputation ➢ Experiments and Robustness Against Attacks ➢ Conclusion 31
  32. 32. Terminology ➢ Given a Worker w who submitted the solution s for task t ➢ Let E = {e1 , e2 , … en }be the set of evaluators for submission s ➢ {ci , qi } rating given by ei ➢ ri reputation of ei ➢ weightq and weightc : weights assigned to quality score and completeness score respectively. ➢ Let P denote the set of potential evaluators. ➢ evalSet (t1 ,t2 ,…, tf ) tasks to be evaluated by w corresponding to t. 32
  33. 33. Potential Evaluators Denoted by P, Workers whose: ➢ Submission is not yet evaluated. ➢ Reputation > average reputation of workers on platform. 33
  34. 34. Selecting Evaluators 34 Let |E| be number of evaluators required for evaluating submission s Divide P into |E| groups based on reputation scores Randomly choose one evaluator from each interval
  35. 35. Review Each evaluator gives 2 types of reviews: ➢ Textual ➢ Feedback, reason supporting rating. ➢ Rating (1-100) ➢ Quality(qi ) ➢ Completeness(ci ) 35
  36. 36. Consensus ➢ Outlier: if score given by evaluator not within 1.25 standard deviation away from mean. outliers set of outliers for submission s. ➢ Consensus majority evaluators are not outliers. ➢ Reviews of only those evaluators who form consensus are considered for computing reputation score of the worker. After removing outlier let, Cm mean completeness score. Qm mean quality score. 36
  37. 37. Updating Reputation Two ways through which the reputation score of worker is updated: ➢ Submission and evaluation ➢ Only evaluation 37
  38. 38. Updating Reputation Through Submission and Evaluation ◆ 38
  39. 39. Updating Reputation Through Submission and Evaluation ◆ 39
  40. 40. Updating Reputation Through Evaluation ◆ 40
  41. 41. Reward ◆ 41
  42. 42. Demo 42 ➢ Post Task ➢ Task Application ➢ Select worker and create agreement ➢ Worker accepts the agreement and start working on it ➢ Submits hash and evaluate task ➢ Get evaluators and send them encrypted solution ➢ Evaluators send review and reputation is updated
  43. 43. Outline ➢ Research Motivation ➢ Research Aim ➢ Proposed Architecture ➢ Unboxing the Steps ➢ Evaluating Submission and Updating Reputation ➢ Experiments and Robustness Against Attacks ➢ Conclusion 43
  44. 44. Various Attacks ➢ Unfair rating attack: is when the rater is biased towards worker and does not give a truthful opinion about him. ➢ Reciprocity: when worker reciprocates negatively for a negative review that he receives. ➢ Ballot stuffing : when the worker tries to increase its own reputation. ➢ Collusion: when a group of workers try to collude together to improve their own reputation or decrease the reputation of others. ➢ Sybil attack:where a malicious worker tries to create multiple identities over the platform to gain influence 45
  45. 45. Robustness Against Various Attacks ➢ Unfair rating attack: ➢ Consensus among evaluators and final rating is being normalized using median. ➢ Reputation of evaluator is decreased if he is an outlier. ➢ Reciprocity: ➢ Outliers removed. ➢ Ballot stuffing and Collusion ➢ Evaluators pseudo randomly chosen. ➢ A trail of evaluation transactions. ➢ Reputation value altered only by contract code. ➢ Sybil Attack 46
  46. 46. Experimental Results Reward Amount($) User type AMT($) Upwork($) WorkerRep($) 50 Worker - 10 0.436 Task Poster 10 1.375 0.195 100 Worker - 20 0.436 Task Poster 20 2.75 0.195 1000 Worker - 200 0.436 Task Poster 200 27.5 0.195 47
  47. 47. Conclusion ➢ How each step of crowdsourcing could be managed as transaction on Blockchain. ➢ Address various attacks possible on centralized Crowdsourcing platform. ➢ Establishing better trust on worker’s of the platform. ➢ Reduce cost for task poster as well as worker. 47
  48. 48. Challenges, Limitation and Future Work ➢ Time delay between creating a transaction and its confirmation - Sharding. ➢ Computation cost on Ethereum network - Computational oracles. ➢ Task poster having a say in the submission evaluation and maintaining its reputation score. ➢ Encrypting evaluation initially until all evaluators have sent. 48
  49. 49. Acknowledgement ➢ Committee members ➢ Abhinav, Accenture Technology Labs ➢ Indira, Shubham, Simran, Shwetanshu ➢ Members of Precog family ➢ Family and friends 49
  50. 50. References ➢ Li, M., Weng, J., Yang, A., Lu, W., Zhang, Y., Hou, L., Liu, J.N., Xiang, Y. and Deng, R.H., 2017. CrowdBC: A Blockchain-based Decentralized Framework for Crowdsourcing. IACR Cryptol. ePrint Arch., Univ. California, Santa Barbara, Santa Barbara, CA, USA, Tech. Rep, 444, p.2017. ➢ Dennis, R. and Owenson, G.H., 2016, February. Rep on the block: A next generation reputation system based on the blockchain. In 10th International Conference for Internet Technology and Secured Transactions (ICITST). IEEE. ➢ Tavakolifard, M. and Almeroth, K.C., 2012. A taxonomy to express open challenges in trust and reputation systems. Journal of Communications, 7(7), pp.538-551. ➢ Jøsang, A. and Golbeck, J., 2009, September. Challenges for robust trust 50
  51. 51. Thanks! gurpriya16021@iiitd.ac.in @gurpriya06 51

×