1. WorkerRep: Building Trust on Crowdsourcing
Platform Using Blockchain
Gurpriya Kaur Bhatia (MT16021)
Advisors:
Dr. Ponnurangam Kumaraguru(IIITD)
Dr. Alpana Dubey(Accenture Technology Labs)
LinkedIn/in/gurpriya-kaur
-562ab89a/
@gurpriya06 gurpriya.bhatia.9

2. Thesis Committee
➢ Dr. Arun Balaji Buduru, IIITD
➢ Mr. Vikrant Kaulgud, Accenture Technology Labs
➢ Dr. Alpana Dubey, Accenture Technology Labs (Co - Chair)
➢ Dr. Ponnurangam Kumaraguru, IIITD (Chair)
2

3. Crowdsourcing Platform Overview
ENTITIES WORKFLOW
3

4. Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Task
4

5. Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Worker
Task
5

6. Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Task Poster
Worker
Task
6

7. Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Task Poster
Worker
Platform
Task
7

8. Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Platform
Task Poster
Worker
Task
Post Task
8

9. Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Platform
Task Poster
Worker
Task
Post Task Task Search
and Apply
9

10. Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Platform
Task Poster
Worker
Task
Post Task Task Search
and Apply
Select worker
10

11. Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Platform
Task Poster
Worker
Task
Post Task Task Search
and Apply
Select worker
Perform task
11

12. Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Platform
Task Poster
Worker
Task
Post Task Task Search
and Apply
Select worker
Submit solution
12
Perform task

13. Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Platform
Task Poster
Worker
Task
Post task Task search
and apply
Select worker
Submit solutionPerform review
13
Perform task

14. Crowdsourcing Platform Overview
ENTITIES WORKFLOW
Platform
Task Poster
Worker
Task
Post Task Task Search
and Apply
Select worker
Submit solutionPerform review and send feedback
14
Perform task

15. Blockchain
15
BankUsers Transaction
Ledger Decentralized Distributed Miner Type

16. Ethereum
16
Blockchain - based distributed computing platform
Ether Smart Contract Ethereum

17. Ethereum
17
User Ethereum Wallet
Smart Contract
9f4195924cb79195ef9b07e
04e641a86e2eaa98340b40
15e8884a9f4ba4a6ec0
d59b04abcbeaf11aeebb6325e0
393bfdd1a016a5ae6b58c31a67
f1aa95030be6b02ae8577e8cbb
e17c4577272fa3055ea2d53917
c07e43d9105e4d7e5f65318e
0x0ca7B9c0E24c
Ac746F29BA984e
C8Ba79856c3a83
Private Key
Public Key
Public Address

18. DISTRIBUTED STORAGE
18
InterPlanetary File System
Storage cost
1. Data
2. Hash
QmfJgmCHkame9yxbNG7VwrzYJf8uZr
dbHHcor6oNPBYQou
User

19. Motivation
19

20. Motivation
20

21. Why Blockchain?
21

22. Research Aim
➢ Modeling crowdsourcing platform with
Blockchain based design principles..
➢ Propose submission evaluation strategy that is
insusceptible to malicious users so as to build
a robust and immutable reputation system for
our platform.
22

23. Outline
➢ Proposed Architecture
➢ Steps in Crowdsourcing Cycle
➢ Evaluating Submission and Updating
Reputation
➢ Experiments and Robustness Against Attacks
➢ Conclusion
23

24. Proposed Crowdsourcing Platform Overview
24
Post Task Task Search Select workerApply for task
Perform task
Submit solutionPerform review and send feedback

25. Application Layer
Storage Layer
Ethereum Layer
IPFS
Front End-Framework
Web3.js
User
Contract
Task
Contract
Submission
Contract
Agreement
Contract
Evaluation
Contract
Meta Mask
Proposed Architecture
Off
Blockchain
On
Blockchain
25

26. User Registration
1. Profile Info
2. Hash
3. Create Worker/
Task poster
INPUT : Hash , Public key
User
User
Contract
4. User Registration
transaction mined
5. Registration Successful
OUTPUT: Worker / Task Poster
26

27. Post Task
2. Hash
1. Task Info Task Poster
3. Post Task ( Hash,
Reward, Skills, Title)
4. Task transaction
mined
Task
Contract
Task
27

28. Task Search and Task Registration
28
Tasks ListWorker
1. Task
search
Task
Contract
2. Task register
(taskID)

29. Task Poster Agreement
1. Create
2. Informs worker
3. Accept
6. Submit hash
Worker
Task Assignment and Submission
Off - Blockchain
hashing
4. Submission
5. Hash
Evaluation
Contract
7. Perform evaluations
29

30. Agreement
4. Hash
8. Update Reputation
Worker
Evaluator N
Evaluator 1
Task Assignment, Evaluation and Updating Reputation
1. Submission
2. Encryption Encryption
algorithm
3. Store
encrypted
submission
5. Hash
6. Assign evaluators
7. Review
30

31. Outline
➢ Research Motivation
➢ Research Aim
➢ Proposed Architecture
➢ Steps in Crowdsourcing Cycle
➢ Evaluating Submission and Updating
Reputation
➢ Experiments and Robustness Against Attacks
➢ Conclusion
31

32. Terminology
➢ Given a Worker w who submitted the solution s for task t
➢ Let E = {e1
, e2
, … en
}be the set of evaluators for submission s
➢ {ci
, qi
} rating given by ei
➢ ri
reputation of ei
➢ weightq
and weightc
: weights assigned to quality score and
completeness score respectively.
➢ Let P denote the set of potential evaluators.
➢ evalSet (t1
,t2
,…, tf
) tasks to be evaluated by w corresponding to t.
32

33. Potential Evaluators
Denoted by P, Workers whose:
➢ Submission is not yet evaluated.
➢ Reputation > average reputation of workers on
platform.
33

34. Selecting Evaluators
34
Let |E| be number of evaluators required for
evaluating submission s
Divide P into |E| groups based on reputation
scores
Randomly choose one evaluator from each
interval

35. Review
Each evaluator gives 2 types of reviews:
➢ Textual
➢ Feedback, reason supporting rating.
➢ Rating (1-100)
➢ Quality(qi
)
➢ Completeness(ci
)
35

36. Consensus
➢ Outlier: if score given by evaluator not within 1.25 standard
deviation away from mean. outliers
set of outliers for
submission s.
➢ Consensus majority evaluators are not outliers.
➢ Reviews of only those evaluators who form consensus are
considered for computing reputation score of the worker.
After removing outlier let,
Cm
mean completeness score.
Qm
mean quality score.
36

37. Updating Reputation
Two ways through which the reputation score of
worker is updated:
➢ Submission and evaluation
➢ Only evaluation
37

38. Updating Reputation Through Submission and
Evaluation
◆
38

39. Updating Reputation Through Submission and
Evaluation
◆
39

40. Updating Reputation Through Evaluation
◆
40

41. Reward
◆
41

42. Demo
42
➢ Post Task
➢ Task Application
➢ Select worker and create agreement
➢ Worker accepts the agreement and start working
on it
➢ Submits hash and evaluate task
➢ Get evaluators and send them encrypted solution
➢ Evaluators send review and reputation is
updated

43. Outline
➢ Research Motivation
➢ Research Aim
➢ Proposed Architecture
➢ Unboxing the Steps
➢ Evaluating Submission and Updating
Reputation
➢ Experiments and Robustness Against Attacks
➢ Conclusion
43

44. Various Attacks
➢ Unfair rating attack: is when the rater is biased
towards worker and does not give a truthful opinion
about him.
➢ Reciprocity: when worker reciprocates negatively for a
negative review that he receives.
➢ Ballot stuffing : when the worker tries to increase its
own reputation.
➢ Collusion: when a group of workers try to collude
together to improve their own reputation or decrease
the reputation of others.
➢ Sybil attack:where a malicious worker tries to create
multiple identities over the platform to gain influence
45

45. Robustness Against Various Attacks
➢ Unfair rating attack:
➢ Consensus among evaluators and final rating is being
normalized using median.
➢ Reputation of evaluator is decreased if he is an outlier.
➢ Reciprocity:
➢ Outliers removed.
➢ Ballot stuffing and Collusion
➢ Evaluators pseudo randomly chosen.
➢ A trail of evaluation transactions.
➢ Reputation value altered only by contract code.
➢ Sybil Attack
46

46. Experimental Results
Reward
Amount($)
User type AMT($) Upwork($) WorkerRep($)
50 Worker - 10 0.436
Task Poster 10 1.375 0.195
100 Worker - 20 0.436
Task Poster 20 2.75 0.195
1000 Worker - 200 0.436
Task Poster 200 27.5 0.195
47

47. Conclusion
➢ How each step of crowdsourcing could be managed as
transaction on Blockchain.
➢ Address various attacks possible on centralized
Crowdsourcing platform.
➢ Establishing better trust on worker’s of the platform.
➢ Reduce cost for task poster as well as worker.
47

48. Challenges, Limitation and Future Work
➢ Time delay between creating a transaction and
its confirmation - Sharding.
➢ Computation cost on Ethereum network -
Computational oracles.
➢ Task poster having a say in the submission
evaluation and maintaining its reputation
score.
➢ Encrypting evaluation initially until all
evaluators have sent. 48

49. Acknowledgement
➢ Committee members
➢ Abhinav, Accenture Technology Labs
➢ Indira, Shubham, Simran, Shwetanshu
➢ Members of Precog family
➢ Family and friends
49

50. References
➢ Li, M., Weng, J., Yang, A., Lu, W., Zhang, Y., Hou, L., Liu, J.N., Xiang, Y. and
Deng, R.H., 2017. CrowdBC: A Blockchain-based Decentralized Framework
for Crowdsourcing. IACR Cryptol. ePrint Arch., Univ. California, Santa Barbara,
Santa Barbara, CA, USA, Tech. Rep, 444, p.2017.
➢ Dennis, R. and Owenson, G.H., 2016, February. Rep on the block: A next
generation reputation system based on the blockchain. In 10th
International Conference for Internet Technology and Secured Transactions
(ICITST). IEEE.
➢ Tavakolifard, M. and Almeroth, K.C., 2012. A taxonomy to express open
challenges in trust and reputation systems. Journal of
Communications, 7(7), pp.538-551.
➢ Jøsang, A. and Golbeck, J., 2009, September. Challenges for robust trust 50

51. Thanks!
gurpriya16021@iiitd.ac.in
@gurpriya06
51