The document describes a simulation model developed to evaluate a community-based distributed fraud detection algorithm. The simulation models the fraud detection process involving multiple levels (L1, L2, L3) of community officers making decisions on applications. Experimental results show that accuracy improves with more officers, but also increases decision time. The identified trade-off between accuracy and efficiency is consistent with previous distributed decision making research. The simulation model is capable of modeling networks of the size considered and can help tune algorithm parameters to improve performance.

1. 1
Jānis Grabis
Arturs Rasnacis
SIMULATION BASED
EVALUATION AND TUNING OF
DISTRIBUTED FRAUD
DETECTION ALGORITHM
Institute of Information Technology, Riga Technical University, Latvia
SIA TrustSearch, Latvia

2. 22
 Community and crowdsourcing based
fraud detection
– CryptoPolice.com > TheTrustSearch.com
 Complex internal dynamics and emergent
collective behavior
 Blockchain as a reward mechanism
Background

3. 3
 Simulation is a suitable technique
dynamics in communities (Bernon
et al. 2007; Aiello et al. 2017)
 The incentives play a major role in
maintaining a viable community
(Chia 2011)
– Blockchains are a suitable
technology (Cai and Zhu
2016)
 Recent simulation studies on
security concerns in distributed
systems (Lee and Wei, 2016;
Panagopoulos et al. 2017)
 Evaluations should be accurate
 Evaluation results should be
obtained as fast as possible
 The optimal number of experts
should be involved
Requirements
Existing Work

4. 44
Objective
• An agent-based approach is used to build the
simulation model
• Decision-making accuracy and response time
are evaluated
To develop a simulation model of the community
based fraud detection algorithm and to conduct
experimental studies to tune the parameters of this
algorithm

5. 55
Community management platform
Community based fraud
detection
Internet user Fraud
application
Issue verdicts
Blockchain – storage of
approved verdicts and
reward
Community of
officers
Multi-stage
evaluation

6. 66
Fraud detection process
Submit
application
Allocate to L1
officers
Issue verdicts
at L1
Evaluate
verdicts at L2
Gather
approvals at
L3
Issue final
decision
Approvals
received
• L1 level officers issue a verdict on
accepting or rejecting the application
• L2 level officers either accept or
reject the verdict
• L3 level officers make the final
approval level by voting
• The verdict is approved as soon as a
number of votes are received

7. 77
Agent based simulation
approach
Officers are
characterized by
their ability to make
correct decisions

8. 88
L3L2
L1
Simulation model
Generate
application
(P correct)
v3
t1
3
w32t2
32
v1
v2
Draw L1 officers (N1)
Generate decision-making time
Generate the verdict with probability P1,
P2
t1
2
t1
1
Assign
verdicts to L2
officers (N2)
Generate
decision-
making time
Verdicts are evaluated by
voting by L3 officers
S approvals are required

9. 99
 Accuracy of fraud identification
– Ratio of correctly evaluated applications to all
applications?
 An application evaluation is correct if officers approve a valid
application or reject a false application;
 Evaluation time
– Time periods to evaluate an application
Experimental studies: objectives

10. 1010
Parameter Value Definition
N1 5;20 Number of officers involved in evaluation at the L1 level
N2 5;20 Number of officers involved in evaluation at the L2 level
P 0.8 Probability of application being correct
P1 0.6; 0.9 Probability that verdict approves the correct application
P2 0.4;0.1 Probability that verdict approves the wrong application
P1 0.7;0.9 Probability that L2 level officer approves the accept verdict for correct
application (correct decision)
P2 0.1;0.2 Probability that L2 level officer declines the accept verdict for correct application
(wrong decision)
P3 0.1;0.3 Probability that L2 level officer declines the reject verdict for wrong application
(wrong decision)
P4 0.5;0.7 Probability that L2 level officer approves the reject verdict for wrong application
(correct decision)
P3 0.02 Probability to make a decision by L3 level officer in any given time period
P4 0.95 Probability of L3 level officer confirming the correct L2 decision
Time to issue a verdict time at L1
S 6 Number of approvals required at the L3 level
Experimental factors
1
it (5,2)LogNormal

11. 1111
Experimental results:
Accuracy by number of officers
0.860.90.940.98
5 10 15 20
Correct
NL1
NL2=5
NL2=20
Number of officers involved in evaluation at the L1 level
Number of officers
involved in evaluation
at the L2 level
Probability P1 that
verdict approves the
correct application
does not have
significant impact in the
range tested (0.6;0.9)

12. 1212
Experimental results:
Accuracy by probability of error
Probability P1 that L2 level officer approves
the Accept verdict for correct application
(correct decision)

13. 1313
Experimental results:
Time by number of officers
Number of officers involved in evaluation at the L1 level

14. 1414
Scenario Parameters Accuracy
S10 P1=0.1, P1=0.1, P2=0.9, P3=0.9, P4=0.1,
P4=0.1
0.04
S50 P1=0.5, P1=0.5, P2=0.5, P3=0.5, P4=0.5,
P4=0.5
0.49
S60 P1=0.6, P1=0.6, P2=0.4, P3=0.4, P4=0.6,
P4=0.6
0.61
S70 P1=0.6, P1=0.6, P2=0.3, P3=0.3, P4=0.7,
P4=0.7
0.78
Experimental results:
Underperforming officers

15. 1515
 Accuracy is satisfactory
 Accuracy and decision-making time can be
improved by increasing the number of officers, at
the L1 level
– Negative consequences on viability of the community
and devaluate tokens issued to motivate the officers
 The identified trade-off between accuracy and
efficiency is consistent with previous findings on
distributed decision-making
 The simulation model is computationally capable
to deal with the networks of the size considered
Conclusion

16. 16
grabis@rtu.lv
http://iti.rtu.lv/vitk/lv/katedra/darbinieki/janis-grabis
Thank you!