This document discusses building a real-time fraud detection system using big data technologies. It outlines the cyber threat landscape, what anomalies and fraud detection are, and proposes an architecture with a data layer to integrate various sources and an analytics layer using stream processing, rules engines, and machine learning to score transactions in real-time and detect fraud. The system aims to scalably and reliably detect threats for increased security.

1. Real-Time Fraud Detection in
Payment Transactions
Christian Gügi, Solution Architect
07.05.2014
Swiss Data Week 2014

2. AGENDA
 Cyber threat landscape
 What are anomalies?
 What is fraud detection?
 Building a fraud detection system
 Q&A

3. WHO I AM
 Christian Gügi, Big Data Solution Architect, YMC
 christian.guegi@ymc.ch
 @chrisgugi
 Founder and organizer Swiss Big Data User Group
 http://www.bigdata-usergroup.ch/

4. WHO WE ARE

5. Source: http://www.viruslist.com/de/analysis?pubid=200883849
CYBER THREAT LANDSCAPE
Countries attacked by phishers in 2013
Switzerland
USA
Russia
Germany
Others

6. TARGETS
Finance-Phishing in 2013 (worldwide)
Quelle: http://www.viruslist.com/de/analysis?pubid=200883849

7. SWITZERLAND AS PHISHING PARADIES
“Wenn Sie in der Schweiz Bank-, Online-Shop
oder E-Payment nutzen, so werden Sie um 45
Prozent häufiger via Phishing attackiert, als im
weltweiten Durchschnitt.“
Source: http://www.finews.ch/news/finanzplatz/14970-phishing-paradies-schweiz

8. WHAT ARE ANOMALIES?
 Anomaly is a pattern that does not conform to the
expected behavior
 Also referred to as fraud, outliers, exceptions, etc.
 Anomalies translate to significant (often critical) real
life entities
 Cyber intrusions
 Credit card fraud

9. REAL WORLD ANOMALIES
 Credit Card Fraud
 An abnormally high purchase made on a credit card
 Cyber Intrusions
 A web server involved in ftp traffic

10. SPOT THE ANOMALY
X
Y
N1
N2
o1
o2
O3
Anomaly

11. WHAT IS FRAUD DETECTION?
 Detection of criminal activities occurring in commercial
organization
 Challenges
 Fast and accurate real-time detection
 Misclassification cost is very high (false positive)

12. REAL-TIME
FRAUD
DETECTION
SYSTEM
BUILDING A

13. THREAT SZENARIO
Quelle: http://www.fedpol.admin.ch/content/fedpol/de/home/dokumentation/information/2013/ref_2013-05-08.html

14. STATUS QUO
 Firewalls protect against attacks
 No detection of anomalous events at transaction level
 No protection from SIM-card fraud (SIM-card swap)

15. WHAT WE REALLY WANT
 Early and automatic detection of anomalies in
real-time
 Augmenting existing fraud detection / security
infrastructure
 Raising efficiency of the whole safety concept
 Reducing costs by detecting fraud

16. STRATEGY
 Use of big data technology
 Integrate all security-relevant data (internal and external)
 Storage of all business transactions
 Detection of anomalies by
 Static business rules
 Machine learning

17. ARCHITECTURE BLUEPRINT
Hadoop
Distributed File System and Processing Framework
Stream Processing
DWH
Analytic SQL
Machine Learning
FraudDetectionSystem
Payment
Transactions
Blacklists
Data
Sources
NoSQL
Others

18. DATA LAYER
 Inclusion of various black-
lists and others
 MapReduce for data
distillation
 Outcomes stored in a
NoSQL database
 Identification of new patterns
by analysis of large data sets
 Simulation of new rules on
historical business data
 Detection rate, error rate
Hadoop
Distributed File System and Processing Framework
Others
DataLayer
Payment
Transactions
Blacklists
Data
Sources
NoSQLMachine Learning
DWH

19. ANALYTICS LAYER
 Streaming data
 Payment transactions
 Stored in a NoSQL database
 Engines for real-time scoring
 Static business rules
 Rules engines / CEP engine
 Machine learning
 Support Vector Machines
 Neuronal Networks
 Score value for each transaction
 Processing of several TB of data
per day using commodity
hardware
Stream Processing
Analytics
Layer
Payment
Transactions
Data
Sources
NoSQL
Score Engine

20. SUMMARY
 Scalable, distributed and reliable system
 Detection in real-time
 Overall safety level adapts to new threats
 Positive side effects for customers
 Methods and technologies can be applied to
other topics

21. YMC AG
Sonnenstrasse 4
CH-8280 Kreuzlingen
Switzerland
@chrisgugi
QUESTIONS
Christian Gügi
christian.guegi@ymc.ch
Tel. +41 (0)71 508 24 76
www.ymc.ch