Assurance Scoring Pydata London 2016

Assurance Scoring:
Using Machine Learning
and Analytics to Reduce
Risk in the Public Sector
Matt Thomson
Natalia Angarita-Jaimes
8/5/2016

2Copyright © Capgemini 2012. All Rights Reserved
Presentation Title | Date
Outline
Introduction
Traditional Fraud Detection
Assurance Scoring
Machine Learning
Business Rules
Anomaly Detection
Graph Links

Who are we?
Matt Thomson
 Senior Data Scientist at Capgemini
 PhD in Astrophysics (http://arxiv.org/abs/1010.3315)
 Several years experience in fraud detection
Natalia Angarita-Jaimes
 Data Scientist at Capgemini
 PhD in Optical Engineering
 Several years experience signal and image processing.
Capgemini
 Big Data Analytics team
 30 Data Scientists, 40 Big Data Engineers
 Focus on Open Source and Big Data technologies to solve client problems
 Sponsor the conference!

Introduction to the Problem
Public sector constantly working in an environment of reduced resources
Want to provide a better service but with greater efficiency
Therefore very important that limited resources are focussed correctly
Assurance Scoring
 Use ML and other analytical methods to identify the least risky people or applications so
that investigators resources can be targeted on the most risky

Hypothetical Example – 2016 Olympics tickets
Running the application process for selling tickets to the 2016 Olympics
Avoid selling tickets to touts/resellers
 Vast majority of people applying for tickets are genuine
 Fraud detection with big class imbalance problem (<0.1%)
 Avoid approach of investigating each person applying
Lets say we know from 2012 Olympics which people ended up reselling
their tickets – training data
Use ML to identify the least risky 30% (say) of people wanting tickets
Investigators focus on the high risk

Traditional Fraud Detection
Identify
Historical
Training Data
Feature
Engineering
Model
Training and
Evaluation
Model
Execution
Feedback

Assurance Scoring
Focus on low-risk
Allows resources to be better focussed
Not limited to Machine Learning
Built using Python!
 Pandas, Scikit-learn etc

Assurance Scoring

POLE ‘Analytical’ Data Layer
Disparate data sources - Atomic Layer
Atomic data is
Transformed and
Loaded into POLE
POLE Layer
EventLocationObjectPerson

POLE ‘Analytical’ Data Layer
POLE contains ALL entities from the Atomic Layer, plus their inter-linkages

Assurance Scoring

Machine learning
Transform Selection Model
Training
Validation
Test
Feature extraction and selection Model Building
Variety of output files: logs, graphics, pickle models, etc
Testing: Unit tests, monitoring tests and integration tests
Vector Build
Input Data
Manipulate, Explore
Data
Framework: Structure, flexibility, consistency

Machine learning : Feature Engineering
SQL, Python
Transform
Explore
Select
Ask
questions,
validate
Refine
features
• Feature Extraction
• Data exploration
• Feature selection
Historical Data

Machine Learning: Model Building
Training
Validation
Test
Split Datasets
Build
Models
Hyper-
parameter
tuning
Selected
features Models
Training
results
Validation
results
Tests
results
Compare
Models

Low risk? High risk? Depends on classifier’s
threshold
• True-positives : applications the
model correctly classifies as high
risk
• True negatives: applications model
correctly classifies as low risk
• False-positives: applications the
model scores as high risk but are
not
• False-negatives: applications the
model scores as low risk but were in
fact high risk

Assurance Scoring

Business Rules
Identifying Fraud often been done using deterministic rules
Look for transactions near a threshold or at the end of the day
Primarily data queries on your feature vector
Olympics example – Anyone applying for more than £10,000 tickets

Assurance Scoring

Anomaly Detection
Use the training data to create a baseline of applications by postcode
(say)
If a particular postcode has a larger than expected number of applications
then those cases pushed into high-risk bucket

Assurance Scoring

Graph Links - Matching
Key part of assurance scoring – bringing data together from disparate
sources
Probability of Match: 80%
Attribute Data Source 1 Data Source 2
Name Matt Thomson Matthew Thosmon
Phone Number 07123 456 789 07123 456 798
Favourite Sport Football Cricket

Assurance Scoring

Jupyter Notebook

Further Details
matt.thomson@capgemini.com / @MattGThomson
Assurance Scoring brochure: http://ow.ly/4nbEUI
Blogs:
 Introduction: https://www.capgemini.com/node/1380596
 Integrating multiple techniques: http://bit.ly/24BmszV
 Machine Learning: http://bit.ly/1QTMGnq
 More coming soon!

We’re Hiring!
Data Science
https://www.uk.capgemini.com/careers/jobs/data-scientist-0
Big Data Engineer
https://www.uk.capgemini.com/careers/jobs/big-data-engineer
Data Visualisation Analyst
https://www.uk.capgemini.com/careers/jobs/data-visualisation-analyst
matt.thomson@capgemini.com

The information contained in this presentation is proprietary.
© 2012 Capgemini. All rights reserved.
www.capgemini.com
About Capgemini
With more than 120,000 people in 40 countries, Capgemini is one
of the world's foremost providers of consulting, technology and
outsourcing services. The Group reported 2011 global revenues
of EUR 9.7 billion.
Together with its clients, Capgemini creates and delivers
business and technology solutions that fit their needs and drive
the results they want. A deeply multicultural organization,
Capgemini has developed its own way of working, the
Collaborative Business ExperienceTM, and draws on Rightshore ®,
its worldwide delivery model.
Rightshore® is a trademark belonging to Capgemini

Assurance Scoring Pydata London 2016

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (6)

Similar to Assurance Scoring Pydata London 2016

Similar to Assurance Scoring Pydata London 2016 (20)

Recently uploaded

Recently uploaded (17)

Assurance Scoring Pydata London 2016