We stand on the shoulder of giants when it comes to using the latest and greatest machine learning libraries. However, not much is said about how to deploy and monitor your beautiful model when it's out in production. I want to talk about the successes and potential perils of building real time machine learning solutions, discuss machine learning in general for the non-technical and discuss the architecture and approach we've taken at Ravelin to use machine learning to stop fraud.

1. building intelligent data products

2. what actually is fraud
architecting flexible data ‘plumbing’
building solid data products on top of them

3. stephen whitworth
2 years at Hailo as data scientist/jack of some trades out of
university
product and marketplace analytics, agent based
modelling, data engineering, ‘ML’ services
data science/engineering at ravelin, specifically
focused on our detection capabilities

4. what is ravelin?
online fraud detection and prevention platform
stream application/server data to our events API
we give fraud probability + beautiful data visualisation
backed by techstars/passion/playfair/amadeus/indeed.com
founder/wonga founder amongst other great investors

5. fraud?

6. $14B
a dollar for every year the universe has existed

7. Same day delivery On-demand services

8. ‘victimless crime’
police ill-equipped to handle
low barrier to entry from dark net
3D secure - conversion killer

9. traditional: human generated rules, born of deep expertise
order-centric view of the world

10. hybrid: augment expertise by learning rules from data
cards don’t commit fraud, people do

11. building
good
plumbing

12. receive firehose through API
decode arbitrary data and store
extract hundreds of features
http/slack/whatever notification to customer
in 100-300ms (ish)
run through N models and rule engine to get probability

13. BUZZWORDS ABOUND
go
postgres
AWS
microservices
zookeeper
NSQ python
event-driven
elasticsearch bigquery dynamodb
redis

15. instrumentation

16. different
databases
for different
needs
kudos if you get The Office reference

17. postgres: solid, start here
dynamodb: very high throughput, low latency data
bigquery: to answer any question you could possibly have
elasticsearch: rich querying in a reasonable amount of time
graph db: haven’t decided, recommendations?

18. asynchronous systemsfirehoses
nice deployment patterns
‘lambda architecture’ - the append only log
services store their own interpretation of events
services are almost entirely decoupled

19. asynchronous systemsfirehoses
error propagation is challenging
no guarantees of SLA - at least as slow as your queue
hard to know who or what is consuming your data

20. building
data
products

21. ‘a random forest is like a room full of
experts who have seen different
cases of fraud from different
perspectives’

22. ‘a random forest is like a room full of
experts who have seen different
cases of fraud from different
perspectives’
N

23. precision: of all of my predictions, what % was I correct?
recall: out of all of the fraudsters, what % did I catch?
implicit tradeoff between conversion and fraud loss
‘accuracy’ a useless metric for fraud

24. 99.8% ACCURATE

26. keep model interfaces simple
hide arbitrarily complex transformations behind it
blend global and client specific models

27. building and training statistical models
currently batch
will combine with online

28. RANDOM FORESTS

29. ‘a random forest is like a room full of
experts who have seen different
cases of fraud from different
perspectives’

30. RANDOM FORESTS
MONITORING

31. probabilistic, not deterministic
dogfood - use live robot customers
run models in ‘dark mode’ to determine performance

32. why not deep learning? ..yet
ability to debug random forests
had nice results with keras

33. serialisation and deployment: an unsolved problem

34. in beta and signing up clients
looking for on-demand services/marketplaces
talk to me afterwards

35. obligatory: we are hiring!
senior machine learning engineers/data scientists
stephen.whitworth@ravelin.com or talk to me after

36. @sjwhitworth
www.ravelin.com - @ravelinhq