Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Big
Data

Practical applications of big data

Business continuity- Cyber crime
SOFE, Thursday 28 November
with

Noam Perski
International Cultivator - Palantir

Kimmo Soramaki
Founder – Financial Network Analytics
Noam Perski - Palantir

MEANING OUT OF MOUNTAINS
Available Data
What we don’t know

Ability to Process
Available Data

What we don’t know

Ability to Process
What about the rest of us?
10 protein-folding puzzles

Video game players vs software
The humans went 7-0-3
Medical test for Nerd Syndrome
• 80% of a saliva test detect Nerd Syndrome when it is
there (so 20% miss it)
• 9.6% of the...
Test Pos
Test Neg

Nerd (1%)
80%
20%

Not Nerd (99%)
9.6%
90.4%

•
•

1% of people have Nerd Syndrome
If you already have ...
Nerd (1%)
Test Pos
Test Neg
•
•
•
•

Not Nerd (99%)

True Pos 1% * 80%
True Neg 1% * 20%

False Pos 99%* 9.6%
False Neg 99...
What if we only test people with symptoms?

• Can name more than 7 characters form Lord of the Rings
• Owns at least one S...
What we already know matters
Dreams

Hunches

Theories

Humans
Think
Hypotheses

Analogies

Guesses
Statistic
s
Visualiza
tion

Calculati
ons

Interpol
ation

Machines
Compute

Informa
tion
Retrieva
l

Tabulati
on

Statist...
NPERSKI@PALANTIR.COM
Kimmo Soramaki
FNA
Amsterdam, 28 November 2013

SWIFT Operations Forum
Network Simulations for Business Continuity

Dr. Kimmo Soramäki
Founde...
Network Simulation – Interactive Demo
Failure Scenario

Black node = can receive
but cannot send

Normal Scenario

Green n...
Real Networks: Fedwire Payment Network ‘Furball’

32
Fedwire Core
Fedwire Interbank
Payment Network
Fall 2001

Around 8000 banks, 66
banks comprise 75% of
value,25 banks compl...
SWIFT Message Flows

34
International Remittances

35
More Network Maps
Federal funds
Bech, M.L. and Atalay, E. (2008), “The Topology of
the Federal Funds Market”. ECB Working ...
Network Theory

Financial
Network Analysis
Social Network
Analysis

Network Science

NETWORK
THEORY
Graph & Matrix
Theory
...
Structure of links between nodes matters
The performance of a node
(bank) cannot be analyzed on
the basis its own properti...
Networks Brings us Beyond the Data Cube
For example:
Entities:
100 banks

Variables:
Liquidity, Opening Balance, …
Time:
D...
Modeling the Flows

40
Predictive Modeling
• Predictive modeling is the process by which a
model is created to try to best predict the
probabilit...
Short History of Payment System Simulations
•

1997 : Bank of Finland
– Evaluate liquidity needs of banks when Finland’s R...
Stress Testing
Basel Committee for Banking Supervision published in
April 2013 document “Monitoring Tools for Intraday
Liq...
Thank you
Blog, library and demos are available at www.fna.fi

Dr. Kimmo Soramäki
kimmo@fna.fi
Twitter: soramaki

44
Demo – business
continuity scenarios
using networks
Demo – cyber
investigation
Sofe 2013 innotribe_big_data
Sofe 2013 innotribe_big_data
Sofe 2013 innotribe_big_data
Sofe 2013 innotribe_big_data
Sofe 2013 innotribe_big_data
Sofe 2013 innotribe_big_data
Sofe 2013 innotribe_big_data
Sofe 2013 innotribe_big_data
Sofe 2013 innotribe_big_data
Sofe 2013 innotribe_big_data
Sofe 2013 innotribe_big_data
Sofe 2013 innotribe_big_data
Sofe 2013 innotribe_big_data
Sofe 2013 innotribe_big_data
Upcoming SlideShare
Loading in …5
×

Sofe 2013 innotribe_big_data

587 views

Published on

SOFE Innovation plenary - slides

Published in: Economy & Finance, Technology
  • Be the first to comment

  • Be the first to like this

Sofe 2013 innotribe_big_data

  1. 1. Big Data Practical applications of big data Business continuity- Cyber crime SOFE, Thursday 28 November
  2. 2. with Noam Perski International Cultivator - Palantir Kimmo Soramaki Founder – Financial Network Analytics
  3. 3. Noam Perski - Palantir MEANING OUT OF MOUNTAINS
  4. 4. Available Data What we don’t know Ability to Process
  5. 5. Available Data What we don’t know Ability to Process
  6. 6. What about the rest of us?
  7. 7. 10 protein-folding puzzles Video game players vs software The humans went 7-0-3
  8. 8. Medical test for Nerd Syndrome • 80% of a saliva test detect Nerd Syndrome when it is there (so 20% miss it) • 9.6% of the test detect Nerd Syndrome when it’s not there (so 90.4% correctly return a negative result) • 1% of people have Nerd Syndrome (so 99% do not) If the test comes out positive, what are the odds I am sick?
  9. 9. Test Pos Test Neg Nerd (1%) 80% 20% Not Nerd (99%) 9.6% 90.4% • • 1% of people have Nerd Syndrome If you already have Nerd, you are in the first column • 80% chance you will test positive • 20% chance you will test negative • If you don’t have Nerd, you are in the second column • 9.6% chance you will test positive • 90.4% chance you will test negative
  10. 10. Nerd (1%) Test Pos Test Neg • • • • Not Nerd (99%) True Pos 1% * 80% True Neg 1% * 20% False Pos 99%* 9.6% False Neg 99% * 90.4% Chances of a true positive = chance you have Nerd * chance test caught it ( 1% * 80% = 0 .008 ) Chances of a false positive = chance you don’t have Nerd * chance test caught it anyway ( 99% * 9.6% = 0.09504 ) Chance of getting any type of positive result is the chance of a true positive + the chance of a false positive (0.008 + 0.09504 = .10304) Probability = desired event / all possibilities Our chance of Nerd is 0.008 / 0.10304 = 0.0776, or about 7.8%.
  11. 11. What if we only test people with symptoms? • Can name more than 7 characters form Lord of the Rings • Owns at least one Star Wars Action figure 40% probability you have Nerd Syndrome + Positive test Chance of Nerd is 85%
  12. 12. What we already know matters
  13. 13. Dreams Hunches Theories Humans Think Hypotheses Analogies Guesses
  14. 14. Statistic s Visualiza tion Calculati ons Interpol ation Machines Compute Informa tion Retrieva l Tabulati on Statistic s
  15. 15. NPERSKI@PALANTIR.COM
  16. 16. Kimmo Soramaki FNA
  17. 17. Amsterdam, 28 November 2013 SWIFT Operations Forum Network Simulations for Business Continuity Dr. Kimmo Soramäki Founder and CEO Financial Network Analytics www.fna.fi
  18. 18. Network Simulation – Interactive Demo Failure Scenario Black node = can receive but cannot send Normal Scenario Green node = Liquidity available Red node = No, liquidity. Queues build up. 31
  19. 19. Real Networks: Fedwire Payment Network ‘Furball’ 32
  20. 20. Fedwire Core Fedwire Interbank Payment Network Fall 2001 Around 8000 banks, 66 banks comprise 75% of value,25 banks completely connected Soramäki, Bech, Beyeler, Glass and Arnold (2007), Physica A, Vol. 379, pp 317-333. 33
  21. 21. SWIFT Message Flows 34
  22. 22. International Remittances 35
  23. 23. More Network Maps Federal funds Bech, M.L. and Atalay, E. (2008), “The Topology of the Federal Funds Market”. ECB Working Paper No. 986. Italian money market Iori G, G de Masi, O Precup, G Gabbi and G Caldarelli (2008): “A network analysis of the Italian overnight money market”, Journal of Economic Dynamics and Control, vol. 32(1), pages 259-278 Unsecured Sterling money market Wetherilt, A. P. Zimmerman, and K. Soramäki (2008), “The sterling unsecured loan market during 2006–2008: insights from network topology“, in Leinonen (ed), BoF Scientific monographs, E 42 Cross-border bank lending Minoiu, Camelia and Reyes, Javier A. (2010). A network analysis of global banking:1978-2009. IMF Working Paper WP/11/74. 36
  24. 24. Network Theory Financial Network Analysis Social Network Analysis Network Science NETWORK THEORY Graph & Matrix Theory Computer Science Biological Network Analysis
  25. 25. Structure of links between nodes matters The performance of a node (bank) cannot be analyzed on the basis its own properties and behavior alone To understand the performance of one node (bank), one must analyze the behavior of nodes that may be several links apart in the network. Each affect each. 38
  26. 26. Networks Brings us Beyond the Data Cube For example: Entities: 100 banks Variables: Liquidity, Opening Balance, … Time: Daily data Links: Bilateral payment flows Links are the 4th dimension to data Information on the links allows us to develop better models for banks' liquidity situation in times of stress 39
  27. 27. Modeling the Flows 40
  28. 28. Predictive Modeling • Predictive modeling is the process by which a model is created to try to best predict the probability of an outcome • “What is the impact if a large bank has an operational disruption at noon?” – Who is affected first? – Who is affected most? – What is the impact on my bank in an hour? • Valuable information for decision making 41
  29. 29. Short History of Payment System Simulations • 1997 : Bank of Finland – Evaluate liquidity needs of banks when Finland’s RTGS system was joined with TARGET – See Koponen-Soramaki (1998) “Liquidity needs in a modern interbank payment systems: • 2000 : Bank of Japan and FRBNY – Test features for BoJ-Net/Fedwire • 2001 - : CLS approval process and ongoing oversight – Test CLS risk management – Evaluate settlement’ members capacity for pay-ins – Understand how the system works • Since: Bank of Canada, Banque de France, Nederlandsche Bank, Norges Bank, TARGET2, and many others • 2010 - : Bank of England new CHAPS – Evaluate alternative liquidity saving mechanisms – Use as platform for discussions with banks – Denby-McLafferty (2012) “Liquidity Saving in CHAPS: A Simulation Study”
  30. 30. Stress Testing Basel Committee for Banking Supervision published in April 2013 document “Monitoring Tools for Intraday Liquidity Management”. It outlines stress scenarios, one of which is: “Counterparty stress: a major counterparty suffers an intraday stress event which prevents it from making payments “ Stress Simulation Demo 43
  31. 31. Thank you Blog, library and demos are available at www.fna.fi Dr. Kimmo Soramäki kimmo@fna.fi Twitter: soramaki 44
  32. 32. Demo – business continuity scenarios using networks
  33. 33. Demo – cyber investigation

×