• Save
Sofe 2013 innotribe_big_data
Upcoming SlideShare
Loading in...5

Sofe 2013 innotribe_big_data



SOFE Innovation plenary - slides

SOFE Innovation plenary - slides



Total Views
Views on SlideShare
Embed Views



1 Embed 10 10


Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Sofe 2013 innotribe_big_data Sofe 2013 innotribe_big_data Presentation Transcript

  • Big Data Practical applications of big data Business continuity- Cyber crime SOFE, Thursday 28 November
  • with Noam Perski International Cultivator - Palantir Kimmo Soramaki Founder – Financial Network Analytics
  • Noam Perski - Palantir MEANING OUT OF MOUNTAINS
  • Available Data What we don’t know Ability to Process
  • Available Data What we don’t know Ability to Process
  • What about the rest of us?
  • 10 protein-folding puzzles Video game players vs software The humans went 7-0-3
  • Medical test for Nerd Syndrome • 80% of a saliva test detect Nerd Syndrome when it is there (so 20% miss it) • 9.6% of the test detect Nerd Syndrome when it’s not there (so 90.4% correctly return a negative result) • 1% of people have Nerd Syndrome (so 99% do not) If the test comes out positive, what are the odds I am sick?
  • Test Pos Test Neg Nerd (1%) 80% 20% Not Nerd (99%) 9.6% 90.4% • • 1% of people have Nerd Syndrome If you already have Nerd, you are in the first column • 80% chance you will test positive • 20% chance you will test negative • If you don’t have Nerd, you are in the second column • 9.6% chance you will test positive • 90.4% chance you will test negative
  • Nerd (1%) Test Pos Test Neg • • • • Not Nerd (99%) True Pos 1% * 80% True Neg 1% * 20% False Pos 99%* 9.6% False Neg 99% * 90.4% Chances of a true positive = chance you have Nerd * chance test caught it ( 1% * 80% = 0 .008 ) Chances of a false positive = chance you don’t have Nerd * chance test caught it anyway ( 99% * 9.6% = 0.09504 ) Chance of getting any type of positive result is the chance of a true positive + the chance of a false positive (0.008 + 0.09504 = .10304) Probability = desired event / all possibilities Our chance of Nerd is 0.008 / 0.10304 = 0.0776, or about 7.8%.
  • What if we only test people with symptoms? • Can name more than 7 characters form Lord of the Rings • Owns at least one Star Wars Action figure 40% probability you have Nerd Syndrome + Positive test Chance of Nerd is 85%
  • What we already know matters
  • Dreams Hunches Theories Humans Think Hypotheses Analogies Guesses
  • Statistic s Visualiza tion Calculati ons Interpol ation Machines Compute Informa tion Retrieva l Tabulati on Statistic s
  • Kimmo Soramaki FNA
  • Amsterdam, 28 November 2013 SWIFT Operations Forum Network Simulations for Business Continuity Dr. Kimmo Soramäki Founder and CEO Financial Network Analytics www.fna.fi
  • Network Simulation – Interactive Demo Failure Scenario Black node = can receive but cannot send Normal Scenario Green node = Liquidity available Red node = No, liquidity. Queues build up. 31
  • Real Networks: Fedwire Payment Network ‘Furball’ 32
  • Fedwire Core Fedwire Interbank Payment Network Fall 2001 Around 8000 banks, 66 banks comprise 75% of value,25 banks completely connected Soramäki, Bech, Beyeler, Glass and Arnold (2007), Physica A, Vol. 379, pp 317-333. 33
  • SWIFT Message Flows 34
  • International Remittances 35
  • More Network Maps Federal funds Bech, M.L. and Atalay, E. (2008), “The Topology of the Federal Funds Market”. ECB Working Paper No. 986. Italian money market Iori G, G de Masi, O Precup, G Gabbi and G Caldarelli (2008): “A network analysis of the Italian overnight money market”, Journal of Economic Dynamics and Control, vol. 32(1), pages 259-278 Unsecured Sterling money market Wetherilt, A. P. Zimmerman, and K. Soramäki (2008), “The sterling unsecured loan market during 2006–2008: insights from network topology“, in Leinonen (ed), BoF Scientific monographs, E 42 Cross-border bank lending Minoiu, Camelia and Reyes, Javier A. (2010). A network analysis of global banking:1978-2009. IMF Working Paper WP/11/74. 36
  • Network Theory Financial Network Analysis Social Network Analysis Network Science NETWORK THEORY Graph & Matrix Theory Computer Science Biological Network Analysis
  • Structure of links between nodes matters The performance of a node (bank) cannot be analyzed on the basis its own properties and behavior alone To understand the performance of one node (bank), one must analyze the behavior of nodes that may be several links apart in the network. Each affect each. 38
  • Networks Brings us Beyond the Data Cube For example: Entities: 100 banks Variables: Liquidity, Opening Balance, … Time: Daily data Links: Bilateral payment flows Links are the 4th dimension to data Information on the links allows us to develop better models for banks' liquidity situation in times of stress 39
  • Modeling the Flows 40
  • Predictive Modeling • Predictive modeling is the process by which a model is created to try to best predict the probability of an outcome • “What is the impact if a large bank has an operational disruption at noon?” – Who is affected first? – Who is affected most? – What is the impact on my bank in an hour? • Valuable information for decision making 41
  • Short History of Payment System Simulations • 1997 : Bank of Finland – Evaluate liquidity needs of banks when Finland’s RTGS system was joined with TARGET – See Koponen-Soramaki (1998) “Liquidity needs in a modern interbank payment systems: • 2000 : Bank of Japan and FRBNY – Test features for BoJ-Net/Fedwire • 2001 - : CLS approval process and ongoing oversight – Test CLS risk management – Evaluate settlement’ members capacity for pay-ins – Understand how the system works • Since: Bank of Canada, Banque de France, Nederlandsche Bank, Norges Bank, TARGET2, and many others • 2010 - : Bank of England new CHAPS – Evaluate alternative liquidity saving mechanisms – Use as platform for discussions with banks – Denby-McLafferty (2012) “Liquidity Saving in CHAPS: A Simulation Study”
  • Stress Testing Basel Committee for Banking Supervision published in April 2013 document “Monitoring Tools for Intraday Liquidity Management”. It outlines stress scenarios, one of which is: “Counterparty stress: a major counterparty suffers an intraday stress event which prevents it from making payments “ Stress Simulation Demo 43
  • Thank you Blog, library and demos are available at www.fna.fi Dr. Kimmo Soramäki kimmo@fna.fi Twitter: soramaki 44
  • Demo – business continuity scenarios using networks
  • Demo – cyber investigation