SlideShare a Scribd company logo
1 of 40
1© Cloudera, Inc. All rights reserved.
Extend Your Analytics Toolkit
with Computational Simulations
2© Cloudera, Inc. All rights reserved.
Agenda
The Need for Governance in Agent Based Modeling
Paige Bartley
Sr. Analyst, Ovum
Justin Lyon
CEO, Simudyne
Make better decisions with Computational
Simulation
Dr. Richard Harmon
Director Financial Services, Cloudera
Model, scale, deploy with Simudyne and Cloudera
3© Cloudera, Inc. All rights reserved.
What are the Trends in IT
Ovum’s ICT Enterprise Insights survey is an annual, global survey of IT decision makers
More than 4,000 participants
More than 900 respondents from the financial industry
1. Firms of all verticals are progressively moving to the cloud
2. Increasing revenue and increasing operating efficiency are top-cited business challenges
3. AI and machine learning are getting deployed in the real world, particularly for decision-
making
4© Cloudera, Inc. All rights reserved.
IT Trends: What about the financial industry?
5© Cloudera, Inc. All rights reserved.
Use cases around AI are growing in popularity
6© Cloudera, Inc. All rights reserved.
Digital capability depends on control of data
The axiom, “garbage in, garbage out” is truer than ever.
Challenges in the modern management of data:
• Architectural silos prevent the ready aggregation of data
• The data lake is becoming virtual, with multiple cloud repositories
• Data protection requires granular lineage, policies, and access controls
• The need to combine historical and streaming “fast data” for analysis
• Self-service era means increasing volume of data consumers and data producers
7© Cloudera, Inc. All rights reserved.
Analytics sophistication compounds the need for well-managed
data
Prescriptive
Analytics
Predictive
Analytics
Descriptive
Analytics
Complexity of
analysis
Magnitude of impact on business decisions
8© Cloudera, Inc. All rights reserved.
ABM: Driving the need for managed data
Agent-based modeling uses “agents” which are virtual copies of people, institutions, or other
autonomous entities
• Requires large amounts of data – ideally suited to the data lake
• Unlike machine learning, ABM and computational simulation work best when the
variables are large and organized
• Even small errors in data can become compounded, leading to substantial error later in
the simulation
Agent based modeling ideally needs to be paired with a strong governance framework or
platform for data within the enterprise
9© Cloudera, Inc. All rights reserved.
The IT-centric governance model is outdated
• IT-driven model doesn’t
adequately scale for self-service
• Need to incorporate data use
context from business users
• Feedback loop between IT and
lines of business is needed
• Need a platform where both
business and IT can collaborate
IT-adjusted
policies for
data
Increased data
access and
leverage
Business
user
activity &
feedback
10© Cloudera, Inc. All rights reserved.
Is the managed data lake the answer?
If done properly, then yes.
• Needs to incorporate both on-premise and cloud data sources
• Must be able to ingest all data, both raw and refined
• Must enable business users to participate in the data curation process
• Needs to be a shared, accessible resource available to all users
• Ideally allows for central management of policies, security, and access
The managed data lake allows for data transparency.
11© Cloudera, Inc. All rights reserved.
Managed data lake reference architecture
Key takeaways:
• Hadoop in isolation does not
perform strong governance
• Needs to accommodate both
business and technical users
• Granular data security is
increasing in importance
• Virtual data lakes complicate
centralized governance
Data platform(s): often Hadoop
Cost optimization & integration
Data-level security
Query/analytics tools, programs
Availability/reliability
Monitoring&Troubleshooting
Perimetersecurity
Data curation
Data inventory
= self service “tier” of data access Source: Ovum
12© Cloudera, Inc. All rights reserved.
Symudine + Cloudera Enterprise Data Hub:
ABM on the managed data lake
By sitting atop of the Cloudera Enterprise Data Hub, Simudyne gains access to a lake of
well-governed data
• Better data quality leads to better models
• Security and access controls facilitate compliance
• Data lineage capabilities provide transparency
• Can scale ABM simulations to manage billions of interacting agents
• Data remains readily available in the lake for use in other analytics applications
13© Cloudera, Inc. All rights reserved.
Polling Questions #1
What is the relative maturity of the information governance initiative
within your organization?
1. Journey has just begun – Key roles are assigned to stakeholders,
but technology is largely disjointed
2. Progress underway – Human processes are in place, and some
architectural changes have been made
3. Mature and functional – People, processes, and technology are
tightly integrated and efficient
14© Cloudera, Inc. All rights reserved.
• The Need for Governance in Agent Based Modeling
• Make better decisions with Computational Simulation
• Cloudera Simudyne Computational Simulation Solution
Agenda
15© Cloudera, Inc. All rights reserved.
DRIVE CUSTOMER INSIGHTS
Customer360
Customer Journey Analytics
Recommendation Engine
PROTECT THE
BUSINESS
Enterprise Risk Management
AML (Anti-Money Laundering)
Fraud Detection/Prevention
Market Surveillance
Regulatory Compliance
Cybersecurity
IMPROVE PRODUCTS & SERVICES
Digital Transformation
Alternative Data
Operational Efficiency
PATTERN
RECOGNITION
DETECTION
PREDICTION
750+CUSTOMERS RUN
ON
SIMULATION
DRIVE BUSINESS INSIGHTS
Prescriptive Analytics
Stress Testing
Business Strategy
The enterprise platform for machine learning
16© Cloudera, Inc. All rights reserved.
Key drivers of transformation in financial services
Proliferation
of Data
Unstructured and
structured data growing
at unprecedented rates,
due to mobile apps and
Io
Digital Disruption and Shift
Towards Omni-Channel
Banking
Consumers demand an
enhanced experience.
FinTech, InsureTech
startups threaten
the status-quo
Escalating Fraud Costs
and Increasing Cyber
Attacks
The complexity and
scale of financial fraud
and
cyber attacks is rising.
Regulations and
Compliance
Changing regulatory
environment requires
firms to be more agile
to meet compliance
17© Cloudera, Inc. All rights reserved.
Risk Management Trends
CONTAGION RISK STRESS TESTING IFRS CHANGES
Understanding counterparty credit risk and
understanding your potential for default has
become a major risk management objective for
financial institutions
Since the financial crisis the levels of stress
testing required by regulators has increased
significantly. Modern stress tests require a high
degree of scenario analysis
Capital Adequacy and related Balance Sheet
calculations are being constantly refined by a
number of IFRS and GAAP Changes
18© Cloudera, Inc. All rights reserved.
Evolution of Risk Management
RISK 1.0 RISK 2.0 RISK 3.0
Historical Data – VAR Models
Understanding potential downside risk by
calculating maximum losses according to
historical data
Static Scenarios – Stress Tests
Running Monte Carlo simulations to uncover
potential losses according to a number of
scenarios
Dynamic Interactions – Agent-based
Creating digital versions of each asset and
liability in order to simulate a wide range of
possible future scenarios
19© Cloudera, Inc. All rights reserved.
Polling Questions #2
Where on this risk management maturity curve is your organization?
1. v1.0
2. v2.0
3. v3.0
4. Prefer not to say
20© Cloudera, Inc. All rights reserved.
• The Need for Governance in Agent Based Modeling
• Make better decisions with Computational Simulation
• Cloudera Simudyne Computational Simulation Solution
Agenda
21© Cloudera, Inc. All rights reserved.
“In the summer of 2007, … the markets for some
mortgage securities stopped functioning. Buyers
and sellers simply couldn’t agree on price, and
this impasse soon spread to other debt markets.
Banks began to doubt one another’s solvency.
Trust evaporated, and not until governments
jumped in, late in 2008, to guarantee that major
banks would not fail did the financial markets
settle down and begin fitfully to function again.”
Source: Harvard Business Review
The Financial Crisis
Source: Financial Times
22© Cloudera, Inc. All rights reserved.
• Past models failed to capture interactions of finance and the real economy
• There was no historical precedent for models to draw data from
• Market conditions moved outside of the known parameter set
• Models assumed no changes in data generation process, which always
produces a future that looks like the past
• Models described reality linearly, which is not realistic
Traditional Models Failed During the Financial Crisis
23© Cloudera, Inc. All rights reserved.
Source: Bank of England
• Computational simulation is the only way to model complex
adaptive systems like the financial system.
• Agent Based Modelling (ABM) is one innovative approach
that can improve our understanding of the markets.
• In an ABM, the modeler creates autonomous agents
that could represent, for example: households,
corporates, banks and funding providers.
• These agents:
− Learn from their experiences
− Adapt their behaviors
− Interact with and influence each other
Advancing Your Tool Kit: Computational Simulation
24© Cloudera, Inc. All rights reserved.
• Heterogeneity: Models explicitly capture differences at the agent level.
• Realistic Behaviors: Accurately reflect real-world behavior by modelling
individual heuristics and deviations from rationality. Decouple the future from
the past – recognizing that future crises will not look like the past.
• Emergent Behavior: Differences in behaviors at the micro-level interact to
influence macroscopic outcomes – recreating emergent phenomena.
• Complexity: Recognize that financial markets are interconnected and exhibit
non-linear behavior that helps propagate stress through the financial system.
The Key Features of Agent Based Models
25© Cloudera, Inc. All rights reserved.
• Capture complex behavior and look for tipping points or knife edges of risk
• Recognize the uncertainty in forward-looking projections
• Ask what-if? -- explore the impact of alternative actions in a virtual world
• Train intelligence, whether human or artificial.
The Key Benefits of Simulation for Risk Management
26© Cloudera, Inc. All rights reserved.
The Challenges of Prescriptive Models
ENGINEERING CHALLENGES MODELLING CHALLENGES
Desktop simulation has limited scale and is
unfit for enterprise use
Supercomputer simulation is expensive, inflexible,
and hard to maintain
Different systems required for different types of
analytics and simulations
Few platforms support behavioural or complex
adaptive systems modelling
• High-definition simulation, needs high-performance compute
• Desktop simulation doesn’t scale and supercomputer simulations are
prohibitively expensive
• There is no single, secure solution for computational simulation built for the
needs of global firms
• Simulating real world complexity requires advanced modelling paradigms
like agent based modelling
• Current tools require specialist expertise and create ‘walled gardens’ of
knowledge within a company
• The adoption of an agreed set of tools across an organisation ensures
greater collaboration and reduces model risk
27© Cloudera, Inc. All rights reserved.
Cloudera/Simudyne Computational Simulation Solution
Ingest ModelTransform Discover Visualise Decide
28© Cloudera, Inc. All rights reserved.
Ingest
Cloudera/Simudyne Computational Simulation Solution
● Single place for all data
● Scale up to the cloud
● Highly governed and protected
● Shared data experience
● Simulations at massive scale
● High fidelity models
● Model complex adaptive systems
● Test decisions in a safe environment
ModelTransform Discover Visualise Decide
Machine Learning Computational Simulation
29© Cloudera, Inc. All rights reserved.
Supply Chain Risk Stress Testing Credit Risk Group Strategy
Omni ChannelPricing Valuation Market Risk Liquidity Risk
Business accelerator for a number of use cases
30© Cloudera, Inc. All rights reserved.
Polling Questions #3
Which use case would be your top choice for ABM?
1. Stress Testing
2. Credit Risk
3. Group Strategy
4. Pricing Valuation
5. Market Risk
6. Liquidity Risk
7. Omni-Channel
8. Supply Chain Risk
9. Do Not Know
31© Cloudera, Inc. All rights reserved.
Simulation SDK for Financial Services
32© Cloudera, Inc. All rights reserved.
Decision Making
33© Cloudera, Inc. All rights reserved.
Decision Making
34© Cloudera, Inc. All rights reserved.
Decision Making
35© Cloudera, Inc. All rights reserved.
Decision Making
36© Cloudera, Inc. All rights reserved.
Decision Making
37© Cloudera, Inc. All rights reserved.
Decision Making
MACHINE LEARNING
38© Cloudera, Inc. All rights reserved.
SIMUDYNE
Simulation SDK for Financial Services
DATA INGESTION
MODELLING
VISUALISATION
• Easily ingest data from a variety of sources
• Platform agnostic SDK deployed on-premise or
in the cloud
• Certified on Cloudera, the leading Hadoop
platform
• Machine Models: Combining data and
algorithms to create models
• Human Models: Describing complex adaptive
systems with agent based modelling
• Console: Out of the box visualisation tool that
helps developers to iterate at pace
• Custom UI: Toolkit for bespoke dashboards,
putting the power of advanced analytics in the
hands of senior decision makers
39© Cloudera, Inc. All rights reserved.
Solution Brief:
https://www.cloudera.com/content/dam/www/marketing/resources/solution-briefs/cloudera-and-simudyne-
computational-simulation-solution.pdf.landing.html
Demo Video: Simulating a Bank’s Mortgage Book over 10 years on Cloudera
https://www.youtube.com/watch?v=vrg9r_cgeso
Cloudera:
Solution Lead: Dr. Richard Harmon, rharmon@cloudera.com
Marketing Lead: Mihaela Risca, mihaela@cloudera.com
Simudyne
Sales Lead: Rodney Taylor, rodney@simudyne.com
Marketing Lead: Dahlia Lamy, dahlia@simudyne.com
Resources
40© Cloudera, Inc. All rights reserved.
Thank you

More Related Content

More from Cloudera, Inc.

More from Cloudera, Inc. (20)

Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
 
Leveraging the cloud for analytics and machine learning 1.29.19
Leveraging the cloud for analytics and machine learning 1.29.19Leveraging the cloud for analytics and machine learning 1.29.19
Leveraging the cloud for analytics and machine learning 1.29.19
 
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
 
Leveraging the Cloud for Big Data Analytics 12.11.18
Leveraging the Cloud for Big Data Analytics 12.11.18Leveraging the Cloud for Big Data Analytics 12.11.18
Leveraging the Cloud for Big Data Analytics 12.11.18
 
Modern Data Warehouse Fundamentals Part 3
Modern Data Warehouse Fundamentals Part 3Modern Data Warehouse Fundamentals Part 3
Modern Data Warehouse Fundamentals Part 3
 
Modern Data Warehouse Fundamentals Part 2
Modern Data Warehouse Fundamentals Part 2Modern Data Warehouse Fundamentals Part 2
Modern Data Warehouse Fundamentals Part 2
 
Modern Data Warehouse Fundamentals Part 1
Modern Data Warehouse Fundamentals Part 1Modern Data Warehouse Fundamentals Part 1
Modern Data Warehouse Fundamentals Part 1
 
Extending Cloudera SDX beyond the Platform
Extending Cloudera SDX beyond the PlatformExtending Cloudera SDX beyond the Platform
Extending Cloudera SDX beyond the Platform
 
Federated Learning: ML with Privacy on the Edge 11.15.18
Federated Learning: ML with Privacy on the Edge 11.15.18Federated Learning: ML with Privacy on the Edge 11.15.18
Federated Learning: ML with Privacy on the Edge 11.15.18
 
Analyst Webinar: Doing a 180 on Customer 360
Analyst Webinar: Doing a 180 on Customer 360Analyst Webinar: Doing a 180 on Customer 360
Analyst Webinar: Doing a 180 on Customer 360
 
Build a modern platform for anti-money laundering 9.19.18
Build a modern platform for anti-money laundering 9.19.18Build a modern platform for anti-money laundering 9.19.18
Build a modern platform for anti-money laundering 9.19.18
 
Introducing the data science sandbox as a service 8.30.18
Introducing the data science sandbox as a service 8.30.18Introducing the data science sandbox as a service 8.30.18
Introducing the data science sandbox as a service 8.30.18
 
Cloudera SDX
Cloudera SDXCloudera SDX
Cloudera SDX
 
Introducing Workload XM 8.7.18
Introducing Workload XM 8.7.18Introducing Workload XM 8.7.18
Introducing Workload XM 8.7.18
 
Get started with Cloudera's cyber solution
Get started with Cloudera's cyber solutionGet started with Cloudera's cyber solution
Get started with Cloudera's cyber solution
 
Spark and Deep Learning Frameworks at Scale 7.19.18
Spark and Deep Learning Frameworks at Scale 7.19.18Spark and Deep Learning Frameworks at Scale 7.19.18
Spark and Deep Learning Frameworks at Scale 7.19.18
 
Cloud Data Warehousing with Cloudera Altus 7.24.18
Cloud Data Warehousing with Cloudera Altus 7.24.18Cloud Data Warehousing with Cloudera Altus 7.24.18
Cloud Data Warehousing with Cloudera Altus 7.24.18
 
How Cloudera SDX can aid GDPR compliance
How Cloudera SDX can aid GDPR complianceHow Cloudera SDX can aid GDPR compliance
How Cloudera SDX can aid GDPR compliance
 
When SAP alone is not enough
When SAP alone is not enoughWhen SAP alone is not enough
When SAP alone is not enough
 
Multi task learning stepping away from narrow expert models 7.11.18
Multi task learning stepping away from narrow expert models 7.11.18Multi task learning stepping away from narrow expert models 7.11.18
Multi task learning stepping away from narrow expert models 7.11.18
 

Recently uploaded

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 

Recently uploaded (20)

[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 

Extend Your Analytics Toolkit with Computational Simulations

  • 1. 1© Cloudera, Inc. All rights reserved. Extend Your Analytics Toolkit with Computational Simulations
  • 2. 2© Cloudera, Inc. All rights reserved. Agenda The Need for Governance in Agent Based Modeling Paige Bartley Sr. Analyst, Ovum Justin Lyon CEO, Simudyne Make better decisions with Computational Simulation Dr. Richard Harmon Director Financial Services, Cloudera Model, scale, deploy with Simudyne and Cloudera
  • 3. 3© Cloudera, Inc. All rights reserved. What are the Trends in IT Ovum’s ICT Enterprise Insights survey is an annual, global survey of IT decision makers More than 4,000 participants More than 900 respondents from the financial industry 1. Firms of all verticals are progressively moving to the cloud 2. Increasing revenue and increasing operating efficiency are top-cited business challenges 3. AI and machine learning are getting deployed in the real world, particularly for decision- making
  • 4. 4© Cloudera, Inc. All rights reserved. IT Trends: What about the financial industry?
  • 5. 5© Cloudera, Inc. All rights reserved. Use cases around AI are growing in popularity
  • 6. 6© Cloudera, Inc. All rights reserved. Digital capability depends on control of data The axiom, “garbage in, garbage out” is truer than ever. Challenges in the modern management of data: • Architectural silos prevent the ready aggregation of data • The data lake is becoming virtual, with multiple cloud repositories • Data protection requires granular lineage, policies, and access controls • The need to combine historical and streaming “fast data” for analysis • Self-service era means increasing volume of data consumers and data producers
  • 7. 7© Cloudera, Inc. All rights reserved. Analytics sophistication compounds the need for well-managed data Prescriptive Analytics Predictive Analytics Descriptive Analytics Complexity of analysis Magnitude of impact on business decisions
  • 8. 8© Cloudera, Inc. All rights reserved. ABM: Driving the need for managed data Agent-based modeling uses “agents” which are virtual copies of people, institutions, or other autonomous entities • Requires large amounts of data – ideally suited to the data lake • Unlike machine learning, ABM and computational simulation work best when the variables are large and organized • Even small errors in data can become compounded, leading to substantial error later in the simulation Agent based modeling ideally needs to be paired with a strong governance framework or platform for data within the enterprise
  • 9. 9© Cloudera, Inc. All rights reserved. The IT-centric governance model is outdated • IT-driven model doesn’t adequately scale for self-service • Need to incorporate data use context from business users • Feedback loop between IT and lines of business is needed • Need a platform where both business and IT can collaborate IT-adjusted policies for data Increased data access and leverage Business user activity & feedback
  • 10. 10© Cloudera, Inc. All rights reserved. Is the managed data lake the answer? If done properly, then yes. • Needs to incorporate both on-premise and cloud data sources • Must be able to ingest all data, both raw and refined • Must enable business users to participate in the data curation process • Needs to be a shared, accessible resource available to all users • Ideally allows for central management of policies, security, and access The managed data lake allows for data transparency.
  • 11. 11© Cloudera, Inc. All rights reserved. Managed data lake reference architecture Key takeaways: • Hadoop in isolation does not perform strong governance • Needs to accommodate both business and technical users • Granular data security is increasing in importance • Virtual data lakes complicate centralized governance Data platform(s): often Hadoop Cost optimization & integration Data-level security Query/analytics tools, programs Availability/reliability Monitoring&Troubleshooting Perimetersecurity Data curation Data inventory = self service “tier” of data access Source: Ovum
  • 12. 12© Cloudera, Inc. All rights reserved. Symudine + Cloudera Enterprise Data Hub: ABM on the managed data lake By sitting atop of the Cloudera Enterprise Data Hub, Simudyne gains access to a lake of well-governed data • Better data quality leads to better models • Security and access controls facilitate compliance • Data lineage capabilities provide transparency • Can scale ABM simulations to manage billions of interacting agents • Data remains readily available in the lake for use in other analytics applications
  • 13. 13© Cloudera, Inc. All rights reserved. Polling Questions #1 What is the relative maturity of the information governance initiative within your organization? 1. Journey has just begun – Key roles are assigned to stakeholders, but technology is largely disjointed 2. Progress underway – Human processes are in place, and some architectural changes have been made 3. Mature and functional – People, processes, and technology are tightly integrated and efficient
  • 14. 14© Cloudera, Inc. All rights reserved. • The Need for Governance in Agent Based Modeling • Make better decisions with Computational Simulation • Cloudera Simudyne Computational Simulation Solution Agenda
  • 15. 15© Cloudera, Inc. All rights reserved. DRIVE CUSTOMER INSIGHTS Customer360 Customer Journey Analytics Recommendation Engine PROTECT THE BUSINESS Enterprise Risk Management AML (Anti-Money Laundering) Fraud Detection/Prevention Market Surveillance Regulatory Compliance Cybersecurity IMPROVE PRODUCTS & SERVICES Digital Transformation Alternative Data Operational Efficiency PATTERN RECOGNITION DETECTION PREDICTION 750+CUSTOMERS RUN ON SIMULATION DRIVE BUSINESS INSIGHTS Prescriptive Analytics Stress Testing Business Strategy The enterprise platform for machine learning
  • 16. 16© Cloudera, Inc. All rights reserved. Key drivers of transformation in financial services Proliferation of Data Unstructured and structured data growing at unprecedented rates, due to mobile apps and Io Digital Disruption and Shift Towards Omni-Channel Banking Consumers demand an enhanced experience. FinTech, InsureTech startups threaten the status-quo Escalating Fraud Costs and Increasing Cyber Attacks The complexity and scale of financial fraud and cyber attacks is rising. Regulations and Compliance Changing regulatory environment requires firms to be more agile to meet compliance
  • 17. 17© Cloudera, Inc. All rights reserved. Risk Management Trends CONTAGION RISK STRESS TESTING IFRS CHANGES Understanding counterparty credit risk and understanding your potential for default has become a major risk management objective for financial institutions Since the financial crisis the levels of stress testing required by regulators has increased significantly. Modern stress tests require a high degree of scenario analysis Capital Adequacy and related Balance Sheet calculations are being constantly refined by a number of IFRS and GAAP Changes
  • 18. 18© Cloudera, Inc. All rights reserved. Evolution of Risk Management RISK 1.0 RISK 2.0 RISK 3.0 Historical Data – VAR Models Understanding potential downside risk by calculating maximum losses according to historical data Static Scenarios – Stress Tests Running Monte Carlo simulations to uncover potential losses according to a number of scenarios Dynamic Interactions – Agent-based Creating digital versions of each asset and liability in order to simulate a wide range of possible future scenarios
  • 19. 19© Cloudera, Inc. All rights reserved. Polling Questions #2 Where on this risk management maturity curve is your organization? 1. v1.0 2. v2.0 3. v3.0 4. Prefer not to say
  • 20. 20© Cloudera, Inc. All rights reserved. • The Need for Governance in Agent Based Modeling • Make better decisions with Computational Simulation • Cloudera Simudyne Computational Simulation Solution Agenda
  • 21. 21© Cloudera, Inc. All rights reserved. “In the summer of 2007, … the markets for some mortgage securities stopped functioning. Buyers and sellers simply couldn’t agree on price, and this impasse soon spread to other debt markets. Banks began to doubt one another’s solvency. Trust evaporated, and not until governments jumped in, late in 2008, to guarantee that major banks would not fail did the financial markets settle down and begin fitfully to function again.” Source: Harvard Business Review The Financial Crisis Source: Financial Times
  • 22. 22© Cloudera, Inc. All rights reserved. • Past models failed to capture interactions of finance and the real economy • There was no historical precedent for models to draw data from • Market conditions moved outside of the known parameter set • Models assumed no changes in data generation process, which always produces a future that looks like the past • Models described reality linearly, which is not realistic Traditional Models Failed During the Financial Crisis
  • 23. 23© Cloudera, Inc. All rights reserved. Source: Bank of England • Computational simulation is the only way to model complex adaptive systems like the financial system. • Agent Based Modelling (ABM) is one innovative approach that can improve our understanding of the markets. • In an ABM, the modeler creates autonomous agents that could represent, for example: households, corporates, banks and funding providers. • These agents: − Learn from their experiences − Adapt their behaviors − Interact with and influence each other Advancing Your Tool Kit: Computational Simulation
  • 24. 24© Cloudera, Inc. All rights reserved. • Heterogeneity: Models explicitly capture differences at the agent level. • Realistic Behaviors: Accurately reflect real-world behavior by modelling individual heuristics and deviations from rationality. Decouple the future from the past – recognizing that future crises will not look like the past. • Emergent Behavior: Differences in behaviors at the micro-level interact to influence macroscopic outcomes – recreating emergent phenomena. • Complexity: Recognize that financial markets are interconnected and exhibit non-linear behavior that helps propagate stress through the financial system. The Key Features of Agent Based Models
  • 25. 25© Cloudera, Inc. All rights reserved. • Capture complex behavior and look for tipping points or knife edges of risk • Recognize the uncertainty in forward-looking projections • Ask what-if? -- explore the impact of alternative actions in a virtual world • Train intelligence, whether human or artificial. The Key Benefits of Simulation for Risk Management
  • 26. 26© Cloudera, Inc. All rights reserved. The Challenges of Prescriptive Models ENGINEERING CHALLENGES MODELLING CHALLENGES Desktop simulation has limited scale and is unfit for enterprise use Supercomputer simulation is expensive, inflexible, and hard to maintain Different systems required for different types of analytics and simulations Few platforms support behavioural or complex adaptive systems modelling • High-definition simulation, needs high-performance compute • Desktop simulation doesn’t scale and supercomputer simulations are prohibitively expensive • There is no single, secure solution for computational simulation built for the needs of global firms • Simulating real world complexity requires advanced modelling paradigms like agent based modelling • Current tools require specialist expertise and create ‘walled gardens’ of knowledge within a company • The adoption of an agreed set of tools across an organisation ensures greater collaboration and reduces model risk
  • 27. 27© Cloudera, Inc. All rights reserved. Cloudera/Simudyne Computational Simulation Solution Ingest ModelTransform Discover Visualise Decide
  • 28. 28© Cloudera, Inc. All rights reserved. Ingest Cloudera/Simudyne Computational Simulation Solution ● Single place for all data ● Scale up to the cloud ● Highly governed and protected ● Shared data experience ● Simulations at massive scale ● High fidelity models ● Model complex adaptive systems ● Test decisions in a safe environment ModelTransform Discover Visualise Decide Machine Learning Computational Simulation
  • 29. 29© Cloudera, Inc. All rights reserved. Supply Chain Risk Stress Testing Credit Risk Group Strategy Omni ChannelPricing Valuation Market Risk Liquidity Risk Business accelerator for a number of use cases
  • 30. 30© Cloudera, Inc. All rights reserved. Polling Questions #3 Which use case would be your top choice for ABM? 1. Stress Testing 2. Credit Risk 3. Group Strategy 4. Pricing Valuation 5. Market Risk 6. Liquidity Risk 7. Omni-Channel 8. Supply Chain Risk 9. Do Not Know
  • 31. 31© Cloudera, Inc. All rights reserved. Simulation SDK for Financial Services
  • 32. 32© Cloudera, Inc. All rights reserved. Decision Making
  • 33. 33© Cloudera, Inc. All rights reserved. Decision Making
  • 34. 34© Cloudera, Inc. All rights reserved. Decision Making
  • 35. 35© Cloudera, Inc. All rights reserved. Decision Making
  • 36. 36© Cloudera, Inc. All rights reserved. Decision Making
  • 37. 37© Cloudera, Inc. All rights reserved. Decision Making MACHINE LEARNING
  • 38. 38© Cloudera, Inc. All rights reserved. SIMUDYNE Simulation SDK for Financial Services DATA INGESTION MODELLING VISUALISATION • Easily ingest data from a variety of sources • Platform agnostic SDK deployed on-premise or in the cloud • Certified on Cloudera, the leading Hadoop platform • Machine Models: Combining data and algorithms to create models • Human Models: Describing complex adaptive systems with agent based modelling • Console: Out of the box visualisation tool that helps developers to iterate at pace • Custom UI: Toolkit for bespoke dashboards, putting the power of advanced analytics in the hands of senior decision makers
  • 39. 39© Cloudera, Inc. All rights reserved. Solution Brief: https://www.cloudera.com/content/dam/www/marketing/resources/solution-briefs/cloudera-and-simudyne- computational-simulation-solution.pdf.landing.html Demo Video: Simulating a Bank’s Mortgage Book over 10 years on Cloudera https://www.youtube.com/watch?v=vrg9r_cgeso Cloudera: Solution Lead: Dr. Richard Harmon, rharmon@cloudera.com Marketing Lead: Mihaela Risca, mihaela@cloudera.com Simudyne Sales Lead: Rodney Taylor, rodney@simudyne.com Marketing Lead: Dahlia Lamy, dahlia@simudyne.com Resources
  • 40. 40© Cloudera, Inc. All rights reserved. Thank you