SlideShare a Scribd company logo
Agenda
Define the problem
Establish the expected outcome
Dive into each pillar
Determine a Solution
Understand the applicability
Financial
Institutions risk
Loss of
Charterand a host of other penalties through
noncompliance with federal money
laundering legislation.
Big Data Evolution
Legacy Systems Current Systems
Big Data
Advanced Analytics
Timely Info Accurate Thoughtful
Marketing Operations Bankers CEOs
• Next Best Action
• Recommended Interventions
• Lifestyle Yield Management
• Seasonal Personal Impact
• Theft Profiling
• Fraudulent Transaction
Identification
• Remote Shutdown
• Site Monitoring
• Recommended Interventions
• Risky Customer Profiling
• Call Center Monitoring
• Churn Scoring
• Payment System Errors
• Money Laundering
prevention
• Compliance
• Data Entry Intervention
?
Personalization of offers &
banking experience
Risk Reduction &
ComplianceCustomer Churn PreventionFraud Detection
Areas of Opportunity for Financial Analytics
Expected Outcome
$
Big Data
Challenges
Architectural Considerations
Fraud Detection Reference Architecture
Apps data
from devices
News and
other alerts
Solution UX
Provisioning API (Pull)
User Profile Information
Stream Processors
Analytics &
Machine Learning
Business
Integration
Connectors
and
Gateway(s)
User Recent Activity Store
Gateway
Data Lake
Gateway
App Backend
Data Path
Optional solution component
Main solution component
Thin Client
Presentation & Business ConnectivityData Processing, Analytics and ManagementDevice Connectivity
Personal
mobile
devices
Trades
and/or
transactions
Business
systems
Reference Architecture with Azure Services
Solution UX
Provisioning API
User Profile Information
Stream Processors
Analytics &
Machine Learning
Business
Integration
Connectors
and
Gateway(s)
User Recent Activity Store Store
Data Lake
Gateway
App Backend
Personal
mobile
devices
Business
systems
Presentation & Business ConnectivityData Processing, Analytics and ManagementDevice Connectivity
Apps data
from devices
News and
other alerts
Gateway
Data Path
Optional solution component
Main solution component
Thin Client
Trades
and/or
transactions
Demo
Woodgrove Financial
User Profile and Metadata Stores
App Backend Solution UX
Provisioning API
User Profile Information
Stream Processors
Analytics &
Machine Learning
Business
Integration
Connectors
and
Gateway(s)
User Recent Activity Information
Data Lake
Gateway
(Kafka,
IoT Hub,
Event Hubs)
Data Path
Optional solution component
Main solution component
Metadata
Store
Gateway
Trades
and/or
transactions
Thin Client
News and
other alerts
Apps data
from
devices
Device Identity, Registry and State Stores
Metadata Store
Authority for all registered sources
Stores identity information and authentication secrets
User Profile Information
Indexed list of all Users and their demographics – Secure, Governed, Audit Controlled
Contains discovery and reference data related to Users
Can define a schema model or use a vertical industry standard schema for metadata
Can contain structured metadata and links to externally stored operational data
User Recent Activity
Contains operational data related to the Users’ most recent activities:
- “Last known values” for each User
- Aggregated or computed values
- Stream of device data events containing Geo location and Time based tagging
Stream Processors
App Backend Solution UX
Provisioning API
Identity and Registry Stores
Stream Processors
Analytics &
Machine Learning
Business
Integration
Connectors
and
Gateway(s)
Device State Store
Data Lake
Data Path
Optional solution component
Main solution component
Gateway
Trades
and/or
transactions
Thin Client
News and
other alerts
Apps data
from
devices
Stream Processing: Data Flow
After ingress through the Gateway (Ingestion), the flow of data
through the system is facilitated by data pumps and analytics tasks
Data flow can be driven by:
• Apache Storm on Azure HDInsight
• Apache Spark on Azure HDInsight
• Azure Stream Analytics
• Custom Event Processors
Each can perform tasks
in flight:
• Data aggregation
• Data enrichment
• Complex event processing
… and can output data
to:
• Azure Data Lake
• Azure Blobs/Tables
• HDInsight / HBase
• Azure SQL DB
• Time Series Databases
• Event Hub
• Service Bus Queues
Stream Processor Examples
Queue
Device Registry Store
Device Metadata
Processor
Data Lake
Device State Store
Device State
Processor
Notification
Processor
Raw Telemetry Processor
App Backend
Rules Processor
Event Hub
Stream Transformation
Processor
Secondary Stream
Processor
Data Path
Optional solution component
Main solution component
Gateway
Trades
and/or
transactions
Thin Client
News and
other alerts
Apps data
from
devices
App Backend
App Backend Solution UX
Provisioning API
Identity and Registry Stores
Stream Processors
Analytics &
Machine Learning
Business
Integration
Connectors
and
Gateway(s)
Device State Store
Storage
Cloud
Gateway
Data Path
Optional solution component
Main solution component
Gateway
Trades
and/or
transactions
Thin Client
News and
other alerts
Apps data
from
devices
High-Scale Compute Models
Scale-appropriate compute models
Actor Frameworks / Service Fabric Reliable Actors: distributed
compute fabric hosting device actors.
Service Fabric Reliable Collections: highly available with
replicated and local state management.
Azure Batch: job scheduling and compute management for
highly parallelizable compute workloads.
Simple programming logic in vastly scalable
compute nodes
Data Analytics
App Backend Solution UX
Provisioning API
Identity and Registry Stores
Stream Processors
Analytics &
Machine Learning
Business
Integration
Connectors
and
Gateway(s)
Device State Store
Data Lake
Cloud
Gateway
Data Path
Optional solution component
Main solution component
Gateway
Trades
and/or
transactions
Thin Client
News and
other alerts
Apps data
from
devices
Data Analytics
Event Hub
NRT Events
Stream Processing
(ASA, Storm or
Spark)
Alerts
Batch Events
Fetching &
Updating
Reference Data
Interceptor (Rules)
Spark
Hive/Pig
U-SQL
Azure Data Lake Store Azure Data Lake Analytics
SQL DB
ML
Reports and
Dashboards
Real Time Scoring
Training ML Models
Relational Data
Data Analytics
Real-Time Analysis
Aggregation/Reduction, Temporal Queries, State
Correlation, Threshold Detection, Alerting
Data-At-Rest Analysis
Time-Series, Map/Reduce, Correlation
Machine Learning
Pattern Detection, Behavior Prediction
Plausibility Analysis, Anomaly and Fraud Detection
Power BI
HDInsight
Stream Analytics
Data Factory
Machine Learning
Presentation and Business Connectivity
App Backend Solution UX
Provisioning API
Identity and Registry Stores
Stream Processors
Analytics &
Machine Learning
Business
Integration
Connectors
and
Gateway(s)
Device State Store
Data Lake
Cloud
Gateway
Data Path
Optional solution component
Main solution component
Gateway
Trades
and/or
transactions
Thin Client
News and
other alerts
Apps data
from
devices
WebHDFS
YARN
U-SQL
Analytics Service HDInsight
(managed Hadoop Clusters)
Analytics
Store
Azure Data Lake
Cortana Intelligence Suite
Action
People
Automated
Systems
Apps
Web
Mobile
Bots
Intelligence
Dashboards &
Visualizations
Cortana
Bot
Framework
Cognitive
Services
Power BI
Information
Management
Event Hubs
Data Catalog
Data Factory
Machine Learning
and Analytics
HDInsight
(Hadoop and
Spark)
Stream Analytics
Intelligence
Data Lake
Analytics
Machine
Learning
Big Data Stores
SQL Data
Warehouse
Data Lake Store
Data
Sources
Apps
Sensors
and
devices
Data
Reference Architecture with Azure Services
Solution UX
Provisioning API
User Profile Information
Stream Processors
Analytics &
Machine Learning
Business
Integration
Connectors
and
Gateway(s)
User Recent Activity Store Store
Data Lake
Gateway
App Backend
Personal
mobile
devices
Business
systems
Presentation & Business ConnectivityData Processing, Analytics and ManagementDevice Connectivity
Apps data
from devices
News and
other alerts
Gateway
Data Path
Optional solution component
Main solution component
Thin Client
Trades
and/or
transactions
Money Laundering Prevention
Fraud Detection
$ $ $
¥
Placement Layering Integration
Process
Know your
Customer
Transaction
Monitoring
Pattern
Detection
Machine Learning
Decision Tree Classification
Cluster
Analysis
Cloud
Anti-Money Laundering
Power BI
Fund monitoring
dashboard
Big Data Storage for
Multiple Sources
HDInsight Azure Data
Lake
Azure Data
Warehouse
SQL Azure Azure Machine
Learning
SQL
Financial Data
Real-time fraud detection feedback
Information Services
HDInsight Streaming
Analytics
Data Science Modeling
• Similar to linear regression
• Weights independent variables
• Useful with categorical
independent variable
• Offers coefficients to inform
management decision-making
• Very useful with internal
analytical teams to interpret
data
• Useful for diagnosing gaps in
data and customer outreach
• Helps drive understanding of
demand drivers
• Uses decision trees & votes
• Forest
• Compares results between
various outcomes
• Votes upon outcomes
• Evaluates based upon a
series of logical questions or
“forest”
• Jungle
• Useful when a forest
produces too many logical
branches
• Produces a series of weighted
edges and nodes
• Trained in input data
• Useful for complex tasks, like
speech recognition when
allowed to train in depth
• Very good with complex
interactions
• Enables retailers to better
identify behaviour patterns &
certain shopping activities
Reference Architecture & Azure Services
Solution UX
Provisioning API
User Profile Information
Stream Processors
Analytics &
Machine Learning
Business
Integration
Connectors
and
Gateway(s)
User Recent Activity Store Store
Data Lake
Gateway
App Backend
Personal
mobile
devices
Business
systems
Presentation & Business ConnectivityData Processing, Analytics and ManagementDevice Connectivity
Apps data
from devices
News and
other alerts
Gateway
Data Path
Optional solution component
Main solution component
Thin Client
Trades
and/or
transactions
nishant.thacker@microsoft.com
© 2016 Microsoft Corporation. All rights reserved.

More Related Content

What's hot

BIG DATA and USE CASES
BIG DATA and USE CASESBIG DATA and USE CASES
BIG DATA and USE CASES
Bhaskara Reddy Sannapureddy
 
Big Data Analytics
Big Data AnalyticsBig Data Analytics
Big Data Analytics
Ghulam Imaduddin
 
AWS Big Data Platform
AWS Big Data PlatformAWS Big Data Platform
AWS Big Data Platform
Amazon Web Services
 
How to govern and secure a Data Mesh?
How to govern and secure a Data Mesh?How to govern and secure a Data Mesh?
How to govern and secure a Data Mesh?
confluent
 
Databricks Platform.pptx
Databricks Platform.pptxDatabricks Platform.pptx
Databricks Platform.pptx
Alex Ivy
 
Big data architectures and the data lake
Big data architectures and the data lakeBig data architectures and the data lake
Big data architectures and the data lake
James Serra
 
Big Data and Analytics on AWS
Big Data and Analytics on AWS Big Data and Analytics on AWS
Big Data and Analytics on AWS
Amazon Web Services
 
Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...
Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...
Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...
Amazon Web Services
 
Data Lakehouse, Data Mesh, and Data Fabric (r2)
Data Lakehouse, Data Mesh, and Data Fabric (r2)Data Lakehouse, Data Mesh, and Data Fabric (r2)
Data Lakehouse, Data Mesh, and Data Fabric (r2)
James Serra
 
Big data ppt
Big data pptBig data ppt
Big data ppt
Thirunavukkarasu Ps
 
Advanced Analytics Platform for Big Data Analytics
Advanced Analytics Platform for Big Data AnalyticsAdvanced Analytics Platform for Big Data Analytics
Advanced Analytics Platform for Big Data Analytics
Arvind Sathi
 
Big Data Architecture and Design Patterns
Big Data Architecture and Design PatternsBig Data Architecture and Design Patterns
Big Data Architecture and Design Patterns
John Yeung
 
Overview of Big data(ppt)
Overview of Big data(ppt)Overview of Big data(ppt)
Overview of Big data(ppt)
Shatavisha Roy Chowdhury
 
Data Governance in a big data era
Data Governance in a big data eraData Governance in a big data era
Data Governance in a big data era
Pieter De Leenheer
 
Data Architecture for Solutions.pdf
Data Architecture for Solutions.pdfData Architecture for Solutions.pdf
Data Architecture for Solutions.pdf
Alan McSweeney
 
Data Mesh for Dinner
Data Mesh for DinnerData Mesh for Dinner
Data Mesh for Dinner
Kent Graziano
 
Hadoop And Their Ecosystem ppt
 Hadoop And Their Ecosystem ppt Hadoop And Their Ecosystem ppt
Hadoop And Their Ecosystem ppt
sunera pathan
 
Hadoop Tutorial For Beginners
Hadoop Tutorial For BeginnersHadoop Tutorial For Beginners
Hadoop Tutorial For Beginners
Dataflair Web Services Pvt Ltd
 
Big Data: The 4 Layers Everyone Must Know
Big Data: The 4 Layers Everyone Must KnowBig Data: The 4 Layers Everyone Must Know
Big Data: The 4 Layers Everyone Must Know
Bernard Marr
 
Architect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh ArchitectureArchitect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh Architecture
Databricks
 

What's hot (20)

BIG DATA and USE CASES
BIG DATA and USE CASESBIG DATA and USE CASES
BIG DATA and USE CASES
 
Big Data Analytics
Big Data AnalyticsBig Data Analytics
Big Data Analytics
 
AWS Big Data Platform
AWS Big Data PlatformAWS Big Data Platform
AWS Big Data Platform
 
How to govern and secure a Data Mesh?
How to govern and secure a Data Mesh?How to govern and secure a Data Mesh?
How to govern and secure a Data Mesh?
 
Databricks Platform.pptx
Databricks Platform.pptxDatabricks Platform.pptx
Databricks Platform.pptx
 
Big data architectures and the data lake
Big data architectures and the data lakeBig data architectures and the data lake
Big data architectures and the data lake
 
Big Data and Analytics on AWS
Big Data and Analytics on AWS Big Data and Analytics on AWS
Big Data and Analytics on AWS
 
Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...
Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...
Big Data Analytics Architectural Patterns and Best Practices (ANT201-R1) - AW...
 
Data Lakehouse, Data Mesh, and Data Fabric (r2)
Data Lakehouse, Data Mesh, and Data Fabric (r2)Data Lakehouse, Data Mesh, and Data Fabric (r2)
Data Lakehouse, Data Mesh, and Data Fabric (r2)
 
Big data ppt
Big data pptBig data ppt
Big data ppt
 
Advanced Analytics Platform for Big Data Analytics
Advanced Analytics Platform for Big Data AnalyticsAdvanced Analytics Platform for Big Data Analytics
Advanced Analytics Platform for Big Data Analytics
 
Big Data Architecture and Design Patterns
Big Data Architecture and Design PatternsBig Data Architecture and Design Patterns
Big Data Architecture and Design Patterns
 
Overview of Big data(ppt)
Overview of Big data(ppt)Overview of Big data(ppt)
Overview of Big data(ppt)
 
Data Governance in a big data era
Data Governance in a big data eraData Governance in a big data era
Data Governance in a big data era
 
Data Architecture for Solutions.pdf
Data Architecture for Solutions.pdfData Architecture for Solutions.pdf
Data Architecture for Solutions.pdf
 
Data Mesh for Dinner
Data Mesh for DinnerData Mesh for Dinner
Data Mesh for Dinner
 
Hadoop And Their Ecosystem ppt
 Hadoop And Their Ecosystem ppt Hadoop And Their Ecosystem ppt
Hadoop And Their Ecosystem ppt
 
Hadoop Tutorial For Beginners
Hadoop Tutorial For BeginnersHadoop Tutorial For Beginners
Hadoop Tutorial For Beginners
 
Big Data: The 4 Layers Everyone Must Know
Big Data: The 4 Layers Everyone Must KnowBig Data: The 4 Layers Everyone Must Know
Big Data: The 4 Layers Everyone Must Know
 
Architect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh ArchitectureArchitect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh Architecture
 

Similar to Big Data Application Architectures - Fraud Detection

WebAction-Sami Abkay
WebAction-Sami AbkayWebAction-Sami Abkay
WebAction-Sami Abkay
Inside Analysis
 
WebAction In-Memory Computing Summit 2015
WebAction In-Memory Computing Summit 2015WebAction In-Memory Computing Summit 2015
WebAction In-Memory Computing Summit 2015
WebAction
 
50 Shades of Data - Dutch Oracle Architects Platform (February 2018)
50 Shades of Data - Dutch Oracle Architects Platform (February 2018)50 Shades of Data - Dutch Oracle Architects Platform (February 2018)
50 Shades of Data - Dutch Oracle Architects Platform (February 2018)
Lucas Jellema
 
A Winning Strategy for the Digital Economy
A Winning Strategy for the Digital EconomyA Winning Strategy for the Digital Economy
A Winning Strategy for the Digital Economy
Eric Kavanagh
 
Agile Mumbai 2022 - Balvinder Kaur & Sushant Joshi | Real-Time Insights and A...
Agile Mumbai 2022 - Balvinder Kaur & Sushant Joshi | Real-Time Insights and A...Agile Mumbai 2022 - Balvinder Kaur & Sushant Joshi | Real-Time Insights and A...
Agile Mumbai 2022 - Balvinder Kaur & Sushant Joshi | Real-Time Insights and A...
AgileNetwork
 
Microsoft SQL Server 2008 R2 and BizTalk Server Presentation
Microsoft SQL Server 2008 R2 and BizTalk Server PresentationMicrosoft SQL Server 2008 R2 and BizTalk Server Presentation
Microsoft SQL Server 2008 R2 and BizTalk Server Presentation
Microsoft Private Cloud
 
Big Data Application Architectures - IoT
Big Data Application Architectures - IoTBig Data Application Architectures - IoT
Big Data Application Architectures - IoT
DataWorks Summit/Hadoop Summit
 
The 4th Generation Kingland platform
The 4th Generation Kingland platformThe 4th Generation Kingland platform
The 4th Generation Kingland platform
Kingland
 
Big Data Analytics Webinar
Big Data Analytics WebinarBig Data Analytics Webinar
Big Data Analytics Webinar
Eckerson Group
 
Event Driven Architecture (EDA), November 2, 2006
Event Driven Architecture (EDA), November 2, 2006Event Driven Architecture (EDA), November 2, 2006
Event Driven Architecture (EDA), November 2, 2006
Tim Bass
 
Azure IoT Suite
Azure IoT Suite Azure IoT Suite
Azure IoT Suite
Samir Arezki ☁
 
Machine Data Analytics
Machine Data AnalyticsMachine Data Analytics
Machine Data Analytics
Nicolas Morales
 
Overview of Composable SaaS Models
Overview of Composable SaaS ModelsOverview of Composable SaaS Models
Overview of Composable SaaS Models
Gabe Pei
 
Enabling Next Gen Analytics with Azure Data Lake and StreamSets
Enabling Next Gen Analytics with Azure Data Lake and StreamSetsEnabling Next Gen Analytics with Azure Data Lake and StreamSets
Enabling Next Gen Analytics with Azure Data Lake and StreamSets
Streamsets Inc.
 
Analytics in Your Enterprise
Analytics in Your EnterpriseAnalytics in Your Enterprise
Analytics in Your Enterprise
WSO2
 
Relevant Pension Portalv4
Relevant Pension Portalv4Relevant Pension Portalv4
Relevant Pension Portalv4
ebstlr
 
Business Analytics Paradigm Change
Business Analytics Paradigm ChangeBusiness Analytics Paradigm Change
Business Analytics Paradigm Change
Dmitry Anoshin
 
Incentius - Portfolio of Capabilities
Incentius - Portfolio of CapabilitiesIncentius - Portfolio of Capabilities
Incentius - Portfolio of Capabilities
Sujeet Pillai
 
Big Data on AWS - Toronto FSI Symposium - October 2016
Big Data on AWS - Toronto FSI Symposium - October 2016Big Data on AWS - Toronto FSI Symposium - October 2016
Big Data on AWS - Toronto FSI Symposium - October 2016
Amazon Web Services
 
Technologies
TechnologiesTechnologies
Technologies
guest6cdabe
 

Similar to Big Data Application Architectures - Fraud Detection (20)

WebAction-Sami Abkay
WebAction-Sami AbkayWebAction-Sami Abkay
WebAction-Sami Abkay
 
WebAction In-Memory Computing Summit 2015
WebAction In-Memory Computing Summit 2015WebAction In-Memory Computing Summit 2015
WebAction In-Memory Computing Summit 2015
 
50 Shades of Data - Dutch Oracle Architects Platform (February 2018)
50 Shades of Data - Dutch Oracle Architects Platform (February 2018)50 Shades of Data - Dutch Oracle Architects Platform (February 2018)
50 Shades of Data - Dutch Oracle Architects Platform (February 2018)
 
A Winning Strategy for the Digital Economy
A Winning Strategy for the Digital EconomyA Winning Strategy for the Digital Economy
A Winning Strategy for the Digital Economy
 
Agile Mumbai 2022 - Balvinder Kaur & Sushant Joshi | Real-Time Insights and A...
Agile Mumbai 2022 - Balvinder Kaur & Sushant Joshi | Real-Time Insights and A...Agile Mumbai 2022 - Balvinder Kaur & Sushant Joshi | Real-Time Insights and A...
Agile Mumbai 2022 - Balvinder Kaur & Sushant Joshi | Real-Time Insights and A...
 
Microsoft SQL Server 2008 R2 and BizTalk Server Presentation
Microsoft SQL Server 2008 R2 and BizTalk Server PresentationMicrosoft SQL Server 2008 R2 and BizTalk Server Presentation
Microsoft SQL Server 2008 R2 and BizTalk Server Presentation
 
Big Data Application Architectures - IoT
Big Data Application Architectures - IoTBig Data Application Architectures - IoT
Big Data Application Architectures - IoT
 
The 4th Generation Kingland platform
The 4th Generation Kingland platformThe 4th Generation Kingland platform
The 4th Generation Kingland platform
 
Big Data Analytics Webinar
Big Data Analytics WebinarBig Data Analytics Webinar
Big Data Analytics Webinar
 
Event Driven Architecture (EDA), November 2, 2006
Event Driven Architecture (EDA), November 2, 2006Event Driven Architecture (EDA), November 2, 2006
Event Driven Architecture (EDA), November 2, 2006
 
Azure IoT Suite
Azure IoT Suite Azure IoT Suite
Azure IoT Suite
 
Machine Data Analytics
Machine Data AnalyticsMachine Data Analytics
Machine Data Analytics
 
Overview of Composable SaaS Models
Overview of Composable SaaS ModelsOverview of Composable SaaS Models
Overview of Composable SaaS Models
 
Enabling Next Gen Analytics with Azure Data Lake and StreamSets
Enabling Next Gen Analytics with Azure Data Lake and StreamSetsEnabling Next Gen Analytics with Azure Data Lake and StreamSets
Enabling Next Gen Analytics with Azure Data Lake and StreamSets
 
Analytics in Your Enterprise
Analytics in Your EnterpriseAnalytics in Your Enterprise
Analytics in Your Enterprise
 
Relevant Pension Portalv4
Relevant Pension Portalv4Relevant Pension Portalv4
Relevant Pension Portalv4
 
Business Analytics Paradigm Change
Business Analytics Paradigm ChangeBusiness Analytics Paradigm Change
Business Analytics Paradigm Change
 
Incentius - Portfolio of Capabilities
Incentius - Portfolio of CapabilitiesIncentius - Portfolio of Capabilities
Incentius - Portfolio of Capabilities
 
Big Data on AWS - Toronto FSI Symposium - October 2016
Big Data on AWS - Toronto FSI Symposium - October 2016Big Data on AWS - Toronto FSI Symposium - October 2016
Big Data on AWS - Toronto FSI Symposium - October 2016
 
Technologies
TechnologiesTechnologies
Technologies
 

More from DataWorks Summit/Hadoop Summit

Running Apache Spark & Apache Zeppelin in Production
Running Apache Spark & Apache Zeppelin in ProductionRunning Apache Spark & Apache Zeppelin in Production
Running Apache Spark & Apache Zeppelin in Production
DataWorks Summit/Hadoop Summit
 
State of Security: Apache Spark & Apache Zeppelin
State of Security: Apache Spark & Apache ZeppelinState of Security: Apache Spark & Apache Zeppelin
State of Security: Apache Spark & Apache Zeppelin
DataWorks Summit/Hadoop Summit
 
Unleashing the Power of Apache Atlas with Apache Ranger
Unleashing the Power of Apache Atlas with Apache RangerUnleashing the Power of Apache Atlas with Apache Ranger
Unleashing the Power of Apache Atlas with Apache Ranger
DataWorks Summit/Hadoop Summit
 
Enabling Digital Diagnostics with a Data Science Platform
Enabling Digital Diagnostics with a Data Science PlatformEnabling Digital Diagnostics with a Data Science Platform
Enabling Digital Diagnostics with a Data Science Platform
DataWorks Summit/Hadoop Summit
 
Revolutionize Text Mining with Spark and Zeppelin
Revolutionize Text Mining with Spark and ZeppelinRevolutionize Text Mining with Spark and Zeppelin
Revolutionize Text Mining with Spark and Zeppelin
DataWorks Summit/Hadoop Summit
 
Double Your Hadoop Performance with Hortonworks SmartSense
Double Your Hadoop Performance with Hortonworks SmartSenseDouble Your Hadoop Performance with Hortonworks SmartSense
Double Your Hadoop Performance with Hortonworks SmartSense
DataWorks Summit/Hadoop Summit
 
Hadoop Crash Course
Hadoop Crash CourseHadoop Crash Course
Hadoop Crash Course
DataWorks Summit/Hadoop Summit
 
Data Science Crash Course
Data Science Crash CourseData Science Crash Course
Data Science Crash Course
DataWorks Summit/Hadoop Summit
 
Apache Spark Crash Course
Apache Spark Crash CourseApache Spark Crash Course
Apache Spark Crash Course
DataWorks Summit/Hadoop Summit
 
Dataflow with Apache NiFi
Dataflow with Apache NiFiDataflow with Apache NiFi
Dataflow with Apache NiFi
DataWorks Summit/Hadoop Summit
 
Schema Registry - Set you Data Free
Schema Registry - Set you Data FreeSchema Registry - Set you Data Free
Schema Registry - Set you Data Free
DataWorks Summit/Hadoop Summit
 
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
DataWorks Summit/Hadoop Summit
 
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
DataWorks Summit/Hadoop Summit
 
Mool - Automated Log Analysis using Data Science and ML
Mool - Automated Log Analysis using Data Science and MLMool - Automated Log Analysis using Data Science and ML
Mool - Automated Log Analysis using Data Science and ML
DataWorks Summit/Hadoop Summit
 
How Hadoop Makes the Natixis Pack More Efficient
How Hadoop Makes the Natixis Pack More Efficient How Hadoop Makes the Natixis Pack More Efficient
How Hadoop Makes the Natixis Pack More Efficient
DataWorks Summit/Hadoop Summit
 
HBase in Practice
HBase in Practice HBase in Practice
HBase in Practice
DataWorks Summit/Hadoop Summit
 
The Challenge of Driving Business Value from the Analytics of Things (AOT)
The Challenge of Driving Business Value from the Analytics of Things (AOT)The Challenge of Driving Business Value from the Analytics of Things (AOT)
The Challenge of Driving Business Value from the Analytics of Things (AOT)
DataWorks Summit/Hadoop Summit
 
Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
Breaking the 1 Million OPS/SEC Barrier in HOPS HadoopBreaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
DataWorks Summit/Hadoop Summit
 
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
DataWorks Summit/Hadoop Summit
 
Backup and Disaster Recovery in Hadoop
Backup and Disaster Recovery in Hadoop Backup and Disaster Recovery in Hadoop
Backup and Disaster Recovery in Hadoop
DataWorks Summit/Hadoop Summit
 

More from DataWorks Summit/Hadoop Summit (20)

Running Apache Spark & Apache Zeppelin in Production
Running Apache Spark & Apache Zeppelin in ProductionRunning Apache Spark & Apache Zeppelin in Production
Running Apache Spark & Apache Zeppelin in Production
 
State of Security: Apache Spark & Apache Zeppelin
State of Security: Apache Spark & Apache ZeppelinState of Security: Apache Spark & Apache Zeppelin
State of Security: Apache Spark & Apache Zeppelin
 
Unleashing the Power of Apache Atlas with Apache Ranger
Unleashing the Power of Apache Atlas with Apache RangerUnleashing the Power of Apache Atlas with Apache Ranger
Unleashing the Power of Apache Atlas with Apache Ranger
 
Enabling Digital Diagnostics with a Data Science Platform
Enabling Digital Diagnostics with a Data Science PlatformEnabling Digital Diagnostics with a Data Science Platform
Enabling Digital Diagnostics with a Data Science Platform
 
Revolutionize Text Mining with Spark and Zeppelin
Revolutionize Text Mining with Spark and ZeppelinRevolutionize Text Mining with Spark and Zeppelin
Revolutionize Text Mining with Spark and Zeppelin
 
Double Your Hadoop Performance with Hortonworks SmartSense
Double Your Hadoop Performance with Hortonworks SmartSenseDouble Your Hadoop Performance with Hortonworks SmartSense
Double Your Hadoop Performance with Hortonworks SmartSense
 
Hadoop Crash Course
Hadoop Crash CourseHadoop Crash Course
Hadoop Crash Course
 
Data Science Crash Course
Data Science Crash CourseData Science Crash Course
Data Science Crash Course
 
Apache Spark Crash Course
Apache Spark Crash CourseApache Spark Crash Course
Apache Spark Crash Course
 
Dataflow with Apache NiFi
Dataflow with Apache NiFiDataflow with Apache NiFi
Dataflow with Apache NiFi
 
Schema Registry - Set you Data Free
Schema Registry - Set you Data FreeSchema Registry - Set you Data Free
Schema Registry - Set you Data Free
 
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
Building a Large-Scale, Adaptive Recommendation Engine with Apache Flink and ...
 
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
Real-Time Anomaly Detection using LSTM Auto-Encoders with Deep Learning4J on ...
 
Mool - Automated Log Analysis using Data Science and ML
Mool - Automated Log Analysis using Data Science and MLMool - Automated Log Analysis using Data Science and ML
Mool - Automated Log Analysis using Data Science and ML
 
How Hadoop Makes the Natixis Pack More Efficient
How Hadoop Makes the Natixis Pack More Efficient How Hadoop Makes the Natixis Pack More Efficient
How Hadoop Makes the Natixis Pack More Efficient
 
HBase in Practice
HBase in Practice HBase in Practice
HBase in Practice
 
The Challenge of Driving Business Value from the Analytics of Things (AOT)
The Challenge of Driving Business Value from the Analytics of Things (AOT)The Challenge of Driving Business Value from the Analytics of Things (AOT)
The Challenge of Driving Business Value from the Analytics of Things (AOT)
 
Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
Breaking the 1 Million OPS/SEC Barrier in HOPS HadoopBreaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
Breaking the 1 Million OPS/SEC Barrier in HOPS Hadoop
 
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
From Regulatory Process Verification to Predictive Maintenance and Beyond wit...
 
Backup and Disaster Recovery in Hadoop
Backup and Disaster Recovery in Hadoop Backup and Disaster Recovery in Hadoop
Backup and Disaster Recovery in Hadoop
 

Recently uploaded

Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Wask
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
Hiike
 
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Tatiana Kojar
 
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Precisely
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
Javier Junquera
 
dbms calicut university B. sc Cs 4th sem.pdf
dbms  calicut university B. sc Cs 4th sem.pdfdbms  calicut university B. sc Cs 4th sem.pdf
dbms calicut university B. sc Cs 4th sem.pdf
Shinana2
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
saastr
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
marufrahmanstratejm
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
Data Hops
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 

Recently uploaded (20)

Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
 
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
 
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
 
dbms calicut university B. sc Cs 4th sem.pdf
dbms  calicut university B. sc Cs 4th sem.pdfdbms  calicut university B. sc Cs 4th sem.pdf
dbms calicut university B. sc Cs 4th sem.pdf
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 

Big Data Application Architectures - Fraud Detection

  • 1.
  • 2. Agenda Define the problem Establish the expected outcome Dive into each pillar Determine a Solution Understand the applicability
  • 3. Financial Institutions risk Loss of Charterand a host of other penalties through noncompliance with federal money laundering legislation.
  • 4. Big Data Evolution Legacy Systems Current Systems Big Data Advanced Analytics Timely Info Accurate Thoughtful
  • 5. Marketing Operations Bankers CEOs • Next Best Action • Recommended Interventions • Lifestyle Yield Management • Seasonal Personal Impact • Theft Profiling • Fraudulent Transaction Identification • Remote Shutdown • Site Monitoring • Recommended Interventions • Risky Customer Profiling • Call Center Monitoring • Churn Scoring • Payment System Errors • Money Laundering prevention • Compliance • Data Entry Intervention ? Personalization of offers & banking experience Risk Reduction & ComplianceCustomer Churn PreventionFraud Detection Areas of Opportunity for Financial Analytics
  • 9. Fraud Detection Reference Architecture Apps data from devices News and other alerts Solution UX Provisioning API (Pull) User Profile Information Stream Processors Analytics & Machine Learning Business Integration Connectors and Gateway(s) User Recent Activity Store Gateway Data Lake Gateway App Backend Data Path Optional solution component Main solution component Thin Client Presentation & Business ConnectivityData Processing, Analytics and ManagementDevice Connectivity Personal mobile devices Trades and/or transactions Business systems
  • 10. Reference Architecture with Azure Services Solution UX Provisioning API User Profile Information Stream Processors Analytics & Machine Learning Business Integration Connectors and Gateway(s) User Recent Activity Store Store Data Lake Gateway App Backend Personal mobile devices Business systems Presentation & Business ConnectivityData Processing, Analytics and ManagementDevice Connectivity Apps data from devices News and other alerts Gateway Data Path Optional solution component Main solution component Thin Client Trades and/or transactions
  • 12. User Profile and Metadata Stores App Backend Solution UX Provisioning API User Profile Information Stream Processors Analytics & Machine Learning Business Integration Connectors and Gateway(s) User Recent Activity Information Data Lake Gateway (Kafka, IoT Hub, Event Hubs) Data Path Optional solution component Main solution component Metadata Store Gateway Trades and/or transactions Thin Client News and other alerts Apps data from devices
  • 13. Device Identity, Registry and State Stores Metadata Store Authority for all registered sources Stores identity information and authentication secrets User Profile Information Indexed list of all Users and their demographics – Secure, Governed, Audit Controlled Contains discovery and reference data related to Users Can define a schema model or use a vertical industry standard schema for metadata Can contain structured metadata and links to externally stored operational data User Recent Activity Contains operational data related to the Users’ most recent activities: - “Last known values” for each User - Aggregated or computed values - Stream of device data events containing Geo location and Time based tagging
  • 14. Stream Processors App Backend Solution UX Provisioning API Identity and Registry Stores Stream Processors Analytics & Machine Learning Business Integration Connectors and Gateway(s) Device State Store Data Lake Data Path Optional solution component Main solution component Gateway Trades and/or transactions Thin Client News and other alerts Apps data from devices
  • 15. Stream Processing: Data Flow After ingress through the Gateway (Ingestion), the flow of data through the system is facilitated by data pumps and analytics tasks Data flow can be driven by: • Apache Storm on Azure HDInsight • Apache Spark on Azure HDInsight • Azure Stream Analytics • Custom Event Processors Each can perform tasks in flight: • Data aggregation • Data enrichment • Complex event processing … and can output data to: • Azure Data Lake • Azure Blobs/Tables • HDInsight / HBase • Azure SQL DB • Time Series Databases • Event Hub • Service Bus Queues
  • 16. Stream Processor Examples Queue Device Registry Store Device Metadata Processor Data Lake Device State Store Device State Processor Notification Processor Raw Telemetry Processor App Backend Rules Processor Event Hub Stream Transformation Processor Secondary Stream Processor Data Path Optional solution component Main solution component Gateway Trades and/or transactions Thin Client News and other alerts Apps data from devices
  • 17. App Backend App Backend Solution UX Provisioning API Identity and Registry Stores Stream Processors Analytics & Machine Learning Business Integration Connectors and Gateway(s) Device State Store Storage Cloud Gateway Data Path Optional solution component Main solution component Gateway Trades and/or transactions Thin Client News and other alerts Apps data from devices
  • 18. High-Scale Compute Models Scale-appropriate compute models Actor Frameworks / Service Fabric Reliable Actors: distributed compute fabric hosting device actors. Service Fabric Reliable Collections: highly available with replicated and local state management. Azure Batch: job scheduling and compute management for highly parallelizable compute workloads. Simple programming logic in vastly scalable compute nodes
  • 19. Data Analytics App Backend Solution UX Provisioning API Identity and Registry Stores Stream Processors Analytics & Machine Learning Business Integration Connectors and Gateway(s) Device State Store Data Lake Cloud Gateway Data Path Optional solution component Main solution component Gateway Trades and/or transactions Thin Client News and other alerts Apps data from devices
  • 20. Data Analytics Event Hub NRT Events Stream Processing (ASA, Storm or Spark) Alerts Batch Events Fetching & Updating Reference Data Interceptor (Rules) Spark Hive/Pig U-SQL Azure Data Lake Store Azure Data Lake Analytics SQL DB ML Reports and Dashboards Real Time Scoring Training ML Models Relational Data
  • 21. Data Analytics Real-Time Analysis Aggregation/Reduction, Temporal Queries, State Correlation, Threshold Detection, Alerting Data-At-Rest Analysis Time-Series, Map/Reduce, Correlation Machine Learning Pattern Detection, Behavior Prediction Plausibility Analysis, Anomaly and Fraud Detection Power BI HDInsight Stream Analytics Data Factory Machine Learning
  • 22. Presentation and Business Connectivity App Backend Solution UX Provisioning API Identity and Registry Stores Stream Processors Analytics & Machine Learning Business Integration Connectors and Gateway(s) Device State Store Data Lake Cloud Gateway Data Path Optional solution component Main solution component Gateway Trades and/or transactions Thin Client News and other alerts Apps data from devices
  • 23. WebHDFS YARN U-SQL Analytics Service HDInsight (managed Hadoop Clusters) Analytics Store Azure Data Lake
  • 24. Cortana Intelligence Suite Action People Automated Systems Apps Web Mobile Bots Intelligence Dashboards & Visualizations Cortana Bot Framework Cognitive Services Power BI Information Management Event Hubs Data Catalog Data Factory Machine Learning and Analytics HDInsight (Hadoop and Spark) Stream Analytics Intelligence Data Lake Analytics Machine Learning Big Data Stores SQL Data Warehouse Data Lake Store Data Sources Apps Sensors and devices Data
  • 25. Reference Architecture with Azure Services Solution UX Provisioning API User Profile Information Stream Processors Analytics & Machine Learning Business Integration Connectors and Gateway(s) User Recent Activity Store Store Data Lake Gateway App Backend Personal mobile devices Business systems Presentation & Business ConnectivityData Processing, Analytics and ManagementDevice Connectivity Apps data from devices News and other alerts Gateway Data Path Optional solution component Main solution component Thin Client Trades and/or transactions
  • 26. Money Laundering Prevention Fraud Detection $ $ $ ¥ Placement Layering Integration Process Know your Customer Transaction Monitoring Pattern Detection Machine Learning Decision Tree Classification Cluster Analysis
  • 27. Cloud Anti-Money Laundering Power BI Fund monitoring dashboard Big Data Storage for Multiple Sources HDInsight Azure Data Lake Azure Data Warehouse SQL Azure Azure Machine Learning SQL Financial Data Real-time fraud detection feedback Information Services HDInsight Streaming Analytics
  • 28. Data Science Modeling • Similar to linear regression • Weights independent variables • Useful with categorical independent variable • Offers coefficients to inform management decision-making • Very useful with internal analytical teams to interpret data • Useful for diagnosing gaps in data and customer outreach • Helps drive understanding of demand drivers • Uses decision trees & votes • Forest • Compares results between various outcomes • Votes upon outcomes • Evaluates based upon a series of logical questions or “forest” • Jungle • Useful when a forest produces too many logical branches • Produces a series of weighted edges and nodes • Trained in input data • Useful for complex tasks, like speech recognition when allowed to train in depth • Very good with complex interactions • Enables retailers to better identify behaviour patterns & certain shopping activities
  • 29. Reference Architecture & Azure Services Solution UX Provisioning API User Profile Information Stream Processors Analytics & Machine Learning Business Integration Connectors and Gateway(s) User Recent Activity Store Store Data Lake Gateway App Backend Personal mobile devices Business systems Presentation & Business ConnectivityData Processing, Analytics and ManagementDevice Connectivity Apps data from devices News and other alerts Gateway Data Path Optional solution component Main solution component Thin Client Trades and/or transactions
  • 31. © 2016 Microsoft Corporation. All rights reserved.

Editor's Notes

  1. Today’s financial services market is highly competitive, complex, and difficult. Particularly with today’s legislation, it is becoming increasingly more important to reduce risk, increase compliance, detect fraud, retain customers, and know your customers better.
  2. Over the course of time, data is evolving. Legacy systems have evolved into the current systems of today. Systems like As systems change and evolve to become more timely, accurate, and thoughtful greater opportunities for return on system investments are realized. Big Data and Advanced Analytics systems offer superior return-on-investment.
  3. A host of opportunities exist to utilize this technology suite in the arena of financial analytics. Left to Right Personalization of offers and tailored banking experiences allow opportunities to engage with customers in a positive way based on their data. Next best action offers surface suspected needs and offer the opportunity for sales lift. Recommended interventions allow for programmatic intervention based upon customer churn. Lifestyle yield management allows for bankers to tailor plans & recommendations based on the life state of customers (retiree versus recent graduate) Many customers of financial institutions are impacted by seasonality in their employment or lifestyle. By recognizing and making offers to these customers based on their needs, banks can increase their profitability. Fraud Detection allows banks to reduce risk and their cost of operations. Theft profiling & fraudulent transaction detection allow for proactive intervention & prevention of fraud. Remote shutdown & site monitoring allow banks to reactively intervene in ATM and physical locations in the event of fraud. Customer Churn Prevention increases revenue by increasing customer lifetime. Churn scoring allows for identification of at-risk customers, and is the basis for all other churn applications. Personalized interventions allow for customized per-customer interventions to be created based upon churn scoring & personalization. Similarly risky customers can be profiled to identify characteristics and intervene. Call center monitoring allows for use of perceptual intelligence to be applied to identify churn behavior based on call center operations. Risk Reduction & Compliance are a key way institutions can reduce operational costs. Prevention of Payment System errors and Money Laundering prevention can substantially reduce risk to fines & lost funds. Data entry is similarly a source of risk; identifying and preventing data entry errors can save time & money.
  4. With the large amounts of data potentially available for analysis, managing data flows efficiently can be a challenge. Huge amounts of data to process (volume) A mixture of structured and unstructured data (variety) New data that’s generated extremely frequently (velocity) Data quality so that it can be trusted (veracity)
  5. Between 2.17 & 3.61 Trillion dollars are laundered annually. The process of detecting and preventing money laundering at a perceptual level is fairly straightforward, but implementation of systems to detect and prevent money laundering are incredibly complex Money laundering has 3 primary processes; placement, layering, & integration. Placement is where funds from illegal activities are introduced to the financial system. Layering is the suite of transactions designed to clean the money. Integration is when funds are redistributed back through business transactions. To prevent this malicious process from happening, process controls can be implemented to prevent money laundering. As you can see, these primarily fall into the 3 categories indicated in process, and can be supported by the various machine learning algorithms mentioned on the right. (Click Again) using Cortana analytics in the process and to drive the machine learning behind money laundering prevention can prevent money laundering.
  6. This diagram shows both the hot path & cold path outlined. The hot path informs directly from the information services layer as data is entered from field services. The cold path involves data storage & use of machine learning to inform hot path development and directly predict into the visualization layer. It is important to stress both the hot path and cold path of the solution here, as both are required to yield superior results. Storm, Spark, & Azure Stream Analytics are the tools useful for the hot path implementations of the rules gleaned from ML. Azure data factory is the orchestration tool. These are referenced again in the Machine Learning layer, as further development here is used to increase hot path value. HDInsight & Azure Data Lake are big data stores. Azure DW and SQL Azure are relational data stores for extracting further value from the big data stores. Azure Machine Learning provides a platform for data evaluation, data science and prediction. This is where the real value for the solution is created.
  7. Data Factory  http://azure.microsoft.com/en-us/services/data-factory/ Data Catalog http://azure.microsoft.com/en-us/services/data-catalog/ Event Hubs http://azure.microsoft.com/en-us/services/event-hubs/ Stream Analytics http://azure.microsoft.com/en-us/services/stream-analytics/
  8. Data Lake http://azure.microsoft.com/en-us/campaigns/data-lake/ SQL Data Warehouse http://azure.microsoft.com/en-us/services/sql-data-warehouse/ HDInsight http://azure.microsoft.com/en-us/services/hdinsight/ Stream Analytics http://azure.microsoft.com/en-us/services/stream-analytics/
  9. Machine Learning https://studio.azureml.net/ HDInsight http://azure.microsoft.com/en-us/services/hdinsight/ Stream Analytics http://azure.microsoft.com/en-us/services/stream-analytics/
  10. Power BI https://powerbi.microsoft.com/ Azure Web Sites http://azure.microsoft.com/en-us/services/app-service/web/