Euronext, the 1st European stock exchange with €3.7 trillion in market cap, built a governed data lake based on Amazon AWS to analyze data from one of the largest databases in Europe enriched with 1.5 billion new messages every day. Euronext uses Talend and AWS services - Amazon S3, Amazon Redshift and Amazon EMR for better agility, elasticity, breadth of functionality and cost savings, compared to the previous Netezza-based solution, while guaranteeing data governance and regulatory compliance.
2. 22
Unleashing analytics with AWS
Euronext at glance
Business drivers and prerequisites
for a cloud first strategy
Implementing the governed data
lake
Business outcomes and next steps
01
02
03
04
05
7. 7 #TalendConnect
• 1st European stock-exchange
• Amsterdam, Brussels, Dublin,
Lisbon, Paris
• 1300 corporate issuers
• €3.7trn market cap
• 6 national regulators
• Home of the CAC 40, BEL 20,
AEX, PSI 20
8. #TalendConnect8
ISSUES AT HAND
Daily Post Trade Processing
AVG Time 24 June 2016
(BREXIT VOTE)
6H 12H
With high data processing
constraints
• Avg order round time < 100 µs
• 1.5 B messages per day
• 400B records on chase trading table
9. #TalendConnect9
BUSINESS DRIVERS
FOR DATA GOVERNANCE
In-depth analysis New data products
Real time operations Mergers & acquisitions
GDPR, MIFID IIAgility for growth
Analytics Monetization
Real-time
monitoring
Consolidation
AI Regulatory
compliance
Data
Governance
INTERNALDEMAND
EXTERNALEXPECTATIONS
10. #TalendConnect10
DATA STRATEGY PROGRAM
Cloud Transformation
Program
Cloud Setup
Cloud Strategy
Data Project
Coordination
DWH Replacement
Data Shop
Analytics
Data Portal
Surveillance
Data Governance
Data Lake Laying out the foundations for future use cases Data Lake
20182019
Data Quality
Data Classification
Data Ownership
Privacy by Design
Data Classification
Retention Policy
Reference Mgmt.
Data Strategy
Data Breach Mgmt.
Data Loss Prevention
GDPR
Info Security
11. #TalendConnect11
EURONEXT DATALAKE
Data Reporting Data Science Monetization Real Time Monitoring
Euronext Data Lake
Orders Reference DataTrade Post Trade 3rd Party Streaming
Euronext Cloud Data
Warehouse
Data Sandboxes with
AI Capabilities
Euronext Data
Shop Surveillance
13. #TalendConnect13
EURONEXT 7 DATA
GOVERNANCE PRINCIPLES
Data mapping
Data protection
Data lineage
Data quality
Regulatory compliance
Change management
Data Catalog
MDM
Enterprise repositories
15. #TalendConnect15
SOME KEY BENEFITS
AGILITY
CI/CD pipelines
Full serverless and/ephemeral resources
Innovation
Amazon Redshift vs Netezza
Use of AWS Batch with Spot instances
TCO: At equal budget with 10x more data usage (stream and storage)
Serverless orchestration with Step Functions/ AWS Batch and Amazon EMR
Amazon S3 Storage/Use of Amazon Redshift Spectrum
Kafka and Data Analytics
Every single identified need for this data lake has its corresponding
service on AWS
COST SAVING
BREADTH OF
FUNCTIONALITY
ELASTICITY
16. #TalendConnect16
INSTANT DATA INSIGHTS
AT SCALE
Data under control
for compliance and
monetization
Elasticity
and limitless
scale-up
On demand
Data Science
capabilities