Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
WIFI SSID:Spark+AISummit | Password: UnifiedDataAnalytics
Omar Hommos & Rodel van Rooijen, Adyen
Building a Modern
FinTech Big Data
Infrastructure
#UnifiedDataAnalytics #SparkAISum...
Adyen
Payments Processor
Tech company w/ banking license
International customers
Omnichannel
Example payment
Example payment
Example payment
Example payment
Example payment
Adyen
Decision Points
Update account?
Fraud?
How to retry?
When to retry?
Do I know him? Weird behaviour?
How to validate card?
...
Update account?
Fraud?
How to retry?
When to retry?
Do I know him? Weird behaviour?
How to validate card?
Strong authentic...
Live Platform
RevenueProtect ShopperDNA RevenueAccelerate MonitoringTokenization
Live Platform
RevenueProtect ShopperDNA RevenueAccelerate MonitoringTokenization
Highly Available Low Latency Sensitive Da...
Enter Data Platform
Live
Platform
Data
Platform
Live Platform <> Data Platform
ETL ML in production
Magic ✨
Products 💸
Live Platform <> Data Platform
ETL ML in production
Magic ✨
Products 💸
ETL Pipelines
Avro
Filebeat
Financial StreamAnalytics Stream
Avro/Parquet
Derived tables
Live Platform <> Data Platform
ETL ML in production
Magic ✨
Products 💸
ML in Production
BeamPush
Push configurations
Alfred
Serve models
ML in Production: Alfred
Alfred
Serve models
Unified Framework
• Build models
• Track experiments
• Re-train
• Rollout
• Se...
Build/Experiment (Re)-training
ML in Production: Alfred
Score Serve,
Rollout
Score
Score
ML in Production: Alfred Rollout
Registered Ghost Test Live Retired Failed
Model available
in live platform
Score and log,...
Scalable Time Series
Forecasting and Monitoring
using Apache Spark and
ElasticSearch at Adyen
Today
17h20
G102
We are hiring!
www.adyen.com/careers
DON’T FORGET TO RATE
AND REVIEW THE SESSIONS
SEARCH SPARK + AI SUMMIT
You’ve finished this document.
Download and read it offline.
Upcoming SlideShare
What to Upload to SlideShare
Next
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

Share

Building a Modern FinTech Big Data Infrastructure

Download to read offline

The cloud is now the first choice for large-scale analytics, but organizations that have sunk investment into Hadoop on-premises are also challenged with maintaining operations. This can make a move to modern analytics platforms like Spark difficult or impossible. Learn about innovations for large-scale migration that can take full advantage of cloud-based analytics without disrupting operations.

Building a Modern FinTech Big Data Infrastructure

  1. 1. WIFI SSID:Spark+AISummit | Password: UnifiedDataAnalytics
  2. 2. Omar Hommos & Rodel van Rooijen, Adyen Building a Modern FinTech Big Data Infrastructure #UnifiedDataAnalytics #SparkAISummit
  3. 3. Adyen Payments Processor Tech company w/ banking license International customers Omnichannel
  4. 4. Example payment
  5. 5. Example payment
  6. 6. Example payment
  7. 7. Example payment
  8. 8. Example payment
  9. 9. Adyen
  10. 10. Decision Points Update account? Fraud? How to retry? When to retry? Do I know him? Weird behaviour? How to validate card? Strong authentication?
  11. 11. Update account? Fraud? How to retry? When to retry? Do I know him? Weird behaviour? How to validate card? Strong authentication? Data-Driven Products and Services RevenueProtect ShopperDNA RevenueAccelerate MonitoringTokenization * few examples of many web services
  12. 12. Live Platform RevenueProtect ShopperDNA RevenueAccelerate MonitoringTokenization
  13. 13. Live Platform RevenueProtect ShopperDNA RevenueAccelerate MonitoringTokenization Highly Available Low Latency Sensitive Data
  14. 14. Enter Data Platform Live Platform Data Platform
  15. 15. Live Platform <> Data Platform ETL ML in production Magic ✨ Products 💸
  16. 16. Live Platform <> Data Platform ETL ML in production Magic ✨ Products 💸
  17. 17. ETL Pipelines Avro Filebeat Financial StreamAnalytics Stream Avro/Parquet Derived tables
  18. 18. Live Platform <> Data Platform ETL ML in production Magic ✨ Products 💸
  19. 19. ML in Production BeamPush Push configurations Alfred Serve models
  20. 20. ML in Production: Alfred Alfred Serve models Unified Framework • Build models • Track experiments • Re-train • Rollout • Serve • …
  21. 21. Build/Experiment (Re)-training ML in Production: Alfred Score Serve, Rollout Score Score
  22. 22. ML in Production: Alfred Rollout Registered Ghost Test Live Retired Failed Model available in live platform Score and log, Async Requests to new model, Specified % Requests to new model, 100% Trash, failed AT, Not to be used Fallback Replace by newer model Performance Tracking (PT) PT PT
  23. 23. Scalable Time Series Forecasting and Monitoring using Apache Spark and ElasticSearch at Adyen Today 17h20 G102
  24. 24. We are hiring! www.adyen.com/careers
  25. 25. DON’T FORGET TO RATE AND REVIEW THE SESSIONS SEARCH SPARK + AI SUMMIT
  • tushar_kale

    Nov. 10, 2019

The cloud is now the first choice for large-scale analytics, but organizations that have sunk investment into Hadoop on-premises are also challenged with maintaining operations. This can make a move to modern analytics platforms like Spark difficult or impossible. Learn about innovations for large-scale migration that can take full advantage of cloud-based analytics without disrupting operations.

Views

Total views

434

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

25

Shares

0

Comments

0

Likes

1

×