Strata Data Conference New York 2019 Speakers: Vitaliy Baklikov, DBS Bank Dipti Borkar, Alluxio

1. Enabling big data & AI workloads on the
object store at DBS
Vitaliy Baklikov (DBS Bank)
Dipti Borkar (Alluxio)
Strata Data New York
2019

2. About DBS
• Headquartered in Singapore
• Largest bank in South East Asia
• Present in 18 markets globally,
including 6 priority markets
• Singapore, Hong Kong, China,
India, Indonesia and Taiwan
• We have a very cool digiBank app
• And lots lots lots of data systems

3. DBS – Top 20 Business Transformations of the Last Decade

4. AWS EnginesOnprem Engines
HDFS
Object Store
Evolution of Data Platforms at DBS
Generation 1
• Boxed data
• Monolithic/Closed Systems
• Proprietary HW/SW
• Data for Targeted Use Cases
Generation 2
• Big Data Explosion
• Hadoop Data Lakes
• Commodity HW and Hadoop
Ecosystem
• Compute tied to Storage
Generation 3
• Data Democratization
• Cloud Native platform… Hybrid! Multi!
• Open Source Engines
• Burst compute in the cloud with data
on-prem for compliance
• AI/ML Centric
Teradata
Informatica
SAS
HadoopTeradata
Informatica
SAS
Teradata
Informatica
SAS
Hadoop

5. Challenges
1. Data Lake built on local Object Store
– Expensive rename operation
– Object listing is slow
– Variable performance
– Data locality is gone
2. Multiple Data Silos
3. Limited on-premise compute capacity
– Legacy ITIL processes for Infra provisioning
– No dynamic scale out/in

6. Alluxio at DBS
Mount HDFS from other
platforms into common
Alluxio cluster
Unified
Namespace
Object store
Analytics
Hybrid
cloud bursting
Caching layer for hot
data to speed up Presto
and Spark jobs
Extend Alluxio cluster into
AWS VPC
Run EMR for model training
and bring the results back to
on-prem

7. Advanced Analytics on Object Storage
The Use Case
• Cash-in-Transit use case
• Forecast cash replenishment schedule for each ATM
• Produce a delivery graph by 4am each morning
The Challenges
• Strict fixed SLA
• Need to load all history data for the forecasting model… daily!

8. Alluxio at DBS
Mount HDFS from other
platforms into common
Alluxio cluster
Unified
Namespace
Object store
Analytics
Hybrid
cloud bursting
Caching layer for hot
data to speed up Presto
and Spark jobs
Extend Alluxio cluster into
AWS VPC
Run EMR for model training
and bring the results back to
on-prem

9. Burst processing into the Cloud
The Use Case
– Call Center project
– Millions of calls annually
– Why do our customers call us?
– What do they do before picking up the phone?
• Reconstruct customer journey
• Predict the reason for the call
The Challenges
– Transcript quality
– Need lots of compute
• >30TB of clickstream, transaction, customer, and product data
• >20TB of audio files
– Need dynamic compute for training and analysis
– Data needs to reside on-prem for compliance

10. High Level Architecture
Data Orchestration Data Orchestration

11. Next Steps
1. Use Cloud offerings for Speech-to-Text
2. Move more workloads to the cloud
3. Extend to other clouds

12. Data Orchestration for the Cloud

13. Data Orchestration for the Cloud
Java File API HDFS Interface S3 Interface REST APIPOSIX Interface
HDFS Driver Swift Driver S3 Driver NFS Driver
Enable innovation with any frameworks
running on data stored anywhere
Data Analyst
Data Engineer
Storage Ops
Data Scientist
Lines of Business

14. Data Orchestration for the Cloud
Java File API HDFS Interface S3 Interface REST APIPOSIX Interface
HDFS Driver Swift Driver S3 Driver NFS Driver
Enable innovation with any frameworks
running on data stored anywhere

15. Problem: HDFS cluster is compute-
bound & complex to maintain
AWS Public Cloud IaaS
Spark Presto Hive TensorFlow
Alluxio Data Orchestration and Control Service
On Premises
Connectivity
Datacenter
Spark Presto Hive
Tensor
Flow
Alluxio Data Orchestration and Control Service
Barrier 1: Prohibitive network latency
and bandwidth limits
• Makes hybrid analytics unfeasible
Barrier 2: Copying data to cloud
• Difficult to maintain copies
• Data security and governance
• Costs of another silo
Step 1: Hybrid Cloud for Burst Compute Capacity
• Orchestrates compute access to on-prem data
• Working set of data, not FULL set of data
• Local performance
• Scales elastically
• On-Prem Cluster Offload (both Compute & I/O)
Step 2: Online Migration of Data Per Policy
• Flexible timing to migrate, with less dependencies
• Instead of hard switch over, migrate at own pace
• Moves the data per policy – e.g. last 7 days
“Zero-copy” bursting to the cloud

16. Using Alluxio with AWS EMR
Presto Hive
Instances
Metadata &
Data cache
Presto Hive
Metadata &
Data cache
HDFS HDFSEMRF
S
EMRF
S
Compute-driven
Continuous sync
Compute-driven
Continuous sync

17. Spark Presto Hive TensorFlow
RAM
Framework
Read file /trades/us
Bucket Trades Bucket Customers
Data requests
Feature Highlight: Data Caching for faster compute
Read file /trades/us again Read file /trades/top
Read file /trades/top
Variable latency
with throttling
Read file /trades/us again Read file /trades/top
Read file /trades/top
Read file /trades/us again Read file /trades/top
Read file /trades/top
Read file /trades/us again Read file /trades/top
Read file /trades/top
Read file /trades/us again

18. Spark Presto Hive TensorFlow
RAM
SSD
Disk
Framework
Bucket Trades Bucket Customers
Data requests
Feature Highlight - Intelligent Tiering for resource efficiency
Read file /customers/145
Out of memory
Variable latency
with throttling
Data moved to another tier

19. Spark Presto Hive TensorFlow
RAM
SSD
Disk
Framework
New Trades
Policy Defined Move data > 90 days old to
Feature Highlight – Policy-driven Data Management
S3 Standard
Policy interval : Every day
Policy applied everyday
HDFS

20. Alluxio
MasterZookeeper /
RAFT
Standby
Master
WAN
Alluxio
Client
Alluxio
Client
Alluxio
Worker
RAM / SSD / HDD
Alluxio
Worker
RAM / SSD / HDD
Alluxio Reference Architecture
…
…
Application
Application
Under Store 1
Under Store 2

21. Questions?