建立創新的現代資料架構，更了解客戶需求 - Get to Know Your Customers – Build and Innovate with a Modern Data Architecture 講師: Ivan Cheng, Solutions Architect, AWS

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Get to Know Your Customers
Build and Innovate with a Modern Data Architecture
Ivan Cheng ( ) AWS Solutions Architect
16th October 2018

2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
A data platform to unleash creativity
Data platform

3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
One home for
all data
– no silos!

4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Traditional Approach
Data warehouse
Business intelligence
OLTP ERP CRM LOB

5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data Lakes Extend the Traditional Approach
Data warehouse
Business intelligence
OLTP ERP CRM LOB
• 3Vs 5Vs 7Vs
• Relational and nonrelational data
• TBs–EBs scale
• Low-cost storage & analytics
Devices Web Sensors Social
Data lake
Big data processing,
real-time, machine learning

6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
When Big
Data meets AI

7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data Lake
on AWS
OLTP ERP CRM LOB Devices Web Sensors Social Images Audio Video
S3
• Unmatched durability, and availability at EB scale
• The most ways to bring data in
• Fastest performance by retrieving subsets of data
Change Data Capture Streaming REST/NFS/Device

8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• PERSONAS
D A T A A N A L Y S T S
B U S I N E S S
P E O P L E
D E V E L O P E R S
D A T A
S C I E N T I S T S
D A T A E N G I N E E R S
B U S I N E S S A N A L Y S T S

9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Right analytic
tools for every
job

10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Machine learningVisualization
Amazon QuickSight
Dashboard
Report
SageMaker
Analytics/Processing
Business user Data Analysts
Redshift
Data engineer Data scientist
SQL
Athena
Hadoop ecosystem
EMR
Application
Software engineer
Ai Service
Rekognition
• Schema on read instead of on write
• EMRFS, S3FS
• 2x as many integrations with partners
Data Lake
on AWS
S3
GUI Python

11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Experiment
quickly &
frequently
Train
DeployEvaluate
More Data

12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster
Data Lake
on AWS
S3

13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster
Cluster Cluster
Cluster Cluster
Data Lake
on AWS
S3

14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
FINRA – Analytics in the cloud
0
2000
4000
6000
8000
10000
12000 2016-10-17T02
2016-10-17T05
2016-10-17T08
2016-10-17T11
2016-10-17T14
2016-10-17T17
2016-10-17T20
2016-10-17T23
2016-10-18T02
2016-10-18T05
2016-10-18T08
2016-10-18T11
2016-10-18T14
2016-10-18T17
2016-10-18T20
2016-10-18T23
2016-10-19T02
2016-10-19T05
2016-10-19T08
2016-10-19T11
2016-10-19T14
2016-10-19T17
2016-10-19T20
2016-10-19T23
2016-10-20T02
2016-10-20T05
2016-10-20T08
2016-10-20T11
2016-10-20T14
2016-10-20T17
2016-10-20T20
2016-10-20T23
2016-10-21T02
2016-10-21T05
2016-10-21T08
2016-10-21T11
2016-10-21T14
2016-10-21T17
2016-10-21T20
2016-10-21T23
2016-10-22T02
2016-10-22T07
2016-10-24T03
2016-10-24T13
2016-10-24T20
2016-10-26T22
ComputeNodes
Hour of Day
Old On-Prem
EMR

15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Modern Data Architecture

16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Speed (Real-time)
IngestData
sources
Scale (Batch) Serving
Data scientists
Data analysts
Business users
Engagement platforms
Automation/events
Transactions
ERP
Connected
devices
Social media
Web logs /
clickstream

17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Let’s run an imaginary
company

18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Speed (Real-time)
IngestData
sources
Scale (Batch) Serving
How is the customer review on the new smart watch
so far by customer segments? i.e. rate, design, quality

19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Speed (Real-time)
IngestData
sources
Scale (Batch)
Application
Data warehouse
Redshift
NFS
CRM
Serving
How is the customer review on the new smart watch
so far by customer segments? i.e. rate, design, quality
Data
Data
Lake
CRM
Visualization
Amazon
QuickSight
Dashboard
Report

20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Speed (Real-time)
IngestData
sources
Scale (Batch)
Application
Data warehouse
Redshift
NFS
CRM
Serving
Data
Data
Lake
CRM
Visualization
Amazon
QuickSight
Dashboard
Report
Can I look at supplier performance over time?
How do quality issue impact cost of production?

21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Speed (Real-time)
IngestData
sources
Scale (Batch)
Application
Data warehouse
Redshift
NFS
ERP
CRM
Analytics
JDBC
ODBC Console
Query
Console
Serving
Can I look at supplier performance over time?
How do quality issue impact cost of production?
Data
Data
Lake
CRM ERP
Direct Query
Athena

22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Speed (Real-time)
IngestData
sources
Scale (Batch)
Data warehouse
Redshift
CRM
Serving
What is the demand forecast of the new smart watch
in each geo?
Data
Data
Lake
ERP

23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Speed (Real-time)
IngestData
sources
Scale (Batch)
Data warehouse
Redshift
CRM
Serving
What is the demand forecast of the new smart watch
in each geo?
Data
Data
Lake
ERP
Transaction
DMS
(CDC)
Sales

24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Speed (Real-time)
IngestData
sources
Scale (Batch)
CRM
Serving
Data
Data
Lake
ERP
Transaction
DMS
(CDC)
Sales
Modeling
Notebook
Time series forecasting
with DeepAR, ARIMA
Modeling
Sagemaker
What is the demand forecast of the new smart watch
in each geo?

25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Start small and iterate

26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Speed (Real-time)
IngestData
sources
Scale (Batch)
CRM
Serving
Data
Data
Lake
ERP
Sales
which products do the existing customers will use in
the next month?

27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Speed (Real-time)
IngestData
sources
Scale (Batch)
CRM
Serving
Data
Data
Lake
ERP
Sales
which products do the existing customers will use in
the next month?
Firehose
Sales
Social
Modeling
Sagemaker
Events
Modeling
Notebook
Feature
engineering
Clickstream
Social
Device events

28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Speed (Real-time)
IngestData
sources
Scale (Batch)
CRM
Serving
Data
Data
Lake
ERP
Sales
which products their existing customers will use in the
next month?
Firehose
Sandbox
Sales
Social
Modeling
Sagemaker
Events
Modeling
Notebook
Feature
engineering
Clickstream
Social
Device events

29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Speed (Real-time)
IngestData
sources
Scale (Batch)
CRM
Serving
Data
Data
Lake
ERP
Sales
which products their existing customers will use in the
next month?
Clickstream
Events
Firehose
Sales
Social
Social
Modeling
Sagemaker
Events
Modeling
Notebook
Analytics/ML Model
Device events

30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Sandbox – fast , cheap, low risk

31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Speed (Real-time)
IngestData
sources
Scale (Batch) Serving
Clickstream
Events
Social
Application
NFS
ERP
CRM
Transaction
DMS
(CDC)
Data
Data
Lake
ML/Analytics
Insights

32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Speed (Real-time)
IngestData
sources
Scale (Batch) Serving
Clickstream
Events
Social
Video
IoT
Events
Action
Process Insights
ML Inference
Stream
Application
Notify the team when the VIPs arrive the store, send a
welcome note.

33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Speed (Real-time)
IngestData
sources
Scale (Batch) Serving
Clickstream
Events
Social
Video
IoT
Events
Action
Process Insights
ML Inference
Stream
Application
Application
NFS
ERP
CRM
Transaction
DMS
(CDC)
Data
Data
Lake
ML/Analytics
Modeling
Notebook
Visualization
Amazon
QuickSight
Dashboard
Report
Analytics
JDBC
ODBC Console
Query
Console
Insights

34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
A data platform to unleash creativity
Right analytic tools for every job
One home for all data – no silos! Experiment quickly & frequently
Flexibility to ingest any type of data
Self-service platforms without gatekeepers
Start small and iterate
Sandbox – fast , cheap, low risk

35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
If you knew
the state of every touchpoints/devices
in your customer journey in real time,
and had infinite compute & storage available,
what problems would you solve?

36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you