AWS Retail & CPG Day - Yan So Data Analysis on AWS 17 Oct 2019

1. -
Data Analysis
On AWS
크로키닷컴㈜ 소성운
2019.10.17

2. (Yan So)
AWSKRUG #datascience
E: 13imso@gmail.com
L: https://www.linkedin.com/in/yanso

3. -
-
- Amazon EMR:
- Amazon Kinesis:
- AWS Glue: ,
- AWS S3:
- Amazon Personalize:

4. 3400
10,000+
1600
(MAU) 250
GB &

5. ,
,

6. ,
,

7. ,
,

8. ,
,

9. ,
,

10. •
•
• Hadoop + Spark / Presto
•
•
•
•
•
• Time/Event Driven
•

11. S3
Amazon
S3
Amazon
S3
Amazon
S3
Amazon
RDS
Amazon
DynamoDB
Client
Amazon Kinesis
Data Firehose
AWS
Lambda
User
Activity
Digestion

12. S3 ,
Amazon
S3
Amazon
S3
Amazon
S3
AWS Glue
Crawler
AWS Glue
Catalog
Amazon
RDS
Amazon
DynamoDB
Client
Amazon Kinesis
Data Firehose
AWS
Lambda
User
Activity
Digestion

13. S3 ,
Amazon
S3
Amazon
S3
Amazon
S3
AWS Glue
Crawler
AWS Glue
Catalog
Amazon
RDS
Amazon
DynamoDB
Client
Amazon Kinesis
Data Firehose
AWS
Lambda
User
Activity
Digestion

14. S3 ,
Amazon
S3
Amazon
S3
Amazon
S3
AWS Glue
Crawler
AWS Glue
Catalog
Amazon
RDS
Amazon
DynamoDB
Client
Amazon Kinesis
Data Firehose
AWS
Lambda
User
Activity
Digestion

15. S3 ,
Amazon
S3
Amazon
S3
Amazon
S3
AWS Glue
Crawler
AWS Glue
Catalog
Amazon
RDS
Amazon
DynamoDB
Client
Amazon Kinesis
Data Firehose
AWS
Lambda
User
Activity
Digestion
Datalake on AWS

16. EMR
AWS Glue
Crawler
Amazon
EMR
AWS Glue
Catalog
Amazon
RDS
Amazon
DynamoDB
Amazon
S3
Client
Amazon Kinesis
Data Firehose
Amazon
S3
Amazon
S3
AWS
Lambda
User
Activity
Digestion

17. EMR + Amazon Athena, QuickSight
Amazon
RDS
Amazon
DynamoDB
Amazon
S3
Client
Amazon Kinesis
Data Firehose
Amazon
S3
Amazon
S3
AWS
Lambda
AWS Glue
Crawler
Amazon
EMR
AWS Glue
Catalog
Amazon
Athena
Amazon
QuickSight
User
Activity
Digestion

18. Tableau
Rstudio
Zeppelin Jupyter
Amazon
EMR
AWS Glue
Catalog
Amazon
S3
- /
-
-

19. -
- TF-IDF
(Term Frequency Inverse Document Frequency)
-
10 20 30

20. -
-
-
-
- ,
10
10,000 100,000 ?

21. - Computer Vision + Machine Learning
Computer Vision
-
-
-
-
&
Machine Learning
- Classification Model
input: RGB
output: Labelled Colour
Amazon
S3
:
URL

22. • Rule-Based
•
•
• Collaborative Filtering, Factorization Machine
•
• (Amazon Sagemaker)
• Deep Learning - Recurrent Neural Networks
•
•

23. • ?
•
• Cold Start - ,
•
•
•
• ,
• ML

24. 🌟 Amazon Personalize 🌟
• Amazon.com Machine Learning
• API Endpoint
• AutoML
• ,

25. 🌟 Amazon Personalize 🌟
• (~1m)
•
• (~1h)
• AWS EMR + Zeppelin + S3
• 2 => User-Item Interaction
• ( 1) ? => Users
=> Campaign#1
• ( 2) ? => Item
=> Campaign#2
• (~10h)
• AutoML
•
• Offline Test:
• Online Test: Solution Metric

26. 🌟 Amazon Personalize 🌟

27. 🌟 Amazon Personalize 🌟
Metric

28. 🌟 Amazon Personalize 🌟

29. -
Thank you
for your attention