This document provides tips and recommendations for using AWS services for data analytics. It discusses using DynamoDB DAX for caching, Kinesis for streaming data, S3 for storage, and Redshift for analytics. It also recommends visualizing data using services like Elasticsearch, Kibana, QuickSight, and open source BI tools. The document concludes by offering tips on join-less analytics and time-sliced tables to improve performance.

1. AW S
Hajime Sano, Marketing & Data Technologist
Data Analytics & CRM Center, B to C Unit, Nikkei Inc.

2. 1.
2.
3. TIPS

4. •
• AWS
DynamoDB DAX
•
• www.linkedin.com/in/hsano

5. 1.
2.
3. ≠

6. 1.
2.
3. TIPS

7. +

9. frequency √ volume

10. 1 2 3 4 5 6 7 8 9 10 11 12 13 14
= LOG2(F√V)

11. • 10
•
•
•
• FT …

12. • GROUP BY 50
•
• Latency
•
•

13. 2014 2015 2016 2017 2018

14. •
•
•
•

15. AT L A S …
•
•
•
• 10 1
•

16. 4

17. R D B
H A D O O P
B I
D A S H B O A R D

19. R D B
H A D O O P
B I
D A S H B O A R D

20. 1.
2.
3. TIPS

21. EndpointTracking Enrichment Consumers S3 AzkabanSQS Kinesis S3 Redshift
Consumers ESS3
ParserAdobe Analytics
Dynamo DB Dynamo DB
S3
Kinesis→ES
DataFeed
Kinesis→S3 S3→RS
E B / E C 2
Rundeck

22. Kinesis S3 Redshift
Athena
Spectrum
ES Kibana
Analytics
Firehose
Quick Sight
(R)
Jupyter
(py)
OSS BI/DS
B I

23. AW S
SQS Kinesis S3 Redshift

24. S Q S
•
-
-
-

25. K I N E S I S
• Kinesis Stream
-
- 7
-
• Firehose Analytics

26. S 3
•
• Redshift
• S3
- Redshift Athena Spectrum
-

27. R E D S H I F T
•
• postgres SQL
- BI
•
-

28. E L A S T I C S E A R C H E C 2
• Elasticsearch Service EC2
- c4.8xlarge x 3
- r4.xlarge x 1
•
- X-Pack Graph ML
- ES OS JVM

29. • Kinesis
- 200ms
- Consumer
• Elasticsearch
-
- INDEX 1
• Redshift
- 15 20

30. 10 20
10 1

31. …
AT L A S
A
G

32. 1.
2.
3. TIPS

33. •
• Lightning Talk
• SQL Data Dojo
•
•
• / IF

34. R Studio Server Shiny Anaconda(JupyterHub)
Chartio
Re:dash Kibana
DOMO
DataSquad Maia
KPI Screens

35. • 2
1.
2.
•
- Slack
-

37. J O I N - L E S S
• OUTER JOIN
-
- JOIN
• 1
- 1
- SELECT

38. J O I N - L E S S
GET URL URL URL

39. •
- time sliced table UNION ALL
- timestamptz
- WHERE
•
- 3 30 …
-

40. last 7 days 7

41. •
- Redash
…
-
•
- Redash
-
KILL

43. T I P S
1.
2.
3.
4.

44. T H A N K Y O U !

45. Data Technologist Data Scientist

46. Q U E S T I O N S ?