Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Redshift Hajime Sano (MarTech & DataTech) Data Team, B to C Unit, Nikkei Inc.
B to C - Atlas 
 https://aws.amazon.com/jp/solutions/case-studies/nikkei-big-data/
 https://www.slideshare.net/HajimeSano1...
- - - - 3
Data Team - - CRM - 4
Data Team 3 5
1.Redshift 2. 3. 4.Spectrum 5. 6
1.Redshift 2. 3. 4. Spectrum 5. 7
AWS 8 1. Redshift Web App Mail APIs Push 3rd Party APIs
DW 2 - - - 
 - 9 1. Redshift CRM - - - - -
1. 2. 3. 4. 5. 10 1. Redshift
DW 1. 2. 3. 4. 5. 11 1. Redshift
1. 2. 3. 4. 5. 12 1. Redshift
DB 13 1. Redshift
DB 14 1. Redshift ( < 200ms)
DB 15 1. Redshift ( < 2sec) Redshift
DB 16 1. Redshift ( < 30sec) 1 2 Dynamic Template Kibana
DB 17 1. Redshift Hourly 9 64KB TEXT & JSON
18 1. Redshift
Redshift 1. 2. SQL 3. 4. 5. 19 1. Redshift
Redshift 20 1. Redshift 3rd Party
1. Redshift 2. 3. 4. Spectrum 5. 21
Read Heavy ds2.xlarge 14 ds2 EBC Read Heavy 22 2.
(WLM) 25 Excel 1縮 23 2.
Short Query Acceleration (SQA) SQA? - 2017 11 - BI - 15 CLI - `aws redshift modify-cluster-parameter-group .. ` 24 2.
Slack 30 Slack - PID XID - - - 25 2.
- 30 - 30 - - - 26 2.
27 2.
Join-less JOIN 
 JOIN - - 28 2.
- CPU I/O - 2016 ANALYZE COMPRESSION LZO VARCHAR LZO Zstandard(ZSTD) - 2017 4 - ANALYZE COMPRESSION ZSTD - LZO 2 29 2.
30 2.
- - JSON - - - 1 5 6 324 31 2.
Aurora - SQA - 90 89 Amazon Aurora Redshift DB 32 2.
33 2.
1. Redshift 2. 3. 4. Spectrum 5. 34
- - 縮 - - - 35 3.
36 3.
- UTC 9 - 1 5 UNION 7 37 3.
38 3. - 1 - -
- 1 3 1 - 1 INSERT SELECT - 39 3.
40 3.
WITH NO SCHEMA BINDING - CREATE OR REPLACE VIEW - 2017 9 - 
 - 41 3.
1. Redshift 2. 3. 4.Spectrum 5. 42
- 7 30 - 13 - - Redshift 2 1 - - 43 4. Spectrum
Glue Parquet Spectrum 44 4. Spectrum
- Redshift - - - 45 4. Spectrum
S3 46 4. Spectrum
- - S3 - Spectrum S3 Redshift S3 - 47 4. Spectrum
- - - Glue Spark 2.2.1 - timestamptz 48 4. Spectrum
1. Redshift 2. 3. 4. Spectrum 5. 49
UPSERT 1 50 5.
SQA 51 5.
縮 Redshift 52
53
Upcoming SlideShare
Loading in …5
×

Redshiftの位置付けと運用のヒント 日経電子版での事例 （AWS Redshift 祭り）

868 views

Published on

日経電子版の運営たのめのデータ分析基盤における、Redshiftの用途と数百人で利用するための最適化について、構成や設定からガバナンスまで実践していることをご紹介します。2018年5月10日開催、AWS主催 「Redshift祭り」 発表資料です。

Published in: Data & Analytics
no profile picture user

  • Be the first to comment

Redshiftの位置付けと運用のヒント 日経電子版での事例 （AWS Redshift 祭り）

  1. 1. Redshift Hajime Sano (MarTech & DataTech) Data Team, B to C Unit, Nikkei Inc.
  2. 2. B to C - Atlas 
 https://aws.amazon.com/jp/solutions/case-studies/nikkei-big-data/
 https://www.slideshare.net/HajimeSano1/jaws-ug-bigdata-branch-oct- 2017 - BI MA API - 連 2
  3. 3. - - - - 3
  4. 4. Data Team - - CRM - 4
  5. 5. Data Team 3 5
  6. 6. 1.Redshift 2. 3. 4.Spectrum 5. 6
  7. 7. 1.Redshift 2. 3. 4. Spectrum 5. 7
  8. 8. AWS 8 1. Redshift Web App Mail APIs Push 3rd Party APIs
  9. 9. DW 2 - - - 
 - 9 1. Redshift CRM - - - - -
  10. 10. 1. 2. 3. 4. 5. 10 1. Redshift
  11. 11. DW 1. 2. 3. 4. 5. 11 1. Redshift
  12. 12. 1. 2. 3. 4. 5. 12 1. Redshift
  13. 13. DB 13 1. Redshift
  14. 14. DB 14 1. Redshift ( < 200ms)
  15. 15. DB 15 1. Redshift ( < 2sec) Redshift
  16. 16. DB 16 1. Redshift ( < 30sec) 1 2 Dynamic Template Kibana
  17. 17. DB 17 1. Redshift Hourly 9 64KB TEXT & JSON
  18. 18. 18 1. Redshift
  19. 19. Redshift 1. 2. SQL 3. 4. 5. 19 1. Redshift
  20. 20. Redshift 20 1. Redshift 3rd Party
  21. 21. 1. Redshift 2. 3. 4. Spectrum 5. 21
  22. 22. Read Heavy ds2.xlarge 14 ds2 EBC Read Heavy 22 2.
  23. 23. (WLM) 25 Excel 1縮 23 2.
  24. 24. Short Query Acceleration (SQA) SQA? - 2017 11 - BI - 15 CLI - `aws redshift modify-cluster-parameter-group .. ` 24 2.
  25. 25. Slack 30 Slack - PID XID - - - 25 2.
  26. 26. - 30 - 30 - - - 26 2.
  27. 27. 27 2.
  28. 28. Join-less JOIN 
 JOIN - - 28 2.
  29. 29. - CPU I/O - 2016 ANALYZE COMPRESSION LZO VARCHAR LZO Zstandard(ZSTD) - 2017 4 - ANALYZE COMPRESSION ZSTD - LZO 2 29 2.
  30. 30. 30 2.
  31. 31. - - JSON - - - 1 5 6 324 31 2.
  32. 32. Aurora - SQA - 90 89 Amazon Aurora Redshift DB 32 2.
  33. 33. 33 2.
  34. 34. 1. Redshift 2. 3. 4. Spectrum 5. 34
  35. 35. - - 縮 - - - 35 3.
  36. 36. 36 3.
  37. 37. - UTC 9 - 1 5 UNION 7 37 3.
  38. 38. 38 3. - 1 - -
  39. 39. - 1 3 1 - 1 INSERT SELECT - 39 3.
  40. 40. 40 3.
  41. 41. WITH NO SCHEMA BINDING - CREATE OR REPLACE VIEW - 2017 9 - 
 - 41 3.
  42. 42. 1. Redshift 2. 3. 4.Spectrum 5. 42
  43. 43. - 7 30 - 13 - - Redshift 2 1 - - 43 4. Spectrum
  44. 44. Glue Parquet Spectrum 44 4. Spectrum
  45. 45. - Redshift - - - 45 4. Spectrum
  46. 46. S3 46 4. Spectrum
  47. 47. - - S3 - Spectrum S3 Redshift S3 - 47 4. Spectrum
  48. 48. - - - Glue Spark 2.2.1 - timestamptz 48 4. Spectrum
  49. 49. 1. Redshift 2. 3. 4. Spectrum 5. 49
  50. 50. UPSERT 1 50 5.
  51. 51. SQA 51 5.
  52. 52. 縮 Redshift 52
  53. 53. 53

×