クックパッドでのemr利用事例

6,024 views
5,906 views

Published on

11/12/15に行われたEMR勉強会で使ったスライドです #emrstudy_jp

Published in: Technology, Business
0 Comments
17 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,024
On SlideShare
0
From Embeds
0
Number of Embeds
1,579
Actions
Shares
0
Downloads
45
Comments
0
Likes
17
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • クックパッドの検索ログを、地域別や季節別に見られるサービスです。\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • クックパッドでのemr利用事例

    1. 1. EMR
    2. 2. • (@sasata299)• Hadoop••• Rails, Hadoop, NoSQL
    3. 3. [PR] NoSQL
    4. 4. 1. Hadoop2. EMR3. EMR4.
    5. 5. 1. Hadoop
    6. 6. 2009/9•• MySQL• GROUP BY …• 7000• Hadoop
    7. 7. 2009/10• EC2 Hadoop• Cloudera CDH1• Ruby Hadoop Streaming• 7000 →30• Hadoop
    8. 8. Hadoop++ ←Hadoop ↓MySQL
    9. 9. 2. EMR
    10. 10. 2010/7• Hadoop• Hadoop• SocketTimeoutException …• CDH2• EMR
    11. 11. EMR vs CDH2 AMI (Amazon Machine Image) UPEMRCDH2
    12. 12. EMR vs CDH2 AMI (Amazon Machine Image) UPEMRCDH2
    13. 13. 2010/8• EMR•• Hadoop•
    14. 14. 3. EMR
    15. 15. DB• xx UU• UU•• , etc...
    16. 16. • MySQL MySQL• MySQL EMR - UU - -
    17. 17. EMR• - ○○ xx• Ruby••
    18. 18. 4.
    19. 19. •• - 1• 5 …
    20. 20. [13930, 29011, 39291, ...] # 50000 1000{ ‘139’ => [13930, 13989, 13991, ...], # 50 ‘290’ => [29011, 29098, 29076, ...], # 50 ‘392’ => [39291, 39244, 39251, ...], # 50 ...}
    21. 21. • …• mapper → reducer → finalize• script-runner.jar•
    22. 22. •• IF•• EMR
    23. 23. • EMR••• Hadoop Streaming• :-)
    24. 24. @sasata299

    ×