Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

クックパッドでのemr利用事例

7,261 views

Published on

11/12/15に行われたEMR勉強会で使ったスライドです #emrstudy_jp

Published in: Technology, Business
  • Dating for everyone is here: ❶❶❶ http://bit.ly/369VOVb ❶❶❶
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Sex in your area is here: ♥♥♥ http://bit.ly/369VOVb ♥♥♥
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

クックパッドでのemr利用事例

  1. 1. EMR
  2. 2. • (@sasata299)• Hadoop••• Rails, Hadoop, NoSQL
  3. 3. [PR] NoSQL
  4. 4. 1. Hadoop2. EMR3. EMR4.
  5. 5. 1. Hadoop
  6. 6. 2009/9•• MySQL• GROUP BY …• 7000• Hadoop
  7. 7. 2009/10• EC2 Hadoop• Cloudera CDH1• Ruby Hadoop Streaming• 7000 →30• Hadoop
  8. 8. Hadoop++ ←Hadoop ↓MySQL
  9. 9. 2. EMR
  10. 10. 2010/7• Hadoop• Hadoop• SocketTimeoutException …• CDH2• EMR
  11. 11. EMR vs CDH2 AMI (Amazon Machine Image) UPEMRCDH2
  12. 12. EMR vs CDH2 AMI (Amazon Machine Image) UPEMRCDH2
  13. 13. 2010/8• EMR•• Hadoop•
  14. 14. 3. EMR
  15. 15. DB• xx UU• UU•• , etc...
  16. 16. • MySQL MySQL• MySQL EMR - UU - -
  17. 17. EMR• - ○○ xx• Ruby••
  18. 18. 4.
  19. 19. •• - 1• 5 …
  20. 20. [13930, 29011, 39291, ...] # 50000 1000{ ‘139’ => [13930, 13989, 13991, ...], # 50 ‘290’ => [29011, 29098, 29076, ...], # 50 ‘392’ => [39291, 39244, 39251, ...], # 50 ...}
  21. 21. • …• mapper → reducer → finalize• script-runner.jar•
  22. 22. •• IF•• EMR
  23. 23. • EMR••• Hadoop Streaming• :-)
  24. 24. @sasata299

×