Hadoop導入事例 in クックパッド
Upcoming SlideShare
Loading in...5
×
 

Hadoop導入事例 in クックパッド

on

  • 20,678 views

4/2, 4/3に #urapad で使った発表資料

4/2, 4/3に #urapad で使った発表資料

Statistics

Views

Total Views
20,678
Views on SlideShare
17,934
Embed Views
2,744

Actions

Likes
15
Downloads
342
Comments
0

18 Embeds 2,744

http://techlife.cookpad.com 2641
http://www.slideshare.net 49
http://paper.li 22
http://webcache.googleusercontent.com 10
http://www.health.medicbd.com 6
http://d.hatena.ne.jp 2
url_unknown 2
http://us-w1.rockmelt.com 2
http://www.google.com 1
https://www.google.co.jp 1
https://abs.twimg.com 1
http://program0.e1blue.net 1
https://si0.twimg.com 1
http://techlife.cookpad.com:80 1
resource://brief-content 1
http://s.deeeki.com 1
http://twitter.com 1
https://cookpadtech.wordpress.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />

Hadoop導入事例 in クックパッド Hadoop導入事例 in クックパッド Presentation Transcript

  • Hadoop in
  • • id:sasata299 ( ) • Hadoop • • http://blog.livedoor.jp/sasata299/
  • Hadoop
  • Hadoop MySQL
  • … Hadoop MySQL
  • :-)
  • 1. Hadoop 2. 3. 4. 5.
  • Hadoop
  • 915 30 3 1
  • ( )
  • ( )
  • ‣ ‣ GROUP BY ( ( Д`) ‣ 7000 ( )
  • !!
  • Hadoop
  • ‣ Google MapReduce ‣ ‣
  • mapper reducer ( ) ( )
  • ‣ Hadoop Streaming ‣ Ruby ‣ Cloudera CDH1 (0.18.3) ‣ EC2 Hadoop ( 10 50 ) ‣ Hadoop S3
  • S3 Native FileSystem (s3n://) ‣ ‣ 5GB S3 Block FileSystem (s3://) ‣ ‣ HDFS ‣
  • ( ) mapper ( )
  • mapper, reducer
  • master ‣ -file master slave hadoop jar xxx.jar -mapper hoge.rb -reducer fuga.rb -file hoge.rb -file fuga.rb -file outdata ‣ mapper, reducer File.open File.open(‘outdata’) {|f| ...}
  • S3 ‣ hadoop cat ※ ‣ -cacheFile S3 slave ( File.open) ※ hadoop jar xxx.jar -mapper hoge.rb -reducer fuga.rb -file hoge.rb -file fuga.rb -cacheFile s3n://path/to/outdata#othername mapper reducer
  • 7000 ( )→
  • 7000 ( )→ 30
  • Hadoop++
  • Hadoop … mapper, reducer … Hadoop …
  • 1. 2. Hadoop 3. Hadoop 4. Hadoop 5. Hadoop ( ) 6. (Excel ) 7. Hadoop
  • Hadoop !!
  • • MySQL Hadoop • Hadoop EC2/S3 • Hadoop Streaming Hadoop