マーケティングのためのHadoop利用
Upcoming SlideShare
Loading in...5
×
 

マーケティングのためのHadoop利用

on

  • 9,006 views

6/4に行われた「クックパッド・PFI共同勉強会」で使った資料です。togetterはこちら。 http://togetter.com/li/26756

6/4に行われた「クックパッド・PFI共同勉強会」で使った資料です。togetterはこちら。 http://togetter.com/li/26756

Statistics

Views

Total Views
9,006
Views on SlideShare
7,206
Embed Views
1,800

Actions

Likes
20
Downloads
197
Comments
0

6 Embeds 1,800

http://techlife.cookpad.com 1777
http://www.slideshare.net 19
http://d.hatena.ne.jp 1
url_unknown 1
http://a0.twimg.com 1
https://cookpadtech.wordpress.com 1

Accessibility

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />

マーケティングのためのHadoop利用 マーケティングのためのHadoop利用 Presentation Transcript

  • Hadoop
  • • sasata299 ( ) • Hadoop • NoSQL • • http://blog.livedoor.jp/sasata299/
  • Hadoop • Google MapReduce OSS • • • •
  • • Hadoop • Hadoop • • & !?
  • • Hadoop • Hadoop • • & !?
  • 896 30 3 1
  • 896 30 3 1
  • 896 30 3 1 ” ”
  • GROUP BY !! MySQL ( 3.5 )
  • GROUP BY !! MySQL ( 3.5 ) 7000 ≒292 ……orz
  • MySQL
  • … ※
  • !?
  • Hadoop
  • 7000 30
  • • Hadoop • Hadoop • • & !?
  • Hadoop • Hadoop Streaming • Ruby • Cloudera CDH1 (0.18.3) • EC2 Hadoop • 10-50 • Hadoop S3
  • Hadoop • Hadoop Streaming • Ruby • Cloudera CDH1 (0.18.3) • EC2 Hadoop • 10-50 • Hadoop S3
  • ○○ ×× ○○ ××
  • Hadoop
  • Hadoop AWS
  • Hadoop
  • Hadoop Hadoop (EC2)
  • hadoop-ec2 push [cluster] mapper.rb hadoop-ec2 push [cluster] reducer.rb Hadoop (EC2)
  • hadoop-ec2 push [cluster] mapper.rb hadoop-ec2 push [cluster] reducer.rb Hadoop (EC2)
  • Hadoop hadoop-ec2 exec [cluster] [command] Hadoop S3 (EC2)
  • Hadoop hadoop-ec2 exec [cluster] [command] Hadoop S3 (EC2)
  • Hadoop hadoop-ec2 exec [cluster] [command] Hadoop S3 (EC2)
  • Hadoop hadoop-ec2 exec [cluster] [command] Hadoop S3 (EC2)
  • S3
  • S3
  • Hadoop Hadoop (EC2)
  • Hadoop
  • !!
  • Hadoop
  • 1) 2) Hadoop 3)
  • • Hadoop • Hadoop • • & !?
  • target_ids # [21310,12902,15321,..] ARGF.each do |log| log.chomp! id, foo, bar, ... = log.split(/,/) next if target_ids.include?(id) end target_ids 5 …
  • :-)
  • # 1000 hash = Hash.new {|h,k| h[k] = []} target_ids.each do |_id| hash[_id.to_s[0,3]] << _id end ARGF.each do |log| log.chomp! id, foo, bar, ... = log.split(/,/) # next if hash[id[0,3]].include?(id) end
  • Mapper Reducer - Mapper - Reducer
  • • Hadoop • Hadoop • • & !?
  • EC2 -> AZ -> JobTracker -> 50030 -> hadoop job -list
  • 10h 8h JobTracker
  • !?
  • Amazon Elastic MapReduce
  • Elastic MapReduce