Your SlideShare is downloading. ×
0
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

ソーシャルアプリでの Amazon Elastic MapReduce 活用事例

9,957

Published on

TokyoWebmining #7 …

TokyoWebmining #7
buhii

0 Comments
11 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
9,957
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
42
Comments
0
Likes
11
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Amazon Elastic MapReduce Takahiro Kamatani gumi, Inc. 2010/09/26 Sunday, September 26, 2010
  • 2. • • • Amazon Elastic MapReduce • Sunday, September 26, 2010
  • 3. • Twitter: @buhii • • gumi @ http://www.kansei.tsukuba.ac.jp/~uchiyamalab/beacon • • beacon • gumi @ynil Sunday, September 26, 2010
  • 4. Sunday, September 26, 2010
  • 5. gumi • mixi, , GREE • python Django • Amazon Web Services (EC2 + RDS) • • DB Sunday, September 26, 2010
  • 6. • PV , UU • DAU Daily Active Users • • • ÷ DAU • ARPU Average Revenue Per User • Sunday, September 26, 2010
  • 7. Amazon Web Service AWS Sunday, September 26, 2010
  • 8. Sunday, September 26, 2010
  • 9. Sunday, September 26, 2010
  • 10. Sunday, September 26, 2010
  • 11. Sunday, September 26, 2010
  • 12. Sunday, September 26, 2010
  • 13. Amazon Elastic MapReduce Sunday, September 26, 2010
  • 14. MapReduce Mapper Key, Value Mapper key Sort / Shuffle Reducer Reducer key, value Mapper, Reducer Sunday, September 26, 2010
  • 15. Amazon Elastic MapReduce • Hadoop • Hadoop Streaming Mapper Reducer Ruby, Perl, Python, PHP, R, Bash, C++ • EC2 job • Sunday, September 26, 2010
  • 16. Example Task • • • Sunday, September 26, 2010
  • 17. • Mapper • Apache Log • ID key value Reducer Sunday, September 26, 2010
  • 18. • Reducer • sort/shuffle ID Reducer • ID Sunday, September 26, 2010
  • 19. Reducer 31758623 2010-08-20 42346572 2010-09-05,2010-09-06 31977736 2010-08-11,2010-08-12,2010-08-13,2010-08-14 14007991 2010-08-16 35995849 2010-08-12,2010-08-13,2010-08-14 34246688 2010-08-21,2010-08-22,2010-08-23,2010-08-27 ... PC Sunday, September 26, 2010
  • 20. Amazon Elastic Mapreduce • AWS • S3 Mapper, Reducer → s3cmd, S3Fox Organizer, Cyberduck • Job OK • Sunday, September 26, 2010
  • 21. Sunday, September 26, 2010
  • 22. Sunday, September 26, 2010
  • 23. Streaming Sunday, September 26, 2010
  • 24. {Input, Output} Location, Mapper, Reducer S3 gzip Hadoop Extra Args -jobconf stream.recordreader.compression=gzip input Location Extra Args -input s3n://(bucket )/( )/access_log.* Sunday, September 26, 2010
  • 25. Sunday, September 26, 2010
  • 26. Sunday, September 26, 2010
  • 27. Sunday, September 26, 2010
  • 28. Debug Sunday, September 26, 2010
  • 29. Sunday, September 26, 2010
  • 30. • Hadoop • MapReduce • • Sunday, September 26, 2010
  • 31. Hadoop • • S3 gzip • • hadoop EC2 • ( 20 ...) Sunday, September 26, 2010
  • 32. @ynil MapReduce http://nlpyutori.g.hatena.ne.jp/yaruki_nil/20100911/1284089305 Sunday, September 26, 2010
  • 33. Sunday, September 26, 2010
  • 34. Sunday, September 26, 2010
  • 35. Sunday, September 26, 2010
  • 36. MapReduce MapReduce Google Map Reduce Map Reduce MapReduce C++ Java Python Wikipedia “MapReduce” http://ja.wikipedia.org/wiki/MapReduce Sunday, September 26, 2010
  • 37. cron • PV, UU NFS CSV • DB → DB • PV, UU Sunday, September 26, 2010

×