• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
ソーシャルアプリでの Amazon Elastic MapReduce 活用事例
 

ソーシャルアプリでの Amazon Elastic MapReduce 活用事例

on

  • 9,778 views

TokyoWebmining #7

TokyoWebmining #7
buhii

Statistics

Views

Total Views
9,778
Views on SlideShare
4,246
Embed Views
5,532

Actions

Likes
11
Downloads
41
Comments
0

3 Embeds 5,532

http://d.hatena.ne.jp 5517
http://webcache.googleusercontent.com 13
https://twitter.com 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    ソーシャルアプリでの Amazon Elastic MapReduce 活用事例 ソーシャルアプリでの Amazon Elastic MapReduce 活用事例 Presentation Transcript

    • Amazon Elastic MapReduce Takahiro Kamatani gumi, Inc. 2010/09/26 Sunday, September 26, 2010
    • • • • Amazon Elastic MapReduce • Sunday, September 26, 2010
    • • Twitter: @buhii • • gumi @ http://www.kansei.tsukuba.ac.jp/~uchiyamalab/beacon • • beacon • gumi @ynil Sunday, September 26, 2010
    • Sunday, September 26, 2010
    • gumi • mixi, , GREE • python Django • Amazon Web Services (EC2 + RDS) • • DB Sunday, September 26, 2010
    • • PV , UU • DAU Daily Active Users • • • ÷ DAU • ARPU Average Revenue Per User • Sunday, September 26, 2010
    • Amazon Web Service AWS Sunday, September 26, 2010
    • Sunday, September 26, 2010
    • Sunday, September 26, 2010
    • Sunday, September 26, 2010
    • Sunday, September 26, 2010
    • Sunday, September 26, 2010
    • Amazon Elastic MapReduce Sunday, September 26, 2010
    • MapReduce Mapper Key, Value Mapper key Sort / Shuffle Reducer Reducer key, value Mapper, Reducer Sunday, September 26, 2010
    • Amazon Elastic MapReduce • Hadoop • Hadoop Streaming Mapper Reducer Ruby, Perl, Python, PHP, R, Bash, C++ • EC2 job • Sunday, September 26, 2010
    • Example Task • • • Sunday, September 26, 2010
    • • Mapper • Apache Log • ID key value Reducer Sunday, September 26, 2010
    • • Reducer • sort/shuffle ID Reducer • ID Sunday, September 26, 2010
    • Reducer 31758623 2010-08-20 42346572 2010-09-05,2010-09-06 31977736 2010-08-11,2010-08-12,2010-08-13,2010-08-14 14007991 2010-08-16 35995849 2010-08-12,2010-08-13,2010-08-14 34246688 2010-08-21,2010-08-22,2010-08-23,2010-08-27 ... PC Sunday, September 26, 2010
    • Amazon Elastic Mapreduce • AWS • S3 Mapper, Reducer → s3cmd, S3Fox Organizer, Cyberduck • Job OK • Sunday, September 26, 2010
    • Sunday, September 26, 2010
    • Sunday, September 26, 2010
    • Streaming Sunday, September 26, 2010
    • {Input, Output} Location, Mapper, Reducer S3 gzip Hadoop Extra Args -jobconf stream.recordreader.compression=gzip input Location Extra Args -input s3n://(bucket )/( )/access_log.* Sunday, September 26, 2010
    • Sunday, September 26, 2010
    • Sunday, September 26, 2010
    • Sunday, September 26, 2010
    • Debug Sunday, September 26, 2010
    • Sunday, September 26, 2010
    • • Hadoop • MapReduce • • Sunday, September 26, 2010
    • Hadoop • • S3 gzip • • hadoop EC2 • ( 20 ...) Sunday, September 26, 2010
    • @ynil MapReduce http://nlpyutori.g.hatena.ne.jp/yaruki_nil/20100911/1284089305 Sunday, September 26, 2010
    • Sunday, September 26, 2010
    • Sunday, September 26, 2010
    • Sunday, September 26, 2010
    • MapReduce MapReduce Google Map Reduce Map Reduce MapReduce C++ Java Python Wikipedia “MapReduce” http://ja.wikipedia.org/wiki/MapReduce Sunday, September 26, 2010
    • cron • PV, UU NFS CSV • DB → DB • PV, UU Sunday, September 26, 2010