Open Network Live - Hack Day Report
Upcoming SlideShare
Loading in...5
×
 

Open Network Live - Hack Day Report

on

  • 1,849 views

 

Statistics

Views

Total Views
1,849
Views on SlideShare
1,781
Embed Views
68

Actions

Likes
1
Downloads
4
Comments
0

2 Embeds 68

http://onlab.jp 65
http://www.slideshare.net 3

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Open Network Live - Hack Day Report Open Network Live - Hack Day Report Presentation Transcript

    • Chirp Hack Day Report ) wada@garage.co.jp @Koichi
    • •  –  wada@garage.co.jp –  @Koichi •  –  http://twinavi.jp
    • Chirp •  4/14, 15 in San Francisco •  –  Chirp –  Hack Day
    • Chirp
    • 1 -Conference
    • 1 - Hack Day Start
    • 1 - Ignite
    • 1 - Coding time
    • 2 -
    • -Sessions
    • 2 -Lunch, Meet The founders
    • •  2 •  • 
    • Hack Day
    • •  • 
    • •  –  –  •  SPAM –  DM •  – 
    • (2) •  –  Tweet Display Guidlines •  http://media.twitter.com/14/tweet-display-guidelines –  Terms Of Service •  http://twitter.com/tos •  •  Awesome –  !
    • (3) •  API Cache •  OAuth Key
    • •  •  at slideshare - #chirppolicy –  We Have Faith in (Most of) You: How Twitter Crafts Policies to Allow Good Apps to Thrive –  http://www.slideshare.net/delbius/ chirppolicy
    • Twitter 7TB/ Chirp GB
    • Challenge •  •  & • 
    • •  syslog-ng • 
    • •  Scribe –  Facebook –  Thrift – 
    • Scribe •  –  •  –  •  HDFS
    • •  7TB/ •  80MB/s •  24.3 •  • 
    • •  Hadoop –  –  MapReduce –  –  Y! 4000 –  1TB 62
    • •  MySQL –  : COUNT, GROUP –  : JOIN •  Hadoop –  5 –  – 
    • •  Java •  –  MapReduce – 
    • •  Pig –  –  SQL –  –  1
    • Pig sample users = load ‘users.csv’ as (username: charaarray, age: int); users_1825 = filter users by age >= 18 and age <=25; pages = load ‘pages.csv’ as (username: chararay, url: chararray) joined = join users_1825 by username, pages by username; grouped = group joined by url; summed = foreach grouped generate group as url, COUNT (joined) AS views; sorted = order summed by views desc; top_5 = limit sorted 5; store top_t into ‘top_5_sites.csv’
    • Java 5%
    • •  –  Scribe –  Hadoop –  Pig •  at slideshare - #chirpdata –  Analyzing Big Data at Twitter –  http://www.slideshare.net/kevinweil/big-data-at- twitter-chirp-2010
    • •  •  •