• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Hadoop for humans
 

Hadoop for humans

on

  • 300 views

 

Statistics

Views

Total Views
300
Views on SlideShare
298
Embed Views
2

Actions

Likes
0
Downloads
0
Comments
0

1 Embed 2

http://eventifier.co 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Hadoop for humans Hadoop for humans Presentation Transcript

    • Hadoop for humans Kien Pham Software Engineer - R&D Anaheim, CA 10/04/2013 Friday, October 4, 13
    • Hadoop? Friday, October 4, 13
    • is a framework HDFS Map /Reduce http://www.flickr.com/photos/d90nikon/6195610430/sizes/o/in/photostream/ Friday, October 4, 13
    • Map / Reduce Friday, October 4, 13
    • Mapper I like SendGrid and email, you like SendGrid and email too 1 1 1 1 1 Friday, October 4, 13
    • Mapper I like SendGrid and email, you like SendGrid and email too 1 1 1 1 1 I like SendGrid and email, you like SendGrid and email too 1 1 1 1 1 I like SendGrid and email, you like SendGrid and email too 1 1 1 1 1 worker 1 worker 2 worker 3 Friday, October 4, 13
    • Reducer 1like SendGrid email SendGrid email 1 1 1 1 1like SendGrid email 2 2 Friday, October 4, 13
    • 1like SendGrid email 2 2 key value Friday, October 4, 13
    • key value {"d": "2013-09-01", "t": "j"} {"d": "2013-09-02", "t": "j"} {"d": "2013-09-01", "t": "x"} {"d": "2013-09-02", "t": "x"} 764872 269661 190889 71693 Friday, October 4, 13
    • HDFS Friday, October 4, 13
    • HDFS Friday, October 4, 13
    • HDFS @ SG 138 TB Friday, October 4, 13
    • 1 TB = 1,024 GB 138TB = 141,312 GB 300GB / day 141,312 GB / 300 GB = 471 days Friday, October 4, 13
    • S3 Friday, October 4, 13
    • 2015 50% of the world’s data Hadoop will process http://www.flickr.com/photos/tisdale53/4737492082/ Friday, October 4, 13
    • custom jobs? Friday, October 4, 13
    • mrgumble Friday, October 4, 13
    • abstract Hadoop process Friday, October 4, 13
    • start stop status result Friday, October 4, 13
    • mrgumble start -j my_cool_job Friday, October 4, 13
    • mrgumble stop -j my_cool_job Friday, October 4, 13
    • mrgumble status --job_id 1234 Friday, October 4, 13
    • mrgumble result -j job_name Friday, October 4, 13
    • excited? Friday, October 4, 13
    • template.py hadoop-jobs repo jobs/ Friday, October 4, 13
    • import mrgumble import sgstats-hadoop Friday, October 4, 13
    • Live Demo Friday, October 4, 13