SlideShare is now on Android. 15 million presentations at your fingertips.  Get the app

×
  • Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
 

Hadoop, Pig, and Twitter (NoSQL East 2009)

by Analytics Lead at Twitter on Oct 30, 2009

  • 107,854 views

A talk on the use of Hadoop and Pig inside Twitter, focusing on the flexibility and simplicity of Pig, and the benefits of that for solving real-world big data problems.

A talk on the use of Hadoop and Pig inside Twitter, focusing on the flexibility and simplicity of Pig, and the benefits of that for solving real-world big data problems.

Statistics

Views

Total Views
107,854
Views on SlideShare
93,389
Embed Views
14,465

Actions

Likes
239
Downloads
2,827
Comments
17

55 Embeds 14,465

http://www.blog.c 5237
http://www.hadoopwizard.com 4943
http://t3n.de 2359
http://www.programadorphp.org 1015
http://www.slideshare.net 306
http://www.scoop.it 170
http://www.linkedin.com 96
http://it.gilbird.com 46
http://paper.li 33
http://www.adambreckler.com 30
http://localhost 25
https://podio.com 16
http://feeds2.feedburner.com 15
http://wiki.weinter.co.kr 14
http://www.rss-ais.com 13
http://marcelmaatkamp.blogspot.com 12
http://juepner-bpm.com 11
http://bigdatacrunching.blogspot.in 9
http://blog.iband.kr 8
http://blog.escalabilidad.net 8
http://obl8.com:8000 8
http://www.iweb34.com 7
http://hadoopmak.blogspot.in 6
http://devpub.kr 6
http://jisi.dreamblog.jp 6
http://bigdatacrunching.blogspot.com 5
http://hadoopbrasil.com 5
http://webcache.googleusercontent.com 5
http://bigdatacrunching.blogspot.fr 5
http://dineshoi.blogspot.in 4
http://twitter.com 4
http://dineshoi.blogspot.com 3
https://twitter.com 3
http://static.slidesharecdn.com 3
http://feeds.feedburner.com 2
https://www.google.com 2
http://translate.googleusercontent.com 2
https://www.linkedin.com 2
http://a0.twimg.com 2
http://irr.posterous.com 2
http://127.0.0.1:8795 2
http://10.150.200.76 2
http://www.tuicool.com 1
http://www.google.es 1
http://bigdatacrunching.blogspot.se 1
http://hadoopmak.blogspot.se 1
https://twimg0-a.akamaihd.net 1
http://cafe.naver.com 1
http://www.e-presentations.us 1
http://www.schoox.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via SlideShare as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

110 of 17 previous next Post a comment

  • bibiki788 bibiki788 Very good, Thanks for sharing. 1 year ago
    Are you sure you want to
    Your message goes here
    Processing…
  • nishantsonar nishantsonar Twitter on hadoop 2 years ago
    Are you sure you want to
    Your message goes here
    Processing…
  • JinhwanLee Jinhwan Lee dd 2 years ago
    Are you sure you want to
    Your message goes here
    Processing…
  • JohnsonRuham Johnson Ruham Excellent slides! Thanks for sharing.
    http://inetworkingskills.blogspot.com/
    2 years ago
    Are you sure you want to
    Your message goes here
    Processing…
  • ShyamVelupula Shyam Velupula PDF version should have provided for downloading 3 years ago
    Are you sure you want to
    Your message goes here
    Processing…
  • HellonoNo Hellono No learn 3 years ago
    Are you sure you want to
    Your message goes here
    Processing…
  • tygarvin tygarvin Hey Kevin,

    I am doing a report of my work with Hadoop in a high performance computing environment for a national laboratory and was wondering if I could use some of your slides and photos in my report?

    If you could get back to me that would be incredible. Thanks in advance.
    Tyler Garvin
    3 years ago
    Are you sure you want to
    Your message goes here
    Processing…
  • xds2000 xds2000 Cool idea 4 years ago
    Are you sure you want to
    Your message goes here
    Processing…
  • satishgaude satishgaude That was great. Cleared some of my doubts 4 years ago
    Are you sure you want to
    Your message goes here
    Processing…
  • kevinweil Kevin Weil, Analytics Lead at Twitter Hitesh,

    We don't read or write data directly from/to MySQL with Pig. The interfaces for load/store aren't quite right yet, although they're getting better (follow along in realtime at https://issues.apache.org/jira/browse/PIG-966). Rather, we export in batch from mysql into hdfs at regular intervals, which is good because then non-pig jobs can access this data too. And we have a little wrapper around pig that can optionally push the results of pig jobs back into databases etc after the job has run. This is something we plan on open sourcing once it hardens a bit.
    4 years ago
    Are you sure you want to
    Your message goes here
    Processing…

110 of 17 previous next

Post Comment
Edit your comment

Hadoop, Pig, and Twitter (NoSQL East 2009) Hadoop, Pig, and Twitter (NoSQL East 2009) Presentation Transcript