Hadoop summit 2010 frameworks panel elephant bird
Upcoming SlideShare
Loading in...5
×
 

Hadoop summit 2010 frameworks panel elephant bird

on

  • 4,803 views

 

Statistics

Views

Total Views
4,803
Views on SlideShare
4,784
Embed Views
19

Actions

Likes
9
Downloads
64
Comments
0

4 Embeds 19

http://www.linkedin.com 8
https://www.linkedin.com 8
https://twitter.com 2
http://www.slideshare.net 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • This is the Title slide. Please use the name of the presentation that was used in the abstract submission.
  • This is the agenda slide. There is only one of these in the deck.
  • This is the agenda slide. There is only one of these in the deck.
  • This is the agenda slide. There is only one of these in the deck.
  • This is the agenda slide. There is only one of these in the deck.
  • This is a topic/content slide. Duplicate as many of these as are needed. Generally, there is one slide per three minutes of talk time.
  • This is a topic/content slide. Duplicate as many of these as are needed. Generally, there is one slide per three minutes of talk time.
  • This is a topic/content slide. Duplicate as many of these as are needed. Generally, there is one slide per three minutes of talk time.
  • This is a topic/content slide. Duplicate as many of these as are needed. Generally, there is one slide per three minutes of talk time.
  • This is a topic/content slide. Duplicate as many of these as are needed. Generally, there is one slide per three minutes of talk time.
  • This is a topic/content slide. Duplicate as many of these as are needed. Generally, there is one slide per three minutes of talk time.

Hadoop summit 2010 frameworks panel elephant bird Hadoop summit 2010 frameworks panel elephant bird Presentation Transcript

  • Hadoop Frameworks
    • Kevin Weil @kevinweil
    Twitter
    • A framework for working with structured data within the Hadoop ecosystem
    Elephant Bird
    • A framework for working with structured data within the Hadoop ecosystem
      • Protocol Buffers
      • Thrift
      • JSON
      • W3C Logs
    Elephant Bird
    • A framework for working with structured data within the Hadoop ecosystem
      • InputFormats
      • OutputFormats
      • Hadoop Writables
      • Pig LoadFuncs
      • Pig StoreFuncs
      • Hbase LoadFuncs
    Elephant Bird
    • A framework for working with structured data within the Hadoop ecosystem… plus:
      • LZO Compression
      • Code Generation
      • Hadoop Counter Utilities
      • Misc Pig UDFs
    Elephant Bird
    • You should only need to specify the data schema
    Why?
    • You should only need to specify the ( flexible, forward-backward compatible, self-documenting ) data schema
    Why?
    • You should only need to specify the ( flexible, forward-backward compatible, self-documenting ) data schema
    • Everything else can be codegen’d.
    Why?
    • You should only need to specify the ( flexible, forward-backward compatible, self-documenting ) data schema
    • Everything else can be codegen’d.
    • Less Code. Efficient Storage. Focus on the Data.
    Why?
    • You should only need to specify the ( flexible, forward-backward compatible, self-documenting ) data schema
    • Everything else can be codegen’d.
    • Less Code. Efficient Storage. Focus on the Data.
    • Underlies 20,000 Hadoop jobs at Twitter every day.
    Why?
    • You should only need to specify the ( flexible, forward-backward compatible, self-documenting ) data schema
    • Everything else can be codegen’d.
    • Less Code. Efficient Storage. Focus on the Data.
    • Underlies 20,000 Hadoop jobs at Twitter every day.
    • http://github.com/kevinweil/elephant-bird : contributors welcome!
    Why?