Yahoo! Hadoop User Group - May Meetup - HBase and Pig: The Hadoop ecosystem at Twitter, Dmitriy Ryaboy, Twitter

Hadoop, Pig, HBase at Twitter ,[object Object],[object Object],[object Object]

Who is this guy, anyway ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

In This Talk ,[object Object],[object Object],[object Object],[object Object],[object Object]

Not In This Talk ,[object Object],[object Object],[object Object],[object Object],[object Object]

Daily workload ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Twitter data pipeline (simplified) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Logs ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Tables ,[object Object],[object Object],[object Object],[object Object]

ETL ,[object Object],[object Object],[object Object]

Mutability ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Aren't you guys Cassandra poster boys? poster boys? ,[object Object],[object Object],[object Object],[object Object],[object Object]

HBase schema for MySQL exports, v1. ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

HBase schema v1, cont. ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

HBase schema, v2. ,[object Object],[object Object],[object Object],[object Object],[object Object]

Why Pig? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

HBase Loader enhancements ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

HBase Loader TODOs ,[object Object],[object Object],[object Object],[object Object],[object Object]

Elephant Bird ,[object Object],[object Object],[object Object],[object Object],[object Object]

Bad records kill jobs ,[object Object],[object Object],[object Object],[object Object]

Runaway UDFs kill jobs ,[object Object],[object Object],[object Object],[object Object],[object Object]

Use Counters ,[object Object],[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],Lazy deserializaton FTW lazy deserialization

Also see ,[object Object],[object Object],[object Object],[object Object],[object Object]

Questions ? Follow me at twitter.com/squarecog TM

Photo Credits ,[object Object],[object Object],[object Object]

Yahoo! Hadoop User Group - May Meetup - HBase and Pig: The Hadoop ecosystem at Twitter, Dmitriy Ryaboy, Twitter

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (19)

Similar to Yahoo! Hadoop User Group - May Meetup - HBase and Pig: The Hadoop ecosystem at Twitter, Dmitriy Ryaboy, Twitter

Similar to Yahoo! Hadoop User Group - May Meetup - HBase and Pig: The Hadoop ecosystem at Twitter, Dmitriy Ryaboy, Twitter (20)

More from Hadoop User Group

More from Hadoop User Group (13)

Recently uploaded

Recently uploaded (20)

Yahoo! Hadoop User Group - May Meetup - HBase and Pig: The Hadoop ecosystem at Twitter, Dmitriy Ryaboy, Twitter