HBase Feed Aggregator Wurbe 25
Upcoming SlideShare
Loading in...5
×
 

HBase Feed Aggregator Wurbe 25

on

  • 1,211 views

 

Statistics

Views

Total Views
1,211
Views on SlideShare
1,207
Embed Views
4

Actions

Likes
1
Downloads
5
Comments
0

2 Embeds 4

http://www.slideshare.net 2
http://coderwall.com 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    HBase Feed Aggregator Wurbe 25 HBase Feed Aggregator Wurbe 25 Presentation Transcript

    • feed aggregator powered by hbase & python Andrei Savu wurbe #25
    • Objectives Highly scalable feed aggregator Play with python & thrift Provide some sample code Provide detailed install instructions Learn new stuff
    • Table Structure 3 tables: Feeds, Urls, UrlsIndex Feeds: all feeds Urls: data extracted from feeds UrlsIndex: index table
    • Source code http://github.com/andreisavu/feedaggregator detailed install instructions
    • Lessons learned
    • Lesson #1: Hbase Game Rules Not relations No joins No sophisticated query engine No column typing No transactions No secondary indices ... all done in application code
    • Lesson #2: Design your index <cat>/<w3c_timestamp> time sorting = lexicographic sorting
    • Lesson #3: No charsets convert everything to bytes ... but store the original charset
    • Questions? http://www.andreisavu.ro http://twitter.com/andreisavu contact@andreisavu.ro