HBase Feed Aggregator Wurbe 25
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

HBase Feed Aggregator Wurbe 25

on

  • 1,253 views

 

Statistics

Views

Total Views
1,253
Views on SlideShare
1,249
Embed Views
4

Actions

Likes
1
Downloads
5
Comments
0

2 Embeds 4

http://www.slideshare.net 2
http://coderwall.com 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

HBase Feed Aggregator Wurbe 25 Presentation Transcript

  • 1. feed aggregator powered by hbase & python Andrei Savu wurbe #25
  • 2. Objectives Highly scalable feed aggregator Play with python & thrift Provide some sample code Provide detailed install instructions Learn new stuff
  • 3. Table Structure 3 tables: Feeds, Urls, UrlsIndex Feeds: all feeds Urls: data extracted from feeds UrlsIndex: index table
  • 4. Source code http://github.com/andreisavu/feedaggregator detailed install instructions
  • 5. Lessons learned
  • 6. Lesson #1: Hbase Game Rules Not relations No joins No sophisticated query engine No column typing No transactions No secondary indices ... all done in application code
  • 7. Lesson #2: Design your index <cat>/<w3c_timestamp> time sorting = lexicographic sorting
  • 8. Lesson #3: No charsets convert everything to bytes ... but store the original charset
  • 9. Questions? http://www.andreisavu.ro http://twitter.com/andreisavu contact@andreisavu.ro