Published on

My presentation at lrug on 10th Feb

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Hi All, my name is Anup.
    I am here to present you birdpie, an experimental project developed using Rails 3 with resque backend.
    This application was released in January.
  • So thats me.
    I am working fulltime on rails since 2007.
    Right now I work for Dynamic50 which is a cool startup based in Croydon.
    You can follow me on twitter & my id is railsbob.
  • You must have seen a ‘tweet this’ link on popular blogs and websites.
    We tweet the links, share it among our network and then forget about it.
  • Won;t it be nice if I was able to look back through my links, tag them with keywords, search them, or see who else have tweeted it?

    There are few websites which aggregate bookmarks, but none of them are offered as personal tools.
  • But there are many limitations while dealing with twitter.
    The search cannot go beyond last 1500 tweets
    We might come accoss different tiny urls pointing to same web page
    Also, the twitter API cannot go beyond 3000 tweets for a user.
  • So we offer you Birdpie, a twitter application which aggregates all your tweeted links.
    It also stores contextual information like tags and allows you to assign a category.
  • It also resolved the tiny urls, and keeps a track of tweeted count unique to a user.
    All dead and non existing links are discarded and the link is labeled with the title of the target page.
  • The sign up is simple, you can log in using your twitter id.
    Private users can sign up too, but their bookmarks are not available for public viewing on the profile pages or search results.
  • Intelligent Tags
    Once the url is saved, birdpie fetches the target page and passes its content to a service to extract tags.
    You can see an example above.

    You can also list all bookmarks by a particular tag.
  • A category is a personal folder to manage your boomark.
    The dashboard allows you to assign a category for a bookmark and define new ones as you want.
  • There is another way to assign a category for a bookmark, and that is using a matching hashtag.
    So, if I tweet a link with hashtag #ruby, it is automatically assigned under ruby cateogy.
  • We developed this application on Rails 3.0
    The idea was to start early and know it better
    This being a minimal app, we did not face any major issue.
    The site is up and running smoothly on production since a month.
  • Birdpie is mostly a background application.
    There are three main tasks
    Crawl for new tweets, check for links
    Resolve target urls
    Tag and save
  • We use Redis to manage bookmark counts and tag counts.
    The in-built increment method eliminates the need of a counter cache.
    It is pretty fast and improves the overall performance of the application.

    The Resque queue manager also relies on Redis for operation.
  • There are few options available for background processing.
    We chose Resque as we wanted to break the background tasks into multiple queues with different priorities.
    Also, the admin app bundled with Resque is a handy tool for monitoring.
  • Basically the process is spread across two queues.
    User queue deals with fetching content from twitter for a particular user.
    Bookmarks queue resolves the raw link, tags it and updates the redis index
  • This is a snapshot of the Resque admin application.
    You can easily see if you require more workers on a particular queue and act accordingly.
    It also gives information about failed jobs if any.
  • We selected redis and resque over delayed job as we wanted to manage the processes using multiple queues.
    I have covered a brief introduction here, but you can follow these links if you want to dig in more details.
  • We have few more features in the pipeline, some are listed here.
    The mozilla plugin will provide you easy access to all your bookmarks right in the browser.
    Retweet within the birdpie application to save someone else’s bookmarks
    And finally a more extensive bookmark search.
  • Thank you for listening to my talk.
    You can keep a watch on dynamic50 blog for new announcements about birdpie features.
  • Birdpie

    1. 1. Anup Narkhede
    2. 2. About Me • Anup Narkhede • On Rails since 2007 • Employee at •
    3. 3. Do you tweet links?
    4. 4. What if you want to look back through your old links?
    5. 5. Limitations • Search only returns last 1500 tweets • Different TinyUrls for same target links • Twitter API goes back for 3000 tweets
    6. 6. • Twitter web application • Aggregates all your tweeted links • Auto link tagging • Categories for link management • Shows popularity of the bookmark
    7. 7. • Resolves the Tiny URLs • Stores the count • 404 Errors and Deadlinks • Gets the page title
    8. 8. Sign Up • Twitter OAuth (Read-Only) • Private Users?
    9. 9. Intelligent Tags • Auto Tags • Listing by tags
    10. 10. Categories • Manage bookmarks by assigning a category • A Category acts like a Folder
    11. 11. Hashtags • Auto Assign a category by adding a hashtag RT @dynamic50: New blog post: Birdpie Launched #ruby Puts the bookmark under Ruby Category
    12. 12. Rails 3.0
    13. 13. Architecture • 75% Background processes Crawl Resolve Tag
    14. 14. Redis • Key value store • Eliminates need for counter cache • Lists, Sets, Ordered Sets • Used to manage bookmark counts
    15. 15. Resque • Queue Manager • Admin Sinatra Application • Inspect Failed Jobs • Add and remove workers
    16. 16. Process User URLs Bookmarks Worker 1 Worker 2 Worker 3 •Crawl •Resolve •Store •Tag •Update count
    17. 17. Resque Admin (Sinatra App)
    18. 18. Redis Resque
    19. 19. Coming soon • Mozilla Plugin to manage your bookmarks • Retweets • Bookmark Search
    20. 20. About Us