Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Parsely: Inside a modern RIA built with Solr

Parse.ly is a rich, adaptive web application that discovers your unique interests to filter and prioritize content from countless news and blog sources on the web. This talk will introduce Parse.ly with a quick demo and then delve right into how the Parse.ly engineering team makes use of the Solr open source search engine. This will include discussion of initial design mistakes that were later revised and "real world issues" that were overcome in scaling a system that currently processes millions of articles per week. Finally, we will discuss the existing Solr and Python landscape, and how we at Parse.ly aim to help the Solr community with the open source release of high-quality, Pythonic components for doing common Solr tasks.

  • Be the first to comment

  • Be the first to like this

Parsely: Inside a modern RIA built with Solr

  1. 1. <ul><li>Inside a modern RIA powered by Solr </li></ul>Andrew Montalenti Co-Founder & Technology Lead [email_address]
  2. 2.
  3. 3. Mainstream Blogs 30,000 900,000 1 From Technorati’s 2008 State of the Blogosphere 1
  4. 4. But, What about your interests?
  5. 5. your interests, your web
  6. 6.
  7. 7. What is ? <ul><li>Your unique interests </li></ul><ul><li>… create a filtered, prioritized, and personalized news feed </li></ul><ul><li>… built just for you! </li></ul><ul><li>120K+ news and blog sources tracked </li></ul><ul><li>The most personally relevant items at the top </li></ul><ul><li>Bottom line: You spend less time skimming headlines, and more time reading relevant content. </li></ul>
  8. 8. Demo! (if possible)
  9. 9. Let’s pop open the hood!
  10. 10. The RIA
  11. 11.
  12. 12.
  13. 13.
  14. 14. Solr in the Real World <ul><li>Storage of &quot;canonical data“ </li></ul><ul><li>Relational vs. Search Index </li></ul><ul><li>Complexity of custom relevancy scoring </li></ul><ul><li>&quot;Near-Real-Time&quot; updates </li></ul><ul><li>Solr in a pipeline </li></ul><ul><li>Pushing bits and marshalling cost </li></ul><ul><li>Index size, corruption, and stability </li></ul><ul><li>Administrability </li></ul>
  15. 16. Scaling Up Parse.ly <ul><li>Custom scoring </li></ul><ul><li>Multicore </li></ul><ul><li>Distributed search </li></ul><ul><li>Celery / Disco </li></ul><ul><li>User-Article Binding Problem </li></ul>
  16. 17.
  17. 18.
  18. 19.
  19. 20.
  20. 21. powered by
  21. 22. Andrew Didier Sachin
  22. 23. Quick Plug <ul><li>Does your company or enterprise </li></ul><ul><li>need our services? </li></ul>
  23. 24.
  24. 25. Twitter @amontalenti Product Twitter @parse_ly Website http://parse.ly Team Blog http://blog.cogtree.com Sign up now! It’s Free! Promo Code SLIDES Andrew Montalenti [email_address]

×