Practical Search in the Cloud - By Marc Krellenstein

884 views

Published on

See conference video - http://www.lucidimagination.com/devzone/events/conferences/revolution/2011

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
884
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Practical Search in the Cloud - By Marc Krellenstein

  1. 1. Practical Search in the Cloud (sneak preview) Martijn Koster, Lucid Imagination martijn.koster@lucidimagination.comWednesday, 25 May 2011
  2. 2. My Background  Martijn Koster  Sr. Software Engineer at Lucid Imagination • Search in early Web – WebCrawler • Customer web services – Excite Mail • Proxies for Mobile – Danger 2Wednesday, 25 May 2011
  3. 3. What I Will Cover  A Solr based Cloud Service  The need  How  Demo  Wrap up 3Wednesday, 25 May 2011
  4. 4. Why? – Getting Started is Hard  Many new concepts  Complex software configuration  System Administration overhead 4Wednesday, 25 May 2011
  5. 5. Search as a Service  A few clicks to get started  Full Power of LucidWorks and Solr  Offload operations  Expertise, tools and processes are transferable to your own deployments. 5Wednesday, 25 May 2011
  6. 6. Usage Model Users Web UI LWE JVM Lucidworks LWE JVM Solr User Code Direct Access 6Wednesday, 25 May 2011
  7. 7. Architecture 3rd party billing Provisioning Users Service (Rails) Web UI (Rails) Heroku Provisioner Heroku (Ruby) User Code Direct Access (Proxy with URL rewriting) LWE JVM LWE JVM LWE JVM SQL Control Monitoring (RDS) (Chef) (Zabbix) 7Wednesday, 25 May 2011
  8. 8. Deployment Automation  AMI  Chef  Custom 8Wednesday, 25 May 2011
  9. 9. Scaling  Heroku  Direct Access  Search Servers  Provisioning/DB  Monitoring and Control 9Wednesday, 25 May 2011
  10. 10. Direct Access Example  The Cloudy Direct Access Solr interface • Simple submission • Simple search First, copy SOLR access URL: SOLR="https://access.cloudy.io/…/solr" 11Wednesday, 25 May 2011
  11. 11. Example - Add $ cat > /tmp/doc.xml <<EOM <add><doc> <field name="id">testdoc_LR2011</field> <field name="text_en">LR rocks!</field> </doc></add> EOM curl $SOLR/my_news/update?commit=true -H "Content-Type: text/xml" --data-binary @/tmp/doc.xml 12Wednesday, 25 May 2011
  12. 12. Example - Search curl $SOLR/my_news/lucid? q=news&role=DEFAULT&wt=json curl $SOLR/my_news/select? wt=json&q=news&start=0&rows=10 13Wednesday, 25 May 2011
  13. 13. Example – Lucidworks API • The Cloudy Direct Access Lucid API interface API="http://access.demo.cloudy.io/…/api” curl $API/collections [ {"name":"lwelogs","instance_dir":"lwelogs” {"name":"collection1","instance_dir":"collection1_0"}, {"name":"my_news","instance_dir":"my_news_1"} ] 14Wednesday, 25 May 2011
  14. 14. Live Demo  Sam Pierson, Director of Product Development  The Web User Interface • Administration UI improvements • Log indexing and graphing  API access • Ruby bindings for the LucidWorks REST API • http://rubygems.org/gems/lucid_works 10Wednesday, 25 May 2011
  15. 15. Lessons  Cloud technologies speed up development  Automate everything  Avoid lock-in  Dogfooding 16Wednesday, 25 May 2011
  16. 16. Wrap Up  Search as a Service  Built in the Cloud, with open source  Full Lucidworks Enterprise and Solr  Powered by Lucid Imagination 17Wednesday, 25 May 2011
  17. 17. When  Early Access Program: July • Interested? cloud@lucidimagination.com 18Wednesday, 25 May 2011
  18. 18. Contact  Martijn Koster • martijn.koster@lucidimagination.com • http://www.lucidimagination.com/ • twitter.com/makuk66 19Wednesday, 25 May 2011

×