KEYNOTE: Solr- Past, Present & Future

771 views
698 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
771
On SlideShare
0
From Embeds
0
Number of Embeds
105
Actions
Shares
0
Downloads
15
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

KEYNOTE: Solr- Past, Present & Future

  1. 1. Solr: Past, Present & Future Yonik Seeley LucidWorks
  2. 2. Origins  of  Solr   •  CNET  driven  to  find  alterna6ves  to  discon6nued   commercial  enterprise  search  product   •  Plan  A:  ATOMICS  (Apache  TO  MySQL  In  CNET   Search)       – Standalone  server  speaking  XML  over  HTTP   – Meet  majority  of  “search”  needs   –  hLp://conferences.oreillynet.com/cs/mysqluc2005/view/e_sess/7066   •  Plan  B:  “Something  based  on  Lucene”   – Started  Summer  2004   – First  prototype  called  “Fusion”,  later  renamed  SOLAR   (Search  On  Lucene  And  Resin)  
  3. 3. Origins  of  the  first  Solr  admin  UI  
  4. 4. New  admin  UI  
  5. 5. Timeline   (up  to  1.4)   Ini6al   prototype   CNET   produc6on   CNET   contributes   Solr  to  ASF   Solr   graduates   from   Incubator   Simple   face6ng   replica6on   highligh6ng,   dismax   Spellchecking,   CSV,  Luke   MLT,  Update   Request   Processors   QParsers   Search   Components   Mul6-­‐core   Distributed   Search   Data  Import   Handler   JMX   1.3   1.4   Sta6s6cs   Component   Java   Replica6on   Terms  and   TermVector   Components   Mul6-­‐select   face6ng   Dynamic   Clustering   1.1   1.0   1.2   4.0   3.1  
  6. 6. Solr  4   •  Solr  Cloud   – Distributed  Indexing   – No  single  points  of  failure   – Near  Real  Time  friendly  (push  replica6on)   •  NoSQL  feature  set   – Update  Durability   – Real-­‐6me  get   – Atomic  Updates   – Op6mis6c  Concurrency   •  Pseudo-­‐join,  Pivot  Face6ng,  Pseudo-­‐fields,  etc  
  7. 7. What  search  solu6on/version  are  you   currently  using?  
  8. 8. Recent  Enhancements  
  9. 9. Document  Rou6ng   80000000-­‐bfffffff   00000000-­‐3fffffff   40000000-­‐7fffffff   c0000000-­‐ffffffff   shard1  shard4   shard3   shard2   id  =  BigCo!doc5   1f27   3c71   (MurmurHash3)   q=my_query   shard.keys=BigCo!   1f27   0000   1f27   ffff  to   (hash)   shard1   numShards=4   router=compositeId  
  10. 10. Seamless  Online  Shard  Splijng   Shard2_0   Shard1   replica   leader   Shard2   replica   leader   Shard3   replica   leader   Shard2_1   1.  New  sub-­‐shards  created  in  “construc6on”  state   2.  Leader  starts  forwarding  applicable  updates,  which   are  buffered  by  the  sub-­‐shards   3.  Leader  index  is  split  and  installed  on  the  sub-­‐shards   4.  Sub-­‐shards  apply  buffered  updates  then  become   “ac6ve”  leaders  and  old  shard  becomes  “inac6ve”   update  
  11. 11. Cloud  Enhancements   •  Request  forwarding   – In  a  mul6-­‐collec6on  cluster,  any  node  can  handle/ forward  requests  for  any  collec6on   •  Collec6on  Aliases   http://localhost:8983/solr/admin/collections   ?action=CREATEALIAS   &name=northeast   &collections=NY,NJ,PA,CT,ME,MA,NH,RI,VT   •  Coming  Soon:  Shard  Aliases    
  12. 12. Schema  REST  API   •  Restlet  is  now  integrated  with  Solr   •  Get  a  specific  field   curl  http://localhost:8983/solr/schema/fields/price   {"field":{          "name":"price",          "type":"float",          "indexed":true,          "stored":true  }}   •  Get  all  fields   curl  http://localhost:8983/solr/schema/fields   •  Get  En6re  Schema!   curl  http://localhost:8983/solr/schema  
  13. 13. Dynamic  Schema   •  Add  a  new  field  (Solr  4.4)   curl  -­‐XPUT  http://localhost:8983/solr/schema/fields/strength  -­‐d  ‘ {"type":”float",  "indexed":"true”}       ‘   •  Works  in  distributed  (cloud)  mode  too!   •  Future:  More  schemaless   – Reality:  there  is  no  such  thing  for  Lucene  based  systems   – Type  guessing  for  fields  we  haven’t  seen  before    
  14. 14. Future   •  Greater  scalability   •  More  “NoSQL”   – More  ways  to  update  &  manipulate  documents   •  Analy6cs   – More  powerful  face6ng,  func6ons,  sta6s6cs   •  Improved  Rela6onal  queries   •  More  dynamic  (sejngs  &  configura6on)   •  Con6nued  focus  on  ease  of  use  
  15. 15. Thank  You!  

×