Code & Beer Topic: Apache Solr

1. by NNNN (周世恩)
Code & Coffee 2013/11/1

2. What is Solr?

3. What is

4. • Full-featured text search
• High performance
• index size: 20-30% the size of text data.
• small RAM requirements(~1MB)
• Powerful, Accurate and Efficient Search
Algorithms
• 100% in Java(^^)

5. Lucene(cont.)
• Multiple Analyzer / Tokenizer
• Fields Searching
• Merge results
• Flexible faceting, highlighting, joins and
result grouping
• Typo-tolerant suggesters(當然要⾃自⼰己建⽴立)
• Customize ranking model..(VSM, BM25)

6. Lucene(Query)
http://lucene.apache.org/core/4_0_0/core/org/apache/lucene/search/Query.html

7. http://www.ibm.com/developerworks/cn/java/j-lo-lucene1/fig001.jpg

8. Where is index file
stored?
• Memory
• File System
• HDFS
• FileSystem config 設定為 HDFS

9. #Note
• 只有被index的field 才可以search
• 可以純store 不index
• ⽀支援多種Type(Long, Int, String, Text...)
• Indexing 就要決定好Tokenizer(Analyzer)
了
• ⽀支援同時searching and indexing?

10. #Note
• 使⽤用前搖⼀一搖
• ⼀一開始就要清楚有哪些Field
• 降低重建index的機會(RDB只要打個指令
就好)

11. Lucene Index file項⺫⽬目很多, 少⼀一個你就GG

12. What is Solr?

13. 超屌
企業級
免費
的

14. Search Platform

15. Lucene 功能該有的
都有了

16. Solr 還多了....
• 漂亮的Admin Interface！
• REST-like API(易與其他App結合)
• Dynamic clustering
• Database integration
• Geospatial search(Google Map?)
• 調整Cache Size

17. 還記得雲端的優勢...
• Highly reliable
• Scalable
• Fault tolerant
• Distributed indexing
• Replication
• Load-balanced
• Automated failover and recovery(?)

19. 常⽤用的config
• schema.xml(定義每個field)
• solrconfig.xml (定義每個handler的URI)
• jetty.xml(!)
• solr.xml(定義core的數量)

20. Real-time indexing?

21. Near Real-time indexing
• Documents are available for search almost
immediately after being indexed...
• 也要有commit 才算數(....)
https://cwiki.apache.org/confluence/display/solr/Near+Real+Time+Searching

22. Searching
• Query
“id: {id} AND name:{Name} OR title:{text}”
• Highlighting
• Projection
• Sorting(asc, desc)
• Output format: JSON, CSV, XML
• Others: spellcheck, Wildcard Query, +-*/

23. Sample Output

24. Import Data From DB
• 在solrconfig.xml 修改
http://wiki.apache.org/solr/DataImportHandler

25. The diff between
Solr and RDB
• Solr is for indexed text or lots of
unstructured docs.
• Solr is optimized for searching, not for
storage and retrieval of individual records.
http://stackoverflow.com/questions/5814050/solr-or-database

26. Distributed Search cluster
• 很多台機器架設 Solr, 選⼀一台來進⾏行聯結
• 需要在config設定嗎?

27. Distributed Solr Cluster
& Load balancer
http://wiki.apache.org/solr/SolrReplication

28. http://wiki.apache.org/solr/SolrReplication

29. #Note
• 你可以先⽤用包有lucene indexing 功能的
java application 先製作好index directory再
給solr ⽤用
• 如果solr要進⾏行update時, 最好先確認沒有
其他application正在進⾏行寫⼊入的程序, 否則
GG
• indexing 時, 不管是solr還是lucene, writelock不要亂刪

30. Live Demo
眾神們曾經說過這很危險的

31. 下載Solr 最新版
$: cd solr-4.4.0/example
$: java -Xmx2048m
-jar start.jar

32. The End

33. 好Tool 分享
• Luke(檢查index⽤用)
• Apache Tika
• Apache hadoop
• Apache Tomcat

34. BBQ(Bonus)
• Customize tokenizer
• Document Boosting
• Field Boosting
• Field aliasing / renaming