This document provides an introduction to enterprise search and its key components. It discusses how search engines work by building indexes on text and answering queries using those indexes. The two main components are indexing, which structures data for easy searching, and search, which returns results based on user queries against the index. It introduces common file formats that can be indexed like text, HTML, PDFs. Lucene and Solr are introduced as open source search libraries, with Solr building on Lucene and adding features like indexing, querying via HTTP, and admin interfaces. The document demonstrates adding, deleting, and searching for documents in Solr.

1. ENTERPRISE SEARCH
an introduction

2. Web Search
Desktop Search
Enterprise Search

3. so what is a
Search Engine?

4. a SOFTWARE
• that builds index on Text
• answers queries using that index

5. Any search application has
two major components
SEARCH component
INDEXING component
- of importance to us developers
(read headache)
- of importance to the users

6. data
INDEX FILES
is indexed
user
sends
search query
receives
search results
INDEXING component SEARCH component

7. Let’s start with
INDEXING

8. is it easy to search here . . .

9. or here . . .

10. • that’s information like garbage
• no structure
• comes in all kinds of
shapes, sizes, formats

11. • And this is what indexing does
• Makes data accessible in a
structured format, easily accessible
through search.

12. so what all needs to be
Indexed and Searched ?

13. various FILE FORMATS
Text Files
HTML
PDF
MS Word
PPT

14. coming from various DATA SOURCES
Emails
CMS
File System
Database
Web Pages

15. data ( documents )
INDEX FILES
user
sends
search query
receives
search results
Analyzer
fed to
text that should be indexed
removing stop words such as "a" or "the"
converting all text to lowercase letters
for case-insensitive searching
Stemming
(A stemming algorithm reduces
the words "fishing", "fished",
"fish", and "fisher" to the root word, "fish". )-
Index Writer
tokenized text

16. Document 1:
Coffee isn't my cup of tea.
Document 2:
Chocolate, men, coffee - some things are better rich.
INDEX
coffee - 1,2
cup - 1
tea - 1
chocolate - 1
men - 1
things - 1
better - 1
rich - 1

17. And now the
SEARCH Component

18. data
INDEX FILES
is indexed
user
receives
search results
sends
search query
search terms

19. Search Request Terms
Taxonomy
Spelling Index
Correct Search Terms + Incorrect Search Terms
Search Terms +
Related Terms from Taxonomy + Concept IDs
Search engine
(INDEX)
Search results with
1) Actual Location of the result
2) Rank
3) Details
4) Facet Categorization
Results’ Page

20. introducing
LUCENE

21. Full-text search library
Open Source
Documents in xml format
Can operate on its own or via Solr

24. Ways of storing fields of any document:
Indexed means it is searchable
Stored you may chose not to make a field searchable, means the content can be
displayed in the search results. Example : “summary associated with a page”
Tokenized means it is run through an Analyzer, that converts the content into
a sequence of tokens

25. introducing
SOLR
Solr
Solr
Lucene
Index

26. • open source
• handles index/Query to Lucene via HTTP and XML
( also JSON )
• manages document update, add and delete
requests to Lucene
• straightforward schema and config files
• comprehensive HTML Admin Interfaces
• highly configurable

27. Adding Documents
to SOLR

28. HTTP POST to /update
<add><doc boost=“2”>
<field name=“type”>05991</field>
<field name=“from”>Apache Solr</field>
<field name=“subject”>An intro...</field>
<field name=“category”>search</field>
<field name=“category”>lucene</field>
<field name=“body”>Solr is a full...</field>
</doc></add>

29. Schema.xml
field indexing and display definition

30. Solrconfig.xml file
defines cache size, faceted field type, request handler customization

31. Deleting Documents
• Delete by Id
<delete><id>05591</id></delete>
• Delete by Query (multiple documents)
<delete>
<query>manufacturer:microsoft</query>
</delete>

32. Search Results
http://localhost:8983/solr/select?q=video&start=0&rows=2&fl=name,price

33. Default Parameters
param default description
q The query
start 0 Offset into the list of matches
rows 10 Number of documents to return
fl * Stored fields to return
qt standard Query type; maps to query handler
df (schema) Default field to search
http://localhost:8983/solr/select?q=video&start=0&rows=2&fl=name,price

34. <response><responseHeader><status>0</status>
<QTime>1</QTime></responseHeader>
<result numFound="16173" start="0">
<doc>
<str name="name">Apple 60 GB iPod with Video</str>
<float name="price">399.0</float>
</doc>
<doc>
<str name="name">ASUS Extreme N7800GTX/2DHTV</str>
<float name="price">479.95</float>
</doc>
</result>
</response>

35. Solr Core
Lucene
Admin
Interface
Standard
Request
Handler
Disjunction
Max
Request
Handler
Custom
Request
Handler
Update
Handler
Caching
XML
Update
Interface
Config
Analysis
HTTP Request Servlet
Concurrency
Update Servlet
XML
Response
Writer
Replication
Schema
Search Requests hit here New document to be added here

37. FAQ
WebSite:www.solr.cc
QQGroup: 187670960