Lightning introduction into full-text search framework in 7 minutes Lukas Vlcek (http://blog.lukas-vlcek.com/), jOpenSpace 2009

1. Compass Framework
Lightning introduction
into full-text search framework in 7 minutes
Lukáš Vlček, jOpenSpace 2009

2. Agenda
• What is Compass
• Code snippets
• Compass integration
• Distributed indexing and searching
• Alternatives
• Resources
• Q&A

3. What it Compass
Compass Goal
quot;Simplify the integration of Search Engine into any applicationquot;
Author: Shay Banon (working for GigaSpaces)
http://www.kimchy.org/
„Using Spring Framework, Hibernate, and all the other tools out there that makes a
developer life simple, he was surprised to find nothing similar in the search engine
department. Now, don't get him wrong, Lucene is an amazing search engine (or IR
library), but developers want simplicity, and Lucene comes with an added complexity that
caused Shay to start Compass.“

4. @Searchable(alias= quot;casequot;, root = true)
Metadata definitions
@SearchableDynamicMetaData(
for the class
name = quot;descriptionquot;,
expression = quot;quot; +
quot;(data.caseComments == null ? '' : data.caseComments) + ' ' + quot; +
quot;(data.caseDescription == null ? '' : data.caseDescription) + ' ' + quot; +
quot;(data.caseDispo == null ? '' : data.caseDispo)quot;,
converter = quot;groovyquot;,
Code
termVector = TermVector.WITH_POSITIONS_OFFSETS) snippets
@Entity
@Table(name=quot;CDT_CASEquot;)
public class Case implements Serializable, LockableEntity{
@SearchableId(boost = 1.2f) Searchable ID
@SearchableMetaData(name = quot;idquot;)
@Id
@GeneratedValue(strategy = GenerationType.SEQUENCE, generator=quot;CDT_CASE_SEQquot;)
@SequenceGenerator(name=quot;CDT_CASE_SEQquot;, sequenceName=quot;CDT_CASE_SEQquot;, allocationSize=1)
@GenericGenerator(
name = quot;CDT_CASE_SEQquot;, strategy = quot;sequencequot;, parameters = {
@Parameter(name = quot;sequencequot;, value = quot;CDT_CASE_SEQquot;),
@Parameter(name = quot;allocationSizequot;, value = quot;1quot;)
}
) Searchable property
private Long id;
@SearchableProperty(name = quot;titlequot;, termVector = TermVector.WITH_POSITIONS_OFFSETS)
@Column(length=1024)
private String caseTitle; Searchable collection
(collection entitiy must have its own mapping)
@SearchableComponent
@OneToMany(fetch=FetchType.EAGER, cascade=javax.persistence.CascadeType.ALL)
@org.hibernate.annotations.Fetch(org.hibernate.annotations.FetchMode.SUBSELECT)
private Set<EventLotId> lotId;

5. Compass integration
Compass integrates with Spring in the same manner ORM Frameworks
integration is done within the Spring Framework code-base. It also
integrates with Spring transaction abstraction layer, AOP support, and MVC
library.
Integration with different ORM frameworks (Hibernate, JPA, JDO, OJB),
allowing for almost transparent integration between a search engine and an
ORM view of content that resides in a database.
Not using ORM or Spring?
Having XML data?
Having no domain model?
… never mind, Compass can still support you!

6. Distributed indexing and searching
~ Slide contributed by Shay Banon ~
Storing the index in a distributed fashion:
Compass supports storing the index on GigaSpaces, Coherence, and
Terracotta. The first two allow to partition the data, the last (tc) works a bit
different. Bind that with optimized local cache on the client side, and you
get a very nice solution where the index is stored in a data grid.
Performing both the indexing and the search in a map reduce
fashion:
You can check the blog I wrote about GigaSpaces, as well as the part
about collocated indexing and searching to understand how the
architecture would look like. This is implemented in Compass for
GigaSpaces, and probably it can be done for Coherence as well.

7. Alternatives
• Database build in full text search (Oracle, Postgres,...)
o close coupling to database
o not using Lucene
• Hibernate Search
o using Lucene
o limited to Hibernate (not for iBatis or JDBCTemplate)
• Solr
o not a direct alternative - used in different scenarios
o Compass could be used in similar scenario as Solr
o http://wiki.apache.org/solr/DataImportHandler
o http://www.sematext.com/product-db-indexer.html

8. Resources
Compass framework homepage:
http://www.compass-project.org/
Shay Banon's Blog:
http://www.kimchy.org/tag/compass/
Lucene:
http://lucene.apache.org/java/docs/index.html
Others:
Katta: Hadoop: Solr:
http://katta.sourceforge.net/ http://hadoop.apache.org/ http://lucene.apache.org/solr/
Hibernate Search:
http://search.hibernate.org/

9. More Resources...
Compass on Google App Engine
http://www.kimchy.org/searchable-google-appengine-with-compass/
Compass in media
http://www.tradingmarkets.com/.site/news/Stock%20News/2301092/

10. Thank you!
Q&A
Lukáš Vlček, lukas.vlcek@gmail.com
http://blog.lukas-vlcek.com/