0
Making	
  Your	
  Domain	
  Objects	
  
            Searchable	
  with	
  Hibernate
                        Search
       ...
Agenda
       Mo#va#ons	
  and	
  Goals
       Indexing
       Retrieval
       Scalability




    Apache	
  Lucene	
...
Hibernate	
  in	
  a	
  nutshell




    Apache	
  Lucene	
  EuroCon   20	
  May	
  2010   IntroducIon	
  ◆	
  Indexing	
 ...
Hibernate	
  in	
  a	
  nutshell
    @Entity                                           @Entity
    public class Author {  ...
Hibernate	
  in	
  a	
  nutshell
    @Entity                                           @Entity
    public class Author {  ...
Hibernate	
  in	
  a	
  nutshell
    @Entity                                           @Entity
    public class Author {  ...
Meet	
  Hibernate	
  Search
       Hibernate	
  extension	
  which	
  uses	
  Lucene	
  internally

       Bring	
  full...
Meet	
  Hibernate	
  Search
       Current	
  version:	
  3.2.0-­‐Final	
  (May/2010)
       LGPL	
  License
       Luc...
Meet	
  Hibernate	
  Search
       Dependencies:
        <dependency>
              <groupId>org.hibernate</groupId>
    ...
Indexing
       Mapping	
  Objects	
  <-­‐>	
  Documents
       Support	
  for	
  types
       Analyzers/Boost	
  
    ...
Indexing	
  -­‐	
  Mapping	
  EnIIes
    @Entity
    public class Author {

    	         @Id @GeneratedValue
    	       ...
Indexing	
  -­‐	
  Mapping	
  EnIIes
    @Indexed
    @Entity
    public class Author {

    	         @Id @GeneratedValue...
Indexing	
  -­‐	
  Mapping	
  EnIIes
    @Indexed
    @Entity
    public class Author {

    	         @Id @GeneratedValue...
Indexing	
  -­‐	
  Mapping	
  EnIIes
    @Indexed(index=”Author_Index”)
    @Entity
    public class Author {
    	
    	 ...
Indexing	
  -­‐	
  Mapping	
  EnIIes
    @Indexed(index=”Author_Index”)
    @Entity
    public class Author {
    	
    	 ...
Indexing	
  -­‐	
  Mapping	
  Fields
    @Indexed(index=”Author_Index”)
    @Entity
    public class Author {
    	
    	 ...
Indexing	
  -­‐	
  Mapping	
  Fields
    @Indexed(index=”Author_Index”)
    @Entity
    public class Author {
    	
    	 ...
Indexing	
  -­‐	
  Mapping	
  Fields
    @Indexed(index=”Author_Index”)
    @Entity
    public class Author {
    	
    	 ...
Indexing	
  -­‐	
  Mapping	
  RelaIonships
    @Indexed(index=”Author_Index”)
    @Entity
    public class Author {
    	
...
Indexing	
  -­‐	
  Types
    @Indexed(index=”Author_Index”)
    @Entity
    public class Author {
    	
    	   @Id @Gener...
Indexing	
  -­‐	
  Boost
    @Indexed(index=”Author_Index”)
    @Entity
    public class Author {
    	
    	   @Id @Gener...
Indexing	
  -­‐	
  Analyzers
    @Entity @Indexed
    public class Author {
    	
    	   @Id @GeneratedValue @DocumentId
...
Indexing	
  -­‐	
  Analyzers
    @Entity @Indexed
    @AnalyzerDef(name=”combinedAnalyzers”,
       tokenizer = @Tokenizer...
Indexing	
  -­‐	
  Analyzers
    @Entity @Indexed
    @AnalyzerDef(name=”combinedAnalyzers”,
       tokenizer = @Tokenizer...
Indexing	
  -­‐	
  Analyzers
    @Entity @Indexed
    @AnalyzerDef(name=”combinedAnalyzers”,
       charFilters = { @CharF...
Indexing	
  -­‐	
  Analyzers
    @Entity @Indexed
    @AnalyzerDef(name=”combinedAnalyzers”,
       charFilters = { @CharF...
Index	
  -­‐	
  Fluent	
  API
             SearchMapping mapping = new SearchMapping();
                       mapping
   ...
Indexing	
  -­‐	
  Backend




                                                                     Source:	
  Hibernate	
...
Indexing	
  -­‐	
  Backend




                                                                       Source:	
  Hibernate...
Indexing	
  -­‐	
  JMS	
  backend	
  




                                                                      Source:	
 ...
Manual	
  Indexing
       Use	
  case
                 Non-­‐exclusive	
  database

       Manual	
  Indexing	
  types:...
Manual	
  Indexing	
  -­‐	
  Single	
  EnIty
    FullTextSession fullTextSession = Search.getFullTextSession(session);

  ...
Mass	
  Indexing
    fullTextSession.createIndexer().startAndWait();
    fullTextSession.createIndexer().start();




    ...
Retrieval	
  -­‐	
  Lucene	
  Queries	
  +	
  Hibernate	
  API
         // Wraps Hibernate Session Object
    org.hibernat...
Retrieval	
  -­‐	
  Hibernate	
  Search
    1.	
  Executes	
  Lucene	
  Query	
  and	
  get	
  the	
  results

    2.	
  R...
Retrieval	
  -­‐	
  Results	
  ManipulaIon
        Pagina#on
        Type	
  restric#on
        Projec#on
        Resu...
Retrieval	
  -­‐	
  IndexReader
        shared	
  strategy:	
  shared	
  IndexReader	
  (default)
         	
  	
  	
  	
...
Scalability


    Sharding




    Clustering




    Apache	
  Lucene	
  EuroCon   20	
  May	
  2010   IntroducIon	
  ◆...
Scalability	
  -­‐	
  Sharding
    • Default:	
  one	
  index	
  per	
  en#ty	
  type

                                   ...
Scalability	
  -­‐	
  Sharding
       Indexes	
  separated	
  physically
       Virtual	
  Index                        ...
Scalability	
  -­‐	
  Sharding
       Configura#on
         hibernate.search.com.sourcesense.Author.sharding_strategy.nbr_...
Scalability	
  -­‐	
  Shard	
  Strategy
       Default	
  algorithm:	
  	
  ID	
  Hash
                                  ...
Custom	
  Sharding	
  Strategy
       Implement	
  IndexShardingStrategy

      hibernate.search.com.sourcesense.Author.s...
Synchronous	
  Clustering
       Every	
  node	
  can	
  read	
  and	
  write	
  to	
  the	
  index
       Pessimist	
  ...
Clustering
       Read-­‐Write	
  Synchronous	
  cluster
                                                       Node 2
  ...
Asynchronous	
  
    Clustering




    Apache	
  Lucene	
  EuroCon   20	
  May	
  2010   IntroducIon	
  ◆	
  Indexing	
  ...
Asynchronous	
  Cluster
       Advantages
                 Only	
  master	
  writes
                 No	
  indexing	
  ...
To	
  learn	
  more...


                                                      hibernate.org/subprojects/search.html

    ...
Thank	
  you
                                                      g.fernandes@sourcesense.com

                          ...
Upcoming SlideShare
Loading in...5
×

Making your domain objects searchable with Hibenate Search

2,577

Published on

Presentation about Hibernate Search done in Lucene Apache Eurocon at Prague, Czech Republic on May 20th

Published in: Technology, Business
1 Comment
3 Likes
Statistics
Notes
No Downloads
Views
Total Views
2,577
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
49
Comments
1
Likes
3
Embeds 0
No embeds

No notes for slide

Transcript of "Making your domain objects searchable with Hibenate Search"

  1. 1. Making  Your  Domain  Objects   Searchable  with  Hibernate Search Gustavo  Fernandes Sunday, 23 May 2010
  2. 2. Agenda  Mo#va#ons  and  Goals  Indexing  Retrieval  Scalability Apache  Lucene  EuroCon 20  May  2010 2 Sunday, 23 May 2010
  3. 3. Hibernate  in  a  nutshell Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 3 Sunday, 23 May 2010
  4. 4. Hibernate  in  a  nutshell @Entity @Entity public class Author { public class Book { @Id @GeneratedValue private Integer id; private Integer id; private String title; private String name; } @OneToMany private Set<Book> books; } Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 4 Sunday, 23 May 2010
  5. 5. Hibernate  in  a  nutshell @Entity @Entity public class Author { public class Book { @Id @GeneratedValue private Integer id; private Integer id; private String title; private String name; } @OneToMany private Set<Book> books; } Author author = new Author(“Stephen King”); Book aBook = new Book(“Blaze”); HashSet<Book> books = new HashSet<Book>(); books.add(aBook); author.setBooks(books); Session session = sessionFactory.openSession(); Transaction tx = session.beginTransaction(); session.save(author); tx.commit(); Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 5 Sunday, 23 May 2010
  6. 6. Hibernate  in  a  nutshell @Entity @Entity public class Author { public class Book { @Id @GeneratedValue private Integer id; private Integer id; private String title; private String name; } @OneToMany private Set<Book> books; } Author author = new Author(“Stephen King”); Book aBook = new Book(“Blaze”); HashSet<Book> books = new HashSet<Book>(); books.add(aBook); author.setBooks(books); Session session = sessionFactory.openSession(); Transaction tx = session.beginTransaction(); session.save(author); tx.commit(); Select * from Author; Select * from Book; Select * from Book_Author; +----+--------------+ +----+----------+ +---------+------------+ | id | name | | id | title | | Book_id | authors_id | +----+--------------+ +----+----------+ +---------+------------+ | 1 | Stephen King | | 1 | Blaze | | 1 | 1 | +----+--------------+ +----+----------+ +---------+------------+ Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 6 Sunday, 23 May 2010
  7. 7. Meet  Hibernate  Search  Hibernate  extension  which  uses  Lucene  internally  Bring  full  text  search  capabiliIes  to  Hibernate  Object-­‐Document  mapping  Take  care  of  the  plumbing  Keep  database  and  index  in  sync  ConvenIon  over  configuraIon  Flexible Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 7 Sunday, 23 May 2010
  8. 8. Meet  Hibernate  Search  Current  version:  3.2.0-­‐Final  (May/2010)  LGPL  License  Lucene  version  supported:  2.9.2  Solr  version  supported:  1.4 Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 8 Sunday, 23 May 2010
  9. 9. Meet  Hibernate  Search  Dependencies: <dependency> <groupId>org.hibernate</groupId> <artifactId>hibernate-search</artifactId> <version>3.2.0.Final</version> </dependency> Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 9 Sunday, 23 May 2010
  10. 10. Indexing  Mapping  Objects  <-­‐>  Documents  Support  for  types  Analyzers/Boost    Transparent/Manual  Indexing Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 10 Sunday, 23 May 2010
  11. 11. Indexing  -­‐  Mapping  EnIIes @Entity public class Author { @Id @GeneratedValue private Integer id; private String name; @OneToMany private Set<Book> books; } Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 11 Sunday, 23 May 2010
  12. 12. Indexing  -­‐  Mapping  EnIIes @Indexed @Entity public class Author { @Id @GeneratedValue private Integer id; private String name; @OneToMany private Set<Book> books; } Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 12 Sunday, 23 May 2010
  13. 13. Indexing  -­‐  Mapping  EnIIes @Indexed @Entity public class Author { @Id @GeneratedValue private Integer id; private String name; @OneToMany private Set<Book> books; } Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 13 Sunday, 23 May 2010
  14. 14. Indexing  -­‐  Mapping  EnIIes @Indexed(index=”Author_Index”) @Entity public class Author { @Id @GeneratedValue private Integer id; private String name; @OneToMany private Set<Book> books; } Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 14 Sunday, 23 May 2010
  15. 15. Indexing  -­‐  Mapping  EnIIes @Indexed(index=”Author_Index”) @Entity public class Author { @Id @GeneratedValue @DocumentId private Integer id; private String name; @OneToMany private Set<Book> books; } Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 15 Sunday, 23 May 2010
  16. 16. Indexing  -­‐  Mapping  Fields @Indexed(index=”Author_Index”) @Entity public class Author { @Id @GeneratedValue @DocumentId private Integer id; @Field private String name; @OneToMany private Set<Book> books; } Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 16 Sunday, 23 May 2010
  17. 17. Indexing  -­‐  Mapping  Fields @Indexed(index=”Author_Index”) @Entity public class Author { @Id @GeneratedValue @DocumentId private Integer id; @Field(name = name_field, store = Store.YES, index = Index.TOKENIZED) private String name; @OneToMany private Set<Book> books; } Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 17 Sunday, 23 May 2010
  18. 18. Indexing  -­‐  Mapping  Fields @Indexed(index=”Author_Index”) @Entity public class Author { @Id @GeneratedValue @DocumentId private Integer id; @Fields( { @Field(index = Index.TOKENIZED), @Field(name= “nameForSort”, index = Index.UN_TOKENIZED) } ) private String name; @OneToMany private Set<Book> books; } Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 18 Sunday, 23 May 2010
  19. 19. Indexing  -­‐  Mapping  RelaIonships @Indexed(index=”Author_Index”) @Entity public class Author { @Id @GeneratedValue @DocumentId private Integer id; @Field(index = Index.TOKENIZED) private String name; @OneToMany @IndexEmbedded private Set<Book> books; } Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 19 Sunday, 23 May 2010
  20. 20. Indexing  -­‐  Types @Indexed(index=”Author_Index”) @Entity public class Author { @Id @GeneratedValue @DocumentId private Integer id; @Field(index = Index.TOKENIZED) private String name; @OneToMany @IndexEmbedded private Set<Book> books; @Field(bridge = @FieldBridge(impl = AddressBridge.class)) private Adress address; } Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 20 Sunday, 23 May 2010
  21. 21. Indexing  -­‐  Boost @Indexed(index=”Author_Index”) @Entity public class Author { @Id @GeneratedValue @DocumentId private Integer id; @Field(index = Index.TOKENIZED) @Boost(1.5f) private String name; @OneToMany @IndexEmbedded private Set<Book> books; @Field(bridge = @FieldBridge(impl = AddressBridge.class)) @Boost(0.75f) private Adress address; } Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 21 Sunday, 23 May 2010
  22. 22. Indexing  -­‐  Analyzers @Entity @Indexed public class Author { @Id @GeneratedValue @DocumentId private Integer id; private String bio; ... } Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 22 Sunday, 23 May 2010
  23. 23. Indexing  -­‐  Analyzers @Entity @Indexed @AnalyzerDef(name=”combinedAnalyzers”, tokenizer = @TokenizerDef(factory = StandardTokenizerFactory.class), ) public class Author { @Id @GeneratedValue @DocumentId private Integer id; private String bio; ... } Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 23 Sunday, 23 May 2010
  24. 24. Indexing  -­‐  Analyzers @Entity @Indexed @AnalyzerDef(name=”combinedAnalyzers”, tokenizer = @TokenizerDef(factory = StandardTokenizerFactory.class), filters = { @TokenFilterDef(factory = LowerCaseFilterFactory.class) } ) public class Author { @Id @GeneratedValue @DocumentId private Integer id; private String bio; ... } Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 24 Sunday, 23 May 2010
  25. 25. Indexing  -­‐  Analyzers @Entity @Indexed @AnalyzerDef(name=”combinedAnalyzers”, charFilters = { @CharFilterDef(factory = MappingCharFilterFactory.class) }, tokenizer = @TokenizerDef(factory = StandardTokenizerFactory.class), filters = { @TokenFilterDef(factory = LowerCaseFilterFactory.class) } ) public class Author { @Id @GeneratedValue @DocumentId private Integer id; private String bio; ... } Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 25 Sunday, 23 May 2010
  26. 26. Indexing  -­‐  Analyzers @Entity @Indexed @AnalyzerDef(name=”combinedAnalyzers”, charFilters = { @CharFilterDef(factory = MappingCharFilterFactory.class) }, tokenizer = @TokenizerDef(factory = StandardTokenizerFactory.class), filters = { @TokenFilterDef(factory = LowerCaseFilterFactory.class) } ) public class Author { @Id @GeneratedValue @DocumentId private Integer id; @Analyzer(definition = “combinedAnalyzers”) private String bio; ... } Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 26 Sunday, 23 May 2010
  27. 27. Index  -­‐  Fluent  API SearchMapping mapping = new SearchMapping(); mapping .analyzerDef("customAnalyzer", StandardTokenizerFactory.class) .filter(LowerCaseFilterFactory.class) .filter(SnowballPorterFilterFactory.class) .param("language", "English") .entity(Author.class) .indexed() .property("id",ElementType.FIELD).documentId() .property("adress", ElementType.FIELD) .field().bridge(AdressBrigde.class).store(Store.YES) .property("books", ElementType.FIELD).indexEmbedded() .property("name", ElementType.METHOD).field().store(Store.YES) .entity(Book.class) .indexed() .property("id", ElementType.METHOD).documentId() .property("title", ElementType.METHOD) .field().analyzer("customAnalyzer"); Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 27 Sunday, 23 May 2010
  28. 28. Indexing  -­‐  Backend Source:  Hibernate  Search  in  AcIon Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 28 Sunday, 23 May 2010
  29. 29. Indexing  -­‐  Backend Source:  Hibernate  Search  in  AcIon  hibernate.work.execu#on    async  hibernate.work.thread_pool_size    10 Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 29 Sunday, 23 May 2010
  30. 30. Indexing  -­‐  JMS  backend   Source:  Hibernate  Search  in  AcIon  hibernate.worker.backend        jms  hibernate.worker.jms.connec#on_factory        /Connec#onFactory  hibernate.worker.jms.queue      queue/hsearch Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 30 Sunday, 23 May 2010
  31. 31. Manual  Indexing  Use  case  Non-­‐exclusive  database  Manual  Indexing  types:  Single  enIty  Mass  indexer Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 31 Sunday, 23 May 2010
  32. 32. Manual  Indexing  -­‐  Single  EnIty FullTextSession fullTextSession = Search.getFullTextSession(session); Transaction tx = fullTextSession.beginTransaction(); Object author = fullTextSession.load( Author.class, 1 ); fullTextSession.index(author); tx.commit(); Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 32 Sunday, 23 May 2010
  33. 33. Mass  Indexing fullTextSession.createIndexer().startAndWait(); fullTextSession.createIndexer().start(); Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 33 Sunday, 23 May 2010
  34. 34. Retrieval  -­‐  Lucene  Queries  +  Hibernate  API // Wraps Hibernate Session Object org.hibernate.seach.FullTextSession fullTextSession = org.hibernate.search.Search.getFullTextSession(session); // Lucene query Version v = Version.LUCENE_29; org.apache.lucene.queryParser.QueryParser queryParser = new org.apache.lucene.queryParser.QueryParser(v, "name", new StandardAnalyzer (v)); org.apache.lucene.search.Query query = queryParser.parse("+King"); // Hibernate search query org.hibernate.Query textQuery = fullTextSession.createFullTextQuery(query, Author.class); Author loadedAuthor = (Author)textQuery.list(); Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 34 Sunday, 23 May 2010
  35. 35. Retrieval  -­‐  Hibernate  Search 1.  Executes  Lucene  Query  and  get  the  results 2.  Retrieves  document  ids  from  the  index 3.  Load  objects  from  database   4.  Return  domain  objects Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 35 Sunday, 23 May 2010
  36. 36. Retrieval  -­‐  Results  ManipulaIon  Pagina#on  Type  restric#on  Projec#on  Result  mapping Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 36 Sunday, 23 May 2010
  37. 37. Retrieval  -­‐  IndexReader  shared  strategy:  shared  IndexReader  (default)          hibernate.search.reader.strategy = shared  not-­‐shared  strategy:  open  IndexReader  for  every  query          hibernate.search.reader.strategy = not-shared  Extensible  by  using  ReaderProvider  Interface hibernate.search.reader.strategy = com.mycompany.CoolReaderProvider Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 37 Sunday, 23 May 2010
  38. 38. Scalability Sharding Clustering Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 38 Sunday, 23 May 2010
  39. 39. Scalability  -­‐  Sharding • Default:  one  index  per  en#ty  type Query Application Index A-Z Index • Shard:  two  or  more  indexes  per  en#ty  type Shard A A-H • Use  cases   Application Shard B I-N • Performance • Maintenance Shard C O-Z Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 39 Sunday, 23 May 2010
  40. 40. Scalability  -­‐  Sharding  Indexes  separated  physically  Virtual  Index Shard A A-H Query Application Virtual I-N Shard B Index Index Shard C O-Z Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 40 Sunday, 23 May 2010
  41. 41. Scalability  -­‐  Sharding  Configura#on hibernate.search.com.sourcesense.Author.sharding_strategy.nbr_of_shard 2 Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 41 Sunday, 23 May 2010
  42. 42. Scalability  -­‐  Shard  Strategy  Default  algorithm:    ID  Hash 1 2 1 Shard 1 4 2 3 f(x) = x % N 4 5 3 Shard 2 5 Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 42 Sunday, 23 May 2010
  43. 43. Custom  Sharding  Strategy  Implement  IndexShardingStrategy hibernate.search.com.sourcesense.Author.sharding_strategy BookTitleStrategy Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 43 Sunday, 23 May 2010
  44. 44. Synchronous  Clustering  Every  node  can  read  and  write  to  the  index  Pessimist  locking  prevents  corrup#on  Single  index  shared  among  every  node  Choose  your  flavour:  NFS,  Database,  distributed  caches Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 44 Sunday, 23 May 2010
  45. 45. Clustering  Read-­‐Write  Synchronous  cluster Node 2 Index Writer Node 3 Node 1 Index Index Writer Writer Node 4 Index Index Writer Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 45 Sunday, 23 May 2010
  46. 46. Asynchronous   Clustering Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 46 Sunday, 23 May 2010
  47. 47. Asynchronous  Cluster  Advantages  Only  master  writes  No  indexing  in  slaves  -­‐>  no  waiIng  for  locks  Downside  Data  is  not  visible  immediately  by  the  slaves Apache  Lucene  EuroCon 20  May  2010 IntroducIon  ◆  Indexing  ◆  Retrieval  ◆  Scaling 47 Sunday, 23 May 2010
  48. 48. To  learn  more... hibernate.org/subprojects/search.html anonsvn.jboss.org/repos/hibernate/search/ Apache  Lucene  EuroCon 20  May  2010 48 Sunday, 23 May 2010
  49. 49. Thank  you g.fernandes@sourcesense.com twicer:  @gustavonalle Apache  Lucene  EuroCon 20  May  2010 49 Sunday, 23 May 2010
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×