SlideShare a Scribd company logo
1 of 18
Responsive Facets
with Apache Solr
Dallas, TX
February 12, 2013




Discussion document – Strictly Confidential & Proprietary
Agenda …


What we will cover tonight

• Concept of Facets

• The Problem with RDBMS

• Solr to the Rescue

• Solr by example

• Architecture basics

• Putting data in

• Getting data out




                             Responsive Facets with Apache Solr
                                             February 12, 2013    2
Concept of facets…


Sometimes the facets only have groups with values




                                                    Responsive Facets with Apache Solr
                                                                    February 12, 2013    3
Concept of facets…


Sometimes, they also include a count for the value




                                                     Responsive Facets with Apache Solr
                                                                     February 12, 2013    4
The Problem with RDBMS…


Easy enough, just attributes of the product…




                                               Responsive Facets with Apache Solr
                                                               February 12, 2013    5
The Problem with RDBMS…


Easy enough, just attributes of the product… Or is it!




                                                         Responsive Facets with Apache Solr
                                                                         February 12, 2013    6
The Problem with RDBMS…


Easy enough, just attributes of the product… Or is it!




                                                         Responsive Facets with Apache Solr
                                                                         February 12, 2013    7
The Problem with RDBMS…


Easy enough, just attributes of the product… Or is it!




                                                         Responsive Facets with Apache Solr
                                                                         February 12, 2013    8
The Problem with RDBMS…


Easy enough, just attributes of the product… Or is it!




                                                         Responsive Facets with Apache Solr
                                                                         February 12, 2013    9
Solr to the rescue…


We need a better way to organize our data for fast queries




Solr is a standalone search server built on the Lucene Search Library. It offers all the
capabilities of the Lucene library and extends the capabilities. Solr interactions are handled
through a REST-like API, allowing you to input and retrieve data in a variety of formats.

Solr can run within most servlet containers including Tomcat and Jetty.

• Full-Text Search Capabilities
• Standards Based Open Interfaces – XML, JSON and HTTP
• Linearly scalable, auto index replication, auto failover and recovery
• Near Real-time indexing
• SolrJ library capable of binary over HTTP communication
• Faceted Search and Filtering




                                                                              Responsive Facets with Apache Solr
                                                                                              February 12, 2013    10
Solr to the rescue…


Products are inserted into Solr as a Document

• Fields and Dynamic Fields

• Unique Key on a specific field

• Fields are typed

   –   String is a literal

   –   Text is a tokenized string

• CopyField allows you to populate multiple fields off a single field insert




                                                                               Responsive Facets with Apache Solr
                                                                                               February 12, 2013    11
Solr by example…


Solr utilizes a REST-like interface with XML, JSON and binary capabilities

Let’s look at a few examples…




                                                                Responsive Facets with Apache Solr
                                                                                February 12, 2013    12
Architecture basics…


Running Solr

• Embedded

   –   Makes your application more bloated

   –   More resource intensive

• Standalone

   –   Run on it’s own server

   –   Network traffic hit

   –   Replication is an option

• SolrCloud

   –   High availability

   –   Sharded with replication

   –   http://bit.ly/TdAJBw

                                             Responsive Facets with Apache Solr
                                                             February 12, 2013    13
Architecture basics…


Our environment




                       Responsive Facets with Apache Solr
                                       February 12, 2013    14
Architecture basics…


 Communicating withSolr

 SolrJ

    •    Java binary communication

    •    Library that abstracts connection, query formation and response processing

 Inject Server
<bean id="solrServer" class="org.apache.solr.client.solrj.impl.LBHttpSolrServer">
<constructor-argvalue="${solr.url.primary}"/>
</bean>


 Inject Search Service
<bean id="blSearchService" class="com.vology.core.service.search.VolSearchServiceImpl">
<constructor-argname="solrServer" ref="${solr.source.primary}" />
</bean>

 Utilize SolrJ Library

    •    ServerContext
                                •   SolrQuery

    •    SolrDocument
                                •   QueryResponse


                                                                                          Responsive Facets with Apache Solr
                                                                                                          February 12, 2013    15
Putting data in…


How do we get the data into Solr?

Transform fields
• Populate transient fields
• Determine terminating object


Query product data from DB
• Utilize Hibernate to retrieve relevant products


Populate SolrJSolrDocument
• Java Reflection to get fields we want
• SolrField data from Field


Add SolrDocument
• Group into a List<SolrDocument>
• Add full set or incremental
• Commit




                                                    Responsive Facets with Apache Solr
                                                                    February 12, 2013    16
Getting data out


How do we get the data out of Solr?

Build SolrQuery
• Query searchable fields from Field table
• Creates ‘&’ delimited list for qf from Field table
• We utilize a boost value (^100.0)


Add Search Facets
• Uses search facet and search facet range tables
• Creates ‘&’ delimited list of facet.field and facet.range


Query on ServerContext
• server.query(SolrQuery)
• QueryResponse returned


Populate ProductSearchResult
• Set facets
• Set results
• Passed back to SearchController



                                                              Responsive Facets with Apache Solr
                                                                              February 12, 2013    17
Contact ...


Tell me what you thought!



              blemons@credera.com

              @brentlemons

              slideshare.net/brentlemons

              linkedin.com/in/brentlemons

              github.com/brentlemons




Do you think this presentation should be at OSCon?

• Tweet @oscon to let them know!




                                                     Responsive Facets with Apache Solr
                                                                     February 12, 2013    18

More Related Content

What's hot

Introduction to Scala
Introduction to ScalaIntroduction to Scala
Introduction to ScalaRahul Jain
 
A Scala tutorial
A Scala tutorialA Scala tutorial
A Scala tutorialDima Statz
 
Scala @ TechMeetup Edinburgh
Scala @ TechMeetup EdinburghScala @ TechMeetup Edinburgh
Scala @ TechMeetup EdinburghStuart Roebuck
 
Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009Martin Odersky
 
Martin Odersky - Evolution of Scala
Martin Odersky - Evolution of ScalaMartin Odersky - Evolution of Scala
Martin Odersky - Evolution of ScalaScala Italy
 
The Evolution of Scala / Scala進化論
The Evolution of Scala / Scala進化論The Evolution of Scala / Scala進化論
The Evolution of Scala / Scala進化論scalaconfjp
 
Java 201 Intro to Test Driven Development in Java
Java 201   Intro to Test Driven Development in JavaJava 201   Intro to Test Driven Development in Java
Java 201 Intro to Test Driven Development in Javaagorolabs
 
Scalaマクロ入門 bizr20170217
Scalaマクロ入門 bizr20170217 Scalaマクロ入門 bizr20170217
Scalaマクロ入門 bizr20170217 dcubeio
 
An Introduction to Scala - Blending OO and Functional Paradigms
An Introduction to Scala - Blending OO and Functional ParadigmsAn Introduction to Scala - Blending OO and Functional Paradigms
An Introduction to Scala - Blending OO and Functional ParadigmsMiles Sabin
 
Introduction to Spark SQL and Catalyst / Spark SQLおよびCalalystの紹介
Introduction to Spark SQL and Catalyst / Spark SQLおよびCalalystの紹介Introduction to Spark SQL and Catalyst / Spark SQLおよびCalalystの紹介
Introduction to Spark SQL and Catalyst / Spark SQLおよびCalalystの紹介scalaconfjp
 
Java 101 Intro to Java Programming - Exercises
Java 101   Intro to Java Programming - ExercisesJava 101   Intro to Java Programming - Exercises
Java 101 Intro to Java Programming - Exercisesagorolabs
 
Scala : language of the future
Scala : language of the futureScala : language of the future
Scala : language of the futureAnsviaLab
 

What's hot (20)

Introduction to Scala
Introduction to ScalaIntroduction to Scala
Introduction to Scala
 
Introduction to Scala
Introduction to ScalaIntroduction to Scala
Introduction to Scala
 
Scala Introduction
Scala IntroductionScala Introduction
Scala Introduction
 
[Start] Scala
[Start] Scala[Start] Scala
[Start] Scala
 
A Scala tutorial
A Scala tutorialA Scala tutorial
A Scala tutorial
 
Scala @ TechMeetup Edinburgh
Scala @ TechMeetup EdinburghScala @ TechMeetup Edinburgh
Scala @ TechMeetup Edinburgh
 
camel-scala.pdf
camel-scala.pdfcamel-scala.pdf
camel-scala.pdf
 
Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009
 
Martin Odersky - Evolution of Scala
Martin Odersky - Evolution of ScalaMartin Odersky - Evolution of Scala
Martin Odersky - Evolution of Scala
 
The Evolution of Scala / Scala進化論
The Evolution of Scala / Scala進化論The Evolution of Scala / Scala進化論
The Evolution of Scala / Scala進化論
 
Java 201 Intro to Test Driven Development in Java
Java 201   Intro to Test Driven Development in JavaJava 201   Intro to Test Driven Development in Java
Java 201 Intro to Test Driven Development in Java
 
Scalax
ScalaxScalax
Scalax
 
Scala-Ls1
Scala-Ls1Scala-Ls1
Scala-Ls1
 
Scalaマクロ入門 bizr20170217
Scalaマクロ入門 bizr20170217 Scalaマクロ入門 bizr20170217
Scalaマクロ入門 bizr20170217
 
Quick introduction to scala
Quick introduction to scalaQuick introduction to scala
Quick introduction to scala
 
An Introduction to Scala - Blending OO and Functional Paradigms
An Introduction to Scala - Blending OO and Functional ParadigmsAn Introduction to Scala - Blending OO and Functional Paradigms
An Introduction to Scala - Blending OO and Functional Paradigms
 
Introduction to Spark SQL and Catalyst / Spark SQLおよびCalalystの紹介
Introduction to Spark SQL and Catalyst / Spark SQLおよびCalalystの紹介Introduction to Spark SQL and Catalyst / Spark SQLおよびCalalystの紹介
Introduction to Spark SQL and Catalyst / Spark SQLおよびCalalystの紹介
 
Java 101 Intro to Java Programming - Exercises
Java 101   Intro to Java Programming - ExercisesJava 101   Intro to Java Programming - Exercises
Java 101 Intro to Java Programming - Exercises
 
Scala : language of the future
Scala : language of the futureScala : language of the future
Scala : language of the future
 
PROGRAMMING IN JAVA
PROGRAMMING IN JAVAPROGRAMMING IN JAVA
PROGRAMMING IN JAVA
 

Similar to Responsive Facets with Apache Solr

Making your Drupal fly with Apache SOLR
Making your Drupal fly with Apache SOLRMaking your Drupal fly with Apache SOLR
Making your Drupal fly with Apache SOLRExove
 
Solr中国8月4日答疑交流v2
Solr中国8月4日答疑交流v2Solr中国8月4日答疑交流v2
Solr中国8月4日答疑交流v2longkeyy
 
"Searching with Solr" - Tyler Harms, South Dakota Code Camp 2012
"Searching with Solr" - Tyler Harms, South Dakota Code Camp 2012"Searching with Solr" - Tyler Harms, South Dakota Code Camp 2012
"Searching with Solr" - Tyler Harms, South Dakota Code Camp 2012Blend Interactive
 
Rapid prototyping with solr - By Erik Hatcher
Rapid prototyping with solr -  By Erik Hatcher Rapid prototyping with solr -  By Erik Hatcher
Rapid prototyping with solr - By Erik Hatcher lucenerevolution
 
Dev8d Apache Solr Tutorial
Dev8d Apache Solr TutorialDev8d Apache Solr Tutorial
Dev8d Apache Solr TutorialSourcesense
 
Building Enterprise Search Engines using Open Source Technologies
Building Enterprise Search Engines using Open Source TechnologiesBuilding Enterprise Search Engines using Open Source Technologies
Building Enterprise Search Engines using Open Source TechnologiesRahul Singh
 
Building Enterprise Search Engines using Open Source Technologies
Building Enterprise Search Engines using Open Source TechnologiesBuilding Enterprise Search Engines using Open Source Technologies
Building Enterprise Search Engines using Open Source TechnologiesAnant Corporation
 
PLAT-4 Understanding the SOLR Integration
PLAT-4 Understanding the SOLR IntegrationPLAT-4 Understanding the SOLR Integration
PLAT-4 Understanding the SOLR IntegrationAlfresco Software
 
Small wins in a small time with Apache Solr
Small wins in a small time with Apache SolrSmall wins in a small time with Apache Solr
Small wins in a small time with Apache SolrSourcesense
 
Apache Solr! Enterprise Search Solutions at your Fingertips!
Apache Solr! Enterprise Search Solutions at your Fingertips!Apache Solr! Enterprise Search Solutions at your Fingertips!
Apache Solr! Enterprise Search Solutions at your Fingertips!Murshed Ahmmad Khan
 
Your Big Data Stack is Too Big!: Presented by Timothy Potter, Lucidworks
Your Big Data Stack is Too Big!: Presented by Timothy Potter, LucidworksYour Big Data Stack is Too Big!: Presented by Timothy Potter, Lucidworks
Your Big Data Stack is Too Big!: Presented by Timothy Potter, LucidworksLucidworks
 
Architecture by Accident
Architecture by AccidentArchitecture by Accident
Architecture by AccidentGleicon Moraes
 
A glimpse of test automation in hadoop ecosystem by Deepika Achary
A glimpse of test automation in hadoop ecosystem by Deepika AcharyA glimpse of test automation in hadoop ecosystem by Deepika Achary
A glimpse of test automation in hadoop ecosystem by Deepika AcharyQA or the Highway
 
Geospatial search with SOLR
Geospatial search with SOLRGeospatial search with SOLR
Geospatial search with SOLRNicolas Leroy
 

Similar to Responsive Facets with Apache Solr (20)

Making your Drupal fly with Apache SOLR
Making your Drupal fly with Apache SOLRMaking your Drupal fly with Apache SOLR
Making your Drupal fly with Apache SOLR
 
Cloudera search
Cloudera searchCloudera search
Cloudera search
 
Solr 8 interview
Solr 8 interview Solr 8 interview
Solr 8 interview
 
Solr中国8月4日答疑交流v2
Solr中国8月4日答疑交流v2Solr中国8月4日答疑交流v2
Solr中国8月4日答疑交流v2
 
"Searching with Solr" - Tyler Harms, South Dakota Code Camp 2012
"Searching with Solr" - Tyler Harms, South Dakota Code Camp 2012"Searching with Solr" - Tyler Harms, South Dakota Code Camp 2012
"Searching with Solr" - Tyler Harms, South Dakota Code Camp 2012
 
Searchlight
SearchlightSearchlight
Searchlight
 
Rapid Prototyping with Solr
Rapid Prototyping with SolrRapid Prototyping with Solr
Rapid Prototyping with Solr
 
Rapid prototyping with solr - By Erik Hatcher
Rapid prototyping with solr -  By Erik Hatcher Rapid prototyping with solr -  By Erik Hatcher
Rapid prototyping with solr - By Erik Hatcher
 
Dev8d Apache Solr Tutorial
Dev8d Apache Solr TutorialDev8d Apache Solr Tutorial
Dev8d Apache Solr Tutorial
 
Building Enterprise Search Engines using Open Source Technologies
Building Enterprise Search Engines using Open Source TechnologiesBuilding Enterprise Search Engines using Open Source Technologies
Building Enterprise Search Engines using Open Source Technologies
 
Building Enterprise Search Engines using Open Source Technologies
Building Enterprise Search Engines using Open Source TechnologiesBuilding Enterprise Search Engines using Open Source Technologies
Building Enterprise Search Engines using Open Source Technologies
 
PLAT-4 Understanding the SOLR Integration
PLAT-4 Understanding the SOLR IntegrationPLAT-4 Understanding the SOLR Integration
PLAT-4 Understanding the SOLR Integration
 
Solr
SolrSolr
Solr
 
Small wins in a small time with Apache Solr
Small wins in a small time with Apache SolrSmall wins in a small time with Apache Solr
Small wins in a small time with Apache Solr
 
Apache Solr! Enterprise Search Solutions at your Fingertips!
Apache Solr! Enterprise Search Solutions at your Fingertips!Apache Solr! Enterprise Search Solutions at your Fingertips!
Apache Solr! Enterprise Search Solutions at your Fingertips!
 
Your Big Data Stack is Too Big!: Presented by Timothy Potter, Lucidworks
Your Big Data Stack is Too Big!: Presented by Timothy Potter, LucidworksYour Big Data Stack is Too Big!: Presented by Timothy Potter, Lucidworks
Your Big Data Stack is Too Big!: Presented by Timothy Potter, Lucidworks
 
Architecture by Accident
Architecture by AccidentArchitecture by Accident
Architecture by Accident
 
Sindice warehousing meetup
Sindice warehousing meetupSindice warehousing meetup
Sindice warehousing meetup
 
A glimpse of test automation in hadoop ecosystem by Deepika Achary
A glimpse of test automation in hadoop ecosystem by Deepika AcharyA glimpse of test automation in hadoop ecosystem by Deepika Achary
A glimpse of test automation in hadoop ecosystem by Deepika Achary
 
Geospatial search with SOLR
Geospatial search with SOLRGeospatial search with SOLR
Geospatial search with SOLR
 

Responsive Facets with Apache Solr

  • 1. Responsive Facets with Apache Solr Dallas, TX February 12, 2013 Discussion document – Strictly Confidential & Proprietary
  • 2. Agenda … What we will cover tonight • Concept of Facets • The Problem with RDBMS • Solr to the Rescue • Solr by example • Architecture basics • Putting data in • Getting data out Responsive Facets with Apache Solr February 12, 2013 2
  • 3. Concept of facets… Sometimes the facets only have groups with values Responsive Facets with Apache Solr February 12, 2013 3
  • 4. Concept of facets… Sometimes, they also include a count for the value Responsive Facets with Apache Solr February 12, 2013 4
  • 5. The Problem with RDBMS… Easy enough, just attributes of the product… Responsive Facets with Apache Solr February 12, 2013 5
  • 6. The Problem with RDBMS… Easy enough, just attributes of the product… Or is it! Responsive Facets with Apache Solr February 12, 2013 6
  • 7. The Problem with RDBMS… Easy enough, just attributes of the product… Or is it! Responsive Facets with Apache Solr February 12, 2013 7
  • 8. The Problem with RDBMS… Easy enough, just attributes of the product… Or is it! Responsive Facets with Apache Solr February 12, 2013 8
  • 9. The Problem with RDBMS… Easy enough, just attributes of the product… Or is it! Responsive Facets with Apache Solr February 12, 2013 9
  • 10. Solr to the rescue… We need a better way to organize our data for fast queries Solr is a standalone search server built on the Lucene Search Library. It offers all the capabilities of the Lucene library and extends the capabilities. Solr interactions are handled through a REST-like API, allowing you to input and retrieve data in a variety of formats. Solr can run within most servlet containers including Tomcat and Jetty. • Full-Text Search Capabilities • Standards Based Open Interfaces – XML, JSON and HTTP • Linearly scalable, auto index replication, auto failover and recovery • Near Real-time indexing • SolrJ library capable of binary over HTTP communication • Faceted Search and Filtering Responsive Facets with Apache Solr February 12, 2013 10
  • 11. Solr to the rescue… Products are inserted into Solr as a Document • Fields and Dynamic Fields • Unique Key on a specific field • Fields are typed – String is a literal – Text is a tokenized string • CopyField allows you to populate multiple fields off a single field insert Responsive Facets with Apache Solr February 12, 2013 11
  • 12. Solr by example… Solr utilizes a REST-like interface with XML, JSON and binary capabilities Let’s look at a few examples… Responsive Facets with Apache Solr February 12, 2013 12
  • 13. Architecture basics… Running Solr • Embedded – Makes your application more bloated – More resource intensive • Standalone – Run on it’s own server – Network traffic hit – Replication is an option • SolrCloud – High availability – Sharded with replication – http://bit.ly/TdAJBw Responsive Facets with Apache Solr February 12, 2013 13
  • 14. Architecture basics… Our environment Responsive Facets with Apache Solr February 12, 2013 14
  • 15. Architecture basics… Communicating withSolr SolrJ • Java binary communication • Library that abstracts connection, query formation and response processing Inject Server <bean id="solrServer" class="org.apache.solr.client.solrj.impl.LBHttpSolrServer"> <constructor-argvalue="${solr.url.primary}"/> </bean> Inject Search Service <bean id="blSearchService" class="com.vology.core.service.search.VolSearchServiceImpl"> <constructor-argname="solrServer" ref="${solr.source.primary}" /> </bean> Utilize SolrJ Library • ServerContext • SolrQuery • SolrDocument • QueryResponse Responsive Facets with Apache Solr February 12, 2013 15
  • 16. Putting data in… How do we get the data into Solr? Transform fields • Populate transient fields • Determine terminating object Query product data from DB • Utilize Hibernate to retrieve relevant products Populate SolrJSolrDocument • Java Reflection to get fields we want • SolrField data from Field Add SolrDocument • Group into a List<SolrDocument> • Add full set or incremental • Commit Responsive Facets with Apache Solr February 12, 2013 16
  • 17. Getting data out How do we get the data out of Solr? Build SolrQuery • Query searchable fields from Field table • Creates ‘&’ delimited list for qf from Field table • We utilize a boost value (^100.0) Add Search Facets • Uses search facet and search facet range tables • Creates ‘&’ delimited list of facet.field and facet.range Query on ServerContext • server.query(SolrQuery) • QueryResponse returned Populate ProductSearchResult • Set facets • Set results • Passed back to SearchController Responsive Facets with Apache Solr February 12, 2013 17
  • 18. Contact ... Tell me what you thought! blemons@credera.com @brentlemons slideshare.net/brentlemons linkedin.com/in/brentlemons github.com/brentlemons Do you think this presentation should be at OSCon? • Tweet @oscon to let them know! Responsive Facets with Apache Solr February 12, 2013 18

Editor's Notes

  1. Field