SlideShare a Scribd company logo
Enterprise Search in
Plone using Solr
Calvin Hendryx-Parker
Plone Conference 2010
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
• Java Based
• Full-Text Search
• Web Services API
• Standards Based Interfaces
• Scalable
• XML Configuration
• Extensible
What is Solr?
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
• Indexing
• Query
Playing with Solr
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
• Data Schema
• Faceted Search
• Administrative
Interface
• Incremental Updates
• Supports Sharding
• Index Databases, Local
Files and Web Pages
• Supports Multiple
Indexes
Solr Features
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
• Stopwords
• Synonyms
• Highlighted Context
Snippets
• Spelling Suggestions
• More Like This
Suggestions
• Supports Rich
Documents
Solr Features
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
Wednesday, October 27, 2010
PLONE CONFERENCE 2010Solr Performance
• Wiktionary Dataset
• 49.5 Millions lines of XML
• 1.3 GB of data
• 1.7 Million Pages Indexed in 5.5 hours
• ZODB Size after import 1.1GB
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
collective.solr
Integration Options
with Plone
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
• Monkey Patching
• Relies on collective.indexing
• Duplicates all indexes
• Sub-Optimal Integration with Zope Transactions
• Relies on Thread Locals
collective.solr Issues
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
What to do?
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
Reevaluate
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
• No Monkey Patching
• Simpler Code
Solr Integration as a
Catalog Index
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
• ZCatalog Index
• Doesn't depend on
Plone
• Utilizes new
foreign_connections
Connection Method
• Pass through Solr
Queries
• Direct access to the
Solr Response
Enter alm.solrindex
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
• Still handled by the ZCatalog
• Could change in the future
Sorting
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
• Handle Parsing Attributes for Indexing
• Translate field-specific queries to Solr
• Registered as Zope Utilities
alm.solrindex Field
Handlers
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
<html>
<body>
<h3>Code Sample</h3>
<p>Replace this text!</p>
</body>
</html>
Example Handler
class TextFieldHandler(DefaultFieldHandler):
def parse_query(self, field, field_query):
name = field.name
request = {name: field_query}
record = parseIndexRequest(request, name, ('query',))
if not record.keys:
return None
query_str = ' '.join(record.keys)
if not query_str:
return None
return {'q': u'+%s:%s' % (name, quote_query(query_str))}
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
• GenericSetup Profile
• Tests
• Uses solrpy instead of
the unsupported
solr.py
Other alm.solrindex
Features
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
• Can replace several ZCatalog indexes
• Remove any indexes you have replaced
• Use it for all Text Indexes
• Still Utilize the ZCatalog Indexes for Everything Else
Tips
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
Demo
Project Gutenburg Data
Wednesday, October 27, 2010
PLONE CONFERENCE 2010
Questions?
Wednesday, October 27, 2010
Check out
sixfeetup.com/demos
Wednesday, October 27, 2010

More Related Content

Similar to Enterprise search in plone using solr

Introduction to Solr
Introduction to SolrIntroduction to Solr
Introduction to Solr
Erik Hatcher
 
20130310 solr tuorial
20130310 solr tuorial20130310 solr tuorial
20130310 solr tuorial
Chris Huang
 
PojoSR or OSGi (µ)Services For the Rest of Us
PojoSR or OSGi (µ)Services For the Rest of UsPojoSR or OSGi (µ)Services For the Rest of Us
PojoSR or OSGi (µ)Services For the Rest of Us
OSGiUsers
 
Lucene BootCamp
Lucene BootCampLucene BootCamp
Lucene BootCamp
GokulD
 
Solr Recipes Workshop
Solr Recipes WorkshopSolr Recipes Workshop
Solr Recipes Workshop
Erik Hatcher
 

Similar to Enterprise search in plone using solr (20)

Instant ECM with SharePoint 2010
Instant ECM with SharePoint 2010Instant ECM with SharePoint 2010
Instant ECM with SharePoint 2010
 
Let's Build an Inverted Index: Introduction to Apache Lucene/Solr
Let's Build an Inverted Index: Introduction to Apache Lucene/SolrLet's Build an Inverted Index: Introduction to Apache Lucene/Solr
Let's Build an Inverted Index: Introduction to Apache Lucene/Solr
 
Introduction to Solr
Introduction to SolrIntroduction to Solr
Introduction to Solr
 
20130310 solr tuorial
20130310 solr tuorial20130310 solr tuorial
20130310 solr tuorial
 
Why RESTful Design for the Cloud is Best
Why RESTful Design for the Cloud is BestWhy RESTful Design for the Cloud is Best
Why RESTful Design for the Cloud is Best
 
Planidoo & Zotonic
Planidoo & ZotonicPlanidoo & Zotonic
Planidoo & Zotonic
 
Building OBO Foundry ontology using semantic web tools
Building OBO Foundry ontology using semantic web toolsBuilding OBO Foundry ontology using semantic web tools
Building OBO Foundry ontology using semantic web tools
 
PojoSR or OSGi (µ)Services For the Rest of Us
PojoSR or OSGi (µ)Services For the Rest of UsPojoSR or OSGi (µ)Services For the Rest of Us
PojoSR or OSGi (µ)Services For the Rest of Us
 
Tagging search solution design Advanced edition
Tagging search solution design Advanced editionTagging search solution design Advanced edition
Tagging search solution design Advanced edition
 
Tagging search solution design
Tagging search solution designTagging search solution design
Tagging search solution design
 
Building Software Backend (Web API)
Building Software Backend (Web API)Building Software Backend (Web API)
Building Software Backend (Web API)
 
Apache Solr
Apache SolrApache Solr
Apache Solr
 
ActiveRecord 2.3
ActiveRecord 2.3ActiveRecord 2.3
ActiveRecord 2.3
 
"How Mozilla Uses Selenium"
"How Mozilla Uses Selenium""How Mozilla Uses Selenium"
"How Mozilla Uses Selenium"
 
Introduction to Zotero: A Free, Open-Source Tool to Manage and Share Citation...
Introduction to Zotero: A Free, Open-Source Tool to Manage and Share Citation...Introduction to Zotero: A Free, Open-Source Tool to Manage and Share Citation...
Introduction to Zotero: A Free, Open-Source Tool to Manage and Share Citation...
 
Ontology Web Services
Ontology Web ServicesOntology Web Services
Ontology Web Services
 
Intro to Apache Solr for Drupal
Intro to Apache Solr for DrupalIntro to Apache Solr for Drupal
Intro to Apache Solr for Drupal
 
Introduction to Solr
Introduction to SolrIntroduction to Solr
Introduction to Solr
 
Lucene BootCamp
Lucene BootCampLucene BootCamp
Lucene BootCamp
 
Solr Recipes Workshop
Solr Recipes WorkshopSolr Recipes Workshop
Solr Recipes Workshop
 

More from Calvin Hendryx-Parker

Social Networking Tools Session Three
Social Networking Tools Session ThreeSocial Networking Tools Session Three
Social Networking Tools Session Three
Calvin Hendryx-Parker
 

More from Calvin Hendryx-Parker (8)

Plone and Drupal -- CMS Coexistance in Higher Education
Plone and Drupal -- CMS Coexistance in Higher EducationPlone and Drupal -- CMS Coexistance in Higher Education
Plone and Drupal -- CMS Coexistance in Higher Education
 
Plone roadmap
Plone roadmapPlone roadmap
Plone roadmap
 
How to seal the deal
How to seal the dealHow to seal the deal
How to seal the deal
 
2010 py ohio supervisor talk
2010 py ohio supervisor talk2010 py ohio supervisor talk
2010 py ohio supervisor talk
 
Social Networking Tools Session Three
Social Networking Tools Session ThreeSocial Networking Tools Session Three
Social Networking Tools Session Three
 
Social Networking Tools Session One
Social Networking Tools   Session OneSocial Networking Tools   Session One
Social Networking Tools Session One
 
Social Networking Tools Session Two
Social Networking Tools   Session TwoSocial Networking Tools   Session Two
Social Networking Tools Session Two
 
Plone's Anatomy
Plone's AnatomyPlone's Anatomy
Plone's Anatomy
 

Recently uploaded

Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Peter Udo Diehl
 

Recently uploaded (20)

Agentic RAG What it is its types applications and implementation.pdf
Agentic RAG What it is its types applications and implementation.pdfAgentic RAG What it is its types applications and implementation.pdf
Agentic RAG What it is its types applications and implementation.pdf
 
UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1
 
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty Secure
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at Comcast
 
Introduction to Open Source RAG and RAG Evaluation
Introduction to Open Source RAG and RAG EvaluationIntroduction to Open Source RAG and RAG Evaluation
Introduction to Open Source RAG and RAG Evaluation
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджера
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
 
Buy Epson EcoTank L3210 Colour Printer Online.pdf
Buy Epson EcoTank L3210 Colour Printer Online.pdfBuy Epson EcoTank L3210 Colour Printer Online.pdf
Buy Epson EcoTank L3210 Colour Printer Online.pdf
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
 
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptxUnpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
Strategic AI Integration in Engineering Teams
Strategic AI Integration in Engineering TeamsStrategic AI Integration in Engineering Teams
Strategic AI Integration in Engineering Teams
 
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through Observability
 
UiPath Test Automation using UiPath Test Suite series, part 2
UiPath Test Automation using UiPath Test Suite series, part 2UiPath Test Automation using UiPath Test Suite series, part 2
UiPath Test Automation using UiPath Test Suite series, part 2
 

Enterprise search in plone using solr