Twin Cities Drupal Users Group - October 22, 2008 EthicShare: Solr + Drupal Under the Hood Tour
EthicShare? <ul><li>Who:  University of Minnesota's Center for Bioethics, the University of Minnesota Libraries, and the U...
The Platform <ul><li>Drupal </li></ul><ul><ul><li>Community Development Framework </li></ul></ul><ul><li>Solr </li></ul><u...
The Process
 
<ul><li>Origin:  Created by CNET and released January 2006 </li></ul><ul><ul><li>Became an Apache Software Foundation proj...
Behind the Scenes - Indexing <ul><li>HTTP/XML API </li></ul><ul><li>  http://localhost:8983/solr/update </li></ul><ul><li>...
Behind the Scenes - Searching <ul><li>Get Contents of …/select URL:  cURL, file_get_contents($url)… </li></ul><ul><li>Apac...
Setup - Solr Directory Layout <ul><ul><li>Tomcat Files: </li></ul></ul><ul><ul><li>… /tomcat/webapps/solr_ethicshare.war (...
Solr Schema - Fields and Types <ul><li>Starter schema:  </li></ul><ul><ul><li>../drupaldir/sites/all/modules/apachesolr/sc...
Solr Schema - <type> Analyzers <ul><li>Tokenize on whitespace, then remove any common words (StopFilterFactory) </li></ul>...
Solr Schema - Dynamic Fields <ul><li><dynamicField name=&quot;smfield*&quot;  type=&quot;string&quot;  indexed=&quot;true&...
Solr Schema - Some Example Options <ul><li>uniqueKey </li></ul><ul><ul><li><!-- Field to use to determine and enforce docu...
ApacheSolr Search Integration Module <ul><li>Core Search  Integrated </li></ul><ul><li>Blocks  for facet configuration </l...
Links <ul><li>Installing Solr + Tomcat </li></ul><ul><ul><li>http: //mikejoconnor . net/content/solr-ubercartorg </li></ul...
Upcoming SlideShare
Loading in...5
×

EthicShare.org (Mostly Solr)

2,586
-1

Published on

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,586
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
29
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

EthicShare.org (Mostly Solr)

  1. 1. Twin Cities Drupal Users Group - October 22, 2008 EthicShare: Solr + Drupal Under the Hood Tour
  2. 2. EthicShare? <ul><li>Who: University of Minnesota's Center for Bioethics, the University of Minnesota Libraries, and the University of Minnesota Department of Computer Science and Engineering </li></ul><ul><ul><li>EthicShare’s pilot implementation builds on a recent planning phase that was a collaboration with the University of Virginia, Georgetown University, Indiana University-Bloomington, and Indiana University-Purdue University, Indianapolis. </li></ul></ul><ul><li>What: A sustainable aggregation of bioethics research and a platform for scholarship </li></ul><ul><li>When: Pilot Phase runs from January 2008 - June 2009 </li></ul><ul><li>How: Funded by the Andrew W. Mellon Foundation </li></ul>
  3. 3. The Platform <ul><li>Drupal </li></ul><ul><ul><li>Community Development Framework </li></ul></ul><ul><li>Solr </li></ul><ul><ul><li>Faceted Search Appliance </li></ul></ul>
  4. 4. The Process
  5. 6. <ul><li>Origin: Created by CNET and released January 2006 </li></ul><ul><ul><li>Became an Apache Software Foundation project shortly thereafter </li></ul></ul><ul><li>Builds on the Lucene Search Engine Library </li></ul><ul><ul><li>Comes with Lucene’s search syntax and features </li></ul></ul><ul><li>Provides simple HTTP/XML API </li></ul><ul><li>Strongly typed field definitions </li></ul><ul><li>Noteworthy Implementations Netflix, CNET Reviews, GameSpot, Digg </li></ul><ul><ul><ul><li>More: http: //wiki .apache. org/solr/PublicServers </li></ul></ul></ul>
  6. 7. Behind the Scenes - Indexing <ul><li>HTTP/XML API </li></ul><ul><li> http://localhost:8983/solr/update </li></ul><ul><li>http://localhost:8983/solr/select </li></ul><ul><li>Indexing = POSTing XML Records to /update </li></ul><ul><li>Commands: <add> <delete><commit/><optimize/> </li></ul><ul><ul><li><add> </li></ul></ul><ul><ul><li><doc> </li></ul></ul><ul><ul><li><field name=”nid&quot;>101</field> </li></ul></ul><ul><ul><li><field name=”vid&quot;>2</field> </li></ul></ul><ul><ul><li> <field name=&quot;title&quot;>Solr Search is Simply Great</field> </li></ul></ul><ul><ul><li><field name=”body&quot;>Solr and Drupal are like PB And J</field> </li></ul></ul><ul><ul><li><field name=&quot;changed&quot;>1224707462</field> </li></ul></ul><ul><ul><li> <field name=”tid&quot;>4</field> </li></ul></ul><ul><ul><li><field name=”name&quot;>libsys</field> </li></ul></ul><ul><ul><li><field name=”uid&quot;>10297</field> </li></ul></ul><ul><ul><li></doc> </li></ul></ul><ul><ul><li></add> </li></ul></ul>
  7. 8. Behind the Scenes - Searching <ul><li>Get Contents of …/select URL: cURL, file_get_contents($url)… </li></ul><ul><li>ApacheSolr makes use of a Solr PHP Client Abstraction Layer </li></ul><ul><ul><li>http: //wiki .apache.org/solr/SolPHP </li></ul></ul>
  8. 9. Setup - Solr Directory Layout <ul><ul><li>Tomcat Files: </li></ul></ul><ul><ul><li>… /tomcat/webapps/solr_ethicshare.war (cp solr.war from example dir) </li></ul></ul><ul><ul><li>… /tomcat/conf/Catalina/localhost/ solr_ethicshare.xml </li></ul></ul>solr_ethicshare.xml - Tell Tomcat About Solr <Context docBase=&quot;solr_ethicshare.war&quot; debug=&quot;0&quot; crossContext=&quot;true&quot; > <Environment name=&quot;solr/home&quot; type=&quot;java.lang.String&quot; value=&quot;/usr/local/solr_home/ethicshare&quot; override=&quot;true&quot; /> </Context>
  9. 10. Solr Schema - Fields and Types <ul><li>Starter schema: </li></ul><ul><ul><li>../drupaldir/sites/all/modules/apachesolr/schema.xml </li></ul></ul><ul><li><types> ex: </li></ul><ul><ul><li>string=solr.StrField </li></ul></ul><ul><ul><li>boolean=solr.BoolField </li></ul></ul><ul><li><fields> </li></ul><ul><ul><li><field name=&quot;title&quot; type=&quot;string&quot; indexed=&quot;true&quot; stored=&quot;true&quot;/> </li></ul></ul>
  10. 11. Solr Schema - <type> Analyzers <ul><li>Tokenize on whitespace, then remove any common words (StopFilterFactory) </li></ul><ul><li>Remove any duplicates (RemoveDuplicatesTokenFilterFactory) </li></ul>
  11. 12. Solr Schema - Dynamic Fields <ul><li><dynamicField name=&quot;smfield*&quot; type=&quot;string&quot; indexed=&quot;true&quot; stored=&quot;true&quot; multiValued=&quot;true&quot;/> </li></ul><ul><li><dynamicField name=&quot;tmfield*&quot; type=&quot;text&quot; indexed=&quot;true&quot; stored=&quot;true&quot; multiValued=&quot;true&quot;/> </li></ul>
  12. 13. Solr Schema - Some Example Options <ul><li>uniqueKey </li></ul><ul><ul><li><!-- Field to use to determine and enforce document uniqueness. </li></ul></ul><ul><ul><li>Unless this field is marked with required=&quot;false&quot;, it will be a required field </li></ul></ul><ul><ul><li>--> </li></ul></ul><ul><ul><ul><li><uniqueKey>nid</uniqueKey> </li></ul></ul></ul><ul><li>defaultSearchField </li></ul><ul><ul><li><!-- field for the QueryParser to use when an explicit fieldname is absent --> </li></ul></ul><ul><ul><ul><li><defaultSearchField>text</defaultSearchField> </li></ul></ul></ul><ul><li>solrQueryParser </li></ul><ul><ul><li><!-- SolrQueryParser configuration: defaultOperator=&quot;AND|OR&quot; -- </li></ul></ul><ul><ul><ul><li><solrQueryParser defaultOperator=&quot;AND&quot;/> </li></ul></ul></ul>
  13. 14. ApacheSolr Search Integration Module <ul><li>Core Search Integrated </li></ul><ul><li>Blocks for facet configuration </li></ul><ul><li>Schedules Indexing (via core search) </li></ul><ul><li>Theme Hooks for overriding look and feel </li></ul><ul><li>CCK Integration </li></ul><ul><ul><li>hook_apachesolr_cck_field_mappings() </li></ul></ul><ul><ul><ul><li>Which Fields to Index </li></ul></ul></ul><ul><ul><ul><li>How to Index them </li></ul></ul></ul><ul><ul><ul><li>Callback to pre-process fields </li></ul></ul></ul><ul><ul><ul><li>Whether or Not to Provide a Facet Block </li></ul></ul></ul><ul><li>Help! We need testers for alpha3! </li></ul><ul><ul><li>http: //drupal . org/project/apachesolr </li></ul></ul>
  14. 15. Links <ul><li>Installing Solr + Tomcat </li></ul><ul><ul><li>http: //mikejoconnor . net/content/solr-ubercartorg </li></ul></ul><ul><li>Google Book Search API </li></ul><ul><ul><li>http://code. google .com/apis/books/ </li></ul></ul><ul><li>unAPI </li></ul><ul><ul><li>http: //unapi .info/ </li></ul></ul><ul><li>ApacheSolr Search Integration </li></ul><ul><ul><li>http: //drupal .org/project/apachesolr </li></ul></ul><ul><li>IBM Developer Works - Solr </li></ul><ul><ul><li>http://www. ibm .com/developerworks/java/library/j-solr1/ </li></ul></ul><ul><li>SolPHP </li></ul><ul><ul><li>http: //wiki .apache. org/solr/SolPHP </li></ul></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×