Your SlideShare is downloading. ×
0
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Couchbase Server 2.0 and Full Text Search Integration
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Couchbase Server 2.0 and Full Text Search Integration

1,103

Published on

-How to set up the integration between a Couchbase Server 2.0 cluster and an ElasticSearch Cluster. …

-How to set up the integration between a Couchbase Server 2.0 cluster and an ElasticSearch Cluster.
-How you can combine the capabilities of both systems, performing fast full text search in ElasticSearch then performing fast document lookup on the results by document ID in Couchbase Server.
-Real world examples and consider how they can be implemented using Couchbase and ElasticSearch.

Published in: Business
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,103
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
13
Comments
0
Likes
3
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • JSON support – natively stored as json, whne you build an app, there is not conversion required. New doc viewing , editing capability. Indexing and querying – look inside your json, build views and query for a key, for ranges or to aggregate data Incremental mapreduce – powers indexing. Build complex views over your data. Great for real-time analytics XDCR – replicate information from one cluster to another cluster
  • ----- Meeting Notes (9/12/12 15:47) -----explain that it will be separate cluster infrastructrejSON schema freepossibly add graphic hereadd the term full-textintegrate with elasticsearch for full-text
  • ----- Meeting Notes (9/12/12 15:47) -----do a better job document typesdocument, field, type, index
  • ----- Meeting Notes (9/12/12 15:47) -----make the text biggeradd another slide with deeper explanation
  • ----- Meeting Notes (9/12/12 15:47) -----clarify that installation is on the ES node
  • ----- Meeting Notes (9/12/12 15:47) -----emphasize full-text querycome up with some way to emphasize which queries are ES and which are couchbase
  • ----- Meeting Notes (9/12/12 15:47) -----emphasize full-text querycome up with some way to emphasize which queries are ES and which are couchbase
  • ----- Meeting Notes (9/12/12 15:47) -----emphasize full-text querycome up with some way to emphasize which queries are ES and which are couchbase
  • ----- Meeting Notes (9/12/12 15:47) -----emphasize full-text querycome up with some way to emphasize which queries are ES and which are couchbase
  • ----- Meeting Notes (9/12/12 15:47) -----emphasize full-text querycome up with some way to emphasize which queries are ES and which are couchbase
  • ----- Meeting Notes (9/12/12 15:47) -----add full architecture slide here
  • ----- Meeting Notes (9/12/12 15:47) -----addd clarification about which come from whichcheck better image
  • ----- Meeting Notes (9/12/12 15:47) -----show some other indexersadd NOTE you need to reindex
  • ----- Meeting Notes (9/12/12 15:47) -----add link to guide
  • ----- Meeting Notes (9/12/12 15:47) -----ask for more feedback here
  • Transcript

    • 1. Couchbase Server 2.0:Full Text Search Integration Marty Schoch Software Engineer 1
    • 2. Couchbase Server 2.0 - Webinar Series Introducing Couchbase Server 2.0 Couchbase Server 2.0 and Indexing/Querying Couchbase Server 2.0 and Incremental Map Reduce for Real-Time Analytics Couchbase Server 2.0 and Cross Data Center Replication Couchbase Server 2.0 and Full-Text Search Integration Couchbase Server 2.0 Use Cases Overview http://www.couchbase.com/webinars 2
    • 3. New in Two Indexing and JSON support Querying Incremental Map Cross data center Reduce replication 3
    • 4. Search Across Full JSON Body { "name": "Abbey Belgian Style Ale", "description": "Winner of four World Beer Cup medals and eight medals at the Great American Beer Fest, Abbey Belgian Ale is the Mark Spitz of New Belgium’s lineup – but it didn’t start out that way." } Search term: abbey 4
    • 5. Search Across Full JSON Body { "name": "Abbey Belgian Style Ale", "description": "Winner of four World Beer Cup medals and eight medals at the Great American Beer Fest, Abbey Belgian Ale is the Mark Spitz of New Belgium’s lineup – but it didn’t start out that way." } Search term: abbey 5
    • 6. Integrate with ElasticSearch for Full Text Search• Based on proven Apache Lucene technology• Apache 2 Licensed with commercial support available• Distributed• Schema Free JSON Documents• RESTful API 6
    • 7. ElasticSearch Terminology• Document – Schema-less JSON… – Contains a set of fields• Type – Contains a set of mappings describing how fields are indexed• Index – Logical namespace for scoping indexing/searching – May contain documents of different types – Uniqueness by ID/Type 7
    • 8. How does it work? Unidirectional Cross Data Center Replication ElasticSearch 8
    • 9. GETTING STARTED 9
    • 10. Install the Couchbase Plug-In• Pre-requisite – Existing Couchbase and ElasticSearch Clusters• Install the ElasticSearch Couchbase Transport Plug-in – bin/plugin -install couchbaselabs/elasticsearch-transport-couchbase/1.0.0-dp• Configure the Plug-in – Set a password – Install the Couchbase Index Template• Restart ElasticSearch• Create an ElasticSearch index for your documents 10
    • 11. Configure XDCR (part 1) 11
    • 12. Configure XDCR (part 2) 12
    • 13. Documents are now being indexed! Document Count Increasing 13
    • 14. WHAT NOW? 14
    • 15. Document from Beer Sample Dataset{ "name": "Pabst Blue Ribbon", "abv": 4.74, "ibu": 0, "srm": 0, "upc": 0, "type": "beer", "brewery_id": "110f1d5dc2", "updated": "2010-07-22 20:00:20", "description": "PBR is not just any beer…", "style": "American-Style Light Lager", "category": "North American Lager"} 15
    • 16. Simple ES Query with HTTP• Search for any beer matching the term “lager” – GET http://127.0.0.1:9200/beer-sample/_search?q=lager { "took": 7, "timed_out": false, "_shards": { ... }, "hits": { "total": 1271, "max_score": 1.1145955, "hits": [...] } } 16
    • 17. Simple ES Query with HTTP• Search for any beer matching the term “lager” – GET http://127.0.0.1:9200/beer-sample/_search?q=lager { "took": 7, Total Search Execution "timed_out": false, Time "_shards": { ... }, "hits": { "total": 1271, "max_score": 1.1145955, "hits": [...] } } 17
    • 18. Simple ES Query with HTTP• Search for any beer matching the term “lager” – GET http://127.0.0.1:9200/beer-sample/_search?q=lager { "took": 7, "timed_out": false, "_shards": { ... }, Total Number of "hits": { Documents Matching "total": 1271, Query "max_score": 1.1145955, "hits": [...] } } 18
    • 19. Simple ES Query with HTTP• Search for any beer matching the term “lager” – GET http://127.0.0.1:9200/beer-sample/_search?q=lager { "took": 7, "timed_out": false, "_shards": { ... }, "hits": { "total": 1271, Maximum Score of All "max_score": 1.1145955, Matching Documents "hits": [...] } } 19
    • 20. Simple ES Query with HTTP• Search for any beer matching the term “lager” – GET http://127.0.0.1:9200/beer-sample/_search?q=lager { "took": 7, "timed_out": false, "_shards": { ... }, "hits": { "total": 1271, "max_score": 1.1145955, Array of Matching "hits": [...] Documents } } 20
    • 21. Single Search Result "hits": [ { "_index": "beer-sample", "_type": "couchbaseDocument", "_id": "110fc4b16b", "_score": 1.1145955, ID of Matching "_source": { Document "meta": { "id": "110fc4b16b", "rev": "1-001ba0044ce30dd50000000000000000", "flags": 0, "expiration": 0 } } }, … ] 21
    • 22. Single Search Result "hits": [ { "_index": "beer-sample", "_type": "couchbaseDocument", "_id": "110fc4b16b", "_score": 1.1145955, "_source": { "meta": { "id": "110fc4b16b", "rev": "1-001ba0044ce30dd50000000000000000", "flags": 0, "expiration": 0 } } }, … ] Where’s the document body? 22
    • 23. Recommended Usage Pattern1. ElasticSearch Query 2. ElasticSearch Result 3. Couchbase Multi-GET 4. Couchbase Result ElasticSearch 23
    • 24. Architecture Overview App Server Couchbase SDK ES queries over HTTP Data Refs MR Query ES Query M MR MR MR MR Views Views Views Views Index Server Cluster Couchbase Server Cluster XDCR Couchbase ES Transport 24
    • 25. MORE ADVANCED CAPABILITIES 25
    • 26. Another Query with HTTP• POST http://127.0.0.1:9200/default/_search { "query": { "query_string": { "query": "style: lambic AND description: blueberry" } } } { "name": "Wild Blue Blueberry Lager", "abv": 8, "type": "beer", "brewery_id": "110f01abce", "updated": "2010-07-22 20:00:20", "description": "…ripe blueberry aroma…", "style": "Belgian-Style Fruit Lambic", "category": "Belgian and French Ale" } 26
    • 27. Faceted Search Categories Items with Counts Range Facets 27
    • 28. Faceted Search Query – Beer Style{ "query": { "query_string":{ "query":"bud” } }, "facets" : { "styles" : { "terms" : { "field" : "style", "size" : 3 } } }} 28
    • 29. Faceted Search Results - Incorrect"terms": [ { "term": "style" "count": 8 } { "term": "lager" "count": 6 } { "term": "american" "count": 4 }] Style was “American-Style Lager” 29
    • 30. Update the Mapping• PUT /beer-sample/couchbaseDocument/_mapping{ "couchbaseDocument":{ "properties":{ "doc":{ "properties":{ "style": { "type":"string", "index": "not_analyzed" } } } } }}NOTE: When you change the mapping you MUST re-index. 30
    • 31. Faceted Search Results - Correct "terms": [ { "term": "American-Style Light Lager”, "count": 5 }, { "term": "American-Style Lager”, "count": 2 }, { "term": "Belgian-Style White”, "count": 1 } ] 31
    • 32. Faceted Search Query – % Alcohol Range{ "query": { "query_string":{ "query":"bud" } }, "facets" : { "abv" : { "range" : { "abv" : [ { "to" : 3 }, { "from" : 3, "to" : 5 }, { "from" : 5 } ] } } }} 32
    • 33. Faceted Search Results - % Alcohol Range "ranges": [ { "to": 3, "count": 1 }, { "from": 3, "to": 5, "count": 5 }, { "from": 5, "count": 3 } ] 33
    • 34. Search Result Scoring• Each matching document is assigned a scored based on how well it matches the query hits: [ { "_index": "default", "_type": "couchbaseDocument", "_id": "35addbc374", "_score": 1.1306798, … 34
    • 35. Custom Scoring – Document Properties • Each document has a numerical field “abv” • Let’s use this field to boost the beers natural score{ "query": { "custom_score" : { "query": { "query_string": { "query": "bud" } }, "script" : "_score * doc[abv].value" } }} 35
    • 36. Custom Scoring – User Preferences• Let users could rank beer styles from 1-10• User with no preferences set searches for “bud” Name Style Score Bud Extra 1.5409653 Bud Light Lime American-Style Light Lager 1.513119 Bud Light Golden Wheat Belgian-Style White 1.3208274 Bud Ice American-Style Lager 1.2839241 Bud Ice Light American-Style Lager 1.2839241 Bud Light American-Style Light Lager 1.245288 Bud Dry American-Style Light Lager 1.1968427 Budweiser Select American-Style Light Lager 0.8559494 Miller Lite American-Style Light Lager 0.7201389 36
    • 37. Custom Scoring – User Preferences • User ranks “Belgian-Style White” with value 10{ "query": { "custom_filters_score" : { "query" : { "text" : { "_all": "bud"} }, "filters" : [ { "filter" : { "term" : { "style" : "Belgian-Style White" } }, "boost" : "10" } ], "score_mode" : "first” } }} 37
    • 38. Custom Scoring – User Preferences Name Style Score Bud Light Golden Wheat Belgian-Style White 13.208274 Bud Extra 1.5409653 Bud Light Lime American-Style Light Lager 1.513119 Bud Light Golden Wheat Belgian-Style White 1.3208274 Bud Ice American-Style Lager 1.2839241 Bud Ice Light American-Style Lager 1.2839241 Bud Light American-Style Light Lager 1.245288 Bud Dry American-Style Light Lager 1.1968427 Budweiser Select American-Style Light Lager 0.8559494 Miller Lite American-Style Light Lager 0.7201389 38
    • 39. Learning Portal – Proof of Concept 39
    • 40. DEMO 40
    • 41. NEXT STEPS 41
    • 42. Explore ElasticSearch Capabilities• Customize Document Mappings – Default behavior isn’t always what you want – Index one field multiple ways• Advanced Cluster Topologies – Dedicate nodes for routing/querying• Rich Query DSL ElasticSearch Guide: http://www.elasticsearch.org/guide/ 42
    • 43. Couchbase ElasticSearch Future• Release 1.0.0• Possible features for future – More fine-grained cluster configuration – More index-level configuration – Pre-index script execution – Indexing non-JSON data• Give us your feedback! 43
    • 44. Resources • Couchbase and Full Text Search Blog – http://blog.couchbase.com/couchbase-and-full-text- search-couchbase-transport-elastic-search • ElasticSearch Plug-in Github Repository – https://github.com/couchbaselabs/elasticsearch- transport-couchbase • Learning Portal Github Repository – https://github.com/couchbaselabs/learningport al 44
    • 45. Q&A 45
    • 46. THANK YOU @MSCHOCHMARTY@COUCHBASE.COM 46

    ×