Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Enabling Full Text Search for Couchbase documents – Connect Silicon Valley 2017

329 views

Published on

Speaker: Marty Schoch

The new Full Text Search service added to Couchbase Server 5.0 enables comprehensive search of documents stored in Couchbase. Creating rapid natural language queries is a game-changer for those needing to find documents rapidly and provide them to their users in an intuitive manner without having to run yet another stack of tools.

The search service takes advantage of the performance and scalability benefits of Couchbase Server, allowing ease of management and increased reliability. Its indexing service is managed using the standard web console as well as a REST API.

In this session, we’ll show how to enable Full Text Search on a node, as well as how to create an index and query it. We’ll also review the primary capabilities of the indexing and querying options, including: analyzers, stemmers, queries, scoring, and more.

Join us to learn how you can take advantage of these powerful new features to extract even more value from your Couchbase documents.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Enabling Full Text Search for Couchbase documents – Connect Silicon Valley 2017

  1. 1. FULL TEXT SEARCH For Couchbase Documents 2017-10-26 Marty Schoch | Principle Engineer
  2. 2. Mobile & IoTKV Query Elastic Scale Architecture Memory-first Architecture Unified Programming Core Database Engine Infrastructure - Cloud & Containers CrossStackSecurity SQL&BigData Integrations Couchbase Data Platform
  3. 3. Couchbase Full-Text Search
  4. 4. Mobile & IoTKV Query Elastic Scale Architecture Memory-first Architecture Unified Programming Core Database Engine Infrastructure - Cloud & Containers CrossStackSecurity SQL&BigData Integrations Couchbase Data Platform
  5. 5. Mobile & IoTKV Query Search Elastic Scale Architecture Memory-first Architecture Unified Programming Core Database Engine Infrastructure - Cloud & Containers CrossStackSecurity SQL&BigData Integrations Couchbase Data Platform
  6. 6. 6 Photo caption here This layout has a WHITE logo for use on a darker photo. Let's Go Back In Time
  7. 7. 7 What a Long Strange Trip It's Been Bleve 2013
  8. 8. Bleve • Full Text Search in Go • Text Analysis • Indexing • Searching • Scoring • Faceting
  9. 9. 9 Text Analysis
  10. 10. 10 Inverted Index Indexed Terms Document ID Postings List cozi hotel_1289, hotel_3376, hotel_5022, hotel_9994 luxuri hotel_0092, hotel_1289, hotel_8989 small hotel_3376 spacious hotel_0092, hotel_1289, hotel_3376, hotel_5022
  11. 11. 11 Searching •Match •Match Phrase •Prefix, Regex, Fuzzy •Conjunction, Disjunction, Boolean •Numeric and Date Ranges •Query String
  12. 12. 12 Relevance Scoring
  13. 13. 13 What a Long Strange Trip It's Been Bleve CBFT 2014 2013
  14. 14. 14 cbft design / index partitioning bucket partitions: 0, 1, 2, 3, 4, … … ,1021, 1022, 1023 (1024 vbuckets)
  15. 15. 15 cbft design / index partitioning bucket partitions: 0, 1, 2, 3, 4, … … ,1021, 1022, 1023 (1024 vbuckets) index partitions: A B C
  16. 16. 16 cbft design / index partitioning bucket partitions: 0, 1, 2, 3, 4, … … ,1021, 1022, 1023 (1024 vbuckets) index partitions: A B C
  17. 17. 17 cbft design / index partitioning bucket partitions: 0, 1, 2, 3, 4, … … ,1021, 1022, 1023 (1024 vbuckets) index partitions: A B C (groups of vbuckets) 0-399 400-799 800-1023
  18. 18. 18 cbft design / index partitioning bucket partitions: 0, 1, 2, 3, 4, … … ,1021, 1022, 1023 (1024 vbuckets) index partitions: A B C (groups of vbuckets) 0-399 400-799 800-1023 cbft nodes: X
  19. 19. 19 cbft design / index partitioning bucket partitions: 0, 1, 2, 3, 4, … … ,1021, 1022, 1023 (1024 vbuckets) index partitions: A B C (groups of vbuckets) 0-399 400-799 800-1023 assign to cbft nodes: cbft nodes: X
  20. 20. 20 cbft design / index partitioning bucket partitions: 0, 1, 2, 3, 4, … … ,1021, 1022, 1023 (1024 vbuckets) index partitions: A B C (groups of vbuckets) 0-399 400-799 800-1023 assign to cbft nodes: cbft nodes: X Y Z
  21. 21. 21 cbft design / index partitioning bucket partitions: 0, 1, 2, 3, 4, … … ,1021, 1022, 1023 (1024 vbuckets) index partitions: A B C (groups of vbuckets) 0-399 400-799 800-1023 assign to cbft nodes: replicas, too: cbft nodes: X Y Z
  22. 22. 22 cbft design / queries cbft cbft a query sent to any cbft node… your application RESTcbft …is scatter / gathered to the other cbft nodes
  23. 23. 23 What a Long Strange Trip It's Been Bleve FTS Dev Preview CBFT 2014 2013 2015
  24. 24. 24 What a Long Strange Trip It's Been CB 4.5 Dev Preview Bleve FTS Dev Preview CBFT 2014 2016 2013 2015
  25. 25. 25 What a Long Strange Trip It's Been CB 4.5 Dev Preview Bleve FTS Dev Preview CBFT 2014 2016 2013 2015 CB 5.0 2017
  26. 26. Using FTS
  27. 27. Add FTS Node
  28. 28. Create FTS Index
  29. 29. Customize Field Behavior
  30. 30. Search within the Couchbase UI
  31. 31. Monitoring
  32. 32. Live Demos
  33. 33. 33 Call Center { "_id": "user_1", "address_1": "3217 Camylle Green Keys", "address_2": "Suite 172", "city": "North Emmalee", "company": null, "country": "LK", "created_on": 1482264366000, "dob": null, "doc_type": "user", "first_name": "Rigoberto",
  34. 34. Mapping
  35. 35. Query { "size": 10, "explain": true, "highlight": {}, "fields": [ "*" ], "query": { "boost": 1, "match": "valley streem", "fuzziness": 1, "field": "address" } }
  36. 36. 36 E-commerce { "title": "Ballet Dress-Up Fairy Tutu", "categories": [ [ "Clothing, Shoes & Jewelry", "Girls", "Clothing", "Active", "Active Skirts" ] ],
  37. 37. Mapping
  38. 38. Keyword Analyzer for Term Facets
  39. 39. Request Facets "facets": { "Category": { "field": "categories", "size": 5 }, "Brand": { "field": "brand", "size": 5 }, "Price": { "field": "price", "size": 3, "numeric_ranges": [ { "name": "Under $100", "max": 100 }, { "name": "$100 - $500", "max": 500, "min": 100 }, { "name": "$500 & Above", "min": 500 } ] } }
  40. 40. Response Facets { "Brand": { "field": "brand", "total": 12582, "missing": 8851, "other": 8162, "terms": [ { "term": "Invicta", "count": 1450 }, { "term": "Casio", "count": 959 }, { "term": "Seiko", "count": 898 }, { "term": "Stuhrling Original", "count": 664 }, { "term": "Timex", "count": 449 } ] }, "Category": { "field": "categories", "total": 156064,
  41. 41. Query "query": { "conjuncts": [ { "query": "watch" } ] },
  42. 42. Query "query": { "conjuncts": [ { "query": "watch" }, { "field": "categories", "term": "Men" } ] },
  43. 43. Query "query": { "conjuncts": [ { "query": "watch" }, { "field": "categories", "term": "Men" }, { "field": "brand", "term": "Casio" } ] },
  44. 44. Query "query": { "conjuncts": [ { "query": "watch" }, { "field": "categories", "term": "Men" }, { "field": "brand", "term": "Casio" }, { "field": "price", "min": 500 } ] },
  45. 45. 45 Geo Local Results (dev preview) { "name": "Tied House Cafe & Brewery - San Jose", "geo": { "accuracy": "ROOFTOP", "lat": 37.3362, "lon": -121.894 }, "address": [ "65 North San Pedro" ], "city": "San Jose",
  46. 46. Mapping
  47. 47. Query { "query": { "conjuncts": [ { "query": "brewery" }, { "location": { "lon": -121.888889, "lat": 37.328611 }, "distance": "20mi", "field": "geo" } ] } }
  48. 48. Real-World Enterprise Example Enabling collaborative storytelling by “transforming your team into your film crew”. Machine learning analyzes media files for audio/visual attributes and tags, captioning, stored as documents in Couchbase for rapid search and retrieval. Very rapid development process… “hardest part was deciding when to stop” Seenit Studio seenit.io
  49. 49. 49 What a Long Strange Trip It's Been CB 4.5 Bleve Developer Preview CBFT 2014 2016 2013 2015 CB 5.0 ??? Future 2017
  50. 50. 50 Photo caption here This layout has a WHITE logo for use on a darker photo.
  51. 51. Future •Performance •N1QL Integration
  52. 52. THANK YOU marty@couchbase.com @mschoch

×