Your SlideShare is downloading. ×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Indexing and querying 2_CouchbaseSF_2013

414
views

Published on

Published in: Technology, Education

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
414
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • example: running the mapper and reducer over all of the docs.Caveat: use careful guards to be sure view execution doesn’t stop due to unbound variables
  • First, walk through the optionsThen mention Observe
  • Mention in summary…Also, note there are other systems such as Hadoop, Elastic search that have a lot of merit to add, and there are system like relational databases that don’tadddresscontemporaryneeds
  • Transcript

    • 1. Power Techniques in Indexing and Querying (Part 2) Jasdeep Jaitla Technical Evangelist
    • 2. Agenda • More about aggregates and JSON - What are schemas in a Document Database? • The modeling behind our sample database - Document structure, metadata in Couchbase Server - An example of how to handle new requirements as needed - Addressing concurrent modifications to documents
    • 3. View Lifecycle: Define – Build - Query
    • 4. Distributed Index Build Phase 5 • Optimized for lookups, in-order access and aggregations • All view reads from disk (different performance profile) • View builds against every document on every node • Automatically kept up to date (on writes and reads) Doc 4 Doc 2 Doc 5 SERVER 1 Doc 6 Doc 4 SERVER 2 Doc 7 Doc 1 SERVER 3 Doc 3 Doc 9 Doc 7 Doc 8 Doc 6 Doc 3 DOC DOC DOC DOC DOC DOC DOC DOC DOC DOC DOC DOC DOC DOC DOC Doc 9 Doc 5 DOC DOC DOC Doc 1 Doc 8 Doc 2 Replica Docs Replica Docs Replica Docs Active Docs Active Docs Active Docs
    • 5. Dynamic Range Queries with Optional Aggregation • Efficiently fetch an row or group of related rows. • Queries use cached values from B-tree inner nodes when possible • Take advantage of in-order tree traversal with group_level queries Doc 4 Doc 2 Doc 5 SERVER 1 Doc 6 Doc 4 SERVER 2 Doc 7 Doc 1 SERVER 3 Doc 3 Doc 9 Doc 7 Doc 8 Doc 6 Doc 3 DOC DOC DOC DOC DOC DOC DOC DOC DOC DOC DOC DOC DOC DOC DOC Doc 9 Doc 5 DOC DOC DOC Doc 1 Doc 8 Doc 2 Replica Docs Replica Docs Replica Docs Active Docs Active Docs Active Docs ?startkey=“J”&endkey=“K” {“rows”:[{“key”:“Juneau”,“value”:null}]}
    • 6. Queries run against stale indexes by default • stale = UPDATE_AFTER (default if nothing is specified) - always get fastest response - can take two queries to read your own writes • stale = OK - auto update will trigger eventually - might not see your own writes for a few minutes - least frequent updates -> least resource impact • stale = FALSE - Use with Persistence observe if data needs to be included in view results - BUT aware of delay it adds, only use when really required
    • 7. Development vs. Production Views • Development views index a subset of the data. • Publishing a view builds the index across the entire cluster. • Queries on production views are scattered to all cluster members and results are gathered and returned to the client.
    • 8. Emergent Schema JSON.org Github API "Capture the user's intent" • Falls out of your key-value usage • Helps to know what's efficient • Deal with unstructured data more easily - Different schemas/APIs
    • 9. Query Pattern: Collation of Related Docs
    • 10. Join Through Collation See Bradley Holt’s presentation from CouchConf Boston: http://www.couchbase.com/couchconf-boston
    • 11. Anti-patterns • Emitting document or too much data into a view - Especially avoid including the doc itself in an emit() call • Reduces that don’t reduce - If you implement a custom reduce, make sure it doesn’t expand! • Expecting a query on an index to be as fast - Secondary indexes need to be built, happen asynchronously, and are (currently) cached at the filesystem level • Trying to do too much with one view - Instead, co-locate views in design documents, or have separate design documents • Note that sometimes, you may need to make requests of multiple views - There is not directly a method of doing a join, but there is a technique
    • 12. What about Geo? • Experimental in the 2.0 release • Currently completely rewritten internally • Supports GeoJSON, will support more rich queries soon. • Java SDK contains Geo support right now!
    • 13. Couchbase Integration
    • 14. Integration with ElasticSearch ElasticSearch 1. ElasticSearch Query 2. ElasticSearch Result 3. Couchbase Multi-GET 4. Couchbase Result
    • 15. The Learning Portal • Designed and built as a collaboration between MHE Labs and Couchbase • Serves as proof-of-concept and testing harness for Couchbase + ElasticSearch integration • Available for download and further development as open source code https://github.com/couchbaselabs/learningportal
    • 16. Integration with Hadoop Logs Couchbase Server Cluster Hadoop Cluster sqoop import Logs Logs Logs Logs Ad Targeting Platform sqoop export flume flow
    • 17. Views Allow Common Methods of Querying Common patterns such as simple secondary indexes, count and average aggregations, and time series rollups are simple and fast. Couchbase Integrates for Full Text and Large Analytics Couchbase integrates with ElasticSearch, Hadoop and other systems. Summary Couchbase has Views for Indexing and Querying Views are incremental map-reduce code that run across all documents.
    • 18. Q&A
    • 19. Thanks!