Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Best Practices with Views in Couchbase Server: Couchbase Connect 2015

2,089 views

Published on

Couchbase views is a very powerful feature that enables enterprises to build real time applications. However, indexing can be a pretty heavy-weight operation on your Couchbase cluster. This session will briefly introduce you to Couchbase views. We’ll then discuss document, database and view design best practices. The session will finish off with tips and tunables for running views in production for a successful Couchbase deployment.

Published in: Technology
  • Be the first to comment

Best Practices with Views in Couchbase Server: Couchbase Connect 2015

  1. 1. BEST PRACTICES WITH VIEWS IN COUCHBASE SERVER David Maier Principal Solutions Engineer
  2. 2. ©2015 Couchbase Inc. 2 Agenda  Introduction  Ways to Query  Database Design Considerations forViews  Configuration Settings and their Effects  Resource Requirements
  3. 3. Introduction
  4. 4. ©2015 Couchbase Inc. 4 Introduction 3.x  Focus  on Indexing viaViews  Not covering  the new Global Secondary Indexes in 4.0
  5. 5. ©2015 Couchbase Inc. 5 Introduction  Views are a powerful feature for real time applications  Indexing can be a pretty heavy weighted operation  Indeed use case dependent! Patch Management Many others.. 90% Views/Queries Key Access 10%
  6. 6. Ways to Query with Couchbase Server
  7. 7. ©2015 Couchbase Inc. 7 Key Patterns  Retrieval via key patterns  e.g. person::$firstname_$lastname  With a lookup document  e.g. just 2 steps to retrieve a user by his email address  Efficiency  B-Tree traversal vs. direct access
  8. 8. ©2015 Couchbase Inc. 8 Key Patterns  Multi-get by using a counter value
  9. 9. ©2015 Couchbase Inc. 9 Indexing and Querying viaViews  Organized in Design Documents  Incremental Map-Reduce  Spread load across nodes  Each node indexes it’s data Map Reduce Process, filter, map and emit a row Aggregate mapped data Built in: _count, _sum, _stats
  10. 10. ©2015 Couchbase Inc. 10 Indexing and Querying viaViews  Multiple roles  Primary Index: All document id-s  Secondary Index:Alternative access path regarding (compound) key attribute  View:Alternative view on data
  11. 11. ©2015 Couchbase Inc. 11 Indexing and Querying viaViews
  12. 12. ©2015 Couchbase Inc. 12 Indexing and Querying viaViews
  13. 13. ©2015 Couchbase Inc. 13 Indexing and Querying viaViews  SimpleView access  Exact match query  Range query  With reduction  With grouping
  14. 14. ©2015 Couchbase Inc. 14 Best Practices - Selection, Projection, Aggregation  Try avoid computing too many things in aView  Check for attribute existence  Pre-Filter data to avoid unnecessary entries in theView  Use document types to makeViews more selective  Project (map) only necessary data by emitting it as part of the value  Do not emit the full document  Back-reference via the original document id  Use the built-in reduce functions if possible
  15. 15. ©2015 Couchbase Inc. 15 Best Practices - Selection, Projection, Aggregation
  16. 16. ©2015 Couchbase Inc. 16 N1QL – Developer Preview  SQL like query language  Extended syntax  Can use several index implementations  ForestDB - Global Secondary Indexes  CouchstoreViews - Only those were created via N1QL CREATE PRIMARY INDEX ON `beer- sample`; CREATE INDEX `beer-sample-type- index` ON `beer-sample`(type); SELECT brewery_id FROM `beer- sample`WHERE name = 'Doppelbock';
  17. 17. How Indexing works in Couchbase 2.x compared to 3.x
  18. 18. ©2015 Couchbase Inc. 18 2.x Architecture
  19. 19. ©2015 Couchbase Inc. 19 3.x Architecture
  20. 20. ©2015 Couchbase Inc. 20 The Semantic of ‘stale = false’  Stale = false  Default is ‘update after’  Used to enforce index update at query time  2.x  Eventual indexed  Eventual consistent  Hit disk before indexed  3.x  Indexed from memory so semantically correct
  21. 21. Database Design Considerations forViews
  22. 22. ©2015 Couchbase Inc. 22 Number of Design Documents per Bucket  Indexers are allocated per Design Document  Bad cases  One Design Document contains allViews AllViews are updated the same time A lot to do for the Indexer  OneView per Design Document Resource intensive because one Indexer perView  Good balance!
  23. 23. ©2015 Couchbase Inc. 23 Separated Buckets for Indexing / Querying  Creating aView for the entire Bucket may be heavy weighted  Separate data to be indexed / queried  Don’t create too many Buckets!
  24. 24. ©2015 Couchbase Inc. 24 XDCR – Separated Cluster for Indexing  Separate the load  Reporting cluster vs. operational one  Active-Passive XDCR
  25. 25. Configuration Settings and their Effects
  26. 26. ©2015 Couchbase Inc. 26 Indexing Settings  Index Path  Separated disks for data and indexes  Improve I/O performance
  27. 27. ©2015 Couchbase Inc. 27 Indexing Settings  Indexing Interval  Controls how up-to-date the index is by default  ‘stale = false’ as explained before
  28. 28. ©2015 Couchbase Inc. 28 Indexing Settings  Max. number of in parallel working indexers  Increase the number of threads per node  Higher level of concurrency  Higher disk and CPU load
  29. 29. ©2015 Couchbase Inc. 29 Rebalance Settings  Index-aware rebalance  Indexing by default as part of rebalancing  Ensures that you get query results from a new node during rebalance that are consistent with the query results you would have received from the node before rebalance started  Performance impact if enabled, so rebalance takes significantly more time
  30. 30. ©2015 Couchbase Inc. 30 Rebalance Settings  Rebalance before compaction  Default is 16, so 16 vBuckets are moved before rebalance is paused for compaction  Higher value may increase rebalance performance  Implicitly increases rebalance priority
  31. 31. ©2015 Couchbase Inc. 31 Rebalance Settings  Rebalance moves per node  Default is 1  Number of vBuckets moved at a time during the rebalance operation
  32. 32. ©2015 Couchbase Inc. 32 Compaction Settings  (Auto) Compaction  Append only storage engine  In-place updates are expensive  Removes tombstone objects and fragmentation  Process Data andView compaction in parallel  Implies a heavier processing and disk I/O load during compaction process
  33. 33. ©2015 Couchbase Inc. 33 Compaction Settings
  34. 34. Resource Requirements
  35. 35. ©2015 Couchbase Inc. 35 Resource Requirements CPU Disk (size, I/O) Number ofViews per Design Document Number of the emitted items Compaction Complexity of Map/Reduce functions Size of the emitted value 0 100 200 ms 0 5000 q / s  More CPU cores are recommended  Configure your OS File System Buffer!  Use SSD-s forViews!
  36. 36. Thank you! Q&A

×