Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Three Things You Need to Know About Data Modeling: Couchbase Connect 2015

1,714 views

Published on

We’re all familiar with modeling data the relational way. When we move to a document database we need to think about things a little differently. In this talk we’ll look how best to plan, model and maintain your data using a document database. By diving into real world case studies of Couchbase users, we’ll look at the three main things you need to know about modeling your data in a document database: document design, key design and querying.

  • Be the first to comment

Three Things You Need to Know About Data Modeling: Couchbase Connect 2015

  1. 1. THREE (OR MORE)THINGS YOU NEEDTO KNOW ABOUT DATA MODELING Matthew Revell, Developer Advocate, Couchbase
  2. 2. WE’RE STILL LEARNING
  3. 3. ©2015 Couchbase Inc. 3
  4. 4. ©2015 Couchbase Inc. 4 Three query methods Application layer computation Off-load computation Predictable queries Key-value: pre-computed answers Views Ad-hoc queries N1QL and key-value with manual indexes N1QL and views
  5. 5. KEY-VALUE
  6. 6. ©2015 Couchbase Inc. 6 Key-value  Pre-compute answers asynchronously  Store object state  Choose when to embed data and when to refer to it  Design your keys well
  7. 7. PRE-COMPUTED ANSWERS
  8. 8. ©2015 Couchbase Inc. 8 We’re used to asking questions
  9. 9. ©2015 Couchbase Inc. 9 Key-value look-ups give us a library of answers
  10. 10. ©2015 Couchbase Inc. 10 Answer oriented databases http://martinfowler.com/bliki/AggregateOrientedDatabase.html
  11. 11. ©2015 Couchbase Inc. 11 Airline bookings
  12. 12. EMBED, OR REFER? How much should we denormalise?
  13. 13. ©2015 Couchbase Inc. 13 An ecommerce order
  14. 14. ©2015 Couchbase Inc. 14 The same order in a document
  15. 15. ©2015 Couchbase Inc. 15 Embedded v referred ©2014 Couchbase, Inc. 15
  16. 16. ©2015 Couchbase Inc. 16 When to embed  You should embed data when:  Speed trumps all else  Slow moving data  No duplication  Application layer can keep multiple copies of same data in sync
  17. 17. ©2015 Couchbase Inc. 17 When to refer  You should refer to data when:  Consistency is a priority  The data has large growth potential
  18. 18. KEY DESIGN
  19. 19. ©2015 Couchbase Inc. 19 Three ways to build a key  Key design is as important as document design.  There are three broad types of key:  Human readable/deterministic: e.g. an email address  Computer generated/random: e.g. UUID  Compound: e.g. UUID with a deterministic portion
  20. 20. ©2015 Couchbase Inc. 20 Human readable/deterministic public class user { private String name; private String email; private String streetAddress; private String city; private String country; private String postCode; private String telephone; private Array orders; private Array productsViewed; } { "name": "Matthew Revell", "address": "11-21 Paul Street", "city": "London", "postCode": "EC2A 4JU", "telephone": "44-20-3837-9130", "orders": [ 1, 9, 698, 32 ], “productsViewed”: [8, 33, 99, 100] } Key: matthew@couchbase.com
  21. 21. ©2015 Couchbase Inc. 21 Random/computer genereated { "name": "Matthew Revell", "email": "matthew@couchbase.com", "address": "11-21 Paul Street", "city": "London", "postCode": "EC2A 4JU", "telephone": "44-20-3837-9130", "orders": [ 1, 9, 698, 32 ], “productsViewed”: [8, 33, 99, 100] } Key: 1001
  22. 22. ©2015 Couchbase Inc. 22 Using counters to generate keys ©2014 Couchbase, Inc. 22 Application user_id = incr(“counter_key") add(user_id, data) Creating a new user add(email_address, user_id) Application key = get("matthew@couchbase.com") get(key) Finding user by email address
  23. 23. ©2015 Couchbase Inc. 23 Multiple look-up documents u::count 1001 u::1001 { "name": “Matthew Revell", "facebook_id": 16172910, "email": “matthew@couchbase.com”, “password”: ab02d#Jf02K "created_at": "5/1/2012 2:30am", “facebook_access_token”: xox0v2dje20, “twitter_access_token”: 20jffieieaaixixj } fb::16172910 1001 nflx::2939202 1001 twtr::2920283830 1001 em::matthew@couchbase.com 1001 em::matthew@understated.co.uk 1001 uname::mrevell 1001
  24. 24. ©2015 Couchbase Inc. 24 Compound keys  Compound keys are look-up documents with a predictable name.  It’s a continuation of the embedded versus referred data discussion.
  25. 25. ©2015 Couchbase Inc. 25 Compound keys: example u::1001 { "name": "Matthew Revell", "email": "matthew@couchbase.com", "address": "11-21 Paul Street", "city": "London", "postCode": "EC2A 4JU", "telephone": "44-20-3837-9130", "orders": [ 1, 9, 698, 32 ], “productsViewed”: [8, 33, 99, 100] }
  26. 26. ©2015 Couchbase Inc. 26 Compound keys: example u::1001 { "name": "Matthew Revell", "email": "matthew@couchbase.com", "address": "11-21 Paul Street", "city": "London", "postCode": "EC2A 4JU", "telephone": "44-20-3837-9130", "orders": [ 1, 9, 698, 32 ] } u::1001::productsviewed { "productsList": [ 8, 33, 99, 100 ] }
  27. 27. ©2015 Couchbase Inc. 27 Compound keys: example u::1001 { "name": "Matthew Revell", "email": "matthew@couchbase.com", "address": "11-21 Paul Street", "city": "London", "postCode": "EC2A 4JU", "telephone": "44-20-3837-9130", "orders": [ 1, 9, 698, 32 ] } u::1001::productsviewed { "productsList": [ 8, 33, 99, 100 ] } p::8 { id": 1, "name": "T-shirt", "description": "Red Couchbase shirt", "quantityInStock": 99, "image": "tshirt.jpg” }
  28. 28. ©2015 Couchbase Inc. 28 Compound keys: example u::1001 { "name": "Matthew Revell", "email": "matthew@couchbase.com", "address": "11-21 Paul Street", "city": "London", "postCode": "EC2A 4JU", "telephone": "44-20-3837-9130", "orders": [ 1, 9, 698, 32 ] } u::1001::productsviewed { "productsList": [ 8, 33, 99, 100 ] } p::8 { id": 1, "name": "T-shirt", "description": "Red Couchbase shirt", "quantityInStock": 99 } p::8::img “http://someurl.com/tshirt.jpg”
  29. 29. N1QL AND VIEWS! The world beyond key-value
  30. 30. ©2015 Couchbase Inc. 30
  31. 31. ©2015 Couchbase Inc. 31 N1QL andViews N1QL Views Ad-hoc querying Predictable queries Nested JSON in and unnested JSON out Number crunching Large growth clusters Multi-dimensional/geospatial queries
  32. 32. ©2015 Couchbase Inc. 32 N1QL  Indexes  Schema and documents types  Keyspaces
  33. 33. ©2015 Couchbase Inc. 33 Indexes  You always need at least one index  PRIMARY index gives you all of the keys in the bucket  Secondary indexes enable ad-hoc querying at speed
  34. 34. ©2015 Couchbase Inc. 34 Indexes
  35. 35. ©2015 Couchbase Inc. 35 Indexes NEW Global Secondary Indexes Views N1QL query to create a new index Javascript map/reduce Option to run dedicated indexing and query nodes Queries always run on database nodes ForestDB backed Couchstore backed Supports multi-dimensional/geospatial queries
  36. 36. ©2015 Couchbase Inc. 36 Maintaining schema and working with keyspaces  JOINs work on primary keys and secondary keys  They JOIN across and within keyspaces (for now, that means buckets) Airlines: airline_24 ← Key (“primary key”) { "id": "24", "type": "airline", "name": "AmericanAirlines", "iata": "AA", "icao": "AAL", "callsign": "AMERICAN", "country": "United States", "active": "Y" } Routes: route_5966 ← Key { "id": "5966", "type": "route", "airline": "AA", "airlineid": "airline_24", ←This is the foreign key "sourceairport": "MCO", "destinationairport": "SEA", "stops": "0", "equipment": "737", "schedule": [... ] }
  37. 37. ©2015 Couchbase Inc. 37 Offloading computation to N1QL N1QL allows you to choose which elements are returned at the top level. This is incredibly useful as it reduces the need to write complex parsing logic:  Within the application.  In Client side front end frameworks (Angular/Backbone etc) SELECT a.name, s.flight, s.utc, r.sourceairport, r.destinationairport, r.equipment FROM `travel-sample` r UNNEST r.schedule s JOIN `travel-sample` a ON KEYS r.airlineidWHERE r.sourceairport='LHR' AND r.destinationairport='LAX' AND s.day=2 ORDER BY a.name
  38. 38. QUESTIONS!
  39. 39. OTHER SESSIONS! IRFC: A design pattern for next gen business applications Steve Yen, 4.30pm Thursday Dev track
  40. 40. THANKS! Matthew Revell matthew@couchbase.com @matthewrevell
  41. 41. Get Started withCouchbase Server 4.0: www.couchbase.com/beta GetTrained on Couchbase: training.couchbase.com

×