Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Couchbase at Cox Automotive – Couchbase Live New York 2015

926 views

Published on

Tony discusses Cox Automotive’s journey with Couchbase. From initially just using it for caching, through specific data storage needs and future optimizations all enabling Cox Automotive to add value for its customers.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Couchbase at Cox Automotive – Couchbase Live New York 2015

  1. 1. Couchbase at Cox Automotive
  2. 2. Who we are 3
  3. 3. Who we are 4
  4. 4. Who we are 5
  5. 5. What we do at 6
  6. 6. What we do at 7
  7. 7. What we do at 8
  8. 8. Our data changes quickly 9
  9. 9. Couchbase at Cox Automotive Keeping pace with our data
  10. 10. 11 What sort of data do we store?
  11. 11. 12 What sort of data do we store?
  12. 12. 13 What sort of data do we store?
  13. 13. 14 How do we keep pace with our data?
  14. 14. 15 How do we keep pace with our data?
  15. 15. 16 How do we keep pace with our data?
  16. 16. 17 How do we keep pace with our data?
  17. 17. 18 How do we keep pace with our data? Two weeks to get to production!
  18. 18. Couchbase at Cox Automotive Accelerating our pace
  19. 19. 20 We churn through truckloads of data
  20. 20. 21 We churn through truckloads of data
  21. 21. 22 An API with Couchbase storage
  22. 22. 23 An API with Couchbase storage
  23. 23. 24 An API with Couchbase storage
  24. 24. 25 Implementation API design features • RESTful endpoints • Use of JSON format and immutable documents • Cached access to SQL Server look ups for configuration • Couchbase storage for speed and flexibility
  25. 25. 26 It’s simple to move our data into documents SQL Server crashes when we try to list all our “staging” tables!
  26. 26. 27 It’s simple to move our data into documents { "ImageURLs": [ "http://www.xhjsdfk.com/15234.jpg", "http://www.xhjsdfk.com/93784.jpg", "http://wwwxhjsdfk.com/13378.jpg" ], "Properties": { "BodyStyle": "2dr Car", "TransmissionSpeed": 6, "TransmissionTypeCode": "6-Speed Ma", "VehicleCondition": "New", "Year": 2021, }, "VIN": "1TAGP28135G148557" }, "Timestamp": "2015-10-05T13:38:46.2979624-04:00", "Version": 1 } JSON lends itself to flexible storage There’s no schema
  27. 27. 28 It’s simple to move our data into documents We coded an additive storage process. Say we get this input first….
  28. 28. 29 It’s simple to move our data into documents And then we get this data, for the same vehicle…
  29. 29. 30 It’s simple to move our data into documents We produce a resulting document…
  30. 30. 31 It’s simple to move our data into documents Now we’ve solved our archiving problem and our rollback problem! Version 1 Version 2
  31. 31. 32 How we got operational in two data centers Now we use Couchbase as our database of record for this data, we want resiliency with XDCR
  32. 32. 33 How we got operational in two data centers Remember that TTL setting for the cached data? Here’s some free advice 
  33. 33. Couchbase at Cox Automotive Staying at the front of the pack
  34. 34. 35 Wow; we’re maxed out again! We have a fancy SQL Server* doing the heavy lifting - 4 TB of primary data - Gigabytes of change per minute - 60 CPU cores! - 2TB RAM - 12TB Fusion IO cards! *cluster of 3 machines
  35. 35. 36 Data model migration Forward plans for SQL and NoSQL OPERATIONAL CACHEDCOMPUTED RAW DATA
  36. 36. 37 What functions should Couchbase serve? • Easing the load for heavy-read objects from SQL Server • Applications where ‘eventually consistent’ is OK • Outbound ‘file’ storage for computed documents • Novel queries against unstructured documents can leverage N1QL • When the query load is unpredictable, it can scale fast
  37. 37. 38 Future steps: How to migrate production data? Our traditional approach PRODUCTION STAGING RESTORE QA ENV. C# APP
  38. 38. 39 Future steps: How to migrate production data? With Couchbase 4.0 PRODUCTION STAGING RESTORE QA ENV. XDCR Filter
  39. 39. Couchbase at Cox Automotive Conclusion
  40. 40. 41 How we began with Couchbase “Start where you are.” - Chris Gardner
  41. 41. Couchbase at Cox Automotive Questions

×