Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Agility with Data Modeling at DIRECTV Using N1QL in Couchbase with Node.js: Couchbase Connect 2015

2,829 views

Published on

Today's companies are challenged to change rapidly and while maintaining high quality standards. Managing Entity–relationship models has become a constraint to these objectives. Now, NoSQL solutions and dynamic languages are offering a simplified approach to modeling real world objects. In particular, JavaScript offers an end-to-end solution – from the user's web browser to the JSON object stored in the database. By integrating Couchbase into the picture, the performance and flexibility offered by projects with these technologies will become the next big thing: fast paced development, agile changes, high performance, easily scalable, and feature rich. This presentation is about the benefits and challenges of an architecture based on: Node.js, Couchbase and N1QL.

  • Be the first to comment

Agility with Data Modeling at DIRECTV Using N1QL in Couchbase with Node.js: Couchbase Connect 2015

  1. 1. Agility with Data Modeling at DIRECTV Using N1QL in Couchbase with Node.js Fidencio Garrido @elfidomx Github: theFido Isidro Salcedo @isidrosalcedo88
  2. 2. About Us DIRECTV is one of the world's leading providers of digital television entertainment services delivering a premium video experience through state-of-the-art technology, superior entertainment experience, and award winning customer service. Offers over 3,000 SD, HD, 3D and 4K channels in the US alone Higher customer satisfaction than the leading cable companies for 14 years running Global presence servicing multiple countries across the entire western hemisphere. Over 38 million subscribers around the world.
  3. 3. The Challenge Developing systems capable of supporting business changes in a fast pace environment. Supports • Additional metadata types (different versions of objects) • Mixed configurations • Reduction in code changes Requirements • Reliable • Scalable • High Performance • Easy Administration (nice to have) Options • RDBMS • Key-Value Stores • Document DB
  4. 4. Architecture Goals The overall architecture must perform as if your business depended on it… because it does! ● High Performance ● Reliability ● Scalability ● Easy to Manage ● Never be a constraint
  5. 5. Why Couchbase? ● Performance beast ● Easy setup / administration: ● Monitoring tools ● REST Administration API ● All levels of scalability ● Almost flat learning curve (Thanks N1QL!) ● Expressive Language for Query
  6. 6. Couchbase Features ● Document Based ● Persistence + Cache ● Support of Views (map-reduce functions also written in JS) ● Separation of Concerns (index, query, data store) ● Fast Rebalancing
  7. 7. Couchbase N1QL Benefits ● Simplified Code ● Reduce Code Complexity: Power of SQL ● Enable Ad-Hoc Queries: interactive queries with low latency with indexes ● Improves granularity of query consistency ● Logical-timestamp at run time ● More processing on the server side
  8. 8. Why NodeJS? ● Fast I/O ● Native JSON Speaker ● Soft Typed ● Single threaded but asynchronous ● Light middleware layer provided by express
  9. 9. API Design ● Stateless ● Fully REST Compliant ● Resource Oriented (Avoiding Verbs) ● Using the Right Verbs ● Returning the Right HTTP Code ● Using HTTP Headers to Provide Metadata
  10. 10. Starting Design API API API LB BL BL BL DAS DAS DAS DB DB N1QL LB API BL DAS N1QL
  11. 11. Clients Growing API API APILB BL BL BL DAS DAS DAS DB DB N1QLLB API BL DAS N1QL API BL DAS
  12. 12. More Read han Write Operations API API API LB get BL BL BL DAS DAS DAS DB DB N1QLLB API BL DAS N1QL N1QLAPI BL DAS LB post put delete
  13. 13. More Writes than Read Operations API API API LB get BL BL BL DAS DAS DAS DB DB DBLB API BL DAS Index N1QLAPI BL DAS LB post put delete
  14. 14. More Key Based Operation API API API LB BL BL BL DAS DAS DAS DB DB DB LB API BL DAS N1QL N1QL Index
  15. 15. Geographic Expansion Enabling an architecture that scales across multiple data centers to support our global business. Data Center #1 Data Center #2 XDCR Data Center #4Data Center #3
  16. 16. Using Documents ● NodeJS is enough for the backend ● Couchbase is great to store the information and retrieve it using the document key but… What happens when you want to use Couchbase as multipurpose Database System? Can you really have the same capabilities as any other traditional SQL solution? Short answer…
  17. 17. Meet Couchbase N1QL ● A language that extends SQL capabilities to search into documents ● Supports search through complex structures ● Join operations for documents even if they belong to a different bucket ● Advanced features such as nesting and unnesting ● Index Support
  18. 18. Gentle Learning Curve ● A Language we were already familiar with ● Classic SQL SELECT first_name, last_name FROM customers WHERE email = "someone@directv.com"; ● N1QL SELECT first_name, last_name FROM bucket WHERE email = "someone@directv.com";
  19. 19. Ad-hoc Querying { "_metadata": { "document_type": "program", "updated": "2015-06-02", "last_user": "admin" }, "data": { "cast_members" : [ { "person_id" : 14531, "role_id" : 1 }, { "person_id " : 5983, "role_id" : 1 } ] } } SELECT * FROM bucket WHERE _metadata.document_type = "program" AND ANY c_0 IN data.cast_members SATISFIES c_0.person_id = 15415 END;
  20. 20. Analyze EXPLAIN SELECT * FROM bucket WHERE _metadata.document_type = "program"; { "resultset": [ { "input": { "expr": { "left": { "left": { "left": { "path": "dtv-ps-store-dtss", "type": "property" }, "right": { "path": "_metadata", "type": "property" }, "type": "dot_member" }, … }, "input": { "as": "bucket", "bucket": "bucket", "ids": null, "input": { "as": "", "bucket": "bucket", "cover": false, "index": "ix_document_type", "pool": "default", …
  21. 21. Challenges ● Thinking in Documents ● Think in the structure ● Visualize the queries you want to perform { “user”: “dtv-user”, “phone”: [“(333) 333-3333”, “(444) 444-4444”] } { “user”: “dtv-user”, “platform”: [“iOS 7”, “Android 4”, “Windows 8.1”] } OR { “user”: “dtv-user”, “platform”: [123, 232, 342] } ● One Data Space { “_metadata”: { “documentType”: “user”, “updated”: “2015-06-02”, “lastUser”: “admin” }, “data”: { “user”: “dtv-002”, “state”: “active”, …. } }
  22. 22. Challenges ● Know your data flow: Use explain ● Plan your infrastructure ● N1QL does processing… therefore it requires processors. Trust me, it knows what to do with them.
  23. 23. Roadmap Support expanding business demands and data growth on common, easy to maintain, platform Upgrade to Couchbase 4.0 to leverage Multi Dimensional Scale Leverage the power of Global Secondary Indices for instant lookups and query at scale
  24. 24. 24 Our development toolkit
  25. 25. Questions

×