Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Why, When, and How of NoSQL - A Practical Approach

784 views

Published on

More and more Fortune 1000 companies like Marriott, Cars.com, Gannett, and PayPal are choosing NoSQL over relational databases like Oracle, SQL Server, and DB2 to power their web, mobile, and IoT applications. Why? Lower costs, higher performance and availability, better agility, and easier scalability. According to The Forrester Wave™: Big Data NoSQL, Q3 2016 report, “NoSQL is no longer an option.” Come see why.

This webinar is intended for developers, architects, and database engineers who are considering NoSQL as an alternative to relational databases. If you’re looking to add NoSQL to your environment, this webinar will show you how to get started and avoid potential pitfalls.

You’ll get practical advice, including:
•Key considerations in moving from relational to NoSQL
•How to identify applications that benefit most from NoSQL
•Data modeling and querying with NoSQL
•Migrating your data to NoSQL
•Best practices for making the switch

Published in: Technology
  • Be the first to comment

The Why, When, and How of NoSQL - A Practical Approach

  1. 1. The Why, When, and How of NoSQL: A Practical Approach David Segleau Dir.Technical Product Marketing Couchbase
  2. 2. ©2016 Couchbase Inc. 2 About the speaker – David Segleau David Segleau DirectorTechnical Product Marketing Couchbase (since Nov 2015) Experience: - Database guy - Oracle, Sleepycat, Informix, Illustra,Teradata - VP Eng, Prod Mgmt, QA, Support,Training, Docs - Technology is only useful when it’s deployed - Expertise: - Database server technology, RDBMS, and NoSQL
  3. 3. ©2016 Couchbase Inc. 3 What’s Couchbase? Couchbase is the company behind Couchbase Server & Couchbase Mobile • Open source JSON database • Founded 2010 • 500+ enterprise customers globally Some of our customers: Couchbase Server can be deployed as: Document database Key-value store Distributed cache
  4. 4. ©2016 Couchbase Inc. 4 Today’s agenda  Why NoSQL?  Identifying the right application  Modeling your data  Accessing your data  Installing and scaling your database  Migrating your data  Q & A
  5. 5. ©2016 Couchbase Inc. 5 What is NoSQL?  NoSQL?  Not only SQL? Non relational?  Distributed (most) – Scaled out, not up • Elasticity and commodity hardware – Partitioned and replicated • Scalability, performance, availability  Schema-less (most) – Flexible model – JSON (some)  Multi-model – Key-value & Document – Columnar & Graph – Graph & Key-value
  6. 6. ©2016 Couchbase Inc. 6 Who is using NoSQL? Enterprises are adopting NoSQL for mission critical applications Media & Publishing eCommerce Hospitality
  7. 7. ©2016 Couchbase Inc. 7 Who is using NoSQL?  Gannett, publisher of 90+ media properties, replaced relational database technology with NoSQL to power its digital publishing platform.  eBay, with over 2 billion page views per day, uses Couchbase + RDBMS for their Listing cache, and Couchbase as database of record forToken management.  Marriott deployed NoSQL to modernize its hotel reservation system that supports $38 billion in annual bookings.  FHLBankTopeka leverages NoSQL on top of SQL Server to speed up access to customer financial data for its 770 member banks.  Cars.com, with over 30 million visits per month, replaced SQL Server with NoSQL to store customer and vehicle data.
  8. 8. ©2016 Couchbase Inc. 8 Why are they using NoSQL? Technology Drivers  Customers are going online  The internet is connecting everything  Big Data is getting bigger  Applications are moving to the cloud  The world has gone mobile Technical Needs  Develop with agility – Flexibility + Simplicity – Easier + Faster  Operate at any scale – Elasticity + Availability – Performance at scale – Always-on, global deployment Business Needs  Innovate and compete – Faster time to market – Reduced costs (operational + hardware) – Increased revenue
  9. 9. ©2016 Couchbase Inc. 9 NoSQL vs. RDBMS  Replace or Complement?  It depends – Replace: NoSQL is often the operational database of record – Complement: NoSQL adds perf, scale, and availability to legacy RDBMS  Customers use RDBMS and NoSQL  NoSQL is adding RDBMS features – Security,QueryLanguage,Analytics  RDBMS is adding NoSQL features – Sharding, JSON, Distributed Processing
  10. 10. ©2016 Couchbase Inc. 10 Why migrate from an RDBMS to NoSQL?  Easier to scale 3 nodes to 100s, 1 data center to many, commodity hardware  Better performance Integrated caching, memory-optimized indexes, memory-based replication  Up to 40x lower cost Open source, subscription-based, per instance (not per core)  Cross-platform Runs onWindows or Linux (Red Hat, Ubuntu, Debian, etc.)  Greater agility JSON-based data model, SQL-based query language
  11. 11. ©2016 Couchbase Inc. 11 How do you get started? 1. Identify the right application 2. Model your data 3. Access your data 4. Install and scale your database 5. Migrate your data
  12. 12. ©2016 Couchbase Inc. 12 Identifying the right application
  13. 13. ©2016 Couchbase Inc. 13 Identifying the right application Have one or more of the following characteristics or requirements:  Innovate and iterate faster  Send and receive JSON  Provide low latency at any throughput  Support many concurrent users  Supports users anywhere and everywhere  Be available 24x7  Store terabytes of data  Read and write to multiple data centers Service RDBMS Service Service NoSQL Application Examples:  High performance, high availability caching service  Independent application with a narrow scope  Logical or physical service within a large application  Global service that powers multiple applications
  14. 14. ©2016 Couchbase Inc. 14 Model your data
  15. 15. ©2016 Couchbase Inc. 15 Demystifying terminology Relational NoSQL (Couchbase) Failover Cluster Cluster Availability Group Cluster Database Bucket Table Bucket Row (Tuple) Document (JSON) Primary Key Object ID IDENTITY or Sequence Counter Indexed View View SQL N1QL
  16. 16. ©2016 Couchbase Inc. 16 Modeling your data: Fixed vs. self-describing schema
  17. 17. ©2016 Couchbase Inc. 17 Modeling your data:The flexibility of JSON Same document type, Different fields • Different types • Optional • On demand Tip:Add a version field to track changes. {“docType”: “user”, “docVersion”: “1”, …} {“docType”: “user”, “docVersion”: “2”, …}
  18. 18. ©2016 Couchbase Inc. 18 Modeling your data: Changing the data model Relational database • Modify the database schema • Modify the application code (e.g., Java) • Modify the interface (e.g., HTML5/JS) Document database • Modify the interface (e.g., HTML5/JS)
  19. 19. ©2016 Couchbase Inc. 19 Modeling your data: Object IDs Best Practices • Natural Keys • Human Readable • Deterministic • Semantic Examples • author::shane • author::shane::blogs • blog::nosql_fueled_hadoop • blog::nosql_fueled_hadoop::comments What about identity columns? 1. Document<Long> nextAuthorIdDoc = bucket.counter(“authorIdCounter”, 1); 2. Long nextAuthorId = nextAuthorIdDoc.content(); 3. String authDocId = “author::” + nextAuthorId; // author::101 Tip: Increment the counter by 10, 20, etc. instead of doing it for every insert.
  20. 20. ©2016 Couchbase Inc. 20 Modeling your data: Relationships Author Blog (FK)Blog (FK) Comment (FK) Comment (FK) Author (FK x2) BlogBlog (FK x2) Comment Comment Bottom up Top down
  21. 21. ©2016 Couchbase Inc. 21 Modeling your data: Relationships
  22. 22. ©2016 Couchbase Inc. 22 Modeling your data: Strategies and best practices If … Then … Relationship is one-to-one or one-to-many Store related data as nested objects Relationship is many-to-one or many-to-many Store related data as separate documents Data reads are mostly parent fields Store children as separate documents Data reads are mostly parent + child fields Store children as nested objects Data writes are mostly parent or child (not both) Store children as separate documents Data writes are mostly parent and child (both) Store children as nested objects
  23. 23. ©2016 Couchbase Inc. 23 Modeling your data: Strategies and best practices  Are there a lot of concurrent writes, continuous updates?  Store children as separate documents Blog  Thread  Comment  Comment  Thread  Comment  Comment Blog { “docType”: “blog”, “author”: “author::shane”, “title”: “Couchbase Wins”, “threads”: [ “blog::couchbase_wins::threads::001”, “blog::couchbase_wins::threads::002” } Thread { “docType”: “thread”, “comments”: [ { “visitor”: “Brendan Bond”, “text”: “This blog is amazing!” “replies”: [ { “user”: “Dustin Johnson”, “text”: “No, it is not.” }] } }
  24. 24. ©2016 Couchbase Inc. 24 Access your data
  25. 25. ©2016 Couchbase Inc. 25 Accessing your data: Options Key-Value (CRUD) N1QL (Query) Views (Query) Documents Indexes MapReduce FullText (Search) Geospatial (Search) We’ll focus on these three for now. Indexes MapReduce
  26. 26. ©2016 Couchbase Inc. 26 Accessing your data: Connecting to the database  Access data via topology-aware smart clients  Maintains an up-to-date cluster map  Communicates directly with database nodes – no proxies, no routers, etc.  Available for Java, Node.js, PHP, .NET, Python, C, Go, and more  With standard, certified JDBC/ODBC drivers (if you want to)
  27. 27. ©2016 Couchbase Inc. 27 Accessing your data: Domain objects vs. document objects * JSON serialization via Boon. Working with document objects requires less code, provides more flexibility.
  28. 28. ©2016 Couchbase Inc. 28 Accessing your data: Key-value operations – referenced data
  29. 29. ©2016 Couchbase Inc. 29 Accessing your data: Key-value operations – nested data
  30. 30. ©2016 Couchbase Inc. 30 Accessing your data: Subdocument operations
  31. 31. ©2016 Couchbase Inc. 31 Accessing your data – N1QL queries: Capabilities Feature SQL N1QL JOIN ✔ ✔ TRANSFORM ✔ ✔ FILTER ✔ ✔ AGGREGATE ✔ ✔ SORT ✔ ✔ SUBQUERIES ✔ ✔ PAGINATION ✔ ✔ OPERATORS ✔ ✔ FUNCTIONS ✔ ✔
  32. 32. ©2016 Couchbase Inc. 32 Accessing your data: N1QL queries – referenced data
  33. 33. ©2016 Couchbase Inc. 33 Accessing your data: N1QL queries – nested data
  34. 34. ©2016 Couchbase Inc. 34 Accessing your data: N1QL queries – CRUD
  35. 35. ©2016 Couchbase Inc. 35 Accessing your data: LINQ _context.Query<Users>().Where(u => u.status == “Platinum”) from user in _context.Query<Users>() join account in _context.Query<Account>() on user.accountId equals N1QlFunctions.Key(account) into userGroup from accounts in userGroup.DefaultIfEmpty() where (account.type == “Visa” || account.type == “MasterCard”) select new { firstName = user.firstName, lastName = user.LastName }; from user in _context.Query<Users>() join address in _context.Query<Address>() on user.addresses.shipping.addressId equals N1QlFunctions.Key(address) into addressGroup from address in addressGroup.DefaultIfEmpty() where address.state == “CA” select new { firstName = user.firstName, lastName = user.LastName };
  36. 36. ©2016 Couchbase Inc. 36 Accessing your data: N1QL queries – indexes Simple Compound Functional Partial
  37. 37. ©2016 Couchbase Inc. 37 Accessing your data:Views What if indexed views worked great with write intensive workloads? And you could use COUNT,ORDER BY, and everything else …
  38. 38. ©2016 Couchbase Inc. 38 Accessing your data:Views
  39. 39. ©2016 Couchbase Inc. 39 Accessing your data:Views – Incremental MapReduce
  40. 40. ©2016 Couchbase Inc. 40 Accessing your data:Views – queries
  41. 41. ©2016 Couchbase Inc. 41 Accessing your data: Strategies and best practices Concept Strategies & Best Practices Key-Value Operations provide the best possible performance • Create an effective key naming strategy • Create an optimized data model Incremental MapReduce (Views) are well suited to aggregation • Ideal for large data sets, the entire data set • Can be used to create complex secondary indexes N1QL queries provide the most flexibility – everything else • Query data regardless of how it is modeled • Remember to create indexes, leverage covering indexes where possible
  42. 42. ©2016 Couchbase Inc. 42 Install and scale your database
  43. 43. ©2016 Couchbase Inc. 43 Installing and scaling your database Admin screenshot – add node 1. Download 2. Install 3. Configure 4. Cluster / Scale 5. Rebalance
  44. 44. ©2016 Couchbase Inc. 44 Installing and scaling your Couchbase database: XDCR  MultipleDataCenters  Cluster per data center  Replicate between clusters  Unidirectional / bidirectional  Master / Master  Local reads and writes  Ring  Hub-and-spoke  Mesh  Combination  Built-in
  45. 45. ©2016 Couchbase Inc. 45 Installing and scaling your Couchbase database: MDS  Multi-Dimensional Scaling  SOA for databases  Independent service scalability  Scale out and scale out  Scale Data, Query, Indexing and FullText services Scale out Scale up
  46. 46. ©2016 Couchbase Inc. 46 Migrate your data
  47. 47. ©2016 Couchbase Inc. 47 So many options! Remember the KISS principle 1) Identify the requirements • ETL vs. Data cleanse vs. Data enrichment • Duration vs. Resources • Data governance 2) Pick your strategy • Batch vs. Incremental • Single threaded vs. multi-threaded 3) Pick your tools • Data migration tools (Informatica, Looker, Talend) • BYO-tool (PHP & Python scripts, Hadoop, Spark) • KISS with Couchbase • Export to CVS; Import as documents; Use N1QL to transform & insert into new bucket • Use SQL to transform & export; Insert into Couchbase • Best Practices • Align with your data model • Plan for failure (bad source data, hardware failure, resource limitations) • Ensure interruptible, restartable, logged, predictable
  48. 48. ©2016 Couchbase Inc. 48 How can you sync NoSQL and relational?  1.ApplicationCode(Manual)  2. Replication (Automatic) – From NoSQL to relational – From relational to NoSQL Couchbase Kafka Queue Producer Consumer RDBMSDCP Stream RDBMS Handler CouchbaseGoldenGate https://github.com/mahurtado/CouchbaseGoldenGateAdapter
  49. 49. ©2016 Couchbase Inc. 49 Where to go next? Conduct a Successful Proof of Concept 1. Select a use case and application 2. Define the success criteria 3. Understand the data 4. Identify the access patterns 5. Review the architecture Measure your Return on Investment  Greater agility?  Faster time to market?  Easier scalability?  Better performance?  Better availability?  Lower costs?
  50. 50. ©2016 Couchbase Inc. 50 Questions?
  51. 51. ©2016 Couchbase Inc. 51 Best practices recap • Pick the right application • Focus on SOA, application/use case specific • Drive data model from data access patterns • Use Document type, Versionid • Create optimized, understandible keys • Weigh nested, referenced or mixed designs • Add indexes: Simple, Compound, Functional, Partial, Covering, Memory Optimized • Match the data access method to requirements • Key-value, Views, N1QL • Proof of Concept • Focus, Success Criteria, Review Architecture
  52. 52. ©2016 Couchbase Inc. 52 Want to learn more? Getting Started guide: http://www.couchbase.com/get-started-developing-nosql Download Couchbase software: http://www.couchbase.com/nosql-databases/downloads Free OnlineTraining http://training.couchbase.com/online “Why NoSQL” white paper http://www.couchbase.com/nosql-resources/why-nosql
  53. 53. ©2016 Couchbase Inc. 53 Thank you

×