Download as PDF, PPTX
![JSON
Document
Structure
meta
{
Meta
Informa@on
“id”:
“u::jasdeep@couchbase.com”,
Including
Key “rev”:
“1-‐0002bce0000000000”,
“flags”:
0,
“expira@on”:
0,
All
Keys
Unique
and
“type”:
“json”
Kept
in
RAM }
document
{
“uid”:
123456,
Document
Value “firstname”:
“jasdeep”,
“lastname”:
“Jaitla”,
Most
Recent
In
Ram
“age”:
22,
“favorite_colors”:
[“blue”,
“black”],
And
Persisted
To
Disk
“email”:
“jasdeep@couchbase.com”
}
Wednesday, March 20, 13](https://image.slidesharecdn.com/normandy-jug-march-2013-130322160230-phpapp02/85/Couche-Base-par-Tugdual-Grall-37-320.jpg)
Uploaded byNormandy JUG
Couche Base par Tugdual Grall
The document outlines a presentation on NoSQL databases, specifically focusing on Couchbase, its features, and its advantages over traditional RDBMS. It discusses the challenges of scaling and flexibility faced by organizations, highlighting Couchbase's ability to provide high scalability and performance with its JSON document model and distributed architecture. The document also covers Couchbase's architecture, client libraries, operations, and indexing capabilities, concluding with a message encouraging engagement and questions from the audience.
More Related Content
Similar to Couche Base par Tugdual Grall
![[Codeurs en seine] management & monitoring cloud](https://cdn.slidesharecdn.com/ss_thumbnails/codeurenseinemanagementmonitoringcloud-131101061123-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
Couche Base par Tugdual Grall
- 1.
- 2. Win some coolstuff, send a mail to winning@couchbase.com with NORMANDYJUG in the subject. Wednesday, March 20, 13
- 3. Normandy JUG -‐ March 19th 2013 Introduc)on to NoSQL with Couchbase Tugdual “Tug” Grall Technical Evangelist email: tug@couchbase.com twi0er: @tgrall Wednesday, March 20, 13
- 4. About me • Tugdual “Tug” Grall • Web -‐ Couchbase -‐ @tgrall -‐ Technical Evangelist -‐ hQp://blog.grallandco.com -‐ eXo -‐ tgrall -‐ CTO • NantesJUG co-‐founder -‐ Oracle • Pet Project : -‐ Developer/Product Manager • hQp://www.resultri.com -‐ Mainly Java/SOA -‐ Developer in consul)ng firms Wednesday, March 20, 13
- 5. INTRO TO NOSQL Wednesday,March 20, 13
- 6. RDBMS ARE NOT ENOUGH? Wednesday, March 20, 13
- 7. Growth is the New Reality • Instagram gained nearly 1 million users overnight when then expanded to Android Wednesday, March 20, 13
- 8. Draw Something -‐ Social Game 35 million monthly active users in 1 month !!! Wednesday, March 20, 13
- 9. By contrast.... The Simpson’s : Tapped Out Daily Active Users (Millions) Wednesday, March 20, 13
- 10. How do you take this growth? Applica@on Scales Out Just add more commodity web servers System Cost Applica)on Performance Web/App Server Tier Users RDBMS Scales Up Get a bigger, more complex server System Cost Applica)on Performance Won’t scale beyond this point Rela@onal Database Users RDBMS is good for many thing, but hard to scale Wednesday, March 20, 13
- 11. Scaling out RDBMS • Run Many SQL Servers Web/App Server Tier • Data could be shared -‐ Done by the applica)on code Memcached Tier • Caching for faster response )me MySQL Tier Wednesday, March 20, 13
- 12. NoSQL Technology Scales Out Applica@on Scales Out Just add more commodity web servers System Cost Applica)on Performance Web/App Server Tier Users NoSQL Database Scales Out Cost and performance mirrors app @er System Cost Applica)on Performance NoSQL Distributed Data Store Users Scaling out fla;ens the cost and performance curves Wednesday, March 20, 13
- 13. A New Technology? Bigtable Dynamo Cassandra Voldemort November 2006 October 2007 August 2008 February 2009 Building new database to answer the following requirements • No schema required before inser)ng data • No schema change required to change data format • Auto-‐sharding without applica)on par)cipa)on • Distributed queries • Integrated main memory caching • Data synchroniza)on ( mul)-‐datacenter) Very few organiza@ons want to (fewer can) build and maintain database soQware technology. But every organiza@on building interac@ve web applica@ons needs this technology. Wednesday, March 20, 13
- 14. What Is Biggest Data Management Problem Driving Use of NoSQL in Coming Year? 49% 35% 29% 16% 12% 11% Lack of flexibility/ Inability to scale Performance Cost All of these Other rigid schemas out data challenges Source: Couchbase Survey, December 2011, n = 1351. Wednesday, March 20, 13
- 15. NO SQL TAXONOMIES Wednesday,March 20, 13
- 16. NoSQL Catalog Key-‐Value Data Structure Document Column Graph (memory only) Cache Memcached Redis (memory/disk) Database Membase Couchbase Cassandra Neo4j MongoDB Wednesday, March 20, 13
- 17. NoSQL Catalog Key-‐Value Data Structure Document Column Graph (memory only) Cache Memcached Redis Coherence (memory/disk) Database Membase Couchbase Cassandra Neo4j MongoDB HBase InfiniteGraph Wednesday, March 20, 13
- 18.
- 19. OperaIonal vs. AnalyIc Databases Real-‐Cme, AnalyCc InteracCve Databases Databases NoSQL Fast access Get insights from to data data Couchbase Cassandra Cloudera MongoDB Hbase Hortonworks Mapr Wednesday, March 20, 13
- 20.
- 21. Couchbase Server Core Principles Easy Consistent High Scalability PE RF O R M A N C E Performance Grow cluster without Consistent sub-‐millisecond applica^on changes, without read and write response ^mes down^me with a single click with consistent high throughput Always On JSON JSON JSO Flexible Data 24x365 Model JSON N JSON No down^me for so`ware JSON document model with no upgrades, hardware fixed schema. maintenance, etc. Wednesday, March 20, 13
- 22. Couchbase 2.0 New Features JSON support Indexing and Querying Incremental Map Cross data center Reduce replication Wednesday, March 20, 13
- 23. Couchbase Handles Real World Scale Wednesday, March 20, 13
- 24. 8092 Couchbase Server 2.0 Architecture 11211 11210 Query API Memcapable 1.0 Memcapable 2.0 Sub^tle Moxi Query Engine REST management API/Web UI vBucket state and replica)on manager Memcached Global singleton supervisor Rebalance orchestrator Configura)on manager Node health monitor Process monitor Couchbase EP Engine Heartbeat Data Manager Cluster Manager storage interface New Persistence Layer hdp on each node one per cluster Erlang/OTP HTTP Erlang port mapper Distributed Erlang 8091 4369 21100 -‐ 21199 Wednesday, March 20, 13
- 25. Couchbase Server 2.0 Architecture 8092 11211 11210 Query API Memcapable 1.0 Memcapable 2.0 Moxi Query Engine REST management API/Web UI vBucket state and replica)on manager Object-‐level Cache Global singleton supervisor RAM Cache, Rebalance orchestrator Configura)on manager Node health monitor Server/Cluster Process monitor Indexing & Heartbeat Couchbase EP Engine Management & Persistence storage interface Communica@on Management (Erlang) New (C Persistence Disk & V8) Persistence Layer hdp on each node one per cluster Erlang/OTP The Unreasonable Effectiveness of C by Damien Katz HTTP Erlang port mapper Distributed Erlang 8091 4369 21100 -‐ 21199 Wednesday, March 20, 13
- 26. Open Source Project Apache 2.0 hQps://github.com/couchbase/ Gerrit: hQp://review.couchbase.org/ hQps://github.com/couchbaselabs/ Wednesday, March 20, 13
- 27. SETTING UP TO DEVELOP Wednesday, March 20, 13
- 28. Install Couchbase Server 2.0 Ubuntu RedHat Mac OS X Windows or build from sources Wednesday, March 20, 13
- 29. Official SDKs Ruby libcouchbase Clojure Python www.couchbase.com/develop Go Wednesday, March 20, 13
- 30. Client SDKs c)on on n e make c y ol og Couchbase eceiv e top r Couchbase Client Topology App Server Update Wednesday, March 20, 13
- 31.
- 32. Write OperaIon Doc 1 App Server 3 3 Managed Cache 2 To other node Replica^on Doc 1 Queue Disk Queue Disk Couchbase Server Node Wednesday, March 20, 13
- 33. Basic OperaIons APP SERVER 1 APP SERVER 2 COUCHBASE Client Library COUCHBASE Client Library CLUSTER MAP CLUSTER MAP • Docs distributed evenly across servers READ/WRITE/UPDATE • Each server stores both ac@ve and replica SERVER 1 SERVER 2 SERVER 3 docs Only one doc ac)ve at a )me ACTIVE ACTIVE ACTIVE • Client library provides app with simple Doc 5 Doc Doc 4 Doc Doc 1 Doc interface to database Doc 2 Doc Doc 7 Doc Doc 2 Doc • Cluster map provides map to which server doc is on Doc 9 Doc Doc 8 Doc Doc 6 Doc App never needs to know • App reads, writes, updates docs REPLICA REPLICA REPLICA • Mul@ple app servers can access same Doc 4 Doc Doc 6 Doc Doc 7 Doc document at same @me Doc 1 Doc Doc 3 Doc Doc 9 Doc Doc 8 Doc Doc 2 Doc Doc 5 Doc COUCHBASE SERVER CLUSTER Wednesday, March 20, 13
- 34. Store & Retrieve OperaIons • get (key) – Retrieve a document • set (key, value) – Store a document, overwrites if exists • add (key, value) – Store a document, error/excep^on if exists • replace (key, value) – Store a document, error/excep^on if doesn’t exist • cas (key, value, cas) – Compare and swap, mutate document only if it hasn’t changed while execu^ng this opera^on Wednesday, March 20, 13
- 35. Atomic Counter OperaIons These opera^ons are always executed in order atomically. • set (key, value) – Use set to ini^alize the counter • cb.set(“my_counter”, 1) • incr (key) – Increase an atomic counter value, default by 1 • cb.incr(“my_counter”) # now it’s 2 • decr (key) – Decrease an atomic counter value, default by 1 • cb.decr(“my_counter”) # now it’s 1 Wednesday, March 20, 13
- 36. Mental Adjustments • In SQL we tend to want to avoid hilng the database as much as possible – Even with caching and indexing tricks, and massive improvements over the years, SQL s^ll gets bogged down by complex joins and huge indexes, so we avoid making database calls • In Couchbase, get’s and set’s are so fast they are trivial, not bo0lenecks, this is hard for many people to accept at first; Mul^ple get statements are commonplace, don’t avoid it! Wednesday, March 20, 13
- 37. JSON Document Structure meta { Meta Informa@on “id”: “u::jasdeep@couchbase.com”, Including Key “rev”: “1-‐0002bce0000000000”, “flags”: 0, “expira@on”: 0, All Keys Unique and “type”: “json” Kept in RAM } document { “uid”: 123456, Document Value “firstname”: “jasdeep”, “lastname”: “Jaitla”, Most Recent In Ram “age”: 22, “favorite_colors”: [“blue”, “black”], And Persisted To Disk “email”: “jasdeep@couchbase.com” } Wednesday, March 20, 13
- 38.
- 39. Add Nodes to Cluster APP SERVER 1 APP SERVER 2 COUCHBASE Client Library COUCHBASE Client Library CLUSTER MAP CLUSTER MAP READ/WRITE/UPDATE READ/WRITE/UPDATE SERVER 1 SERVER 2 SERVER 3 SERVER 4 SERVER 5 ACTIVE ACTIVE ACTIVE ACTIVE ACTIVE • Two servers added Doc 5 Doc Doc 4 Doc Doc 1 Doc One-‐click opera@on • Docs automa@cally Doc 2 Doc Doc 7 Doc Doc 2 Doc rebalanced across cluster Even distribu)on of docs Doc 9 Doc Doc 8 Doc Doc 6 Doc Minimum doc movement • Cluster map updated REPLICA REPLICA REPLICA REPLICA REPLICA • App database Doc 4 Doc Doc 6 Doc Doc 7 Doc calls now distributed over larger number of Doc 1 Doc Doc 3 Doc Doc 9 Doc servers Doc 8 Doc Doc 2 Doc Doc 5 Doc COUCHBASE SERVER CLUSTER User Configured Replica Count = 1 Wednesday, March 20, 13
- 40. Fail Over Node APP SERVER 1 APP SERVER 2 COUCHBASE Client Library COUCHBASE Client Library CLUSTER MAP CLUSTER MAP SERVER 1 SERVER 2 SERVER 3 SERVER 4 SERVER 5 • App servers accessing docs ACTIVE ACTIVE ACTIVE ACTIVE ACTIVE Doc 5 Doc Doc 4 Doc Doc 1 Doc Doc 9 Doc Doc 6 Doc • Requests to Server 3 fail Doc 2 Doc Doc 7 Doc Doc 2 Doc Doc 8 Doc Doc • Cluster detects server failed Promotes replicas of docs to ac)ve Updates cluster map Doc 1 Doc 3 • Requests for docs now go to appropriate server REPLICA REPLICA REPLICA REPLICA REPLICA Doc 4 Doc Doc 6 Doc Doc 7 Doc Doc 5 Doc Doc 8 Doc • Typically rebalance would follow Doc 1 Doc Doc 3 Doc Doc 9 Doc Doc 2 Doc COUCHBASE SERVER CLUSTER User Configured Replica Count = 1 Wednesday, March 20, 13
- 41. Indexing and Querying APP SERVER 1 APP SERVER 2 COUCHBASE Client Library COUCHBASE Client Library CLUSTER MAP CLUSTER MAP Query SERVER 1 SERVER 2 SERVER 3 ACTIVE • Indexing work is distributed amongst ACTIVE ACTIVE nodes Doc 5 Doc Doc 5 Doc Doc 5 Doc • Large data set possible Doc 2 Doc Doc 2 Doc Doc 2 Doc • Parallelize the effort Doc 9 Doc Doc 9 Doc Doc 9 Doc • Each node has index for data stored on it REPLICA REPLICA REPLICA • Queries combine the results from required nodes Doc 4 Doc Doc 4 Doc Doc 4 Doc Doc 1 Doc Doc 1 Doc Doc 1 Doc Doc 8 Doc Doc 8 Doc Doc 8 Doc COUCHBASE SERVER CLUSTER Wednesday, March 20, 13
- 42.
- 43. Cross Data Center ReplicaIon (XDCR) Couchbase Cluster Couchbase Cluster West Coast Data Center East Coast Data Center SERVER 1 SERVER 2 SERVER 3 SERVER 1 SERVER 2 SERVER 3 RAM CACHE RAM CACHE RAM CACHE RAM CACHE RAM CACHE RAM CACHE Doc 1 Doc Doc 1 Doc Doc 1 Doc Doc 1 Doc Doc 1 Doc Doc 1 Doc Doc 2 Doc Doc 2 Doc Doc 2 Doc Doc 2 Doc Doc 2 Doc Doc 2 Doc Doc 6 Doc Doc 6 Doc Doc 6 Doc Doc 6 Doc Doc 6 Doc Doc 6 Doc DISK DISK DISK DISK DISK DISK Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Wednesday, March 20, 13
- 44.
- 45.
- 46. ElasIc Search Adaptor • Elastic Search is good for ad-hoc queries and faceted browsing • Our adapter is aware of changing Couchbase topology • Indexed by Elastic Search after stored to disk in Couchbase Elas@cSearch Wednesday, March 20, 13
- 47. I’m Excited to See What You Build, Q & A Couchbase Docs Contact me on Twider www.couchbase.com/docs/index-‐full.html @tgrall Couchbase Forums Contact me by Email www.couchbase.com/forums tug@couchbase.com IRC Learn More About Design Paderns #couchbase CouchbaseModels.com #libcouchbase Senng up for Ruby on Rails CouchbaseOnRails.com Wednesday, March 20, 13
- 48. Win some coolstuff, send a mail to winning@couchbase.com with NORMANDYJUG in the subject. Wednesday, March 20, 13
- 49.