Einführung inCouchbase Server 2.0
• Developer Advocate at Couchbase, Inc.• Maintainer of the Couchbase Java SDK• Speaking at Conferences and Meetups• Living...
1. Introduction to Couchbase Server 2.02. Cluster Management3. Operations & Document Design4. Introduction to ViewsAgenda
Couchbase Server 2.0
Big Release in December 2012
Couchbase Open Source Project• One of the leading NoSQLdatabase projects focused ondistributed database technologyand surr...
EasyScalabilityConsistent HighPerformanceAlwaysOn24x365Grow cluster withoutapplication changes, withoutdowntime with a sin...
New in 2.0JSON support Indexing and QueryingCross data center replicationIncremental Map ReduceJSONJSONJSONJSONJSON
Couchbase Server 2.0 ArchitectureHeartbeatProcessmonitorGlobalsingletonsupervisorConfigurationmanageron each nodeRebalance...
33 2Single node - Couchbase WriteOperationManaged CacheDiskQueueDiskReplicationQueueApp ServerCouchbase Server NodeDoc 1Do...
GETDoc133 2Single node - Couchbase ReadOperationDiskQueueReplicationQueueApp ServerDoc 1Doc 1Doc 1Managed CacheDiskTo othe...
COUCHBASE SERVER CLUSTERBasic Operation• Docs distributed evenly acrossservers• Each server stores both active andreplica ...
Add Nodes to Cluster• Two servers addedOne-click operation• Docs automaticallyrebalanced acrossclusterEven distribution of...
Fail Over NodeREPLICAACTIVEDoc 5Doc 2DocDocDoc 4Doc 1DocDocSERVER 1REPLICAACTIVEDoc 4Doc 7DocDocDoc 6Doc 3DocDocSERVER 2RE...
Q & A
Operations & Document Design
Fundamentals• Every Document has a Key assigned to it• Keys- must be max. 255 UTF-8 chars long- must be unique in a bucket...
Basic Store & Retrieve Operations• get(key)Retrieve a document• set(key, value)Store a document or replace if it exists• a...
Lots of other Operations• View operations• Atomic Counters (increment, decrement)• Append/Prepend• CAS (Compare and Set)- ...
Couchbase 2.0 Bonus Points: JSON• JSON is a lightweight format to represent document structurein a language-independent ma...
The BIG mental adjustment• In SQL, we tend to avoid hitting the database as much aspossible.- Use JOINs and let the DB opt...
meta{“id”: “u::michael@couchbase.com”,“rev”: “1-0002bce0000000000”,“flags”: 0,“expiration”: 0,“type”: “json”}document{“uid...
Views & Indexes• Views can cover a few different use cases- Simple secondary indexes (the most common)- Complex secondary,...
A View in Action
Developing with Couchbase
Aggregate View of Datahttp://martinfowler.com/bliki/AggregateOrientedDatabase.html
What is an aggregate?When you need to retrieve data from RDBMS, youare "aggregating" or "denormalizing" the data foryour a...
Store and Retrieve Aggregates• Easier to Distribute Data• More Flexibility• Reduced Latencyorder::1001{uid: ji22jd,custome...
Relational vs Document Model31Relational data modelHighly-structured table organizationwith rigidly-defined data formats a...
SQL Normalized Tables32Addresses1 DEN 30303CO2 MV 94040CA3 CHI 60609ILUsersKEY First ZIP_IDLast4 NY 10010NY1 Jasdeep 2Jait...
Documents are Aggregates33+Addresses1 DEN 30303CO23 CHI 60609IL4 NY 10010NYZIP_ID CITY ZIPSTATEUsersKEY First ZIP_IDLast22...
Document KeysKeys have to be unique in a Bucket (database)which acts as a namespace.Keys tend to have three categories:hum...
www.couchbase.com/developPythonRubyGo ClojureOfficial SDKsCommunity & Libraries
Couchbase Views
Indexing Architecture33 2Managed CacheDiskQueueDiskReplicationQueueApp ServerCouchbase Server NodeDoc 1Doc 1Doc 1To other ...
Buckets >> Design Documents >> ViewsCouchbase BucketDesign Document 1 Design Document 2View ViewViewViewViewIndexers Are A...
Map() Function => Indexfunction(doc, meta) {emit(doc.username, doc.email)} indexed key output value(s)create rowjson doc d...
Single Element Keys (Text Key)function(doc, meta) {emit(doc.email, null)}text keyMapdoc.email meta.idabba@couchbase.com u:...
Compound Keys (Array)function(doc, meta) {emit(dateToArray(doc.timestamp), 1)}array keyArray Based Index Keys get sorted a...
Reduce Values (doc.abv) with _statsadd _stats built-in reduction
Result Set - Brewery ID’s by Beerbrewery_iddocument key (of the beer)alcohol by volume (abv)
Q & A
Thank you!michael.nitschinger@couchbase.com@daschlGet Couchbase Server athttp://www.couchbase.com/download
Webinar - Einfuehrung in Couchbase
Webinar - Einfuehrung in Couchbase
Upcoming SlideShare
Loading in …5
×

Webinar - Einfuehrung in Couchbase

609 views

Published on

Couchbase Server 2.0 ist eine dokumentenorientierte NoSQL Datenbank. Features wie native JSON Unterstützung, umfangreiche Abfragemechanismen und die Möglichkeit zwischen mehreren Datacentern zu replizieren sind seit der neuen Version enthalten. Zusätzlich wurden alt bekannte Funktionen wie Performanz und einfache Skalierbarkeit weiter verbessert.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
609
On SlideShare
0
From Embeds
0
Number of Embeds
47
Actions
Shares
0
Downloads
8
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • JSON support – natively stored as json, whne you build an app, there is not conversion required. New doc viewing , editing capability. Indexing and querying – look inside your json, build views and query for a key, for ranges or to aggregate data Incremental mapreduce – powers indexing. Build complex views over your data. Great for real-time analytics XDCR – replicate information from one cluster to another cluster
  • 1.  A set request comes in from the application .2.  Couchbase Server responses back that they key is written3. Couchbase Server then Replicates the data out to memory in the other nodes4. At the same time it is put the data into a write que to be persisted to disk
  • 1.  A set request comes in from the application .2.  Couchbase Server responses back that they key is written3. Couchbase Server then Replicates the data out to memory in the other nodes4. At the same time it is put the data into a write que to be persisted to disk
  • Bulletize the text. Make sure the builds work.
  • Bulletize the text. Make sure build work properly.
  • Bulletize the text. Make sure build work properly.
  • Per data bucket, we have multiple Design Docs which contain the view definitions for a number of views.  This means our views are all batched together to be incrementally updated.  Best practise is splitting our views up into relevant ownerships / writers.  So i.e. 1 Design Document holds all the views for the Frontend UI of the website, and another Design Document holds the views for the Backend Admin interface (used to list and edit users, or posts etc etc.)In a worst case Design Doc scenario, there would be a 1 view in a dozen design documents, meaning we have 12 view functions to run, whereas we should structure it as multiple views per design document.  But, getting the balance right is important, as we also wouldn't want to have a design document with 100 views in it!When we change 1 view definition, it will update the index for the ENTIRE design doc, this is why it's logical to split views into relevant Design Doc categories etc.
  • Built on the Javascript V8 engine. Our query language is simple Javascript, so very easy to write our map functions.
  • How we can write our map functions: We could use a Single Element key, or “Primary key.”
  • Group_level queriesWe can use the built in dateToArray Javascript helper function to find a rollup of i.e. documents edited by date (by year, month, day, hour etc.)Group_level=2 we'd segment by year,month   and    Group_level=3  we'd segment by year,month,day etc.
  • Webinar - Einfuehrung in Couchbase

    1. 1. Einführung inCouchbase Server 2.0
    2. 2. • Developer Advocate at Couchbase, Inc.• Maintainer of the Couchbase Java SDK• Speaking at Conferences and Meetups• Living and Working in Vienna, Austria{“about”: “me”}
    3. 3. 1. Introduction to Couchbase Server 2.02. Cluster Management3. Operations & Document Design4. Introduction to ViewsAgenda
    4. 4. Couchbase Server 2.0
    5. 5. Big Release in December 2012
    6. 6. Couchbase Open Source Project• One of the leading NoSQLdatabase projects focused ondistributed database technologyand surrounding ecosystem• Supports both key-value anddocument-oriented use cases• All components are availableunder the Apache 2.0 PublicLicense• Obtained as packaged software inboth enterprise and communityeditions.
    7. 7. EasyScalabilityConsistent HighPerformanceAlwaysOn24x365Grow cluster withoutapplication changes, withoutdowntime with a single clickConsistent sub-millisecondread and write response timeswith consistent high throughputNo downtime for softwareupgrades, hardwaremaintenance, etc.JSONJSONJSONJSONJSONFlexible DataModelJSON document model withno fixed schema.Core Principles
    8. 8. New in 2.0JSON support Indexing and QueryingCross data center replicationIncremental Map ReduceJSONJSONJSONJSONJSON
    9. 9. Couchbase Server 2.0 ArchitectureHeartbeatProcessmonitorGlobalsingletonsupervisorConfigurationmanageron each nodeRebalanceorchestratorNodehealthmonitorone per clustervBucketstateandreplicationmanagerhttpRESTmanagementAPI/WebUIHTTP8091Erlang port mapper4369Distributed Erlang21100 - 21199Erlang/OTPstorage interfaceCouchbase EP Engine11210Memcapable 2.0Moxi11211Memcapable 1.0MemcachedNew Persistence Layer8092Query APIQueryEngineData Manager Cluster Manager
    10. 10. 33 2Single node - Couchbase WriteOperationManaged CacheDiskQueueDiskReplicationQueueApp ServerCouchbase Server NodeDoc 1Doc 1Doc 1To other node
    11. 11. GETDoc133 2Single node - Couchbase ReadOperationDiskQueueReplicationQueueApp ServerDoc 1Doc 1Doc 1Managed CacheDiskTo other nodeCouchbase Server Node
    12. 12. COUCHBASE SERVER CLUSTERBasic Operation• Docs distributed evenly acrossservers• Each server stores both active andreplica docsOnly one server active at a time• Client library provides app withsimple interface to database• Cluster map provides mapto which server doc is onApp never needs to know• App reads, writes, updates docs• Multiple app servers can access samedocument at same timeUser Configured Replica Count = 1READ/WRITE/UPDATEACTIVEDoc 5Doc 2DocDocDocSERVER 1ACTIVEDoc 4Doc 7DocDocDocSERVER 2Doc 8ACTIVEDoc 1Doc 2DocDocDocREPLICADoc 4Doc 1Doc 8DocDocDocREPLICADoc 6Doc 3Doc 2DocDocDocREPLICADoc 7Doc 9Doc 5DocDocDocSERVER 3Doc 6APP SERVER 1COUCHBASE Client LibraryCLUSTER MAPCOUCHBASE Client LibraryCLUSTER MAPAPP SERVER 2Doc 9
    13. 13. Add Nodes to Cluster• Two servers addedOne-click operation• Docs automaticallyrebalanced acrossclusterEven distribution of docsMinimum doc movement• Cluster map updated• App databasecalls now distributedover larger number ofserversREPLICAACTIVEDoc 5Doc 2DocDocDoc 4Doc 1DocDocSERVER 1REPLICAACTIVEDoc 4Doc 7DocDocDoc 6Doc 3DocDocSERVER 2REPLICAACTIVEDoc 1Doc 2DocDocDoc 7Doc 9DocDocSERVER 3 SERVER 4 SERVER 5REPLICAACTIVEREPLICAACTIVEDocDoc 8 DocDoc 9 DocDoc 2 DocDoc 8 DocDoc 5 DocDoc 6READ/WRITE/UPDATE READ/WRITE/UPDATEAPP SERVER 1COUCHBASE Client LibraryCLUSTER MAPCOUCHBASE Client LibraryCLUSTER MAPAPP SERVER 2COUCHBASE SERVER CLUSTERUser Configured Replica Count = 1
    14. 14. Fail Over NodeREPLICAACTIVEDoc 5Doc 2DocDocDoc 4Doc 1DocDocSERVER 1REPLICAACTIVEDoc 4Doc 7DocDocDoc 6Doc 3DocDocSERVER 2REPLICAACTIVEDoc 1Doc 2DocDocDoc 7Doc 9DocDocSERVER 3 SERVER 4 SERVER 5REPLICAACTIVEREPLICAACTIVEDoc 9Doc 8Doc Doc 6 DocDocDoc 5 DocDoc 2Doc 8 DocDoc• App servers accessing docs• Requests to Server 3 fail• Cluster detects server failedPromotes replicas of docs toactiveUpdates cluster map• Requests for docs now go toappropriate server• Typically rebalancewould followDocDoc 1 Doc 3APP SERVER 1COUCHBASE Client LibraryCLUSTER MAPCOUCHBASE Client LibraryCLUSTER MAPAPP SERVER 2User Configured Replica Count = 1COUCHBASE SERVER CLUSTER
    15. 15. Q & A
    16. 16. Operations & Document Design
    17. 17. Fundamentals• Every Document has a Key assigned to it• Keys- must be max. 255 UTF-8 chars long- must be unique in a bucket (“database”)- are completely under the control of the application• Values- can be any binary blob (but bonus points when JSON!)- can be up to 20MB in size
    18. 18. Basic Store & Retrieve Operations• get(key)Retrieve a document• set(key, value)Store a document or replace if it exists• add(key, value)Store a document and fail if it exists• replace(key, value)Replace a document and fail if it does not exist
    19. 19. Lots of other Operations• View operations• Atomic Counters (increment, decrement)• Append/Prepend• CAS (Compare and Set)- Optimistic Locking• “Get with Lock”- Write Lock on Objects• Bulk Operations- Saves network overhead• Stats
    20. 20. Couchbase 2.0 Bonus Points: JSON• JSON is a lightweight format to represent document structurein a language-independent manner.• If JSON documents are stored, the View engine can be used.• Allows to build secondary indexes on your datasets.• Makes it possible to ask questions like“Give me all user documents by lastname”.
    21. 21. The BIG mental adjustment• In SQL, we tend to avoid hitting the database as much aspossible.- Use JOINs and let the DB optimizer figure out what to do• In Couchbase, get’s and set‘s are so fast they are trivial, notbottlenecks, this is hard for many people to accept at first;Multiple get statements are commonplace, don’t avoid it!
    22. 22. meta{“id”: “u::michael@couchbase.com”,“rev”: “1-0002bce0000000000”,“flags”: 0,“expiration”: 0,“type”: “json”}document{“uid”: 123456,“firstname”: “Michael”,“lastname”: “Nitschinger”,“age”: 25,“favorite_colors”: [“blue”, “black”],“email”: “michael.nitschinger@...”}Meta InformationIncluding KeyAll Keys Unique andKept in RAMDocument ValueMost Recent In RamAnd Persisted To DiskDocument Structure with JSON
    23. 23. Views & Indexes• Views can cover a few different use cases- Simple secondary indexes (the most common)- Complex secondary, tertiary and composite indexes- Aggregation functions (reduction)• Example: count the number of North American Ales- Organizing related data• Built using Map/Reduce- Map function creates the index (b-tree)- Reduce function summarizes (reduces) information- Written using superfast Javascript (V8)
    24. 24. A View in Action
    25. 25. Developing with Couchbase
    26. 26. Aggregate View of Datahttp://martinfowler.com/bliki/AggregateOrientedDatabase.html
    27. 27. What is an aggregate?When you need to retrieve data from RDBMS, youare "aggregating" or "denormalizing" the data foryour application through queries with joins, whereclauses and order by clauses.In Document Databases, instead of breaking datainto tables and foreign keys, you store theaggregate data together in JSON document(s).
    28. 28. Store and Retrieve Aggregates• Easier to Distribute Data• More Flexibility• Reduced Latencyorder::1001{uid: ji22jd,customer: Ann,line_items: [{ sku: 0321293533, quan: 3, unit_price: 48.0 },{ sku: 0321601912, quan: 1, unit_price: 39.0 },{ sku: 0131495054, quan: 1, unit_price: 51.0 }],payment: { type: Amex, expiry: 04/2001,last5: 12345 }}
    29. 29. Relational vs Document Model31Relational data modelHighly-structured table organizationwith rigidly-defined data formats andrecord structure.C1 C2 C3 C4Document data modelCollection of complex documents witharbitrary, nested data formats andvarying “record” format.JSONJSONJSON{}
    30. 30. SQL Normalized Tables32Addresses1 DEN 30303CO2 MV 94040CA3 CHI 60609ILUsersKEY First ZIP_IDLast4 NY 10010NY1 Jasdeep 2Jaitla2 Joe 2Smith3 Ali 2Dodson4 John 3DoeZIP_ID CITY ZIPSTATETo get information about specific user, you perform a join across two tablesforeign keySELECT * FROM Users u INNER JOIN Addresses a ON u.zip_id = a.zip_id WHERE key=1
    31. 31. Documents are Aggregates33+Addresses1 DEN 30303CO23 CHI 60609IL4 NY 10010NYZIP_ID CITY ZIPSTATEUsersKEY First ZIP_IDLast22 Joe 2Smith3 Ali 2Dodson4 John 3DoeAll data in a single document{“ID”: 1,“First”:“Jasdeep”,“Last”: “Jaitla”,“ZIP”: “94103”,“CITY”: “SF”,“STATE”: “CA”}JSON=couchbase.get(“user::1”)Document Data is anAggregate1 Jasdeep Jaitla94103CASF1 Jasdeep Jaitla
    32. 32. Document KeysKeys have to be unique in a Bucket (database)which acts as a namespace.Keys tend to have three categories:human-readable, randomly generated, and hybridHuman readable keys use unique descriptors:email addresses, social media id’s, phone numbers, blogpost titles, usernames, sku’s, etc.
    33. 33. www.couchbase.com/developPythonRubyGo ClojureOfficial SDKsCommunity & Libraries
    34. 34. Couchbase Views
    35. 35. Indexing Architecture33 2Managed CacheDiskQueueDiskReplicationQueueApp ServerCouchbase Server NodeDoc 1Doc 1Doc 1To other nodeView EngineDoc 1Doc Updated in RAMCache FirstIndexer Updates IndexesAfter On Disk, in BatchesAll Documents & Updates PassThrough View Engine
    36. 36. Buckets >> Design Documents >> ViewsCouchbase BucketDesign Document 1 Design Document 2View ViewViewViewViewIndexers Are Allocated PerDesign DocAll Updated at Same TimeAll Updated at Same TimeAll Updated at Same TimeCan Only Access Data inthe Bucket NamespaceCan Only Access Data inthe Bucket Namespace
    37. 37. Map() Function => Indexfunction(doc, meta) {emit(doc.username, doc.email)} indexed key output value(s)create rowjson doc doc metadataEvery Document passes through View Map() functionsMap
    38. 38. Single Element Keys (Text Key)function(doc, meta) {emit(doc.email, null)}text keyMapdoc.email meta.idabba@couchbase.com u::1jasdeep@couchbase.com u::2zorro@couchbase.com u::3
    39. 39. Compound Keys (Array)function(doc, meta) {emit(dateToArray(doc.timestamp), 1)}array keyArray Based Index Keys get sorted as Strings,but can be grouped by array elementsMapdateToArray(doc.timestamp)value[2012,10,9,18,45] 1[2012,9,26,11,15] 1[2012,8,13,2,12] 1
    40. 40. Reduce Values (doc.abv) with _statsadd _stats built-in reduction
    41. 41. Result Set - Brewery ID’s by Beerbrewery_iddocument key (of the beer)alcohol by volume (abv)
    42. 42. Q & A
    43. 43. Thank you!michael.nitschinger@couchbase.com@daschlGet Couchbase Server athttp://www.couchbase.com/download

    ×