Webinar - Einfuehrung in Couchbase
 

Webinar - Einfuehrung in Couchbase

on

  • 495 views

Couchbase Server 2.0 ist eine dokumentenorientierte NoSQL Datenbank. Features wie native JSON Unterstützung, umfangreiche Abfragemechanismen und die Möglichkeit zwischen mehreren Datacentern zu ...

Couchbase Server 2.0 ist eine dokumentenorientierte NoSQL Datenbank. Features wie native JSON Unterstützung, umfangreiche Abfragemechanismen und die Möglichkeit zwischen mehreren Datacentern zu replizieren sind seit der neuen Version enthalten. Zusätzlich wurden alt bekannte Funktionen wie Performanz und einfache Skalierbarkeit weiter verbessert.

Statistics

Views

Total Views
495
Views on SlideShare
452
Embed Views
43

Actions

Likes
1
Downloads
6
Comments
0

1 Embed 43

http://www.couchbase.com 43

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • JSON support – natively stored as json, whne you build an app, there is not conversion required. New doc viewing , editing capability. Indexing and querying – look inside your json, build views and query for a key, for ranges or to aggregate data Incremental mapreduce – powers indexing. Build complex views over your data. Great for real-time analytics XDCR – replicate information from one cluster to another cluster
  • 1.  A set request comes in from the application .2.  Couchbase Server responses back that they key is written3. Couchbase Server then Replicates the data out to memory in the other nodes4. At the same time it is put the data into a write que to be persisted to disk
  • 1.  A set request comes in from the application .2.  Couchbase Server responses back that they key is written3. Couchbase Server then Replicates the data out to memory in the other nodes4. At the same time it is put the data into a write que to be persisted to disk
  • Bulletize the text. Make sure the builds work.
  • Bulletize the text. Make sure build work properly.
  • Bulletize the text. Make sure build work properly.
  • Per data bucket, we have multiple Design Docs which contain the view definitions for a number of views.  This means our views are all batched together to be incrementally updated.  Best practise is splitting our views up into relevant ownerships / writers.  So i.e. 1 Design Document holds all the views for the Frontend UI of the website, and another Design Document holds the views for the Backend Admin interface (used to list and edit users, or posts etc etc.)In a worst case Design Doc scenario, there would be a 1 view in a dozen design documents, meaning we have 12 view functions to run, whereas we should structure it as multiple views per design document.  But, getting the balance right is important, as we also wouldn't want to have a design document with 100 views in it!When we change 1 view definition, it will update the index for the ENTIRE design doc, this is why it's logical to split views into relevant Design Doc categories etc.
  • Built on the Javascript V8 engine. Our query language is simple Javascript, so very easy to write our map functions.
  • How we can write our map functions: We could use a Single Element key, or “Primary key.”
  • Group_level queriesWe can use the built in dateToArray Javascript helper function to find a rollup of i.e. documents edited by date (by year, month, day, hour etc.)Group_level=2 we'd segment by year,month   and    Group_level=3  we'd segment by year,month,day etc.

Webinar - Einfuehrung in Couchbase Webinar - Einfuehrung in Couchbase Presentation Transcript

  • Einführung inCouchbase Server 2.0
  • • Developer Advocate at Couchbase, Inc.• Maintainer of the Couchbase Java SDK• Speaking at Conferences and Meetups• Living and Working in Vienna, Austria{“about”: “me”}
  • 1. Introduction to Couchbase Server 2.02. Cluster Management3. Operations & Document Design4. Introduction to ViewsAgenda
  • Couchbase Server 2.0
  • Big Release in December 2012
  • Couchbase Open Source Project• One of the leading NoSQLdatabase projects focused ondistributed database technologyand surrounding ecosystem• Supports both key-value anddocument-oriented use cases• All components are availableunder the Apache 2.0 PublicLicense• Obtained as packaged software inboth enterprise and communityeditions.
  • EasyScalabilityConsistent HighPerformanceAlwaysOn24x365Grow cluster withoutapplication changes, withoutdowntime with a single clickConsistent sub-millisecondread and write response timeswith consistent high throughputNo downtime for softwareupgrades, hardwaremaintenance, etc.JSONJSONJSONJSONJSONFlexible DataModelJSON document model withno fixed schema.Core Principles
  • New in 2.0JSON support Indexing and QueryingCross data center replicationIncremental Map ReduceJSONJSONJSONJSONJSON
  • Couchbase Server 2.0 ArchitectureHeartbeatProcessmonitorGlobalsingletonsupervisorConfigurationmanageron each nodeRebalanceorchestratorNodehealthmonitorone per clustervBucketstateandreplicationmanagerhttpRESTmanagementAPI/WebUIHTTP8091Erlang port mapper4369Distributed Erlang21100 - 21199Erlang/OTPstorage interfaceCouchbase EP Engine11210Memcapable 2.0Moxi11211Memcapable 1.0MemcachedNew Persistence Layer8092Query APIQueryEngineData Manager Cluster Manager
  • 33 2Single node - Couchbase WriteOperationManaged CacheDiskQueueDiskReplicationQueueApp ServerCouchbase Server NodeDoc 1Doc 1Doc 1To other node
  • GETDoc133 2Single node - Couchbase ReadOperationDiskQueueReplicationQueueApp ServerDoc 1Doc 1Doc 1Managed CacheDiskTo other nodeCouchbase Server Node
  • COUCHBASE SERVER CLUSTERBasic Operation• Docs distributed evenly acrossservers• Each server stores both active andreplica docsOnly one server active at a time• Client library provides app withsimple interface to database• Cluster map provides mapto which server doc is onApp never needs to know• App reads, writes, updates docs• Multiple app servers can access samedocument at same timeUser Configured Replica Count = 1READ/WRITE/UPDATEACTIVEDoc 5Doc 2DocDocDocSERVER 1ACTIVEDoc 4Doc 7DocDocDocSERVER 2Doc 8ACTIVEDoc 1Doc 2DocDocDocREPLICADoc 4Doc 1Doc 8DocDocDocREPLICADoc 6Doc 3Doc 2DocDocDocREPLICADoc 7Doc 9Doc 5DocDocDocSERVER 3Doc 6APP SERVER 1COUCHBASE Client LibraryCLUSTER MAPCOUCHBASE Client LibraryCLUSTER MAPAPP SERVER 2Doc 9
  • Add Nodes to Cluster• Two servers addedOne-click operation• Docs automaticallyrebalanced acrossclusterEven distribution of docsMinimum doc movement• Cluster map updated• App databasecalls now distributedover larger number ofserversREPLICAACTIVEDoc 5Doc 2DocDocDoc 4Doc 1DocDocSERVER 1REPLICAACTIVEDoc 4Doc 7DocDocDoc 6Doc 3DocDocSERVER 2REPLICAACTIVEDoc 1Doc 2DocDocDoc 7Doc 9DocDocSERVER 3 SERVER 4 SERVER 5REPLICAACTIVEREPLICAACTIVEDocDoc 8 DocDoc 9 DocDoc 2 DocDoc 8 DocDoc 5 DocDoc 6READ/WRITE/UPDATE READ/WRITE/UPDATEAPP SERVER 1COUCHBASE Client LibraryCLUSTER MAPCOUCHBASE Client LibraryCLUSTER MAPAPP SERVER 2COUCHBASE SERVER CLUSTERUser Configured Replica Count = 1
  • Fail Over NodeREPLICAACTIVEDoc 5Doc 2DocDocDoc 4Doc 1DocDocSERVER 1REPLICAACTIVEDoc 4Doc 7DocDocDoc 6Doc 3DocDocSERVER 2REPLICAACTIVEDoc 1Doc 2DocDocDoc 7Doc 9DocDocSERVER 3 SERVER 4 SERVER 5REPLICAACTIVEREPLICAACTIVEDoc 9Doc 8Doc Doc 6 DocDocDoc 5 DocDoc 2Doc 8 DocDoc• App servers accessing docs• Requests to Server 3 fail• Cluster detects server failedPromotes replicas of docs toactiveUpdates cluster map• Requests for docs now go toappropriate server• Typically rebalancewould followDocDoc 1 Doc 3APP SERVER 1COUCHBASE Client LibraryCLUSTER MAPCOUCHBASE Client LibraryCLUSTER MAPAPP SERVER 2User Configured Replica Count = 1COUCHBASE SERVER CLUSTER
  • Q & A
  • Operations & Document Design
  • Fundamentals• Every Document has a Key assigned to it• Keys- must be max. 255 UTF-8 chars long- must be unique in a bucket (“database”)- are completely under the control of the application• Values- can be any binary blob (but bonus points when JSON!)- can be up to 20MB in size
  • Basic Store & Retrieve Operations• get(key)Retrieve a document• set(key, value)Store a document or replace if it exists• add(key, value)Store a document and fail if it exists• replace(key, value)Replace a document and fail if it does not exist
  • Lots of other Operations• View operations• Atomic Counters (increment, decrement)• Append/Prepend• CAS (Compare and Set)- Optimistic Locking• “Get with Lock”- Write Lock on Objects• Bulk Operations- Saves network overhead• Stats
  • Couchbase 2.0 Bonus Points: JSON• JSON is a lightweight format to represent document structurein a language-independent manner.• If JSON documents are stored, the View engine can be used.• Allows to build secondary indexes on your datasets.• Makes it possible to ask questions like“Give me all user documents by lastname”.
  • The BIG mental adjustment• In SQL, we tend to avoid hitting the database as much aspossible.- Use JOINs and let the DB optimizer figure out what to do• In Couchbase, get’s and set‘s are so fast they are trivial, notbottlenecks, this is hard for many people to accept at first;Multiple get statements are commonplace, don’t avoid it!
  • meta{“id”: “u::michael@couchbase.com”,“rev”: “1-0002bce0000000000”,“flags”: 0,“expiration”: 0,“type”: “json”}document{“uid”: 123456,“firstname”: “Michael”,“lastname”: “Nitschinger”,“age”: 25,“favorite_colors”: [“blue”, “black”],“email”: “michael.nitschinger@...”}Meta InformationIncluding KeyAll Keys Unique andKept in RAMDocument ValueMost Recent In RamAnd Persisted To DiskDocument Structure with JSON
  • Views & Indexes• Views can cover a few different use cases- Simple secondary indexes (the most common)- Complex secondary, tertiary and composite indexes- Aggregation functions (reduction)• Example: count the number of North American Ales- Organizing related data• Built using Map/Reduce- Map function creates the index (b-tree)- Reduce function summarizes (reduces) information- Written using superfast Javascript (V8)
  • A View in Action
  • Developing with Couchbase
  • Aggregate View of Datahttp://martinfowler.com/bliki/AggregateOrientedDatabase.html
  • What is an aggregate?When you need to retrieve data from RDBMS, youare "aggregating" or "denormalizing" the data foryour application through queries with joins, whereclauses and order by clauses.In Document Databases, instead of breaking datainto tables and foreign keys, you store theaggregate data together in JSON document(s).
  • Store and Retrieve Aggregates• Easier to Distribute Data• More Flexibility• Reduced Latencyorder::1001{uid: ji22jd,customer: Ann,line_items: [{ sku: 0321293533, quan: 3, unit_price: 48.0 },{ sku: 0321601912, quan: 1, unit_price: 39.0 },{ sku: 0131495054, quan: 1, unit_price: 51.0 }],payment: { type: Amex, expiry: 04/2001,last5: 12345 }}
  • Relational vs Document Model31Relational data modelHighly-structured table organizationwith rigidly-defined data formats andrecord structure.C1 C2 C3 C4Document data modelCollection of complex documents witharbitrary, nested data formats andvarying “record” format.JSONJSONJSON{}
  • SQL Normalized Tables32Addresses1 DEN 30303CO2 MV 94040CA3 CHI 60609ILUsersKEY First ZIP_IDLast4 NY 10010NY1 Jasdeep 2Jaitla2 Joe 2Smith3 Ali 2Dodson4 John 3DoeZIP_ID CITY ZIPSTATETo get information about specific user, you perform a join across two tablesforeign keySELECT * FROM Users u INNER JOIN Addresses a ON u.zip_id = a.zip_id WHERE key=1
  • Documents are Aggregates33+Addresses1 DEN 30303CO23 CHI 60609IL4 NY 10010NYZIP_ID CITY ZIPSTATEUsersKEY First ZIP_IDLast22 Joe 2Smith3 Ali 2Dodson4 John 3DoeAll data in a single document{“ID”: 1,“First”:“Jasdeep”,“Last”: “Jaitla”,“ZIP”: “94103”,“CITY”: “SF”,“STATE”: “CA”}JSON=couchbase.get(“user::1”)Document Data is anAggregate1 Jasdeep Jaitla94103CASF1 Jasdeep Jaitla
  • Document KeysKeys have to be unique in a Bucket (database)which acts as a namespace.Keys tend to have three categories:human-readable, randomly generated, and hybridHuman readable keys use unique descriptors:email addresses, social media id’s, phone numbers, blogpost titles, usernames, sku’s, etc.
  • www.couchbase.com/developPythonRubyGo ClojureOfficial SDKsCommunity & Libraries
  • Couchbase Views
  • Indexing Architecture33 2Managed CacheDiskQueueDiskReplicationQueueApp ServerCouchbase Server NodeDoc 1Doc 1Doc 1To other nodeView EngineDoc 1Doc Updated in RAMCache FirstIndexer Updates IndexesAfter On Disk, in BatchesAll Documents & Updates PassThrough View Engine
  • Buckets >> Design Documents >> ViewsCouchbase BucketDesign Document 1 Design Document 2View ViewViewViewViewIndexers Are Allocated PerDesign DocAll Updated at Same TimeAll Updated at Same TimeAll Updated at Same TimeCan Only Access Data inthe Bucket NamespaceCan Only Access Data inthe Bucket Namespace
  • Map() Function => Indexfunction(doc, meta) {emit(doc.username, doc.email)} indexed key output value(s)create rowjson doc doc metadataEvery Document passes through View Map() functionsMap
  • Single Element Keys (Text Key)function(doc, meta) {emit(doc.email, null)}text keyMapdoc.email meta.idabba@couchbase.com u::1jasdeep@couchbase.com u::2zorro@couchbase.com u::3
  • Compound Keys (Array)function(doc, meta) {emit(dateToArray(doc.timestamp), 1)}array keyArray Based Index Keys get sorted as Strings,but can be grouped by array elementsMapdateToArray(doc.timestamp)value[2012,10,9,18,45] 1[2012,9,26,11,15] 1[2012,8,13,2,12] 1
  • Reduce Values (doc.abv) with _statsadd _stats built-in reduction
  • Result Set - Brewery ID’s by Beerbrewery_iddocument key (of the beer)alcohol by volume (abv)
  • Q & A
  • Thank you!michael.nitschinger@couchbase.com@daschlGet Couchbase Server athttp://www.couchbase.com/download