Building_a_node_application_with_Couchbase_Node_and_Angular_Barcelona_JS

747 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
747
On SlideShare
0
From Embeds
0
Number of Embeds
97
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Well, the game launched on February 6, 2012. Like most new games, social media integration (Facebook in this case) makes it easy to both invite your friends to play, and to highlight that you are playing the game. This “social component” helps build popularity. A few weeks into its life, Draw Something began to get a lot of attention, including from celebrities who also used social media (facebook, twitter and pinterest) to talk about their experience. One of the stars of Jersey Shore tweeted about the game in early March, kicking off the initial round of growth – to 1 million daily active uers. Miley Cyrus tweeted about her Draw Something “addiction” on March 8 and growth accelerated – from over 4 million daily users. Two weeks later, at over 14 million daily active users, the company behind Draw Something was acquired by Zynga for a purported $200 plus million.
  • The network/cluster management is written in Erlang with several components: - OTP : Open Telecom Platform - all the administartion console is created in Erlang : - REST API - Web UI (many JS with JQuery and Sparkline) - The data server is written in C Includes memcache as main cache for the DB The storage itself is also written in C, based on a Append only
  • The SDKs are the component where we have most of the contribution from the community. I have to say that it is most of the time easier since you can use your “favorite language”
  • Load balancing in Couchbase is achieved as follows: Keys are uniformly distributed based on the hash function When machines are added and removed in the cluster.  The administrator can request a redistribution of vBucket so that data are evenly spread across physical machines.
  • Building_a_node_application_with_Couchbase_Node_and_Angular_Barcelona_JS

    1. 1. BarcelonaJS / April 4th, 2013Couchbase & JavascriptMapReduce, Node.js, AngularMapReduce, Node.js, AngularTugdual “Tug” GrallTechnical EvangelistBarcelonaJS / April 4th, 2013
    2. 2. BarcelonaJS / April 4th, 2013• Tugdual “Tug” Grall­ Couchbase• Technical Evangelist­ eXo• CTO­ Oracle• Developer/Product Manager• Mainly Java/SOA­ Developer in consulting firms• Web• @tgrall• http://blog.grallandco.com• tgrall• NantesJUG co-founder• Pet Project :• http://www.resultri.com
    3. 3. BarcelonaJS / April 4th, 2013191715131197533/12826242220181614121082/6Draw Something by OMGPOPDaily Active Users (millions)21246810121416
    4. 4. BarcelonaJS / April 4th, 2013RDBMS is good for many thing, but hard to scaleRDBMS is good for many thing, but hard to scaleRDBMS is good for many thing, but hard to scaleRDBMS is good for many thing, but hard to scaleRDBMS Scales UpRDBMS Scales UpGet a bigger, more complex serverGet a bigger, more complex serverUsersApplication Scales OutApplication Scales OutJust add more commodity web serversJust add more commodity web serversUsersSystem CostApplication PerformanceRelational DatabaseWeb/App Server TierSystem CostApplication PerformanceWon’t scalebeyond thispointHow do you take the growth?
    5. 5. BarcelonaJS / April 4th, 2013Scaling out flattens the costScaling out flattens the cost andand performance curvesperformance curvesScaling out flattens the costScaling out flattens the cost andand performance curvesperformance curvesNoSQL Database Scales OutNoSQL Database Scales OutCost and performance mirrors app tierCost and performance mirrors app tierUsersNoSQL Distributed Data StoreWeb/App Server TierApplication Scales OutApplication Scales OutJust add more commodity web serversJust add more commodity web serversUsersSystem CostApplication PerformanceApplication PerformanceSystem CostNoSQL Technology Scales Out
    6. 6. BarcelonaJS / April 4th, 2013DynamoOctober 2007CassandraAugust 2008BigtableNovember 2006VoldemortFebruary 2009Very few organizations want to (fewer can) build and maintain database software technology.Very few organizations want to (fewer can) build and maintain database software technology.But every organization building interactive web applications needs this technology.But every organization building interactive web applications needs this technology.Very few organizations want to (fewer can) build and maintain database software technology.Very few organizations want to (fewer can) build and maintain database software technology.But every organization building interactive web applications needs this technology.But every organization building interactive web applications needs this technology.A new technology?• Building new database to answer the followingrequirements­ No schema required before inserting data­ No schema change required to change data format­ Auto-sharding without application participation­ Distributed queries­ Integrated main memory caching­ Data synchronization ( multi-datacenter)
    7. 7. BarcelonaJS / April 4th, 2013Lack of flexibility/rigid schemasInability to scaleout dataPerformancechallengesCost All of these Other49%35%29%16%12%11%Source: Couchbase Survey, December 2011, n = 1351.What is driving NoSQL adoption?
    8. 8. Couchbase Open Source Project• Leading NoSQL database projectfocused on distributed databasetechnology and surroundingecosystem• Supports both key-value anddocument-oriented use cases• All components are availableunder the Apache 2.0 PublicLicense• Obtained as packaged software inboth enterprise and communityeditions.CouchbaseOpen Source Project
    9. 9. BarcelonaJS / April 4th, 2013EasyScalabilityConsistent HighPerformanceAlways On24x365Grow cluster without applicationchanges, without downtime with asingle clickConsistent sub-millisecondread and write response timeswith consistent high throughputNo downtime for softwareupgrades, hardware maintenance,etc.Flexible DataModelJSON document model with nofixed schema.Couchbase Server Core Principles
    10. 10. BarcelonaJS / April 4th, 2013HeartbeatProcessmonitorGlobalsingletonsupervisorConfigurationmanageron each nodeRebalanceorchestratorNodehealthmonitorone per clustervBucketstateandreplicationmanagerhttpRESTmanagementAPI/WebUIHTTP8091Erlang port mapper4369Distributed Erlang21100 - 21199Erlang/OTPstorage interfaceCouchbase EP Engine11210Memcapable 2.0Moxi11211Memcapable 1.0MemcachedNew Persistence Layer8092Query APIQueryEngineData Manager Cluster ManagerCouchbase Server 2.0 Architecture
    11. 11. BarcelonaJS / April 4th, 2013New Persistence Layerstorage interfaceCouchbase EP Engine11210Memcapable 2.0Moxi11211Memcapable 1.0Object-level CacheDisk Persistence8092Query APIQueryEngineHTTP8091Erlang port mapper4369Distributed Erlang21100 - 21199HeartbeatProcessmonitorGlobalsingletonsupervisorConfigurationmanageron each nodeRebalanceorchestratorNodehealthmonitorone per clustervBucketstateandreplicationmanagerhttpRESTmanagementAPI/WebUIErlang/OTPServer/ClusterServer/ClusterManagement &Management &CommunicationCommunication(Erlang)(Erlang)RAM Cache,RAM Cache,Indexing &Indexing &PersistencePersistenceManagementManagement(C & V8)(C & V8)The Unreasonable Effectiveness of C by Damien KatzCouchbase Server 2.0 Architecture
    12. 12. BarcelonaJS / April 4th, 2013https://github.com/couchbase/https://github.com/couchbaselabs/http://review.couchbase.org/Gerrit:Open Source ProjectApache 2.0
    13. 13. BarcelonaJS / April 4th, 2013WHAT ABOUT MY APP?
    14. 14. BarcelonaJS / April 4th, 2013www.couchbase.com/developClojurePythonRubylibcouchbaseGoCouchbase SDK
    15. 15. BarcelonaJS / April 4th, 201333 2Managed CacheDiskQueueDiskQueueDiskReplicationReplicationQueueQueueApp ServerCouchbase Server NodeDoc 1Doc 1Doc 1To other nodeWrite Operation
    16. 16. BarcelonaJS / April 4th, 2013COUCHBASE SERVER CLUSTER• Docs distributed evenly across servers• Each server stores both active andreplica docsOnly one doc active at a time• Client library provides app with simpleinterface to database• Cluster map provides mapto which server doc is onApp never needs to know• App reads, writes, updates docs• Multiple app servers can access samedocument at same timeREAD/WRITE/UPDATEACTIVEACTIVEDoc 5Doc 2DocDocDocSERVER 1ACTIVEACTIVEDoc 4Doc 7DocDocDocSERVER 2Doc 8ACTIVEACTIVEDoc 1Doc 2DocDocDocREPLICAREPLICADoc 4Doc 1Doc 8DocDocDocREPLICAREPLICADoc 6Doc 3Doc 2DocDocDocREPLICAREPLICADoc 7Doc 9Doc 5DocDocDocSERVER 3Doc 6APP SERVER 1COUCHBASE Client LibraryCOUCHBASE Client LibraryCOUCHBASE Client LibraryCOUCHBASE Client LibraryCLUSTER MAPCLUSTER MAPCOUCHBASE Client LibraryCOUCHBASE Client LibraryCOUCHBASE Client LibraryCOUCHBASE Client LibraryCLUSTER MAPCLUSTER MAPAPP SERVER 2Doc 9Basic Operations
    17. 17. BarcelonaJS / April 4th, 2013• get (key)­ Retrieve a document• set (key, value)­ Store a document, overwrites if exists• add (key, value)­ Store a document, error/exception if exists• replace (key, value)­ Store a document, error/exception if doesn’t exist• cas (key, value, cas)­ Compare and swap, mutate document only if it hasn’t changedwhile executing this operationStore & Retrieve Operations
    18. 18. BarcelonaJS / April 4th, 2013These operations are always executed in order atomically.•set (key, value)­ Use set to initialize the counter• cb.set(“my_counter”, 1)•incr (key)­ Increase an atomic counter value, default by 1• cb.incr(“my_counter”) # now it’s 2•decr (key)­ Decrease an atomic counter value, default by 1• cb.decr(“my_counter”) # now it’s 1Atomic Counter Operations
    19. 19. BarcelonaJS / April 4th, 2013• In SQL we tend to want to avoid hitting the database as much aspossible­ Even with caching and indexing tricks, and massiveimprovements over the years, SQL still gets bogged down bycomplex joins and huge indexes, so we avoid making databasecalls• In Couchbase, get’s and set’s are so fast they are trivial, notbottlenecks, this is hard for many people to accept at first;Multiple get statements are commonplace, don’t avoid it!Mental Adjustments
    20. 20. BarcelonaJS / April 4th, 2013Operations with Node
    21. 21. BarcelonaJS / April 4th, 2013NPM{"name": "my-node-application","version": "1.0.0","private": true,"dependencies":{"express": "3.x","couchbase": "0.0.11","ejs": ">= 0.0.1"}}
    22. 22. BarcelonaJS / April 4th, 2013Connect to the clustervar driver = require(couchbase);dbConfiguration = {"hosts": ["localhost:8091"],"bucket": "ideas"};driver.connect(dbConfiguration, function(err, cb) {if (err) {throw (err)}// your application code here}
    23. 23. BarcelonaJS / April 4th, 2013Insert Datavar meetup = {"type" : "meetup", "language" : "javascript"};cb.set("barcelonajs",meetup, function(err, meta) {});var tmp = {"message" : "hello world!"};cb.set("tmp", tmp, {"expiry" : 5},function(err, meta) {});
    24. 24. BarcelonaJS / April 4th, 2013Insert / Delete Datavar meetup = {"type" : "meetup", "language" : "javascript"};cb.set("barcelonajs",meetup, function(err, meta) {});var tmp = {"message" : "hello world!"};cb.set("tmp", tmp, {"expiry" : 5},function(err, meta) {});cb.set("todelete", tmp, function(err, meta) {});cb.remove("todelete", function(err, meta) {});
    25. 25. BarcelonaJS / April 4th, 2013Retrieve the Datacb.get("product:45", function(errs, doc, metas) {console.log("=== get the document ===");console.log( doc ); }); var keys = new Array();keys.push("product:1"); keys.push("product:45");keys.push("product:65"); keys.push("product:80");cb.get(keys, null, function(errs, docs, metas) {console.log("n=== get List of documents ===");console.log( docs ); });
    26. 26. BarcelonaJS / April 4th, 2013Retrieve the DataWhat if I want all products or meetups?key : barcelonajs{ "type": "meetup","language": "javascript"}key : product:10{ "type": "product", "name":"Product with id 10"}
    27. 27. BarcelonaJS / April 4th, 2013Calling a view from your appvar queryParams = {stale: false,key : "meetup"};cb.view("my_views", "by_type", queryParams, function(err, view) {var keys = new Array();for (var i = 0; i < view.length; i++) {keys.push(view[i].id);}cb.get(keys, null, function(errs, docs, metas) {console.log(docs);});});
    28. 28. BarcelonaJS / April 4th, 2013Indexing and Querying• Define materialized views on JSON documents and thenquery across the data set• Using views you can define­ Primary indexes­ Simple secondary indexes (most common use case)­ Complex secondary, tertiary and composite indexes­ Aggregations (reduction)• Indexes are eventually indexed• Queries are eventually consistent with respect todocuments• Built using Map/Reduce technology­ Map and Reduce functions are written in Javascript
    29. 29. BarcelonaJS / April 4th, 2013Map Function
    30. 30. BarcelonaJS / April 4th, 2013Q&A

    ×