Introduc)on	
  to	
  NoSQL
with	
  Couchbase
Tugdual	
  “Tug”	
  Grall
Technical	
  Evangelist

tug@couchbase.com
Monday, ...
{“about”	
  :	
  “me”}
• Tugdual	
  “Tug”	
  Grall

-­‐

Couchbase

-­‐
-­‐

eXo

-­‐
-­‐

Technical	
  Evangelist
CTO

Or...
RDBMS	
  are	
  not	
  Enough?

Monday, October 28, 13
Growth	
  is	
  the	
  New	
  Reality
• Instagram	
  gained	
  nearly	
  1	
  million	
  users	
  overnight	
  when	
  the...
50	
  Million	
  Users	
  in	
  50	
  Days
Draw	
  Something	
  by	
  OMGPOP
Daily	
  Ac)ve	
  Users	
  (millions)
16

14
...
How	
  do	
  you	
  take	
  this	
  growth?
Applica;on	
  Scales	
  Out
Just	
  add	
  more	
  commodity	
  web	
  servers...
Scaling	
  out	
  RDBMS
• Run	
  Many	
  SQL	
  Servers
Web/App	
  Server	
  Tier

• Data	
  could	
  be	
  sharded
Memcac...
NoSQL	
  Technology	
  Scales	
  Out
Applica;on	
  Scales	
  Out
Just	
  add	
  more	
  commodity	
  web	
  servers
System...
A	
  New	
  Technology?
Bigtable

November	
  2006

Dynamo

October	
  2007

Cassandra
August	
  2008

Voldemort
February	...
What	
  Is	
  Biggest	
  Data	
  Management	
  Problem	
  Driving	
  
Use	
  of	
  NoSQL	
  in	
  Coming	
  Year?

49%
35%...
NoSQL	
  Catalog
Cache
(memory	
  only)

Key-­‐Value

Memcached

Data	
  Structure

Document

Column

Graph

Redis

Databa...
Opera)onal	
  vs.	
  Analy)c	
  Databases
Real-­‐Pme,	
  
InteracPve	
  Databases

AnalyPc
Databases

NoSQL
Fast	
  access...
Document	
  Database

Monday, October 28, 13
Rela;onal	
  vs	
  Document	
  Data	
  Model
C1

C2

C3

C4

{

JSON
JSON

}
JSON

RelaPonal	
  data	
  model

Document	
 ...
Rela;onal	
  Approach

Monday, October 28, 13
Document	
  Database
o::1001
{
uid: “ji22jd”,
customer: “Ann”,
line_items: [
{ sku: 0321293533, qty: 2, price: 48.0 },
{ s...
Schema	
  Update	
  :	
  RDBMS
Event	
  Info
ID
1

Title

Speaker

Speaker_id

Paris	
  JUG

ID

Name

rbin

rbin

Robin	
...
Schema	
  Update	
  :	
  Document
user:rbin
{ user:rbin
{ user:rbin
type: “user”,
{
type: “user”,
name: “Robin Johnson”,
t...
Couchbase	
  Server

Monday, October 28, 13
Couchbase	
  Server	
  Core	
  Principles
Easy	
  
Scalability

PE

RF O R M A N C E

Grow	
  cluster	
  without	
  applic...
Monday, October 28, 13
Couchbase	
  Architecture
8092

11211

Query	
  API

Memcapable	
  	
  1.0

11210
Memcapable	
  	
  2.0

New	
  Persistenc...
Couchbase	
  Architecture
8092

11211

Query	
  API

Memcapable	
  	
  1.0

11210
Memcapable	
  	
  2.0

New	
  Persistenc...
Open	
  Source	
  Project
Apache	
  2.0

hEps://github.com/couchbase/
hEps://github.com/couchbaselabs/

Monday, October 28...
Cluster-­‐wide	
  Basic	
  Opera)on
App	
  Server	
  1

App	
  Server	
  2

COUCHBASE	
  Client	
  Library

COUCHBASE	
  C...
Add	
  Nodes	
  to	
  Cluster
App	
  Server	
  1

App	
  Server	
  2

COUCHBASE	
  Client	
  Library

COUCHBASE	
  Client	...
Fail	
  Over	
  Node
App	
  Server	
  1

App	
  Server	
  2

COUCHBASE	
  Client	
  Library

COUCHBASE	
  Client	
  Librar...
Indexing	
  and	
  Querying
APP	
  SERVER	
  1

APP	
  SERVER	
  2

COUCHBASE	
  Client	
  Library

COUCHBASE	
  Client	
 ...
Couchbase	
  SDK
Official	
  SDKs
Ruby

Python

Community	
  SDKs
Go

Clojure

www.couchbase.com/develop
Monday, October 28,...
Full	
  Text	
  Search
• Elastic Search is good for ad-hoc queries and faceted browsing
• Our adapter is aware of changing...
The	
  Complete	
  Mobile	
  Solu;on

Available	
  now	
  in	
  Beta	
  :
hfp://mobile.couchbase.com
Monday, October 28, 1...
Couchbase	
  Docs
www.couchbase.com/docs/
Couchbase	
  Forums
www.couchbase.com/communi;es
Meetup
hbp://meetup.com/Couchba...
Introduc)on	
  to	
  NoSQL
with	
  Couchbase
Tugdual	
  “Tug”	
  Grall
Technical	
  Evangelist

tug@couchbase.com
Monday, ...
Upcoming SlideShare
Loading in …5
×

Softshake 2013: Introduction to NoSQL with Couchbase

867 views

Published on

This presentation was delivered during Softshake 2013. Learn why RDBMS are not enought and why NoSQL help developers to scale their applications and provide agility.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
867
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Softshake 2013: Introduction to NoSQL with Couchbase

  1. 1. Introduc)on  to  NoSQL with  Couchbase Tugdual  “Tug”  Grall Technical  Evangelist tug@couchbase.com Monday, October 28, 13 TwiBer:  tgrall
  2. 2. {“about”  :  “me”} • Tugdual  “Tug”  Grall -­‐ Couchbase -­‐ -­‐ eXo -­‐ -­‐ Technical  Evangelist CTO Oracle • Web -­‐ -­‐ -­‐    @tgrall      hEp://blog.grallandco.com      tgrall • NantesJUG  co-­‐founder • Pet  Project  : • hEp://www.resultri.com -­‐ • tug@couchbase.com -­‐ -­‐ Developer/Product   Manager Mainly  Java/SOA • tugdual@gmail.com Developer  in  consulAng  firms Monday, October 28, 13
  3. 3. RDBMS  are  not  Enough? Monday, October 28, 13
  4. 4. Growth  is  the  New  Reality • Instagram  gained  nearly  1  million  users  overnight  when  then   expanded  to  Android Monday, October 28, 13
  5. 5. 50  Million  Users  in  50  Days Draw  Something  by  OMGPOP Daily  Ac)ve  Users  (millions) 16 14 12 10 8 6 4 2 2/6 8 10 Monday, October 28, 13 12 14 16 18 20 22 24 26 28 3/1 3 5 7 9 11 13 15 17 19 21
  6. 6. How  do  you  take  this  growth? Applica;on  Scales  Out Just  add  more  commodity  web  servers System  Cost Applica;on  Performance   Web/App  Server  Tier Users RDBMS  Scales  Up Get  a  bigger,  more  complex  server System  Cost Applica;on  Performance   Won’t  scale   beyond  this   point Rela2onal  Database Users RDBMS  is  good  for  many  thing,  but  hard  to  scale Monday, October 28, 13
  7. 7. Scaling  out  RDBMS • Run  Many  SQL  Servers Web/App  Server  Tier • Data  could  be  sharded Memcached  Tier ­ Done  by  the  applicaAon  code • Caching  for  faster  response  )me MySQL  Tier Monday, October 28, 13
  8. 8. NoSQL  Technology  Scales  Out Applica;on  Scales  Out Just  add  more  commodity  web  servers System  Cost Applica;on  Performance   Web/App  Server  Tier Users NoSQL  Database  Scales  Out Cost  and  performance  mirrors  app  ;er System  Cost Applica;on  Performance   NoSQL  Distributed  Data  Store Users Scaling  out  fla?ens  the  cost  and  performance  curves Monday, October 28, 13
  9. 9. A  New  Technology? Bigtable November  2006 Dynamo October  2007 Cassandra August  2008 Voldemort February  2009 Building  new  database  to  answer  the  following  requirements • No  schema  required  before  inser)ng  data • No  schema  change  required  to  change  data  format • Auto-­‐sharding  without  applica)on  par)cipa)on • Distributed  queries • Integrated  main  memory  caching • Data  synchroniza)on  (  mul)-­‐datacenter) Very  few  organiza)ons  want  to  (fewer  can)  build  and  maintain  database  so]ware  technology. But  every  organiza)on  building  interac)ve  web  applica)ons  needs  this  technology. Monday, October 28, 13
  10. 10. What  Is  Biggest  Data  Management  Problem  Driving   Use  of  NoSQL  in  Coming  Year? 49% 35% 29% 16% Lack  of  flexibility/ rigid  schemas Inability  to  scale   Performance  challenges out  data Source:  Couchbase  Survey,  December  2011,  n  =  1351. Monday, October 28, 13 Cost 12% 11% All  of  these Other
  11. 11. NoSQL  Catalog Cache (memory  only) Key-­‐Value Memcached Data  Structure Document Column Graph Redis Database (memory/disk) Coherence Membase Cassandra Neo4j MongoDB Monday, October 28, 13 Couchbase HBase InfiniteGraph
  12. 12. Opera)onal  vs.  Analy)c  Databases Real-­‐Pme,   InteracPve  Databases AnalyPc Databases NoSQL Fast  access   to  data Couchbase MongoDB Monday, October 28, 13 Get  insights  from   data Cloudera Cassandra Hortonworks Hbase Mapr
  13. 13. Document  Database Monday, October 28, 13
  14. 14. Rela;onal  vs  Document  Data  Model C1 C2 C3 C4 { JSON JSON } JSON RelaPonal  data  model Document  data  model Highly-­‐structured  table  organiza;on  with   rigidly-­‐defined  data  formats  and  record   structure. Collec;on  of  complex  documents  with arbitrary,  nested  data  formats  and varying  “record”  format. Monday, October 28, 13
  15. 15. Rela;onal  Approach Monday, October 28, 13
  16. 16. Document  Database o::1001 { uid: “ji22jd”, customer: “Ann”, line_items: [ { sku: 0321293533, qty: 2, price: 48.0 }, { sku: 0321601912, qty: 1, price: 39.0 }, { sku: 0131495054, qty: 1, price: 51.0 } ], payment: { type: “Amex”, expiry: “04/2001”, last5: 12345 } } Aggregate  Oriented  Database hEp://marAnfowler.com/bliki/AggregateOrientedDatabase.html   Monday, October 28, 13
  17. 17. Schema  Update  :  RDBMS Event  Info ID 1 Title Speaker Speaker_id Paris  JUG ID Name rbin rbin Robin  Johnson 1.  Create  a  new  table 2 tgrall QCON  UK 2.“Prepare”  the  Data    (move  to  new  tgrall drop  column) table,   Tug  Grall A  talk  could  be  done  by  one  or  more  speakers! 3.  Change  the  constraints   3 4 NoSQL  CLN johnz johnz John  Zablo fweigel fweigel Frank  Weigel 4.  Change  the  app Devoxx Monday, October 28, 13
  18. 18. Schema  Update  :  Document user:rbin { user:rbin { user:rbin type: “user”, { type: “user”, name: “Robin Johnson”, type: “user”, name: “Robin Johnson”, } } name: “Robin Johnson”, } event:1 { event:1 1. Change  The  ApplicaPon { event:1 type: “event”, { event:1 type: “event”, the   • Update   { event:1 event:1 title: “Paris JUG”, Data  when  “you”  want type: “Paris JUG”, { title: “event”, the  index  to  query  the  new  format “event”, {type: “Paris • speaker: “rbin” JUG”, title: Change   type:“rbin” JUG”, title: “event”, } speaker: “Paris JUG”, type: “Paris title: “event”, } speaker: “rbin” JUG”, title: “rbin” } speaker: “Paris speaker: “rbin” } speaker: [“rbin”,”tgrall”] } } Monday, October 28, 13
  19. 19. Couchbase  Server Monday, October 28, 13
  20. 20. Couchbase  Server  Core  Principles Easy   Scalability PE RF O R M A N C E Grow  cluster  without  applicaAon   changes,  without  downAme  with   a  single  click Always  On   24x365 No  downAme  for  sodware   upgrades,  hardware  maintenance,   etc. Monday, October 28, 13 Consistent  High   Performance Consistent  sub-­‐millisecond   read  and  write  response  Ames   with  consistent  high  throughput JSON JSON JSO JSON N JSON Flexible  Data   Model JSON  document  model  with  no   fixed  schema.
  21. 21. Monday, October 28, 13
  22. 22. Couchbase  Architecture 8092 11211 Query  API Memcapable    1.0 11210 Memcapable    2.0 New  Persistence  Layer vBucket  state  and  replica;on  manager Node  health  monitor Rebalance  orchestrator Global  singleton  supervisor storage  interface Configura;on  manager Data  Manager Process  monitor Couchbase  EP  Engine Heartbeat Memcached REST  management  API/Web  UI Query  Engine Moxi Cluster  Manager hRp on  each  node one  per  cluster Erlang/OTP HTTP 8091 Monday, October 28, 13 Erlang  port  mapper 4369 Distributed  Erlang 21100  -­‐  21199
  23. 23. Couchbase  Architecture 8092 11211 Query  API Memcapable    1.0 11210 Memcapable    2.0 New  Persistence  Layer Disk  Persistence vBucket  state  and  replica;on  manager Node  health  monitor Rebalance  orchestrator Global  singleton  supervisor Configura;on  manager storage  interface Server/Cluster   Management  &   Communica2on (Erlang) hRp Process  monitor RAM  Cache,  Indexing   Couchbase  EP  Engine &  Persistence   Management (C  &  V8) Heartbeat Object-­‐level  Cache REST  management  API/Web  UI Query  Engine Moxi on  each  node one  per  cluster Erlang/OTP The Unreasonable Effectiveness of C by Damien Katz HTTP 8091 Monday, October 28, 13 Erlang  port  mapper 4369 Distributed  Erlang 21100  -­‐  21199
  24. 24. Open  Source  Project Apache  2.0 hEps://github.com/couchbase/ hEps://github.com/couchbaselabs/ Monday, October 28, 13 Gerrit: hEp://review.couchbase.org/
  25. 25. Cluster-­‐wide  Basic  Opera)on App  Server  1 App  Server  2 COUCHBASE  Client  Library COUCHBASE  Client  Library Cluster  Map Cluster  Map READ/WRITE/UPDATE Server  1 Server  2 Server  3 Ac2ve Ac2ve Ac2ve Doc  5 Doc Doc  4 Doc Doc  1 Doc Doc  2 Doc Doc  7 Doc Doc  2 Doc Doc  9 Doc Doc  8 Doc Doc  6 Doc REPLICA REPLICA REPLICA • Docs  distributed  evenly  across   servers   • Each  server  stores  both  ac)ve  and   replica  docs Only  one  server  acAve  at  a  Ame • Client  library  provides  app  with   simple  interface  to  database • Cluster  map  provides  map   to  which  server  doc  is  on Doc  4 Doc Doc  6 Doc Doc  7 Doc Doc  1 Doc Doc  3 Doc Doc  9 Doc • App  reads,  writes,  updates  docs Doc  8 Doc Doc  2 Doc Doc  5 Doc • Mul)ple  app  servers  can  access  same   document  at  same  )me Couchbase  Server    Cluster User  Configured  Replica  Count  =  1 Monday, October 28, 13 App  never  needs  to  know
  26. 26. Add  Nodes  to  Cluster App  Server  1 App  Server  2 COUCHBASE  Client  Library COUCHBASE  Client  Library Cluster  Map Cluster  Map READ/WRITE/UPDATE READ/WRITE/UPDATE Server  1 Server  2 Server  3 Server  4 Server  5 Ac2ve Ac2ve Ac2ve Ac2ve Ac2ve Doc  5 Doc Doc  4 Doc Doc  1 Doc Doc  2 Doc Doc  7 Doc Doc  2 Doc Doc  9 Doc Doc  8 Doc Doc  6 Doc REPLICA REPLICA REPLICA Doc  4 Doc Doc  6 Doc Doc  7 Doc Doc  1 Doc Doc  3 Doc Doc  9 Doc Doc  8 Doc Doc  2 Doc Doc  5 Doc Couchbase  Server    Cluster User  Configured  Replica  Count  =  1 Monday, October 28, 13 • Two  servers  added One-­‐click  opera)on • Docs  automa)cally   rebalanced  across   cluster Even  distribuAon  of  docs Minimum  doc  movement REPLICA REPLICA • Cluster  map  updated • App  database   calls  now  distributed   over  larger  number  of   servers
  27. 27. Fail  Over  Node App  Server  1 App  Server  2 COUCHBASE  Client  Library COUCHBASE  Client  Library Cluster  Map Cluster  Map Server  1 Server  2 Server  3 Server  4 Server  5 Ac2ve Ac2ve Ac2ve Ac2ve Ac2ve Doc  5 Doc Doc  4 Doc Doc  1 Doc Doc  9 Doc Doc  2 Doc Doc  7 Doc Doc  2 Doc Doc  8 Doc Doc  1 Doc  6 Doc Doc Doc  3 REPLICA REPLICA REPLICA REPLICA Doc  4 Doc Doc  6 Doc Doc  7 Doc Doc  5 Doc  1 Doc Doc  3 Doc Doc  9 Doc Doc  2 Couchbase  Server    Cluster User  Configured  Replica  Count  =  1 Monday, October 28, 13 Doc REPLICA Doc  8 Doc Doc • App  servers  accessing  docs • Requests  to  Server  3  fail • Cluster  detects  server  failed Promotes  replicas  of  docs  to   acAve Updates  cluster  map • Requests  for  docs  now  go  to   appropriate  server • Typically  rebalance   would  follow
  28. 28. Indexing  and  Querying APP  SERVER  1 APP  SERVER  2 COUCHBASE  Client  Library COUCHBASE  Client  Library CLUSTER  MAP CLUSTER  MAP Query SERVER  1 SERVER  2 ACTIVE ACTIVE SERVER  3 ACTIVE • Indexing  work  is  distributed  amongst   nodes Doc  5 Doc Doc  5 Doc Doc  5 Doc Doc  2 Doc Doc  2 Doc Doc  2 Doc • Parallelize  the  effort Doc  9 Doc Doc  9 Doc Doc  9 Doc • Each  node  has  index  for  data  stored  on  it REPLICA REPLICA REPLICA Doc  4 Doc Doc  4 Doc Doc  4 Doc Doc  1 Doc Doc  1 Doc Doc  1 Doc Doc  8 Doc Doc  8 Doc Doc  8 Doc COUCHBASE  SERVER    CLUSTER Monday, October 28, 13 • Large  data  set  possible • Queries  combine  the  results  from  required   nodes
  29. 29. Couchbase  SDK Official  SDKs Ruby Python Community  SDKs Go Clojure www.couchbase.com/develop Monday, October 28, 13
  30. 30. Full  Text  Search • Elastic Search is good for ad-hoc queries and faceted browsing • Our adapter is aware of changing Couchbase topology • Indexed by Elastic Search after stored to disk in Couchbase ElasPcSearch Monday, October 28, 13
  31. 31. The  Complete  Mobile  Solu;on Available  now  in  Beta  : hfp://mobile.couchbase.com Monday, October 28, 13
  32. 32. Couchbase  Docs www.couchbase.com/docs/ Couchbase  Forums www.couchbase.com/communi;es Meetup hbp://meetup.com/Couchbase-­‐France/ Mobile  (beta) hbp://mobile.couchbase.com/ N1QL  -­‐  New  Query  Engine  (DP) hbp://query.couchbase.com Monday, October 28, 13 Contact  me  on  Twi?er @tgrall Contact  me  by  Email tug@couchbase.com IRC #couchbase #libcouchbase
  33. 33. Introduc)on  to  NoSQL with  Couchbase Tugdual  “Tug”  Grall Technical  Evangelist tug@couchbase.com Monday, October 28, 13 TwiBer:  tgrall

×