Couchbase Server 2.0 in Production Perry Krug, Sr. Solutions Architect, Couchbase Inc.                                    ...
Couchbase Server 2.0: Overview                          =                                 Simple. Fast. Elastic.   •   Mem...
Let’s build a social game…                     www.facebook.com/farm_town_wars                                            ...
COUCHBASE SERVER 2.0 IN PRODUCTION:             DEV/OPS LIFECYCLE                                                     Clie...
Couchbase Server 2.0 in Production: Initial Setup  Extremely easy to get up and running:  • RPM/deb/OSX/exe installation  ...
Couchbase Server 2.0 in Production: Data and Indexes• Data goes “in and out” via the Memcached protocol• Queries/indexes a...
Couchbase Server 2.0 in Production: Data and IndexesExample JSON values:                     Example view:User_id:{       ...
Couchbase Server 2.0 in Production: Data and Indexes• Indexes/views are based on incremental map-reduce:   • Indexes are u...
Couchbase Server 2.0 in Production: Data and Indexes• Views are “developed” off of a random (or specific)  sampling of the...
COUCHBASE SERVER 2.0 IN PRODUCTION:        DEV/OPS LIFECYCLE                         Client setup                         ...
Couchbase Server 2.0 in production: Sizing Sizing Question: How many nodes do I need? Considerations: RAM Disk Network Dat...
Couchbase Server 2.0 in production: Sizing Sizing Question: How many nodes do I need? Considerations: RAM • Metadata      ...
Couchbase Server 2.0 in production: Sizing             500,000 keys to begin with:               -20 bytes in length      ...
Couchbase Server 2.0 in production: Sizing Sizing Question: How many nodes do I need? Considerations: Disk: • Sustained wr...
Couchbase Server 2.0 in production: Sizing Sizing Question: How many nodes do I need? Considerations: Network: • Client tr...
Couchbase Server 2.0 in production: Sizing Sizing Question: How many nodes do I need? Considerations: Data Safety1 node = ...
Couchbase Server 2.0 in production: Sizing Sizing Question: How many nodes do I need? Considerations: RAM Disk Network Dat...
Couchbase Server 2.0 in production: Client-side Deployment       Application server                            Farm Town W...
COUCHBASE SERVER 2.0 IN PRODUCTION:        DEV/OPS LIFECYCLE            Monitor    Grow                 Maintain        Up...
Couchbase Server 2.0 in production: MonitoringIMMENSE amount of information available-Real-time traffic graphs-REST API ac...
21
Couchbase Server 2.0 in production: Growth• Going from 1 hundred users to 2 million users…   – RAM usage is growing:      ...
Couchbase Server 2.0 in production: General Maintenance• Persistence is CouchDB:   – Append-only B-tree   – Reliability an...
Couchbase Server 2.0 in production: General Maintenance• Backup/Restore:   – Backup is as simple as a file-level copy (tha...
Couchbase Server 2.0 in production: Failures  • Failures happen:     – Hardware     – Network     –   Bugs  • Failover to ...
COUCHBASE SERVER 2.0 IN PRODUCTION:             DEV/OPS LIFECYCLE                                                     Clie...
THANK YOU!        Q&APerry Krug - perry@couchbase.com                                   27
Upcoming SlideShare
Loading in...5
×

CouchConf-NYC-couchbase-in-production

693

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
693
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
14
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Based upon real-world experience from customers like Zynga, Quepasa, TribalCrossing and many others.
  • Slow down on demo, describe each section of setup
  • CouchConf-NYC-couchbase-in-production

    1. 1. Couchbase Server 2.0 in Production Perry Krug, Sr. Solutions Architect, Couchbase Inc. 1
    2. 2. Couchbase Server 2.0: Overview = Simple. Fast. Elastic. • Membase + CouchDB • Managed memory caching layer = super high performance • Clustering and online data redistribution (Rebalancing) • Indexing and Querying via JSON Map-Reduce • New SDK’s and client libraries • Developer Preview available today! • http://www.couchbase.com/downloads 2
    3. 3. Let’s build a social game… www.facebook.com/farm_town_wars Load Balancer Web Servers Couchbase Servers 3
    4. 4. COUCHBASE SERVER 2.0 IN PRODUCTION: DEV/OPS LIFECYCLE Client setup Initial Setup Test DeployView Development Sizing Monitor Grow Maintain Upgrade Backup/Restore Failures 4
    5. 5. Couchbase Server 2.0 in Production: Initial Setup Extremely easy to get up and running: • RPM/deb/OSX/exe installation • Simple Web UI and setup wizard 5
    6. 6. Couchbase Server 2.0 in Production: Data and Indexes• Data goes “in and out” via the Memcached protocol• Queries/indexes are created and accessed via an HTTP protocol• Both are available separately, Couchbase-provided SDK’s will expose a single API to the developer and abstract the actual traffic 6
    7. 7. Couchbase Server 2.0 in Production: Data and IndexesExample JSON values: Example view:User_id:{ function (doc) { “farms”: *“one_farm”, “two_farm”+ for(var i in doc.animals) “animals”: *, if(doc.animals[i].sheep=="black") “pig”:”pink”, “sheep”:”black”, “chicken”:”yellow” + } emit(doc) }] “enemies”: *“you”, “U2”+} = List of black sheep owners 7
    8. 8. Couchbase Server 2.0 in Production: Data and Indexes• Indexes/views are based on incremental map-reduce: • Indexes are updated with incremental changes (not batch)• View processing is done per-vbucket: • Parallel processing on subset of data • Couchbase provides scatter-gather aggregation 8
    9. 9. Couchbase Server 2.0 in Production: Data and Indexes• Views are “developed” off of a random (or specific) sampling of the overall dataset and then deployed • Faster • Less load on system• Updated views can be applied without rescanning entire dataset 9
    10. 10. COUCHBASE SERVER 2.0 IN PRODUCTION: DEV/OPS LIFECYCLE Client setup Deploy Sizing 10
    11. 11. Couchbase Server 2.0 in production: Sizing Sizing Question: How many nodes do I need? Considerations: RAM Disk Network Data Safety 11
    12. 12. Couchbase Server 2.0 in production: Sizing Sizing Question: How many nodes do I need? Considerations: RAM • Metadata Active+Replica • Working set • Buffer/overhead 12
    13. 13. Couchbase Server 2.0 in production: Sizing 500,000 keys to begin with: -20 bytes in length -average value size of 2k -metadata of about 150 bytes per key = ~1Gb to store active data, an extra 1Gb to store replica dataAdding in some headroom: Give 3GB RAM to Couchbase to start and grow with 13
    14. 14. Couchbase Server 2.0 in production: Sizing Sizing Question: How many nodes do I need? Considerations: Disk: • Sustained write rate • Index generation (space and IO) • Append-only B-Tree 14
    15. 15. Couchbase Server 2.0 in production: Sizing Sizing Question: How many nodes do I need? Considerations: Network: • Client traffic • Replication • Rebalancing 15
    16. 16. Couchbase Server 2.0 in production: Sizing Sizing Question: How many nodes do I need? Considerations: Data Safety1 node = single point of failure (bad)2 nodes = 1 replica copy (better)3+ nodes = 1 replica copy AND data/load distribution 16
    17. 17. Couchbase Server 2.0 in production: Sizing Sizing Question: How many nodes do I need? Considerations: RAM Disk Network Data Safety 17
    18. 18. Couchbase Server 2.0 in production: Client-side Deployment Application server Farm Town Wars Farm Town Wars Application server App Code App Code Couchbase PHP Client Coucbase Java LIbrary client library Moxi (Couchbase proxy) OR 11210 5984 11210 5984 Query API Query API Couchbase Server Couchbase Server Couchbase Couchbase Client-side Moxi (“smart”) library 18
    19. 19. COUCHBASE SERVER 2.0 IN PRODUCTION: DEV/OPS LIFECYCLE Monitor Grow Maintain Upgrade Backup/Restore Failures 19
    20. 20. Couchbase Server 2.0 in production: MonitoringIMMENSE amount of information available-Real-time traffic graphs-REST API accessible-Per bucket, per node and aggregate statistics-Application and inter-node traffic-RAM <-> Disk-Inter-system timing 20
    21. 21. 21
    22. 22. Couchbase Server 2.0 in production: Growth• Going from 1 hundred users to 2 million users… – RAM usage is growing: • Ejecting data to and fetching data from disk • Resident item ratios decreasing – might impact failover – Disk write queue grows higher than usual• Need to add a few more nodes...• Now we have more RAM and more disk throughput without any downtime 22
    23. 23. Couchbase Server 2.0 in production: General Maintenance• Persistence is CouchDB: – Append-only B-tree – Reliability and data integrity – Constantly growing on-disk files• Compaction is the answer: – Automatic – “Fragmentation” data – Scheduled compaction – Per-vbucket database (saves disk space and time) 23
    24. 24. Couchbase Server 2.0 in production: General Maintenance• Backup/Restore: – Backup is as simple as a file-level copy (thanks CouchDB!) – Server will automatically “warmup” from disk files upon reboot• Upgrade: – Add nodes of new version, rebalance… – Remove nodes of old version, rebalance… – Done! – No disruption – Upgrade from existing Membase 1.7 installations to Couchbase 2.0 24
    25. 25. Couchbase Server 2.0 in production: Failures • Failures happen: – Hardware – Network – Bugs • Failover to replica data for immediate access • Remove and rebalance “malfunctioning” node 25 (Demo)
    26. 26. COUCHBASE SERVER 2.0 IN PRODUCTION: DEV/OPS LIFECYCLE Client setup Initial Setup Test DeployView Development Sizing Monitor Grow Maintain Upgrade Backup/Restore Failures 26
    27. 27. THANK YOU! Q&APerry Krug - perry@couchbase.com 27
    1. Gostou de algum slide específico?

      Recortar slides é uma maneira fácil de colecionar informações para acessar mais tarde.

    ×