Understanding, Choosing & Instrumenting NOSQL

Understanding Choosing& INSTRUMENTING NOSQL A PRESENTATION BY @timangladE FROM YOUR FRIENDS AT @cloudant

@timanglade

Tin/taɪn/A database engine so tiny,its name had to be shortened

NOSQLTAPES.com@NOSQLTAPES

You havea scaling problem

So you decide togive NOSQL a try

Now you HAveTWO Problems

That’s the realityof NOSQL TODAYSo why bother? Oh, Don’t worry, I’m going to tell you…

Why & HOWdo I NOSQL?

Understanding Choosing& INSTRUMENTING NOSQL

! UNDERSTANDING Your problem& NOSQL

NOSQL is about Two things1. Performance 2. Distribution N.B. To varying degrees in each NOSQL project…

Performance?

PerformancePerformance/$

LOOK AT YOURRDBMS FIRST…

Tip #1Cost: close to $0…A-B-C

A Always B Be C CACHING

Always BeCACHING

Always

CACHING

If you don’tI’ve NO SYMPATHYFOR YOU, PALAlso, always be rewatching GlenGarry Glen Ross.Great Movie. Fantastic Cast.

Tip #2Still For close to $0…INDEXFor God’s SAKE, INDEX

Tip #3Did you go to school? Then yes, Still close to zeroIf You know sqlYou can nosql

Tip #4For a Fistful of dollars…Hirea DatabaseConsultant

Tip #5For a few dollars more…Buy a bigger box

5.1 TB 1.2 MIOPS 100 K Dollars

Tip #6Say, That’s a really nice shirt you’re wearing thereI’m sure we could come to an arrangement…A word about“the Cloud”

You have a scalingproblem,So you decide togive the Cloud a try

Now you HAveX ProblemsWhere X is arbitrarily large

The greatest sandboxever madeThe BEST Bootcampever designed

BTWTHESE TIPS APPLYTO NOSQL TOO…

Did all that?don’t have the $$$?

Welcome TO NOSQL

NOSQL lets youUSE skillsINSTEAD of $$$But do you have the skills? Can you get them?

Distribution

What if you Can’tBUY a BOX ANY BIGGER?

NETWORK PARTITIONSHAPPEN

BROWNoutsHAPPEN

Distribution is a(mostly) efficient wayTO ADD moreCAPACITY & AvailabilityTo youR DB

A Word aboutMASTER-SLAVEREPLICATION

A WORD ABOUTTHE CAP THEOREM

! CHOOSING A NOSQL DATABASE

How do I chose?

Same 2 parametersDistribution+ “Performance”

What doesPerformanceMean?

Performance! Data / Query Model! Disk Structure

the Moon Methodology™1. Distribution 2. Data / Query Model 3. Disk Structure

DistributionDynamo-StyleMaster-slaveMASTER-MASTER

Data / Query ModelMap/ReduceEverywhere?

Disk StructuresThe Devil is in the(implementation)Details

Now let’s lookat major NOSQL DBsthrough this lens…

CouchdbMaster-masterDoc + Persistent M/RAppend-only B+ Tree

Ideal SCENARIOSMALL SCALEQUERIES don’t CHANGEHTTP IS A MUST

CouchBase 2.0Master-slaveK/V + Persistent M/RAppend-only B+ Tree

IDEAL SCENARIO?

BigCouchDynamoDoc + Persistent M/RAppend-only B+ Tree

IDEAL SCENARIOSame as COUCHDB,BIGGER SCALE

CassandraDynamoColumn FamiliesLog + SSTable

ideal scenariofast writes +Don’t mind hacking

RiakMulti-DC DynamoK/V + M/R + 2aryLog-Struct. Hash Table(& others)

ideal scenarioKnowledgeable team,really large scale

MongoDBMaster-slaveDocS + M/R + 2aryLog + B-Tree

ideal scenarioPrototyping

RedisMaster-slave (?)ManyLOG + Many (?)

ideal scenarioAPPLICATION “GLUE”

A plea for mercyplease don’t sniffyour REDIS

NEO4JMaster-slave (~)OO + RESTCustom Graph Struct.

ideal scenarioLOTS OF SELF-JOINS

! INSTRUMENTING YOUR STACK

You WILL FAILOh yes, Yes you will…

Any advancedDistributed SystemWill behaveLIKE a BLACK BOX

TESTING IS FINE butMEASURING IS MOREUSEFULIn this context

Some stuff to keep an eye onPercentileRESPONSE TIMES

Some stuff to keep an eye onERROR RATES

Some stuff to keep an eye onMemory usage& stack depth

Some stuff to keep an eye onCPU usage& Number of Processes

Some stuff to keep an eye onDISK USAGE& IOPS

Hawk the graphsOVER LONG PERIODS

Instrumentation& METROLOGYare still DARK ARTS

Next Steps

Coda Hale’smetrics EVERYWHEREpivotallabs.com/talks/139-metrics-metrics-everywhere

Find a monitoringsystem that worksFOR YOU

THe NOSQL Handbooknosqlhandbook.com

Recap!1.! ! NOSQL IS HARD2.! ! KNOW YOUR RDBMS, KNOW YOUR PROBLEM3.! ! PICK A DB By DISTRIBUTION, Query & Disk Models4.! ! Instrument the heck out of it5.! ! Rinse6.! ! REPEAT

GoEXPERIMENTDeployMEASUREIMPROVEHAVE FUN

? cloudant.com tim@cloudant.com@timanglade

http://weblogs.vpro.nl	1486
http://paper.li	65
http://redactie-live.test.vpro.nl	47
http://lanyrd.com	22
http://twitter.com	5
https://twitter.com	4
http://trunk.ly	3
http://127.0.0.1:8795	1
http://redactie-test.vpro.omroep.nl	1

Understanding, Choosing & Instrumenting NOSQL

by Tim Anglade

Accessibility

Categories

Upload Details

Usage Rights

9 Embeds 1,634

Statistics

Understanding, Choosing & Instrumenting NOSQL Presentation Transcript