The document discusses data service level agreements (SLAs) in public cloud environments. It explains that achieving availability, consistency, and scalability is challenging due to Brewer's CAP theorem. It reviews strategies for relational and NoSQL databases to handle these tradeoffs, including dropping consistency or availability depending on needs. Code examples demonstrate typical operations for Cassandra, MongoDB, and Neo4J NoSQL databases. The conclusion recommends choosing solutions based on requirements and migrating to NoSQL as needed to address scaling issues.

1. Data SLA in the cloud

2. About UsScaleBase is a new startup targeting the database-as-a-service market (DBaaS)We offer unlimited database scalability and availability using our Database Load BalancerWe launch in September, 2010. Stay tuned at our site.

3. AgendaThe requirements for data SLA in public cloud environmentsAchieving data SLA with NOSQLAchieving data SLA with relational databases

4. The requirements for data SLA in public cloud environments

5. What We NeedAvailabilityConsistencyScalability

6. Brewer's (CAP) TheoremIt is impossible for a distributed computer system to simultaneously provide all three of the following guarantees:Consistency (all nodes see the same data at the same time)Availability (node failures do not prevent survivors from continuing to operate)Partition Tolerance (the system continues to operate despite arbitrary message loss)http://en.wikipedia.org/wiki/CAP_theorem

7. What It Meanshttp://guyharrison.squarespace.com/blog/2010/6/13/consistency-models-in-non-relational-databases.html

8. Dealing With CAPDrop Partition ToleranceRun everything on one machine.This is, of course, not very scalable.

9. Dealing With CAPDrop AvailabilityIf a partition fail, everything waits until the data is consistent again. This can be very complex to handle over a large number of nodes.

10. Dealing With CAPDrop ConsistencyWelcome to the “Eventually Consistent” term.At the end – everything will work out just fine - And hi, sometimes this is a good enough solutionWhen no updates occur for a long period of time, eventually all updates will propagate through the system and all the nodes will be consistentFor a given accepted update and a given node, eventually either the update reaches the node or the node is removed from serviceKnown as BASE (Basically Available, Soft state, Eventual consistency), as opposed to ACID

11. Reading More On CAPThis is an excellent read, and some of my samples are from this bloghttp://www.julianbrowne.com/article/viewer/brewers-cap-theorem

12. Achieving data SLA with relational databases

13. Databases And CAPACID – ConsistencyAvailability – tons of solutions, most of them not cloud orientedOracle RACMySQL ProxyEtc.Replication based solutions can solve at least read availability and scalability (see Azure SQL)

14. Database Cloud SolutionsAmazon RDSNaviSite Oracle RACNot that popularCosts to cloud providers (complexity, not standard)

15. So Where Is The Problem?Partition Tolerance just doesn’t workScaling problems (usually write but also read)BigData problems

16. Scaling UpIssues with scaling up when the dataset is just too bigRDBMS were not designed to be distributedBegan to look at multi-node database solutionsKnown as ‘scaling out’ or ‘horizontal scaling’Different approaches include:Master-slaveSharding

17. Scaling RDBMS – Master/SlaveMaster-SlaveAll writes are written to the master. All reads performed against the replicated slave databasesCritical reads may be incorrect as writes may not have been propagated downLarge data sets can pose problems as master needs to duplicate data to slaves

18. Scaling RDBMS - ShardingPartition or shardingScales well for both reads and writesNot transparent, application needs to be partition-awareCan no longer have relationships/joins across partitionsLoss of referential integrity across shards

19. Other ways to scale RDBMSMulti-Master replicationINSERT only, not UPDATES/DELETESNo JOINs, thereby reducing query timeThis involves de-normalizing dataIn-memory databases

20. Achieving data SLA with NOSQL

21. NoSQLA term used to designate databases which differ from classic relational databases in some way. These data stores may not require fixed table schemas, and usually avoid join operations and typically scale horizontally. Academics and papers typically refer to these databases as structured storage, a term which would include classic relational databases as a subset.http://en.wikipedia.org/wiki/NoSQL

22. NoSQL TypesKey/ValueA big hash tableExamples: Voldemort, Amazon DynamoBig TableBig table, column familiesExamples: Hbase, CassandraDocument basedCollections of collectionsExamples: CouchDB, MongoDBGraph databasesBased on graph theoryExamples: Neo4JEach solves a different problem

23. NO-SQLhttp://browsertoolkit.com/fault-tolerance.png

24. Pros/ConsPros:PerformanceBigDataMost solutions are open sourceData is replicated to nodes and is therefore fault-tolerant (partitioning)Don't require a schemaCan scale up and downCons:Code changeNo framework supportNot ACIDEco system (BI, Backup)There is always a database at the backendSome API is just too simple

25. Amazon S3 Code SampleAWSAuthConnection conn = new AWSAuthConnection(awsAccessKeyId, awsSecretAccessKey, secure, server, format);Response response = conn.createBucket(bucketName, location, null);final String text = "this is a test";response = conn.put(bucketName, key, new S3Object(text.getBytes(), null), null);

26. Cassandra Code SampleCassandraClient cl = pool.getClient() ;KeySpaceks = cl.getKeySpace("Keyspace1") ;// insert valueColumnPathcp = new ColumnPath("Standard1" , null, "testInsertAndGetAndRemove".getBytes("utf-8")); for(int i = 0 ; i < 100 ; i++){ks.insert("testInsertAndGetAndRemove_"+i, cp , ("testInsertAndGetAndRemove_value_"+i).getBytes("utf-8"));}//get valuefor(inti = 0 ; i < 100 ; i++){ Column col = ks.getColumn("testInsertAndGetAndRemove_"+i, cp); String value = new String(col.getValue(),"utf-8") ;}//remove valuefor(int i = 0 ; i < 100 ; i++){ks.remove("testInsertAndGetAndRemove_"+i, cp);}

27. Cassandra Code Sample – Cont’try{ks.remove("testInsertAndGetAndRemove_not_exist", cp);}catch(Exception e){ fail("remove not exist row should not throw exceptions");}//get already removed valuefor(int i = 0 ; i < 100 ; i++){try{ Column col = ks.getColumn("testInsertAndGetAndRemove_"+i, cp); fail("the value should already being deleted");}catch(NotFoundException e){}catch(Exception e){ fail("throw out other exception, should be NotFoundException." + e.toString() ); }}pool.releaseClient(cl) ;pool.close() ;

28. Cassandra StatisticsFacebook SearchMySQL > 50 GB DataWrites Average : ~300 msReads Average : ~350 msRewritten with Cassandra > 50 GB DataWrites Average : 0.12 msReads Average : 15 ms

29. MongoDBMongo m = new Mongo();DB db = m.getDB( "mydb" );Set<String> colls = db.getCollectionNames();for (String s : colls) {System.out.println(s);}

30. MongoDB – Cont’BasicDBObjectdoc = new BasicDBObject();doc.put("name", "MongoDB");doc.put("type", "database");doc.put("count", 1);BasicDBObject info = new BasicDBObject();info.put("x", 203);info.put("y", 102);doc.put("info", info);coll.insert(doc);

31. Neo4JGraphDatabaseServicegraphDb = new EmbeddedGraphDatabase("var/base");Transaction tx = graphDb.beginTx();try { Node firstNode = graphDb.createNode(); Node secondNode = graphDb.createNode(); Relationship relationship = firstNode.createRelationshipTo(secondNode, MyRelationshipTypes.KNOWS);firstNode.setProperty("message", "Hello, ");secondNode.setProperty("message", "world!");relationship.setProperty("message", "brave Neo4j ");tx.success();System.out.print(firstNode.getProperty("message"));System.out.print(relationship.getProperty("message"));System.out.print(secondNode.getProperty("message"));}finally {tx.finish();graphDb.shutdown();}

32. The Bottom Line

33. Data SLAThere is no golden hammerChoose your tool wisely, based on what you needUsuallyStart with RDBMS (shortest TTM, which is what we really care about)When scale issues occur – start moving to NoSQL based on your needsYou can get Data SLA in the cloud – just think before you code!!!