26. amazon dynamo
predecessor to dynamoDB
replicated DHT with consistent hashing
optimistic replication
“sloppy quorum”
anti-entropy mechanism
object versioning
specialist tool :
•limited querying capabilities
•simpler consistency
@swami_79 @ksshams
27. dynamo had many benefits
• higher availability
• we traded it off for consistency
• incremental scalability
• no more repartitioning
• no need to architect apps for peak
• just add boxes
• simpler querying model ==>> predictable performance
@swami_79 @ksshams
28. but dynamo was not perfect...
lacked strong consistency
@swami_79 @ksshams
29. but dynamo was not perfect...
scaling was easier, but...
@swami_79 @ksshams
30. but dynamo was not perfect...
steep learning curve
@swami_79 @ksshams
31. but dynamo was not perfect...
dynamo was a library ==>> not a service...
@swami_79 @ksshams
33. Managed NoSQL Database
ADMIN
DynamoDB
Built for Scale
Fast & Predictable Performance
@swami_79 @ksshams
34. DynamoDB
“Even though we have years of experience with large,
complex NoSQL architectures, we are happy to be finally
out of the business of managing it ourselves.” - Don
MacAskill, CEO
@swami_79 @ksshams
35. DynamoDB Goals and Philosophies
durability and availability
scale is our problem
easy to use
scale in rps
consistent and low latencies
@swami_79 @ksshams
38. scale is our problem, not yours..
@swami_79 @ksshams
39. Fault Tolerant Design
Infrastructure Fails - deal with it!
Planning for failures is not easy
How do you ensure your recovery strategies work correctly?
@swami_79 @ksshams
44. Easy?
Writes from
client X
Reads and
Writes from
client Y
Classic Split Brain Issue in Replicated systems leading to lost writes!
@swami_79 @ksshams
45. Building correct distributed systems is not straight forward..
Handle replica failures
Handle partial failures of replicas
Handle concurrent failures
Ensure there isn’t a parallel quorum
@swami_79 @ksshams
http://www.swissknifeshop.com/media/catalog/product/cache/1/image/5e06319eda06f020e43594a9c230972d/W/G/WG16999.jpg
They seriously don’t scale well
and if you’re not careful and try to use all the features at once, you can get cut in the process
it was like playing minesweeper
RDBMs focus way too much on consistency - and compromise availability
And seriously - are they really all that consistent? ever had slaves fall behind? or how about a recent bug we had where all slaves got updates, but the master didn’t take the update
Like Swiss army knives - there is waaaay too much functionality in the knives. Let’s move it to the app (easier to debug and change).
Developers often use transactions when they shouldn’t - very annoying - it is like bringing a swiss army knife through a TSA checkpoint. Just foolish
Khawaja
Swami:
K:
required action from developers
Q4 scaling: While not as scary as phase 1, still was work
Need benchmarking, patching, adding hardware, rebalancing clusters..
developers still had to learn to set it up
if you are building a recommendation system, you want to be learning k-clustering and collaborative filtering algorithms rather than focusing on lamport’s papers
Swami:
like s3 - and it required developers to be admins.
this is the really key point - our developers wanted a service, not a product. If you look at mongodb, that is what they are doing today.
K
</K>
2 dc writes
on disk
continuous replication
we should brag about availability
multi-region service!
Regional service
spans multiple availability zones
All data is continuously replicated to multiple AZ’s