Learn more about how to scale your application with Amazon DynamoDB.

1. Scale Your App for the Holidays
with DynamoDB
David Pearson, Business Development Manager
build high scale applications in days
fast | scalable | cost-effective
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

2. What Is NoSQL?
CAP theorem
BASE Not Only SQL
schema-free
horizontally scalable
key-value
eventually consistent
unstructured
open-source non-relational
distributed
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

3. scalability
big data challenge:
RDBMS the cost of scalability
infrastructure scaling
scale-up = +
bigger servers
application scaling
versatile
scale-out = more servers
change application to use…
 read slaves
 read caches
 data shards
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

4. scalability
NoSQL
RDBMS
infrastructure scaling
=
application scaling
versatile specialized
scale-out without
application changes
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

5. General Characteristics
scalability
RDBMS
complexity
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

6. NoSQL @ Amazon
a story in three parts…
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

7. key-value access
NoSQL @ Amazon complex queries
transactions
part one - early days… analytics
RDBMS used for “all kinds of access patterns”
data (re-)partitioning
bigger hardware is tempting
versatile scalability
easy to learn
availability
easy to query trade-off with consistency
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

8. specialized technology
NoSQL @ Amazon limited query capabilities
simpler consistency
part two - dynamo…
replicated DHT with consistency management
• Consistent hashing
• Optimistic replication
• “Sloppy quorum”
• Anti-entropy mechanisms
• Object versioning
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

9. NoSQL @ Amazon
part two - dynamo…
replicated DHT with consistency management
eventual consistency
higher availability no need to re-architect applications
incremental scalability just add hardware!
predictable performance
simpler query model
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

10. NoSQL @ Amazon
part two - dynamo…
replicated DHT with consistency management
much better, but…
higher availability no strong consistency
still required developers to
incremental scalability scaling effort required be administrators…
predictable performance operational complexity
install, patch, upgrade,
balance clusters
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

11. NoSQL @ Amazon nosql database service
part three - Amazon DynamoDB…
 Fast & Predictable Performance
 Seamless Scalability
 Zero Administration
“Even though we have years of experience with large, complex
NoSQL architectures, we are happy to be finally out of the
business of managing it ourselves.” – Don MacAskill, CEO
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

12. consistent, disk-only
NoSQL @ Amazon writes (not memory)
continuous replication
Design Philosophy across multiple AZ’s
• No compromise on durability or availability for performance
• Express scale needs in simpler terms, not servers and disks
• Scale will be our problem and not our customers
• Extremely easy to use with no administration
• Provide consistently low latencies provisioned
throughput
backed by SSDs
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

13. November Traffic at Amazon.com
Capacity needed before DynamoDB
76% = wasted hardware
Actual
traffic
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

14. November Traffic at Amazon.com
Capacity needed before DynamoDB
Capacity we can provision
with DynamoDB
Actual
traffic
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

15. Case Study small objects
move faster!
metadata in DynamoDB
Stores user objects in cloud
objects in S3
Queries and object searches are list all the objects in my drive
served by DynamoDB find all my music albums
“We were excited by how fast we were able to put DynamoDB into production and how much
developer time we saved. In addition, DynamoDB lets us scale up and down easily by simply
reserving increased throughput capacity when we need it and dialing it back when we don’t”
- Russel Dicker, Amazon CloudDrive
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

16. Cloud Drive Photos
#customers X
Goal = enable customers to see Cloud Drive #photos
photos on their Kindle
Need = low latency access to the metadata, at any scale
Result = lower total cost of ownership (tco)
lower admin effort required to scale
hardware costs +
operational costs +
future tco opportunity cost of
feature development
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

17. KINDLE
DynamoDB access pattern
list all images for photos uploaded
by a customer
CLOUD DRIVE
Process Process
Process Process
Process
Cloud Drive App Process
Cloud Drive
Servers Metadata Service
image thumbnails
album cover art
AWS links to S3
S3 DynamoDB
image Thumbnails
files Dynamo
S3 Images URLs
DB
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

18. Getting started with DynamoDB…
Two decisions + three
clicks = ready for use
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

19. Two decisions + three
clicks = ready for use
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

20. Provisioned Throughput effectively assigns each table
its own set of servers
• Reserve throughput for each table
• Set at table creation, increased and decreased via an API call
$0.01 per hour for every 10 units of Write Capacity
$0.01 per hour for every 50 units of Read Capacity =10 writes
per second…
$1.00 per GB-month of Storage
=50 strongly consistent
free tier… reads per second…
100MB storage + 5 writes/sec ... for items up to
+ 10 reads/sec each month 1KB in size
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

21. Items are indexed by
primary key
single hash keys and composite keys
hash + range
key-value access =
extreme performance
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

22. table = collection of items
attribute (key value pair)
item (max size 64k) = string, number, binary
collection of attributes
deviceid = 21EC2020-3AEA-1069-A2DD-08002B30309D total = 25.00
deviceid = 74ED9134-3FEC-9902-E8BA-19733F49779C total = 50.00
hash key
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

23. mapping 1:M relations
userid = 100 date = 2012-10-24-09-00-10 total = 25.00
userid = 100 date = 2012-10-24-09-00-11 total = 50.00
hash key range key
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

24. DynamoDB
APIs CreateTable PutItem
UpdateTable GetItem
read and
manage tables DeleteTable write items
UpdateItem
DescribeTable
DeleteItem
ListTables
query specific Query BatchGetItem
bulk select or
items OR scan
Scan BatchWriteItem update
the full table
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

25. Query Patterns
Available for hash-and-range primary key tables
Retrieve all items by hash key
Range key conditions:
==, <, >, >=, <=, begins with, between
Counts. Top and bottom n values. Paged responses
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

26. Getting started with DynamoDB…
Designing a Photo Store
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

27. Photo Store
use case usage patterns data design
enable users to upload and
view photos from any device
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

28. Photo Store
use case usage patterns data design
upload photos
view photos by time
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

29. Photo Store
use case usage patterns data design
table photos
hash userid data model is optimized
for retrieval performance
range timestamp + photoid
attributes location, resolution,
comments, tags, s3link tags enable search integration
s3link points to raw files
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

30. Photo Store
use case usage patterns data design
table photos
Potential Enhancements…
hash userid
• Albums
range timestamp + photoid
• Search integration (CloudSearch)
attributes location, resolution,
• Global caching and delivery of
comments, tags, s3link
media from S3 (CloudFront)
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

31. Extreme scalability…
Enhancing relational DB
performance… and cost
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

32. Scaling Relational DB’s
Infrastructure Scaling bigger hardware (scaling up)
Read Scaling #1 read replicas (slaves)
Read Scaling #2 (hot keys) read caching (memcached)
Write Scaling data sharding
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

33. Scaling Relational DB’s
Infrastructure Scaling bigger hardware (scaling up)
Read Scaling #1 read replicas (slaves)
Read Scaling #2 (hot keys) read caching (memcached)
Write Scaling data sharding
application changes required
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

34. Scaling Relational DB’s with DynamoDB
move simple, high-scale workloads
Identify Design the new Implement code
candidates DynamoDB table changes in app
maximize access performance hopefully, for
the last time
Tables with high transaction volume (esp writes)
Primary key-only is preferred (one non-PK index ok)
No dependencies (FK’s, triggers, procedures)
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

35. DynamoDB Scalability @ HalfBrick Studios
• Fruit Ninja Frenzy (facebook) 8M
• Moved game data into DynamoDB
• Grew from 1 million to 8 million active
monthly users in two weeks
“it’s really tough to quickly scale a normal database
system to handle that kind of rapid increase in load”
1M
- Glen Arrowsmith, Systems Architect
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

36. DynamoDB Scalability
“Creating a table that can serve
100,000 writes/second is no more
work than creating a table that
can serve 10 writes/second”
Werner Vogels, Amazon CTO
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

37. Super Bowl promotion Weatherbug app – lightning
detection & alerting for 6M phones
Millions of interactions over
a relatively short period Extending to 40M users/month
Built the app in 3 days – from Developed and tested in weeks
design to production-ready “1/20th of the cost of the
traditional DB approach”
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

38. DynamoDB • minimal development time
and effort
• consistently low latency
Fast
• effortless scaling to meet
workload demand
Scalable
• Reduces DB costs and
increases reliability
Cost-Effective • Free tier reduces initial
development costs
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

39. Recommended Resources
• Building Applications with DynamoDB (Matt Wood)
 google search = youtube "building applications with dynamodb"
http://www.youtube.com/watch?v=4jZthAFKAE4
http://www.slideshare.net/AmazonWebServices/building-applications-with-dynamodb
• From the Super Bowl to WeatherBug (Werner Vogels)
http://www.allthingsdistributed.com/2012/06/amazon-dynamodb-growth.html
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.

40. Questions?
aws.amazon.com/dynamodb
© 2011 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.