AWS Database Options and Decision FactorsBest Practice Tips and Techniques • Optimizing for Manageability and Scale Edmodo • Optimizing for App Velocity and Scale Obama for America • Leveraging YesSQL and NoSQL BrandVerityQ&A
Easily and rapidly analyzepetabytes of data1/10 the cost of traditionaldata warehousesAutomated deployment &administrationCompatible with popularBI tools – MicroStrategy,Jaspersoft
1. Zero to App in ____ Minutes2. Zero to Millions of users in ____ Days3. Zero to “Hero” in ____ Months
Should I useShould I use SQL MySQL on EC2 or or NoSQL? RDS? Should I use MongoDB, ? Should I use Redis, Cassandra, or Memcache, or DynamoDB? ElastiCache?
What are myWhat are my scale transactional andand latency needs? consistency needs? What are my ? What are my time toread/write, storage market and server and IOPS needs? control needs?
Factors SQL NoSQLApplication • App with complex business logic? • Web app with lots of users?Transactions • Complex txns, joins, updates? • Simple data model, updates, queries?Scale • Developer managed • Automatic, on-demand scalingPerformance • Developer architected • Consistent, high performance at scaleAvailability • Architected for fail-over • Seamless and transparentCore Skills • SQL + Java/Ruby/Python/PhP • NoSQL + Java/Ruby/Python/PhP Best of both worlds: Possible to Use SQL and NoSQL models in one App
Amazon RDS is a fully managed SQL database service. Choice of Database engines Simple to deploy and scale Reliable and cost effective Without any operational burden.
Migration Backup and recoverySchema design PatchingQuery construction ConfigurationQuery optimization Software upgrades Storage upgrades Frequent server upgrades Focus on the “innovation” Hardware crash Off load the “administration”
MySQL Manual for Read Replica Multiple databases per instance Use MySQL tools & drivers Quickly set up Read Replicas High availability Multi-AZ option OR Amazon RDS console Ability to promote Read replicas, Rename as Master Diagnostics Native MySQL replication SSL for encryption over the wire Monitor metrics Shell, super user or direct file system access (Think security!)
AZ location for both Primary and Stand-by are now visible You can place 3rd replica in an AZ different from Primary and Stand-by Ability to promote Read replicas Rename as Master after promotion
ElastiCache is a fully managed Memcachedcaching service.Easy to set up and operateScale cache clusters with push button easeUltra fast response time for read scalingWithout any operational burden.
Amazon DynamoDB is a fully managed NoSQLdatabase service.Store and retrieve any amount of dataScale throughput to millions of IOSingle digit millisecond latenciesWithout any operational burden.
CreateTable PutItem UpdateTable GetItem DeleteTable UpdateItem “Select”, “insert”, “update” DescribeTable itemsManage tables DeleteItem ListTables BatchGetItem Query Bulk select or update Query specific items OR Scan BatchWriteItem (max 1MB) scan the full table
So, what are the tips and techniques forsuccessful deployments?
KimoEducates millions of students RosenbaumReaches millions of citizensAnalyzes billions of Ads
Learning 101• Largest, fastest growing social platform for education• Secure learning network for teachers and students• Browser, iOS, Android• Free for teachers and students
Stats 101• 100,000 schools• 14 million users• 7 million new users in the last year• 1 million visits daily
Web Instance Auto scaling Group Amazon CloudWatchAmazon Route 53 Elastic Load Balancer Cache Cache Instance InstanceAmazon Cloudfront Instances Amazon S3 RDS DB Instance RDS DB Instance RDS DB Instance Read Replica Read Replica Read Replica Availability Zone RDS DB Instance RDS DB Instance RDS DB Instance Read Replica MySQL DB Instance Read Replica MySQL DB Instance Read Replica MySQL DB Instance
DBA 101• Restore from snapshot• Replica creation• Parameter tuning• Metrics collection• Know your app/data
Educates millions of students JayReaches millions of citizens EdwardsAnalyzes billions of Ads
Me.• Twitter: First dedicated DBA• OFA: Lead Database Engineer• PalominoDB: CTO & VP/Operations
Obama for America.• Technically sophisticated for a campaign • Not “web-scale”• Hockey-stick++ growth• Downtime hurts. A lot…really, really, really a lot.
• Managed services let you focus on creating value• Amazon S3 - Very robust, handles large items, but you filter• Amazon DynamoDB - Extremely fast, scalable, good value • Must cast your problem as kvs or key + range• Amazon RDS - MySQL, without the headaches• Amazon ElastiCache - As memcached, fast kvs for small data• Multi column queries on big data? • Looking forward to the AWS solution