Scalable Database Options on AWS

AWS Database Options and Decision FactorsBest Practice Tips and Techniques • Optimizing for Manageability and Scale  Edmodo • Optimizing for App Velocity and Scale  Obama for America • Leveraging YesSQL and NoSQL  BrandVerityQ&A

Easily and rapidly analyzepetabytes of data1/10 the cost of traditionaldata warehousesAutomated deployment &administrationCompatible with popularBI tools – MicroStrategy,Jaspersoft

1. Zero to App in ____ Minutes2. Zero to Millions of users in ____ Days3. Zero to “Hero” in ____ Months

Focus on your App

Load balancerApplication tierDatabase tier

Load balancer Security, Scale, Availability… Application tier Security, Innovation, Scale, Performance, Availability… Database tierSecurity, Innovation, Scale, Transactions, Performance, Durability, Availability, Skills..

SQL NoSQLDo-it Yourself Fully Managed Not available on AWSLow Cost High Cost

SQL NoSQLDo-it Yourself Fully Managed

SQL NoSQLDo-it Yourself Fully ManagedMySQL MySQLOracle OracleSQL Server SQL ServerMariaDBPostgres…

SQL NoSQL Do-it Yourself Fully ManagedMongoDB DynamoDBCassandra ElastiCacheRedis SimpleDBMemcache

Should I useShould I use SQL MySQL on EC2 or or NoSQL? RDS? Should I use MongoDB, ? Should I use Redis, Cassandra, or Memcache, or DynamoDB? ElastiCache?

What are myWhat are my scale transactional andand latency needs? consistency needs? What are my ? What are my time toread/write, storage market and server and IOPS needs? control needs?

Factors SQL NoSQLApplication • App with complex business logic? • Web app with lots of users?Transactions • Complex txns, joins, updates? • Simple data model, updates, queries?Scale • Developer managed • Automatic, on-demand scalingPerformance • Developer architected • Consistent, high performance at scaleAvailability • Architected for fail-over • Seamless and transparentCore Skills • SQL + Java/Ruby/Python/PhP • NoSQL + Java/Ruby/Python/PhP Best of both worlds: Possible to Use SQL and NoSQL models in one App

Amazon RDS is a fully managed SQL database service. Choice of Database engines Simple to deploy and scale Reliable and cost effective Without any operational burden.

Migration Backup and recoverySchema design PatchingQuery construction ConfigurationQuery optimization Software upgrades Storage upgrades Frequent server upgrades Focus on the “innovation” Hardware crash Off load the “administration”

MySQL Manual for Read Replica Multiple databases per instance Use MySQL tools & drivers Quickly set up Read Replicas High availability Multi-AZ option OR Amazon RDS console Ability to promote Read replicas, Rename as Master Diagnostics Native MySQL replication SSL for encryption over the wire Monitor metrics Shell, super user or direct file system access (Think security!)

 AZ location for both Primary and Stand-by are now visible You can place 3rd replica in an AZ different from Primary and Stand-by Ability to promote Read replicas Rename as Master after promotion

ElastiCache is a fully managed Memcachedcaching service.Easy to set up and operateScale cache clusters with push button easeUltra fast response time for read scalingWithout any operational burden.

Amazon DynamoDB is a fully managed NoSQLdatabase service.Store and retrieve any amount of dataScale throughput to millions of IOSingle digit millisecond latenciesWithout any operational burden.

CreateTable PutItem UpdateTable GetItem DeleteTable UpdateItem “Select”, “insert”, “update” DescribeTable itemsManage tables DeleteItem ListTables BatchGetItem Query Bulk select or update Query specific items OR Scan BatchWriteItem (max 1MB) scan the full table

So, what are the tips and techniques forsuccessful deployments?

KimoEducates millions of students RosenbaumReaches millions of citizensAnalyzes billions of Ads

Where learning happens. Kimo Rosenbaum kimo@edmodo.com AWS re: Invent 2012

Learning 101• Largest, fastest growing social platform for education• Secure learning network for teachers and students• Browser, iOS, Android• Free for teachers and students

Stats 101• 100,000 schools• 14 million users• 7 million new users in the last year• 1 million visits daily

Web Instance Auto scaling Group Amazon CloudWatchAmazon Route 53 Elastic Load Balancer Cache Cache Instance InstanceAmazon Cloudfront Instances Amazon S3 RDS DB Instance RDS DB Instance RDS DB Instance Read Replica Read Replica Read Replica Availability Zone RDS DB Instance RDS DB Instance RDS DB Instance Read Replica MySQL DB Instance Read Replica MySQL DB Instance Read Replica MySQL DB Instance

DBA 101• Restore from snapshot• Replica creation• Parameter tuning• Metrics collection• Know your app/data

Educates millions of students JayReaches millions of citizens EdwardsAnalyzes billions of Ads

Me.• Twitter: First dedicated DBA• OFA: Lead Database Engineer• PalominoDB: CTO & VP/Operations

Obama for America.• Technically sophisticated for a campaign • Not “web-scale”• Hockey-stick++ growth• Downtime hurts. A lot…really, really, really a lot.

Hockey-stick++

OFA Architecture RDS Read Replica ElastiCache RDS with DynamoDB Multi-AZ ELB

Problems!• You always need more databases • OFA had 24+ schemas & 100+ RDS instances• You never have enough DBAs • OFA had 1 – 2 x 0.5 fulltime MySQL DBAs

Why RDS?• Makes operational issues very easy • Need more replicas? BAM! • Upsize hardware? KAPOW! • Point in time restore? BIF!

Why not RDS?• Hardware cap (vertical v. horizontal)• Sophisticated use-cases • Frequent topology changes • Multi-region replication (on their roadmap)• DBAs need busy work

Educates millions of studentsReaches millions of citizens AndyAnalyzes billions of Ads Skalet

Managed Services Bias

New Products/Markets – YesSQL!

Big Data? Cast your problem

AWS Options

Case Study: Crawl history

andy@brandverity.com

• Managed services let you focus on creating value• Amazon S3 - Very robust, handles large items, but you filter• Amazon DynamoDB - Extremely fast, scalable, good value • Must cast your problem as kvs or key + range• Amazon RDS - MySQL, without the headaches• Amazon ElastiCache - As memcached, fast kvs for small data• Multi column queries on big data? • Looking forward to the AWS solution

Thank youFree aws.amazon.com/dynamodbTier aws.amazon.com/rds aws.amazon.com/elasticache

Scalable Database Options on AWS

by Amazon Web Services on Jan 21, 2013

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 1

Statistics

Scalable Database Options on AWS Presentation Transcript