Building Scalable Databases on AWS - AWS Summit 2012 - NYC

Building Scalable DatabaseApplications on AWSSundar Raghavan, AmazonGreg Scallan, Architect, FlipboardEric Weller, Director, Earth NetworksEdward Dingels, Technical Lead, Earth Networks

Database Services: One Size Does Not Fit All Amazon RDS + DynamoDB ElastiCache MySQL, Oracle, Memcached Apps that need Scalable Relational databases Apps that need Massive Scalability YesSQL NoSQL

Building Database Applications – The Old WayDemand Human Layer • Stuck with peak capacity • Human driven • Time consuming Q1 Time

What We Hear From Customers “Help us focus on applications – Shift database maintenance time to more productive application development time” Security planning Frequent server upgrades License/doc training Backup rec 5% 5% Constant storage upgrades load/unload 20% 5% Scripting coding Backup and recovery Software upgrade and patching Performance and 25% 40% tuning Hardware failure replacements Install, upgrade, patch, migration Distribution of timeSource: http://www.forrester.com/Events/Content/0,5180,-1110,00.ppt

Building Database Applications – The New Way Application LayerDemand • Elastic capacity • App driven • Immediate Time

Amazon Relational Database ServiceRDS is a fully managed Relational database service that is simple to deploy, easy to scale, reliable and cost-effective Choice of Database Engines Fully Managed Service Push Button Scalability Fault Tolerance with Multi-AZ Works with EC2 & ElastiCache Amazon Relational Database Service (RDS)

High Performance Relational Databases Amazon RDS Improve Increase Reduce Configuration Availability Throughput Latency Push-Button Scaling Multi AZ Read Replicas ElastiCachePush-Button Scaling Read Replicas ElastiCache

Push Button ScalingScale up your instanceand storageScale out as your appgrowsAutomatic backupEasy restore

Read Scaling with Amazon ElastiCache and Read Replicas Improve performance with ElastiCache Read replica for read scaling Off load BI and reporting from production

Reliability For Production Applications

High Availability – Multi-AZ Deployments Enterprise grade fault tolerance Synchronous replication Automatic failover

Amazon RDS Customer Highlights Gaming Web Apps Mobile/Social Media

IntroductionYour Social Magazine for Apple Mobile DevicesDelivers relevant articles and photos based on usage andinteractions within your social networksLaunched 6 months after the initial team put togetherOver 5 million users and 2 billion page “flips” per month

Operating in the Cloud : Managing Complex, Real Time Data 6 months to deploy a real-time, socially relevant magazineChallenge Constantly changing user interestsArchitecture Ability to change all hardware and software elastically Frequently changing system requirementsApp needs Complex queries on user and relevancy data Milliseconds count. So, does uptime Solution Highly performant, reliable, proven database technology Amazon RDS MySQL

The Data View of the World Flipboard Application ElastiCache Memcache for performance Reliable andOperational Complex data,Configuration Queriesand State Data SimpleDB RDS MySQL

Friends, Magazines … anyone relevant to me who published something very recently that I care about seeing Show Me More, Please!A friend who recently shared a photo

Amazon RDS Tips For Success1. Leverage the Cloud for what it does best. Don’t bring old DC habits.2. Scale up for better performance.3. Use RDS for complex, real-time data.4. Use Read Replicas to augment write heavy databases. They are awesome.5. Leverage existing SQL knowledge and experience.6. Use copies of your database for testing new code. It’s trivial and saves time.7. Scale horizontally with sharding. Plan for it before you need it.

SQL or NoSQL?

Relational Or Non-Relational?Factors Relational (RDS) NoSQL (DynamoDB) • Existing database apps • New Web scale applicationsApplication type • Business process-centric apps • Large # of small writes and readsApplication • Relational data models, transactions • Simple data models, transactionscharacteristics • Complex queries, joins and updates • Range queries, simple updatesScaling • Application or DBA architected • Seamless, on-demand scaling • Performance – developer architected • Performance – AutomaticQoS • Reliability and availability • Reliability and availabilitySkillset • SQL + Programming languages • Web style programming Possible to use both SQL and NoSQL models in one application

The “Big Data” Scalability Challenge Requirement: predictable, consistent performancePerformance  Hardware purchase and provisioning $!   Data sharding Data caching  Cluster management Reality: performance  Fault management degrades with scale Scalability

Kit, go fasterTransformation 3: From Scaling by Architecture … to Scaling By Yes Command Michael

Amazon DynamoDBDynamoDB is a fully managed NoSQL database service that providesextremely fast and predictable performance with seamless scalability Easy Administration Low Latency SSD’s Reserved Capacity Unlimited Potential Storage and Throughput

Amazon DynamoDB NoSQL Database Fast & predictable performance Seamless Scalability ADMIN Easy administration“Even though we have years of experience with large, complexNoSQL architectures, we are happy to be finally out of thebusiness of managing it ourselves.” - Don MacAskill, CEO

DynamoDB Highlights Low Latency • SSD-based storage nodes • Typical latency =Get latency single-digit millisecondsCPU utilization

Provisioned Throughput Reserve the IOPS needed for each table Set at table creation Increase / decrease any time via API call Pay for throughput and storage (not instances) • $0.01 per hour for every 10 units of Write Capacity • $0.01 per hour for every 50 units of Read Capacity • $1.00 per GB-month of Storage

DynamoDB Customer Highlights

Earth Networks Case StudyEric Weller – Director of DevelopmentEdward Dingels – Technical Lead

IntroductionGathers and analyzes atmospheric observations from aglobal sensor network to promote a better understandingof the planetProprietary lightning network output used to pinpointlightning activity - best indicator of dangerous weatherOwner of the WeatherBug brand (mobile, desktop, Web)

Problem Generate lightning alert notifications, in proximity Need to the user’s location, on a mobile device. Geospatial queries ScalableConstraints • 6 million existing mobile users • 100% YOY mobile growth • Severe Weather Outbreaks Fast • Speed + Accuracy = Safety Reduce Time to Market Cost of Ownership

Analysis Provider Product Throughput Engineering Cost of per Instance Cost OwnershipMicrosoft SQL Server 2008 Medium Low HighMySQL MySQL Medium High* High*Earth Networks In Memory Quadtree High High MediumAmazon Mem-Cache High Medium MediumAmazon DynamoDB High Low Low * Not currently supported by Earth Networks

SolutionGeo-Hash lightning dataBackend Windows service for loading/managementof data inside DynamoDBFront-end web service for encoding/decodingUse geometric instead of trigonometric calculationsClient computes actual range and produces alerts

ReflectionDynamoDB Advantages• Easy to Provision• Built-in Consistency• Scalable• High AvailabilityDynamoDB Wish List• Automated regional DR failover• Durability across regions in addition to inside a region• Auto-scale down without a throttle on change• Item size limit of 64k

Lightning Demo

Try Amazon DB Services!Amazon RDS – Free usage• For Details Visit Aws.amazon.com/rdsDynamoDB – Free usage• For Details Visit Aws.amazon.com/dynamodb

http://www.scoop.it	105
http://www.gabrielemittica.com	33

Building Scalable Databases on AWS - AWS Summit 2012 - NYC

by Amazon Web Services

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 138

Statistics

Building Scalable Databases on AWS - AWS Summit 2012 - NYC Presentation Transcript