Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Building Scalable Databases on AWS - AWS Summit 2012 - NYC


Published on

Published in: Technology
  • RDS rocks always and it has come upto mature level since 2012 with various offerings.

    I would like to add a nice blog on Scaling with RDS,
    if you are thinking for Horizontal Scaling with RDS , Scalebase is one of the best answers.
    Are you sure you want to  Yes  No
    Your message goes here

Building Scalable Databases on AWS - AWS Summit 2012 - NYC

  1. 1. Building Scalable DatabaseApplications on AWSSundar Raghavan, AmazonGreg Scallan, Architect, FlipboardEric Weller, Director, Earth NetworksEdward Dingels, Technical Lead, Earth Networks
  2. 2. Database Services: One Size Does Not Fit All Amazon RDS + DynamoDB ElastiCache MySQL, Oracle, Memcached Apps that need Scalable Relational databases Apps that need Massive Scalability YesSQL NoSQL
  3. 3. Building Database Applications – The Old WayDemand Human Layer • Stuck with peak capacity • Human driven • Time consuming Q1 Time
  4. 4. What We Hear From Customers “Help us focus on applications – Shift database maintenance time to more productive application development time” Security planning Frequent server upgrades License/doc training Backup rec 5% 5% Constant storage upgrades load/unload 20% 5% Scripting coding Backup and recovery Software upgrade and patching Performance and 25% 40% tuning Hardware failure replacements Install, upgrade, patch, migration Distribution of timeSource:,5180,-1110,00.ppt
  5. 5. Building Database Applications – The New Way Application LayerDemand • Elastic capacity • App driven • Immediate Time
  6. 6. Amazon Relational Database ServiceRDS is a fully managed Relational database service that is simple to deploy, easy to scale, reliable and cost-effective Choice of Database Engines Fully Managed Service Push Button Scalability Fault Tolerance with Multi-AZ Works with EC2 & ElastiCache Amazon Relational Database Service (RDS)
  7. 7. High Performance Relational Databases Amazon RDS Improve Increase Reduce Configuration Availability Throughput Latency Push-Button Scaling Multi AZ Read Replicas ElastiCachePush-Button Scaling Read Replicas ElastiCache
  8. 8. Push Button ScalingScale up your instanceand storageScale out as your appgrowsAutomatic backupEasy restore
  9. 9. Read Scaling with Amazon ElastiCache and Read Replicas Improve performance with ElastiCache Read replica for read scaling Off load BI and reporting from production
  10. 10. Reliability For Production Applications
  11. 11. High Availability – Multi-AZ Deployments Enterprise grade fault tolerance Synchronous replication Automatic failover
  12. 12. Amazon RDS Customer Highlights Gaming Web Apps Mobile/Social Media
  13. 13. IntroductionYour Social Magazine for Apple Mobile DevicesDelivers relevant articles and photos based on usage andinteractions within your social networksLaunched 6 months after the initial team put togetherOver 5 million users and 2 billion page “flips” per month
  14. 14. Operating in the Cloud : Managing Complex, Real Time Data 6 months to deploy a real-time, socially relevant magazineChallenge Constantly changing user interestsArchitecture Ability to change all hardware and software elastically Frequently changing system requirementsApp needs Complex queries on user and relevancy data Milliseconds count. So, does uptime Solution Highly performant, reliable, proven database technology Amazon RDS MySQL
  15. 15. The Data View of the World Flipboard Application ElastiCache Memcache for performance Reliable andOperational Complex data,Configuration Queriesand State Data SimpleDB RDS MySQL
  16. 16. Friends, Magazines … anyone relevant to me who published something very recently that I care about seeing Show Me More, Please!A friend who recently shared a photo
  17. 17. Amazon RDS Tips For Success1. Leverage the Cloud for what it does best. Don’t bring old DC habits.2. Scale up for better performance.3. Use RDS for complex, real-time data.4. Use Read Replicas to augment write heavy databases. They are awesome.5. Leverage existing SQL knowledge and experience.6. Use copies of your database for testing new code. It’s trivial and saves time.7. Scale horizontally with sharding. Plan for it before you need it.
  18. 18. SQL or NoSQL?
  19. 19. Relational Or Non-Relational?Factors Relational (RDS) NoSQL (DynamoDB) • Existing database apps • New Web scale applicationsApplication type • Business process-centric apps • Large # of small writes and readsApplication • Relational data models, transactions • Simple data models, transactionscharacteristics • Complex queries, joins and updates • Range queries, simple updatesScaling • Application or DBA architected • Seamless, on-demand scaling • Performance – developer architected • Performance – AutomaticQoS • Reliability and availability • Reliability and availabilitySkillset • SQL + Programming languages • Web style programming Possible to use both SQL and NoSQL models in one application
  20. 20. The “Big Data” Scalability Challenge Requirement: predictable, consistent performancePerformance  Hardware purchase and provisioning $!   Data sharding Data caching  Cluster management Reality: performance  Fault management degrades with scale Scalability
  21. 21. Kit, go fasterTransformation 3: From Scaling by Architecture … to Scaling By Yes Command Michael
  22. 22. Amazon DynamoDBDynamoDB is a fully managed NoSQL database service that providesextremely fast and predictable performance with seamless scalability Easy Administration Low Latency SSD’s Reserved Capacity Unlimited Potential Storage and Throughput
  23. 23. Amazon DynamoDB NoSQL Database Fast & predictable performance Seamless Scalability ADMIN Easy administration“Even though we have years of experience with large, complexNoSQL architectures, we are happy to be finally out of thebusiness of managing it ourselves.” - Don MacAskill, CEO
  24. 24. DynamoDB Highlights Low Latency • SSD-based storage nodes • Typical latency =Get latency single-digit millisecondsCPU utilization
  25. 25. Provisioned Throughput Reserve the IOPS needed for each table Set at table creation Increase / decrease any time via API call Pay for throughput and storage (not instances) • $0.01 per hour for every 10 units of Write Capacity • $0.01 per hour for every 50 units of Read Capacity • $1.00 per GB-month of Storage
  26. 26. DynamoDB Customer Highlights
  27. 27. Earth Networks Case StudyEric Weller – Director of DevelopmentEdward Dingels – Technical Lead
  28. 28. IntroductionGathers and analyzes atmospheric observations from aglobal sensor network to promote a better understandingof the planetProprietary lightning network output used to pinpointlightning activity - best indicator of dangerous weatherOwner of the WeatherBug brand (mobile, desktop, Web)
  29. 29. Problem Generate lightning alert notifications, in proximity Need to the user’s location, on a mobile device. Geospatial queries ScalableConstraints • 6 million existing mobile users • 100% YOY mobile growth • Severe Weather Outbreaks Fast • Speed + Accuracy = Safety Reduce Time to Market Cost of Ownership
  30. 30. Analysis Provider Product Throughput Engineering Cost of per Instance Cost OwnershipMicrosoft SQL Server 2008 Medium Low HighMySQL MySQL Medium High* High*Earth Networks In Memory Quadtree High High MediumAmazon Mem-Cache High Medium MediumAmazon DynamoDB High Low Low * Not currently supported by Earth Networks
  31. 31. SolutionGeo-Hash lightning dataBackend Windows service for loading/managementof data inside DynamoDBFront-end web service for encoding/decodingUse geometric instead of trigonometric calculationsClient computes actual range and produces alerts
  32. 32. ReflectionDynamoDB Advantages• Easy to Provision• Built-in Consistency• Scalable• High AvailabilityDynamoDB Wish List• Automated regional DR failover• Durability across regions in addition to inside a region• Auto-scale down without a throttle on change• Item size limit of 64k
  33. 33. Lightning Demo
  34. 34. Try Amazon DB Services!Amazon RDS – Free usage• For Details Visit – Free usage• For Details Visit