Relational Databases Redefined with AWS


Published on

Relational Databases Redefined on AWS webinar series as presented by Joe Ziegler

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • From the beginning, Flipboard has run its infrastructure on Amazon Web Services (AWS). Greg Scallan, Chief Architect at Flipboard says, “Our service currently runs 100% on AWS in multiple availability zones.” He adds, “We chose AWS because they were able to provide a majority of the solution we needed as we built our data center. Also, we appreciated the flexibility as we tried out various solutions to our business vision.” Flipboard has incorporated AWS services as follows:Amazon Elastic Compute Cloud (Amazon EC2) for all server side processing from handling end user requests to data crunching, including a good sized Hadoop infrastructureAmazon SimpleDB for operational dataAmazon Relational Database Service (Amazon RDS) for mission critical dataAmazon Simple Queue Service (Amazon SQS)Amazon CloudWatchAmazon Elastic Load Balancing (Amazon ELB)Amazon Simple Storage Service (Amazon S3) and Amazon CloudFront for static and dynamic content distributionAmazon Mechanical Turk To build their solution, the team used Python/Ruby AWS SDKs in their operational tools and administrative servers. Scallan explains, “We leverage the Java AWS SDKs for core engineering services. We utilize many different languages, choosing the language that best meets the requirements at hand.”Flipboard has many hundreds of Amazon EC2 instances running and that number is continually growing with their increasing user base and feature sets. Scallan says, “We were able to go from concept to delivered product in approximately six months with just a handful of engineers, in large part because of the ability to deploy within a working cloud infrastructure very quickly.” Scallan would give the following advice to other developers: “If you plan to deploy in the cloud, embrace and leverage it for what it has to offer, and do not try to shoehorn traditional data center deployment strategies. One of the major perks of being in the cloud is flexibility and leveraging that can do wonders for operational costs.”The Flipboard team is always investigating additional AWS solutions and is open to using additional products as their needs evolve.The team has most appreciated the flexibility AWS has offered for prototyping before their first release. Scallan says, “This truly helped us release a working product on a highly aggressive schedule in a market with many emerging players.”
  • Mr.Horiuchi, CTO of gumiInc, describes what the company was using for their infrastructure prior to signing on with Amazon Web Services (AWS): “We used to rent racks at a datacenter, and ran our service using one Web server and one database server.” He notes that the company looked for a new offering mainly because they faced difficulty meeting demand: “So much traffic was coming from the social apps we were running on a mixi [a Japanese social networking site] platform, but we did not have the physical IT resources to manage it.”One of the deciding factors in their choice to use AWS was the capability to easily add servers as traffic increases, or remove them if it decreases. Horiuchi says, “Using AWS, we do not have to be limited by the number of physical servers. For example, we can easily carry out distribution of traffic using memcached or tokyotyrant when the database load gets higher, and if traffic decreases, we can reduce costs by canceling those instances. Even if there is no one who understands our in-house infrastructure very well, we are able to carry out this process automatically using AWS.” The number of gumi Inc. users is still increasing, and they are able to handle the increase in traffic.“Now,” says Horiuchi, “we are operating all services on AWS, except the delivery of static images.” The company uses Amazon Elastic Compute Cloud (Amazon EC2), Amazon Relational Database Service (Amazon RDS) with the Multi-AZ deployment option, and Amazon Simple Storage Service (Amazon S3) for running social applications in a very efficient way that requires only a small number of people to manage the system. “In particular,” notes Horiuchi, “Amazon RDS is a really good solution to reduce the time and heavy-lifting involved in database management.” The company is using the following server spec—the most high-spec instances they can choose—for their social application:Load Balancer:Amazon Elastic Load BalancingApp Server:Amazon EC2CentOS 5.4 + Apache2.2.3 + mod_wsgi + Python2.5.4Instances: c1.xlarge High-CPU Extra Large Instance x 8Memcached Server:Amazon EC2CentOS 5.4 + memcachedInstances: m1.xlarge Standard Extra Large Instance x 1Tokyotyrant (key-value store):Amazon EC2CentOS 5.4 + tokyotyrantInstances: m1.xlarge Standard Extra Large Instance x 2DB Server:Amazon RDSMySQL 5.1.38Instances: Quadruple Extra Large DB Instance: 68 GB of memory, 26 ECUs (8 virtual cores with 3.25 ECUs each), 64-bit platform x 2 (Multi-AZ HA Option)Horiuchi says gumiInc will use AWS for future social applications. The company is particularly pleased that AWS allows them to quickly perform testing and evaluation, reducing time to market and development costs. Horiuchi offers an example: “We quickly inspected and tested several candidates when we examined the introduction of the key-value store [KVS] for an anti-load measure of our database.” They look forward to quickly building their services with additional solutions from AWS, especially KVS-type services.
  • 80-90% cost savings compared to traditional hosting environment.Minimum of 99.9% service availability using Multi-Availability zones without increasing costs.Easy expansion into other geographical regions as neededFocus on improving products and applications rather than managing infrastructure.Customer Quote:“If we were to use the traditional on-premise datacenter, we would have spent $34 million dollars in hardware and maintenance expenses during the first two years. With AWS cloud, we met our reliability and performance objectives at a fraction of the cost.”Mr. Chun KangPrincipal Engineer, Visual Display Division
  • Relational Databases Redefined with AWS

    1. 1. Relational Databases Redefinedon AWS Joseph Ziegler Technical @jiyosub
    2. 2. Focus on your App with Amazon RDS
    3. 3. Focus on your App with Amazon RDS
    4. 4. Load balancerApplication tierDatabase tier
    5. 5. Load balancer Application tier Invest more time making this innovative Database tierSpend less time managing this
    6. 6. Migration Backup andSchema design recovery Patching Query constructionConfiguration Software upgrades Frequent server Storage upgrades upgrades Hardware crash Query optimization
    7. 7. Migration Backup and recoverySchema design PatchingQuery construction ConfigurationQuery optimization Software upgrades Storage upgrades Focus on the “buck” Frequent server upgrades Hardware crash Off load the “muck”
    8. 8. Source: Forrester Security planning License training Backup, recovery load and unload Script automationPerformance Installation, upgrade,and tuning patching, migration
    9. 9. Undifferentiated Heavy Lifting
    10. 10. Amazon RDSRelational Database Service Quick introduction
    11. 11. Amazon RDS OverviewAmazon RDS is a fully managed relationaldatabase service. Choice of Database engines Simple to deploy Easy to operate and scale Reliable Cost effective
    12. 12. Choice of database engines MySQL Oracle Microsoft SQL Server
    13. 13. Choice of database engines Community Edition MySQL 5.1.x & 5.5.x Oracle Microsoft SQL Server
    14. 14. Choice of database engines MySQL Standard & Enterprise Edition Oracle 11.2.x Microsoft SQL Server
    15. 15. Choice of database engines MySQL Oracle Express, Microsoft SQL Server Web, & Standard Edition 2008 R2 & 2012
    16. 16. Three decisions + a few clicks= ready for use
    17. 17. Easy to changeThree decisions + a few clicks= ready for use
    18. 18. Three decisions + a few clicks= ready for use
    19. 19. Three decisions + a few clicks= ready for development
    20. 20. Three decisions + a few clicks= ready for scale
    21. 21. Three decisions + a few clicks= ready for production
    22. 22. Three ways to remove the “muck”
    23. 23. Productivity PerformanceData Protection
    24. 24. ProductivityMost admin tasks are managed by RDS Provisioning Database backups Patching Performance management
    25. 25. Productivity Multiple databases per instance Standard user accounts Connect and query using common MySQL tools & drivers Tune engine parameters Import and export data using standard MySQL tools (mysqldump) Diagnostics Native MySQL replication SSL for encryption over the wire Monitor metrics Shell, super user or direct file system access (Think security!)
    26. 26. ProductivityNear zero administration Painless patching, automatic upgrades Cloudwatch monitoring, Metric alarms One click. High Availability.
    27. 27. ProductivityOne click. High availability with Multi-AZ Automated deployment across multiple AZs Synchronous replication from master to replica Automatic fail-over; replica promoted to master Test fail-over
    28. 28. PerformancePush-button scale, high performance Scale storage from 5Gb to 1Tb of storage Scale instance from small to 4XL (better I/O) Add Read Replicas with asynchronous replication Add ElastiCache for performance
    29. 29. PerformanceAmazon ElastiCache In-memory cache service Ideal front end to RDS for read-heavy applications Low latency
    30. 30. PerformanceRead Replicas Scale beyond capacity constraints of a single instance Asynchronous replication to replicas Perfect for read-heavy loads and reporting
    31. 31. PerformanceReplicas vs Caches Caches are good for low latency access over a limited dataset Replicas provide relatively less performance over a substantially larger dataset
    32. 32. PerformanceProvisioned IOPS Fast, predictable, and consistent I/O 1,000 IOPS to 10,000 IOPS 100GB to 1 TB
    33. 33. Case study6 months to deploy a real time, social magazine5 million users, 2 billion page flips per monthRun on EC2, RDS and ElastiCache. Multi-AZ.Better response time (milliseconds) and uptimeComplex queries on use and relevance data
    34. 34. Case studyOutgrew existing IT environmentElasticity to respond to peaks; reduce costsRun on EC2, RDS10 million visitors a dayReduced time to market via
    35. 35. Data ProtectionAutomatic snapshots Daily database back-ups (save up to 35 days) Transaction log backups every 5 minutes Everything up to final 5 minutes can be recovered
    36. 36. Data ProtectionAutomatic backups Automated point-in-time backups 1 day retention (extendable to 35 days) On by default. No additional charge.
    37. 37. Data ProtectionDB Security groups DB instances are isolated by default Control access by source IP address or EC2 security group Specify IP range: discouraged!
    38. 38. Data ProtectionRDS in the Virtual Private Cloud Virtual network, bring your own IP addresses Complete virtual network control: public, private subnets Includes Multi-AZ (for MySQL and Oracle)
    39. 39. Data ProtectionEncryption via SSL RDS generates SSL certificates for each DB instance Encryption of data between client and DB instance
    40. 40. Samsung saved $34M Problem: • Needed to reduce IT costs and were looking to create a more flexible IT environment Solution: • AWS and RDS services. With every request, the application authenticates devices, delivers apps and content, and pushes notifications. Business Benefits: • Saved $34M in hardware and maintenance expenses, 85% less than running on-premises
    41. 41. BonusReserved instances One time, up front payment Reserve capacity for your DB instance Discounted hourly rate Low, medium and high utilization rates
    42. 42. BonusInstant Development Data Use RDS for Production and Development Leverage Backups as Test Data
    43. 43. BonusPromoting RDS Read Replicasto Primary Fast Sharding Instant Replication of Fully Provisioned Database
    44. 44. BonusFree Tier Run RDS free for 1 Year! MySQL, Oracle (BYOL), SQL Server Express
    45. 45. Focus on your App with Amazon RDS
    46. 46. Instance scaling Multi-AZ PerformanceFocus on your App with Amazon RDS Backups DR Storage scaling
    47. 47. Instance scaling Multi-AZ Faster updates New features PerformanceFocus on your App with Amazon RDS Backups Happy customers : ) DR Storage scaling
    48. 48. Questions?
    49. 49. Thank Joseph Ziegler Technical Evangelist @jiyosub