Download to read offline
Scaling on AWS to the First 10 Million Users
- 1. © 2014 Amazon.com,Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc. Spain Tech Talk Scaling on AWS for the First 10 Million Users Matt Auerbach, Associate Solutions Architect April 24, 2015
- 2. Scaling on AWSfor the First 10 Million Users • Me: – Matt Auerbach; mauerbac@amazon.com • YOU: Here to learn more about scaling infrastructure on AWS • TODAY: about best practices and things to think about when building for large scale
- 3. So how dowe scale?
- 6. a lot ofthings to read
- 7. a lot ofthings to read not where we want to start
- 8.
- 9. What do weneed first?
- 10. Lets start withthe basics…
- 11. US-WEST (Oregon) EU-WEST (Ireland) ASIAPAC (Tokyo) US-WEST (N. California) SOUTH AMERICA (Sao Paulo) US-EAST (Virginia) AWS GovCloud (US) ASIA PAC (Sydney) Regions ASIA PAC (Singapore) EU-WEST (Frankfurt)
- 12. US-WEST (Oregon) EU-WEST (Ireland) ASIAPAC (Tokyo) US-WEST (N. California) SOUTH AMERICA (Sao Paulo) US-EAST (Virginia) AWS GovCloud (US) ASIA PAC (Sydney) Availability Zones ASIA PAC (Singapore) EU-WEST (Frankfurt)
- 13.
- 14. • $7B+ retailbusiness • 8,000+ employees • A whole lot of servers Every day, AWS adds enough server capacity to power that entire $7B enterprise 2004 2015
- 15. Storage & Content Delivery AWSGlobal Infrastructure Database App Services Deployment & Management Compute & Networking Analytics
- 16. Storage & Content Delivery AWSGlobal Infrastructure Database App Services Deployment & Management Compute & Networking Analytics Amazon DynamoDB Amazon RDS Amazon ElastiCache Amazon RedShift AWS Storage Gateway Amazon S3 Amazon Glacier Amazon CloudFront Amazon CloudWatch AWS IAM AWS CloudFormation Amazon Elastic Beanstalk AWS OpsWorks AWS CloudTrail Amazon SQS Amazon Elastic Transcoder Amazon SNS Amazon SES Amazon CloudSearch Amazon SWF Amazon AppStream Amazon EC2 Amazon VPC Amazon Route 53 AWS Direct Connect Amazon WorkSpaces Amazon EMR Amazon Kinesis AWS Data Pipeline
- 17. So let’s startfrom day one, user one (you)
- 18. Day One, UserOne: • A single EC2 Instance – With full stack on this host • Web App • Database • Management • Etc. • A single Elastic IP • Route53 for DNS EC2 Instance Elastic IP Amazon Route 53 User
- 19. “We’re gonna needa bigger box” • Simplest approach • Can now leverage PIOPs • High I/O instances • High memory instances • High CPU instances • High storage instances • Easy to change instance sizes • Will hit an endpoint eventually c3.8xlarge m3.2xlarge t2.micro
- 20. “We’re gonna needa bigger box” • Simplest approach • Can now leverage PIOPs • High I/O instances • High memory instances • High CPU instances • High storage instances • Easy to change instance sizes • Will hit an endpoint eventually c3.8xlarge m3.2xlarge t2.micro
- 21. Day One, UserOne: • We could potentially get to a few hundred to a few thousand depending on application complexity and traffic • No failover • No redundancy • Too many eggs in one basket EC2 Instance Elastic IP Amazon Route 53 User
- 22. Day One, UserOne: • We could potentially get to a few hundred to a few thousand depending on application complexity and traffic • No failover • No redundancy • Too many eggs in one basket EC2 Instance Elastic IP Amazon Route 53 User
- 23. Day Two, User>1: First let’s separate out our single host into more than one. • Web • Database – Make use of a database service? Web Instance Database Instance Elastic IP Amazon Route 53 User
- 24. Self-Managed Fully-Managed Database Server onAmazon EC2 Your choice of database running on Amazon EC2 Bring Your Own License (BYOL) Amazon DynamoDB Managed NoSQL database service using SSD storage Seamless scalability Zero administration Amazon RDS Microsoft SQL, Oracle, MySQL or PostgreSQL as a managed service Flexible licensing BYOL or License Included Amazon Redshift Massively parallel, petabyte-scale, data warehouse service Fast, powerful and easy to scale Database Options
- 25. But how doI choose what DB technology I need? SQL? NoSQL?
- 26.
- 27. Why start withSQL? • Established and well worn technology • Lots of existing code, communities, books, background, tools, etc • You aren’t going to break SQL DBs in your first 10 million users. No really, you won’t* • Clear patterns to scalability *Unless you are doing something SUPER weird with the data or MASSIVE amounts of it, even then SQL will have a place in your stack
- 28. AH HA! You said“massive amounts”, I will have massive amounts!
- 29. If your usageis such that you will be generating several TB ( >5 ) of data in the first year OR have an incredibly data intensive workload, then you might need NoSQL
- 30. Why else mightyou need NoSQL? • Super low latency applications • Metadata driven datasets • Highly non-relational data • Need schema-less data constructs* • Massive amounts of data (again, in the TB range) • Rapid ingest of data ( thousands of records/sec ) *Need != “its easier to do dev without schemas”
- 31. User >100: First let’sseparate out our single host into more than one. • Web • Database – Use Amazon RDS to make your life easier Web Instance Elastic IP RDS DB Instance Amazon Route 53 User
- 32. User > 1000: Nextlet’s address our lack of failover and redundancy issues: • Elastic Load Balancer • Another Web Instance – In another Availability Zone • Enable RDS Multi-AZ Web Instance RDS DB Instance Active (Multi-AZ) Availability Zone Availability Zone Web Instance RDS DB Instance Standby (Multi-AZ) Elastic Load Balancer Amazon Route 53 User
- 33. • Create highlyscalable applications • Distribute load across EC2 instances in multiple availability zones Feature Details Available Load balance across instances in multiple Availability Zones Health checks Automatically checks health of instances and takes them in or out of service Session stickiness Route requests to the same instance Secure sockets layer Supports SSL offload from web and application servers with flexible cipher support Monitoring Publishes metrics to CloudWatch and can get logs of requests processed Elastic Load Balancer Elastic Load Balancing
- 34. Scaling this horizontally andvertically will get us pretty far ( 10s-100s of thousands )
- 35. User >10ks-100ks: RDS DBInstance Active (Multi-AZ) Availability Zone Availability Zone RDS DB Instance Standby (Multi-AZ) Elastic Load Balancer RDS DB Instance Read Replica RDS DB Instance Read Replica RDS DB Instance Read Replica RDS DB Instance Read Replica Web Instance Web Instance Web Instance Web Instance Web Instance Web Instance Web Instance Web Instance Amazon Route 53 User
- 36. This will takeus pretty far honestly, but we care about performance and efficiency, so let’s improve this further
- 37. Shift some loadaround: Let’s lighten the load on our web and database instances: • Move static content from the web instance to Amazon S3 and Amazon CloudFront • Move session/state and DB caching to Amazon ElastiCache or Amazon DynamoDB Web Instance RDS DB Instance Active (Multi-AZ) Availability Zone Elastic Load Balancer Amazon S3 Amazon Cloudfront Amazon Route 53 User ElastiCache DynamoDB
- 38. Working with AmazonS3 • Object based storage for the web • 11 9s of durability • Good for things like: – Static assets ( css, js, images, videos ) – Backups – Logs – Ingest of files for processing • “Infinitely scalable” • Supports fine grained permission control • Ties in well with CloudFront • Ties in with EMR • Acts as a logging endpoint for S3/CloudFront/Billing • Supports Encryption at transit and at rest • Reduced Redundancy 1/3 cheaper • Amazon Glacier for super long term storage
- 39. CloudFront Amazon CloudFront isa web service for scalable content delivery: • Cache static content at the edge for faster delivery • Helps lower load on origin infrastructure • Dynamic and static content • Streaming video • Zone apex support • Custom SSL certificates • Low TTLs (as short as 0 seconds) • Lower costs for origin fetches (between Amazon S3 / Amazon EC2 and CloudFront) • Optimized to work with Amazon EC2, Amazon S3, Elastic Load Balancing, and Amazon Route 53 ResponseTime ServerLoad ResponseTime Server Load ResponseTime Server Load No CDN CDN for Static Content CDN for Static & Dynamic Content 0 10 20 30 40 50 60 70 80 8:00 AM 9:00 AM 10:00 AM 11:00 AM 12:00 PM 1:00 PM 2:00 PM 3:00 PM 4:00 PM 5:00 PM 6:00 PM 7:00 PM 8:00 PM 9:00 PM VolumeofData Delivered(Gbps)
- 40. Shift some loadaround: Web Instance RDS DB Instance Active (Multi-AZ) Availability Zone Elastic Load Balancer Amazon S3 Amazon Cloudfront Amazon Route 53 User ElastiCache DynamoDB Let’s lighten the load on our web and database instances: • Move static content from the web instance to Amazon S3 and Amazon CloudFront • Move session/state and DB caching to Amazon ElastiCache or Amazon DynamoDB
- 41. Amazon DynamoDB • Managed,provisioned throughput NoSQL database • Fast, predictable performance • Fully distributed, fault tolerant architecture • Considerations for non-uniform data Feature Details Provisioned throughput Dial up or down provisioned read/write capacity. Predictable performance Average single digit millisecond latencies from SSD-backed infrastructure. Strong consistency Be sure you are reading the most up to date values. Fault tolerant Data replicated across Availability Zones. Monitoring Integrated with Amazon CloudWatch. Secure Integrates with AWS Identity and Access Management (IAM). Amazon EMR Integrates with Amazon EMR for complex analytics on large datasets
- 42. Amazon ElastiCache • HostedMemcached & Redis – Speaks same API as traditional open source Memcached and Redis • Scale from one to many nodes • Self healing ( replaces dead instance ) • Very fast ( single digit ms speeds usually (or less) ) • Local to a single AZ for Memcache, with no persistence or replication • With Redis can put a replica in a different AZ with persistence • Use AWS’s Auto Discovery client to simplify clusters growing and shrinking without affecting your application
- 43. Shift some loadaround: Web Instance RDS DB Instance Active (Multi-AZ) Availability Zone Elastic Load Balancer Amazon S3 Amazon Cloudfront Amazon Route 53 User ElastiCache DynamoDB Let’s lighten the load on our web and database instances: • Move static content from the web instance to Amazon S3 and Amazon CloudFront • Move dynamic content from the ELB to Amazon CloudFront • Move session/state and DB caching to ElastiCache or DynamoDB
- 44. Now that ourWeb tier is much more lightweight, we can revisit the beginning of our talk…
- 45.
- 46. Automatic resizing ofcompute clusters based on demand Feature Details Control Define minimum and maximum instance pool sizes and when scaling and cool down occurs. Integrated to Amazon CloudWatch Use metrics gathered by CloudWatch to drive scaling. Instance types Run Auto Scaling for On-Demand and Spot Instances. Compatible with VPC. aws autoscaling create-auto-scaling-group --auto-scaling-group-name MyGroup --launch-configuration-name MyConfig --min-size 4 --max-size 200 --availability-zones us-west-2c Auto-Scaling Trigger auto-scaling policy Amazon CloudWatch
- 47. Sunday Monday TuesdayWednesday Thursday Friday Saturday Typical weekly traffic to Amazon.com
- 48. Sunday Monday TuesdayWednesday Thursday Friday Saturday Typical weekly traffic to Amazon.com Provisioned capacity
- 49. November traffic toAmazon.com November
- 50. November traffic toAmazon.com Provisioned capacity November
- 51. November traffic toAmazon.com 76% 24% Provisioned capacity November
- 52. November traffic toAmazon.com November
- 53. Auto Scaling lets youdo this!
- 54. User >500k+: Availability Zone Amazon Route53 User Amazon S3 Amazon Cloudfront Availability Zone Elastic Load Balancer DynamoDB RDS DB Instance Read Replica Web Instance Web Instance Web Instance ElastiCache RDS DB Instance Read Replica Web Instance Web Instance Web Instance ElastiCacheRDS DB Instance Standby (Multi-AZ) RDS DB Instance Active (Multi-AZ)
- 56. Use Tools: Managing yourinfrastructure will become an ever increasing important part of your time. Use tools to automate repetitive tasks. • Tools to manage AWS resources • Tools to manage software and configuration on your instances • Automated data analysis of logs and user actions
- 57. AWS Application ManagementSolutions Convenience Control Higher-level Services Do it yourself AWS Elastic Beanstalk AWS OpsWorks AWS CloudFormation Amazon EC2
- 58. User >500k+: You’ll potentiallystart to run into issues with speed and performance of your applications. • Make sure you have monitoring/metrics/logging in place – If you can’t build it internally, outsource it! ( 3rd party SaaS ) • Pay attention to what customers are saying works well vs. what doesn’t, use this direction • Try to work on squeezing as much performance out of each service/component
- 59.
- 60. Not having proper monitoring/metricsis like flying a plane with an eye mask on in a thunderstorm. Oh and your wing is on fire.
- 62. AWS Marketplace &Partners Can Help • Customer can find, research, and buy software • Simple pricing, aligns with Amazon EC2 usage model • Launch in minutes • AWS Marketplace billing integrated into your AWS account • 1300+ products across 20+ categories Learn more at: aws.amazon.com/marketplace
- 63. There’s further improvements tobe made in breaking apart our web/app layer
- 65. SOA = ServiceOriented Architecture
- 66. SOA’ing Move services intotheir own tiers/modules. Treat each of these as 100% whole-y separate pieces of your infrastructure and scale them independently. Amazon.com and AWS do this extensively! It offers flexibility and greater understanding of each component.
- 67. Loose coupling +SOA = winning DON’T REINVENT THE WHEEL Examples: • Email • Queuing • Transcoding • Search • Databases • Monitoring • Metrics • Logging Amazon CloudSearch Amazon SQSAmazon SNS Amazon Elastic Transcoder Amazon SWF Amazon SES In the early days, if someone has a service for it already, opt to use that instead of building it yourself.
- 68. On re-inventing thewheel: If you find yourself writing your own: queue, DNS server, database, storage system, monitoring tool
- 69. Take a deepbreath and stop it. Now.
- 70. User >1mil+: Reaching amillion and above is going to require some bit of all the previous things: • Multi-AZ • Elastic Load Balancer between tiers • Auto Scaling • Service oriented architecture • Serving content smartly (Amazon S3/CloudFront ) • Caching off DB • Moving state off tiers that auto-scale
- 71. User >1mil+: RDS DBInstance Active (Multi-AZ) Availability Zone Elastic Load Balancer RDS DB Instance Read Replica RDS DB Instance Read Replica Web Instance Web Instance Web Instance Web Instance Amazon Route 53 User Amazon S3 Amazon Cloudfront DynamoDB Amazon SQS ElastiCache Worker Instance Worker Instance Amazon CloudWatch Internal App Instance Internal App Instance Amazon SES
- 72. The next bigsteps
- 73. User >5mil –10mil: You’ll potentially start to run into issues with your database around contention on the write master. How can you solve it? • Federation ~ splitting into multiple DBs based on function • Sharding ~ splitting one data set up across multiple hosts • Moving some functionality to other types of DBs ( NoSQL )
- 74. Database Federation • Splitup Databases by function/purpose • Harder to do cross function queries • Essentially delaying the need for something like sharding/NoSQL until much further down the line • Won’t help with single huge functions/tables ForumsDB UsersDB ProductsDB
- 75. Sharded Horizontal Scaling •More complex at the application layer • ORM support can help • No practical limit on scalability • Operation complexity/sophistication • Shard by function or key space • RDBMS or NoSQL User ShardID 002345 A 002346 B 002347 C 002348 B 002349 A A B C
- 76. Shifting functionality toNoSQL • Similar in a sense to federation • Again, think about the earlier points for when you need NoSQL vs SQL • Leverage hosted services like DynamoDB • Some use cases: – Leaderboards/scoring – Rapid ingest of clickstream/log data – Temporary data needs ( cart data ) – “Hot” tables – Metadata/lookup tables DynamoDB
- 77.
- 78. A Quick Review •Multi-AZ your infrastructure • Make use of self scaling services - ELB, Amazon S3, Amazon SNS, Amazon SQS, Amazon SWF, Amazon SES, etc • Build in redundancy at every level. • Start SQL. Seriously. • Cache data both inside and outside your infrastructure • Use automation tools in your infrastructure
- 79. A Quick Review •Make sure you have good metrics/monitoring/logging tools in place • Split tiers into individual services ( SOA ) • Use Auto Scaling once you’re ready for it • Don’t reinvent the wheel • Move to NoSQL if/when it makes sense
- 80. Putting all thistogether means we should now easily be able to handle 10+ million users!
- 81.
- 82. User >10mil: Iterating ontop of the patterns seen here will get you up and over 100 million users.
- 83. User >10mil: • Morefine tuning of your application • More SOA of features/functionality • Going from Multi-AZ to Multi-Region • Needing to start potentially building custom solutions • Deep analysis of your whole stack
- 84. Next steps? READ! – •aws.amazon.com/documentation • aws.amazon.com/architecture • aws.amazon.com/start-ups
- 85. Next steps? START USINGAWS – aws.amazon.com/free/
- 86. Next steps? ASK FORHELP! • forums.aws.amazon.com • aws.amazon.com/support • Your Account Manager • A Solutions Architect
- 87. THANKS FOR LISTENING! Matt Auerbach –mauerbac@amazon.com
Editor's Notes
- #3 Introduce yourself, who the crowd is, and our goal for today
- #4 -new to aws -> “how do I scale” If you are new to scaling on AWS, you might ask yourself this question: “So how do I scale?”
- #5 Hard to start with new service and so much available Go to a search engine… “And if you are like most people, its really hard to know where to start. Again, there are all these resources, twitter based experts, and blog posts preaching to you how to scale”.. “so again, where do we start?”
- #6 Most start with a search engine and something like "scaling on aws" If you are like me, you’ll start where I usually start when I want to learn how to do something. A search engine. In this case I’ve gone and searched for “scaling on AWS” using my favorite search engine.
- #7 With any search there are many results This is hard It’s important to note something about the results here. First off, there are a lot of things to read. This search was from a few months ago, and there were over half a million posts on how to scale on AWS.
- #8 This is actually not what we want as auto scaling is a feature – which we will discuss later Unfortunately for us and our search engine here however, the first response back is actually not what we are looking for. Auto-scaling IS an AWS services, and its great, but its not really the place to start
- #9 Interestingly enough our presentation is actually listed in the results A quick side note, this presentation however has been delivered now a few times, and you can currently find the presentation at #4 of the results!
- #10 Back to Auto-scaling. “Auto-scaling is a tool and a destination for your infrastructure. It isn’t a single thing. Its not a check-box you can click when launching something. Your infrastructure really has to be built with the right properties in mind for Auto-scaling to work.”.. “So again, where do we start?”
- #11 What do we need first?
- #12 We need some basics to lay the foundations we’ll need to build our knowledge of AWS on top of.
- #13 Lets start with our infrastructure -> datacenters First we have regions which are geographical areas throughout the world Latency, compliance, privacy First up we have AWS regions. This is the most macro level concept we have at AWS today. ( describe regions, their number, locations ). https://aws.amazon.com/about-aws/globalinfrastructure
- #14 Regions are made up of Azs. AZ are logically separated, different ISP, power , but interconnected by private link Next up we have Availability Zones, these are part of our regions, and existing within them. There will be at a minimum 2 of these in every AZ, and generally speaking your infrastructure will live in one or more AZ’s inside a given region. We’ll be talking a lot about Multi-AZ architectures today, as it’s a core-component to having a highly available, highly redundant, and highly durable infrastructure on AWS. https://aws.amazon.com/about-aws/globalinfrastructure
- #15 Our edge locations are apart of the CDN Last up we have our Edge locations, which are the homes to our CloudFront CDN, and Route 53 DNS service. There are over 50 of these around the globe right now, and we’re constantly trying to add these to new places to help best meet our customer’s needs. We also sometimes have multiple edge locations in a single area to help handle the capacity, IE New York, which has 3 today. https://aws.amazon.com/about-aws/globalinfrastructure
- #16 This might help understand our scale This is a point of reference about our scale- in 2004, Amazon.com was a $7+ Billion technology powered retail business. When we measured the capacity growth rate of our current AWS environments, we found that we add enough server capacity to power all of the 2004 era Amazon business, EVERY DAY.
- #17 Our services Over 30 services as represented in groupings in the console
- #18 -just a subset now -> new service -many new services released
- #19 So let’s start from day one, user one, of our new infrastructure and application.
- #20 Very basic setup to serve a web application Route 53 for DNS Ec2 is running the web app and database This here is the most basic set up you would need to serve up a web application. We have Route53 for DNS, an EC2 instance running our webapp and database, and an Elastic IP attached to the EC2 instance so Route53 can direct traffic to us. Now in scaling this infrastructure, the only real option we have is to get a bigger EC2 instance…
- #21 Bigger box is scaling vertically Add more cpu/memory/storage Scaling the one EC2 instance we have to a larger one is the most simple approach to start with. There are a lot of different AWS instance types to go with depending on your work load. Some have high I/O, CPU, Memory, or local storage. You can also make use of EBS-Optimized instances and Provisioned IOPs to help scale the storage for this instance quite a bit.
- #22 Issue is availability and cost All in one basket The key concern here, is that you WILL hit an endpoint, where we just don’t have a bigger instance class out yet, and so scaling this way while it can get you over an initial hump, really isn’t going to get us very far.
- #23 So while we could reach potentially a few hundred or few thousand users supported by this single instance, its not a long term play.
- #24 We also have to consider some other issues with this infrastructure; No Failover, No redundancy, and too many eggs in one basket, since we have both the database and webapp on the same instance.
- #25 Let's think about reducing load on our webserver We can rethink our database Many advantages Operate independently - > de-couple Let's look at our database options… The first thing we can do to address the issues of too many eggs in one basket, and to over come the “no bigger boat” problem, is to split out our Webapp and Database into two instances. This gives us more flexibility in scaling these two things independently. And since we are breaking out the Database, this is a great time to think about maybe making use of a database services instead of managing this ourselves…
- #26 Many options for db at aws At AWS there are a lot of different options to running databases. One is to just install pretty much any database you can think of on an EC2 instance, and manage all of it yourself. If you are really comfortable doing DBA like activities, like backups, patching, security, tuning, this could be an option for you. If not, then we have a few options that we think are a better idea: First is Amazon RDS, or Relational Database Service. With RDS you get a managed database instance of either MySQL, Oracle, or SQL Server, with features such as automated daily backups, simple scaling, patch management, snapshots and restores, High availability, and read replicas, depending on the engine you go with. Next up we have DynamoDB, a NoSQL database, built ontop of SSDs. DynamoDB is based on the Dynamo whitepaper published by Amazon.com back in 2003, considered the grandfather of most modern NoSQL databases like Cassandra and Riak. DynamoDB that we have here at AWS is kind of like a cousin of the original paper. One of the key concepts to DynamoDB is what we call “Zero Administration”. With DynamoDB the only knobs to tweak are the reads and writes per second you want the DB to be able to perform at. You set it, and it will give you that capacity with query responses averaging in single millisecond. We’ve had customers with loads such as half a million reads and writes per second without DynamoDB even blinking. Lastly we have Amazon Redshift, a multi-petabyte-scale data warehouse service. With Redshift, much like most AWS services, the idea is that you can start small, and scale as you need to, while only paying for what scale you are at. What this means is that you can get 1TB of of data per year at less than a thousand dollars with Redshift. This is several times cheaper than most other dataware house providers costs, and again, you can scale and grow as your business dictates without you needing to sign an expensive contract upfront.
- #27 Given that we have all these different options, from running pretty much anything you want yourself, to making use of one of the database services AWS provides, how do you choose? How do you decide between SQL and NoSQL?
- #28 Read as is
- #29 Read as is
- #30 I know right now you are saying you have lots of excuses as to why they won’t work for you…
- #31 Read as is
- #32 Read as is
- #33 SQL is well established Strong community/codebase Most likely won't break So Why start with SQL databases? Generally speaking SQL based databases are established and well worn technology. There’s a good chance SQL is older than most people in this room. It has however continued to power most of the largest web applications we deal with on a daily basis. There are a lot of existing code, books, tools, communities, and people who know and understand SQL. Some of these newer nosql databases might have a handful, tops, of companies using them at scale. You also aren’t going to break SQL databases in your first 10 million users. And yes there is an astrisk here, and we’ll get to that in a second. Lastly, there are a lot of clear patterns for scalability that we’ll discuss a bit through out this talk. So as for my point here at the bottom, I again strongly recommend SQL based technology, unless your application is doing something SUPER weird with the data, or you’ll have MASSIVE amounts of it, even then, SQL will be in your stack.
- #34 Let's say you do have massive amounts of data…. Let's explain this blurry line AH HA! You say. I said ‘massive amounts”, and we all assume we’ll have massive amounts, so that means that you must be the lone exclusion in this room… well lets clarify this a bit.
- #35 If your usage is such that you will be generating several terabytes ( greater than 5) of data in the first year, OR you will have an incredibly data intensive workload, then, you might need NoSQL
- #36 So why else might you need NoSQL? There are definitely usecases where it makes sense to go NoSQL right off the bat. Some examples: Super low latency applications. Metadata driven datasets High-unrelational data Kind of going along with the previous is where you really need schema-less data constructs. And lets highlight the word NEED here. This isn’t just developers saying its easy to make apps without schemas. That’s just laziness Massive amounts of data, again from the previous slide, in the several TB range. Rapid ingest of data. Where you need to ingest potentially thousands of records per second into a single dataset
- #37 Read this slide…
- #38 Read this slide…
- #39 So for this scenario today, we’re going to go with RDS and MYSQL as our database engine.
- #40 Address the lack of failover and redundancy Next up we need to address the lack of failover and redundancy in our infrastructure. We’re going to do this by adding in another webapp instance, and enabling the Multi-AZ feature of RDS, which will give us a standby instance in a different AZ from the Primary. We’re also going to replace our EIP with an Elastic Load Balancer to share the load between our two web instances
- #41 -ELB sets the stage for horizontal scaling
- #42 Read this slide.
- #43 Bit complicated to look at Adding read replicas -> async copies of master Most of you will get to this point and be pretty well off honestly. You can take this really pretty far for most web applications. We could scale this out over another AZ maybe. Add in another tier of read replicas.
- #44 but its not that efficient in both performance or cost, and since those are important too, let’s clean up this infrastructure a bit.
- #45 Move static content to s3 and cloudfront Move session state to elasticache and Dynamo The biggest things we can do, and these are incredibly important, is lighten up some of the work our webapp is doing, as well as make life easier for our database. We can start by moving any static assets from our webapp instances to S3, and then serving those objects via CloudFront. We can also move things like session information, and any other temporary application data to a memory based cache like one supported by ElastiCache or DynamoDB. We can also use this same cache to store some of our database query results which will help us from hitting the database too much.
- #46 Talk about S3
- #47 -talk about charts -cloudfront can save you money
- #48 Serving our static assets through CloudFront is going to be a massive performance boost to our end-users, but CloudFront can do much more.
- #49 So let’s actually go and pump the entire site through CloudFront. This could allow us to cache all sorts of page content at the edge, and greatly speed up our site performance to our end users, while significantly lowering the load on our infrastructure.
- #50 Talk about DynamoDB. We could use it in this work flow to store session information from our web application.
- #51 Talk about elasticache. Now supports Redis which is new! We could use Memcache/Redis as a place to store database query information for content that doesn’t change often, like our end-users’s name or email, or what is in their cart. We should try and do this as often as possible.
- #52 -now move our dynamic content to ELB Imagine for instance if you cached the search pages for highly requested queries. This could take load off your search, off your web application, your database, etc. So now we can see here that we’ve got CloudFront in front of both S3 and our ELB. Now that we’ve got that covered, lets move back to the session information, and database queries we can be caching as well.
- #53 Read slide
- #54 Read slide
- #55 Talk about auto-scaling.
- #56 Imagine a “typical” week of traffic to Amazon.com. This pattern might look a lot like your own traffic, with a peak during the middle of the day, and a low down in the middle of the night.
- #57 Given this pattern, it becomes really easy to do capacity planning. We can provision say 15% over what we see our normal peak to be, and be happy with the capacity we have for a while, so long as our traffic matches this pattern. BUT, what if I told you that this was the first week of November for Amazon.com
- #58 And this was the month of November! We can see a pretty big growth here at the end of the month with Black Friday and Cyber Monday sales in the US.
- #59 IF we attempted to do our ‘add 15% capacity for spikes” rule, we’d be in trouble for the month of November.
- #60 That’s a lot of potential wasted infrastructure and cost. 76% wasted potentially, while only 24% of it on average for the month gets utilized. Traditionally this is how IT did things. You bought servers for a 6-12 month vision on what growth might be. So, since we can all agree this is bad, what is the solution.
- #61 Well what if we could map our capacity directly to our needs based on our end-users? We could make sure that at any point in time, we were only provisioning what we needed, vs some arbitrary capacity guess.
- #62 Read slide.
- #63 If we add in auto-scaling, our caching layer(both inside, and outside our infrastructure), and the read-replicas with MySQL, we can now handle a pretty serious load. This could potentially even get us into the millions of users by itself if continued to be scaled horizontally and vertically.
- #64 Deployment tool
- #65 Read slide
- #66 -Discuss lightly pros/cons of each. -chef with opsworks -ec2 own automation platofrom – install what software you would like , connect to our services using API/SDK Elastic Beanstalk is easiest to start with, but offers less control. Opsworks gives you more tools, with a bit more work on your part. CloudFormation is a template driven tool with its own language, so a bit of a learning curve, but very very powerful. Lastly you could do all this manually, but at scale its nearly impossible without a huge team.
- #67 Read slide
- #68 Read slide
- #69 Metrics are extremely important Also cloudwatch has a log agent Pay attention to what your metrics say to you. Host Level metrics are great for deep diving on problems, but aggregate level metrics will be more valuable as a bigger picture of what is going on with your infrastructure. Log analysis is also very much needed, and incredibly powerful to have in your infrastructure. Don’t skim on it. Log everything centrally. Lastly we have external site metrics. Its amazing how many people don’t think about this last one here. You need to understand how your site is performing from the view of your end users. ( top two are from CloudWatch, bottom left is from Kibana/logstash, bottom right is Pingdom)
- #70 Read slide
- #72 Read slide, talk about how awesome the marketplace is to find the kind of tools you need to help you scale.
- #73 There’s even further that we can go beyond what we have so far. So far we’ve had just a single webapp tier doing all of our application workload. While that works for some sites and applications, for many it doesn’t. Which brings us on to our next topic…
- #74 Say nothing, go quick from this slide to next one.
- #75 Service Oriented Architecture!
- #76 Read slide, sum up SOA, and mention that Amazon.com and AWS have hundreds of services under the hood that represent the sites and services you see. It’s a core principle in application/service development at Amazon.
- #77 Talk about loose coupling and how it pertains to SOA architectures. Describe the SQS as a buffer example.
- #78 Combining Loose coupling, SOA, and prebuilt services, can also really have some huge advantages. Instead of writing all these mini services yourself, try and leverage already existing services and applications, especially when you are starting out. DON’T REINVENT THE WHEEL! For example, at AWS we have services to help you with Email, Queues, Transcoding, Search, Databases, and Monitoring and Metrics. Lean on other 3rd parties for more.
- #79 Read slide.
- #80 Read slide
- #81 Read slide
- #82 Read slide
- #83 We let our users upload their photos to us, we put the initial image into an S3 bucket for ingest, and then put a message through SNS, which in turn feeds 3 different SQS queues. Each queue represents a different size of our image we’re going to resize it to. We then have an auto-scaling group set up with EC2 instances that will poll each SQS queue, do the resize, and put the new images into S3. The original could be put back into a different S3 bucket. This is a highly SOA and decoupled setup here making use of all these different services, that can scale independently, be unaffected by failures in each other, and automatically scale and heal themselves as traffic increases, or in case of a failure. This is pretty ideal.
- #84 In being highly SOA, there is another great AWS service that can help you out. Simple Work Flow (talk about SWF).
- #85 Going back to our photo-upload infrastructure again, let’s see what it would look like potentially with SWF..
- #86 What we have done is stripped out SNS and SQS, replaced it with SWF, and an instance running the decider application we’d write to help us out.
- #88 This diagram is missing the other AZ, but we’ve only got so much room on the slide. But we can see we’ve added in some internal pools for different tasks perhaps. Maybe we’re not using SQS for something, and have SES for sending our out bound email. Again our users will still talk to Route53, and then to CloudFront to get to our site and our content hosted back by our ELB and S3.
- #89 Read slide
- #90 Read slide
- #91 Read slide, talk about federation
- #92 Read slide, talk about sharding. Sharding is used heavily by companies like Etsy, Facebook, Flickr, Tumblr, Pinterest, Instagram
- #93 Read slide
- #94 Read slide
- #95 Read slide.
- #97 Read slide
- #98 So, beyond 10mil
- #99 Read slide
- #100 Read slide
- #101 Read slide
- #102 Read slide
- #103 Read slide
- #104 Thanks for listening, and I hope you found this to be really useful for you today. Credits