AWS Cloud Kata 2013 | Singapore - Getting to Scale on AWS


Published on

This session will focus on how to get from 'Minimum Viable Product' (MVP) to scale. It will also explain how to deal with unpredictable demand and how to build a scalable business. Attend this session to learn how to:
Scale web servers and app services with Elastic Load Balancing and Auto Scaling on Amazon EC2
Scale your storage on Amazon S3 and S3 Reduced Redundancy Storage
Scale your database with Amazon DynamoDB, Amazon RDS, and Amazon ElastiCache
Scale your customer base by reaching customers globally in minutes with Amazon CloudFront

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

AWS Cloud Kata 2013 | Singapore - Getting to Scale on AWS

  1. 1. Getting to Scale Calvin Kizana CEO, PicMix
  2. 2. What is PicMix? PicMix is an advanced mobile photo sharing app and integrated platform for brand engagement and advertising to serve brand activations and marketplace digitally with clear and solid monetization model.
  3. 3. on the Amazon Cloud Use many AWS Services: EC2, S3, SES, SNS, Cloudwatch +Experimenting with CloudFront and Auto-scaling AWS enables us to focus on product & code, and user acquisition – not infra better performance, reliability & infinite scale Result? go global easily Hyper growth without worries Costs? only $0.05 user/year
  4. 4. Getting to Scale Dhruv Parpia & Joe Ziegler
  5. 5. 503 Service Temporarily Unavailable The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.
  6. 6. With AWS, scale from one instance…
  7. 7. …to thousands Fully automated!
  8. 8. BUT… How do I scale my architecture to support my first 10M users?
  9. 9. “Think Big, Start Small, Scale Fast” Eric Ries, author of NY Times bestseller “The Lean Startup”
  10. 10. 01 02 03 04 Idea MVP ProfitabilityScale
  11. 11. Getting to Scale By building a scalable Architecture to support your first 10M users
  12. 12. 1. Dev & Test 2. Alpha Release 3. Beta Release
  13. 13. Production 1.0 Architecture
  14. 14. Database Options Self-Managed Fully-Managed Database Server on Amazon EC2 Your choice of database running on Amazon EC2 Bring Your Own License (BYOL) Amazon DynamoDB Managed NoSQL database service using SSD storage Seamless scalability Zero administration Amazon RDS Relational Database as a managed service Flexible licensing: BYOL or License Included
  15. 15. But how do I choose what DB technology I need? SQL? NoSQL?
  16. 16. Some folks won’t like this. But…
  17. 17. Start with SQL databases
  18. 18. But, but, but, but…
  19. 19. No. You don’t.
  20. 20. Start with SQL databases
  21. 21. Established and well worn technology Lots of existing code, communities, books, tools, etc Clear patterns to scalability You aren’t going to break SQL DBs in your first 10 million users. No really, you won’t Why SQL?
  22. 22. • Database-as-a-Service • No need to install or manage database instances • Scalable and fault tolerant configurations Feature Details Platform support Create MySQL, SQL Server and Oracle Preconfigured Get started instantly with sensible default settings Automated patching Keep your database platform up to date automatically Backups Automatic backups and point in time recovery using snapshots Manual DB snapshots Failover Automated failover to slave hosts in event of a failure Replication Easily create read-replicas of your data and seamlessly replicate data across availability zones Amazon Relational Database Service (RDS)
  23. 23. Automatic resizing of compute clusters based on demand Trigger auto-scaling policy Feature Details Control Define minimum and maximum instance pool sizes and when scaling and cool down occurs. Integrated to Amazon CloudWatch Use metrics gathered by CloudWatch to drive scaling. Instance types Run Auto Scaling for On-Demand and Spot Instances. Compatible with VPC. as-create-auto-scaling-group MyGroup --launch-configuration MyConfig --availability-zones us-east-1a --min-size 4 --max-size 200 Auto-Scaling Amazon CloudWatch
  24. 24. Production 1.0 Architecture
  25. 25. Production 1.0 Architecture Well-designed, 2 Tier architecture Highly Available due to Multiple Availability Zone Load Balancing & Auto-Scaling for full scalability Fully managed Database included Capable of serving >10K-100Ks users
  26. 26. BUT…
  27. 27. Production 1.0 Architecture Wasted server capacity for static content Reliability and durability are not yet optimal End-user experience could be improved thru offloading & caching
  28. 28. SO…
  29. 29. Let’s add Simple Storage Service (S3) CloudFront to optimize the end-user experience
  30. 30. Durable storage, any object 99.999999999% durability of objects Unlimited storage of objects of any type Up to 5TB size per object Feature Details Flexible object store Buckets act like drives, folder structures within Access control Granular control over object permissions Server-side encryption 256bit AES encryption of objects Multi-part uploads Improved throughput & control Object versioning Archive old objects and version new ones Object expiry Automatically remove old objects Access logging Full audit log of bucket/object actions Web content hosting Serve content as web site with built in page handling Notifications Receive notifications on key events Import/Export Physical device import/export service Simple Storage Service (S3)
  31. 31. • World-wide content distribution network • Easily distribute content to end users with low latency, high data transfer speeds, and no commitments Feature Details Fast Multiple world-wide edge locations to serve content as close to your users as possible Integrated with other services Works seamlessly with S3 and EC2 origin servers Dynamic content Supports static and dynamic content from origin servers Streaming Supports rtmp from S3 and includes support for live streaming from Adobe FMS and Microsoft Media Server CloudFront
  32. 32. Production 1.2 Architecture
  33. 33. Production 1.2 Architecture Well-designed, 2 Tier architecture Highly Available due to Multiple Availability Zone Load Balancing & Auto-Scaling for full scalability Fully managed Database included Static content stored in durable, consistent way Improved end-user experience through CDN Capable of serving >100K-1M+ users
  34. 34. BUT…
  35. 35. Production 1.2 Architecture You are now at Scale… …with lots of data… …and need to optimize continuously. But how and where?
  36. 36. SO…
  37. 37. Let’s add Big Data for analytics of web, mobile, gaming, and log data
  38. 38. Multiple managed AWS services for Big Data
  39. 39. • Managed, elastic Hadoop cluster • Integrates with S3 & DynamoDB • Leverage Hive & Pig analytics scripts Feature Details Scalable Use as many or as few compute instances running Hadoop as you want. Modify the number of instances while your job flow is running Integrated with other services Works seamlessly with S3 as origin and output. Integrates with DynamoDB Comprehensive Supports languages such as Hive and Pig for defining analytics, and allows complex definitions in Cascading, Java, Ruby, Perl, Python, PHP, R, or C++ Cost effective Works with Spot instance types Monitoring Monitor job flows from with the management console Elastic MapReduce (EMR)
  40. 40. Foursquare… Founded in 2009 112M in Venture Capital 33 million users 1.3 million businesses using the service …generates a lot of Data 3.5 billion check-ins 15M+ venues, Terabytes of log data
  41. 41. Uses EMR for Evaluation of new features Machine learning Exploratory analysis Daily customer usage reporting Long-term trend analysis
  42. 42. Benefits of EMR Ease-of-Use “We have decreased the processing time for urgent data-analysis” Flexibility To deal with changing requirements & dynamically expand reporting clusters Costs “We have reduced our analytics costs by over 50%”
  43. 43. Production 1.3 Architecture
  44. 44. Production 1.3 Architecture Well-designed, 2 Tier architecture Highly Available due to Multiple Availability Zone Load Balancing & Auto-Scaling for full scalability Static content stored in durable, consistent way Improved end-user experience through CDN Big Data analytics built in for continuous optimization Capable of serving >1m-10M+ users
  45. 45. DEMO Getting to Scale
  46. 46. Thank you!
  47. 47. AWS Coding Challenge