• Save
AWS Cloud Kata | Bangkok - Getting to Scale on AWS
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,203
On Slideshare
1,203
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Kuala Lumpur Ge#ng  to  Scale  on  AWS   Pattrawoot S, CTO, Wongnai.com
  • 2. •  •  •  •  Launched in 2010 Restaurant Review Website (Wongnai.com) Social Network for Food Lovers Key Features •  Location-based search for restaurants •  Write Reviews / Upload Photos / Eat Here (Check-in) •  Coupons / Guides / Articles •  Web & Mobile applications (iOS / Android / WP8 / BB10) •  > 820,000 users •  > 120,000 restaurants
  • 3. •  Number of Users •  Jan 2013: 360k users •  Mar 2013: 470k users (increase more than 100k) •  Number of User Contributions •  3x increase •  Dynamic and Personalized Contents •  Time to Scale!
  • 4. •  Amazon SES (Simple Email Service) •  > 500,000 email subscribers •  Over 1 million copies of e-newsletter sent every week •  Email blast ==> Bandwidth Consumption issue •  SEO performance •  Amazon EC2 (Elastic Computing Cloud) •  Past: One Server (Single Point of Failure) •  Single Physical Server ==> Multiple Virtual Servers •  Load Balancing
  • 5. •  Amazon RDS (Relational Database Service) •  MySQL •  Automated Backup •  Amazon S3 (Simple Storage Service) •  Versioning (S3 Object)
  • 6. Architecture Diagram www.wongnai.com Auto scaling Group Web App Server Elastic Load Balancing EC2 Instance Auto scaling Group Alarm Solr Server Data Volume Security Group EC2 Instance Data Volume Security Group EMM Server Amazon S3 Bucket Amazon RDS EC2 Instance Region Amazon SES
  • 7. What Has Been Improved •  •  •  •  •  Speed Scalability Flexibility Manageability Time / Cost Savings & Effectiveness Plan to Use: •  Elastic MapReduce (Hadoop) •  Simple Notification Service (SNS) •  ElastiCache
  • 8. Getting to Scale
  • 9. 503 Service Temporarily Unavailable The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.
  • 10. With AWS, scale from one instance…
  • 11. …to thousands Fully automated!
  • 12. How do I scale my architecture to support my first 10M users?
  • 13. “Think Big, Start Small, Scale Fast” Eric Ries, author of NY Times bestseller “The Lean Startup”
  • 14. Idea MVP Scale Profitability 01 02 03 04
  • 15. Getting to Scale By building a scalable Architecture to support your first 10M users
  • 16. 1.  Dev  &  Test   3.  Beta  Release   2.  Alpha  Release  
  • 17. Production 1.0 Architecture
  • 18. Database Options Self-Managed Database Server on Amazon EC2 Your choice of database running on Amazon EC2 Bring Your Own License (BYOL) Fully-Managed Amazon RDS Relational Database as a managed service Flexible licensing: BYOL or License Included Amazon DynamoDB Managed NoSQL database service using SSD storage Seamless scalability Zero administration
  • 19. But how do I choose what DB technology I need? SQL? NoSQL?
  • 20. Some folks won’t like this. But…
  • 21. Start with SQL databases
  • 22. But, but, but, but…
  • 23. No. You don’t.
  • 24. Start with SQL databases
  • 25. Why SQL? Established and well worn technology Lots of existing code, communities, books, tools, etc Clear patterns to scalability You aren’t going to break SQL DBs in your first 10 million users. No really, you won’t
  • 26. Amazon  Rela&onal  Database  Service  (RDS)   Feature   PlaAorm  support   Preconfigured   Automated  patching   Details   Create  MySQL,  SQL  Server  and  Oracle   Get  started  instantly  with  sensible  default   seAngs   Keep  your  database  plaEorm  up  to  date   automa&cally   Backups   Failover   •  Database-as-a-Service •  No need to install or manage database instances •  Scalable and fault tolerant configurations Automa&c  backups  and  point  in  &me   recovery  using  snapshots   Manual  DB  snapshots   Automated  failover  to  slave  hosts  in  event  of   a  failure   ReplicaFon   Easily  create  read-­‐replicas  of  your  data  and   seamlessly  replicate  data  across  availability   zones  
  • 27. Auto-Scaling Automatic resizing of compute clusters based on demand Feature   Amazon   CloudWatch   Trigger  auto-­‐scaling  policy   Details   Control   Define  minimum  and  maximum  instance  pool   sizes  and  when  scaling  and  cool  down  occurs.   Integrated  to  Amazon   CloudWatch   Use  metrics  gathered  by  CloudWatch  to  drive   scaling.   Instance  types   Run  Auto  Scaling  for  On-­‐Demand  and  Spot   Instances.  Compa&ble  with  VPC.   as-create-auto-scaling-group MyGroup --launch-configuration MyConfig --availability-zones us-east-1a --min-size 4 --max-size 200
  • 28. ProducFon  1.0   Architecture  
  • 29. Production 1.0 Architecture Well-­‐designed,  2  Tier  architecture   Highly  Available  due  to  Mul&ple  Availability  Zone   Load  Balancing  &  Auto-­‐Scaling  for  full  scalability   Fully  managed  Database  included     Capable  of  serving  >10K-­‐100Ks  users    
  • 30. BUT…
  • 31. Production 1.0 Architecture Wasted  server  capacity  for  sta&c  content   Reliability  and  durability  are  not  yet  op&mal   End-­‐user  experience  could  be  improved  thru   offloading  &  caching  
  • 32. SO…
  • 33. Let’s add Simple Storage Service (S3) CloudFront to optimize the end-user experience
  • 34. Simple Storage Service (S3) Feature   Flexible  object  store   Access  control   Server-­‐side  encrypFon   MulF-­‐part  uploads   Object  versioning   Object  expiry   Durable storage, any object 99.999999999% durability of objects Unlimited storage of objects of any type Up to 5TB size per object Access  logging   Web  content  hosFng   NoFficaFons   Import/Export   Details   Buckets  act  like  drives,  folder  structures  within   Granular  control  over  object  permissions   256bit  AES  encryp&on  of  objects   Improved  throughput  &  control   Archive  old  objects  and  version  new  ones   Automa&cally  remove  old  objects   Full  audit  log  of  bucket/object  ac&ons   Serve  content  as  web  site  with  built  in  page  handling   Receive  no&fica&ons  on  key  events   Physical  device  import/export  service  
  • 35.  CloudFront   •  World-wide content distribution network •  Easily distribute content to end users with low latency, high data transfer speeds, and no commitments Feature   Fast   Integrated  with  other  services   Dynamic  content   Streaming   Details   Mul&ple  world-­‐wide  edge  loca&ons  to  serve  content  as  close  to  your  users  as  possible   Works  seamlessly  with  S3  and  EC2  origin  servers   Supports  sta&c  and  dynamic  content  from  origin  servers   Supports  rtmp  from  S3  and  includes  support  for  live  streaming  from  Adobe  FMS  and  Microsod   Media  Server  
  • 36. Production 1.2 Architecture
  • 37. Production 1.2 Architecture Well-­‐designed,  2  Tier  architecture   Highly  Available  due  to  Mul&ple  Availability  Zone   Load  Balancing  &  Auto-­‐Scaling  for  full  scalability   Fully  managed  Database  included     Sta&c  content  stored  in  durable,  consistent  way   Improved  end-­‐user  experience  through  CDN   Capable  of  serving  >100K-­‐1M+  users  
  • 38. BUT…
  • 39. Production 1.2 Architecture You are now at Scale… …with lots of data… …and need to optimize continuously. But how and where?
  • 40. SO…
  • 41. Let’s add Big Data for analytics of web, mobile, gaming, and log data
  • 42. Multiple managed AWS services for Big Data
  • 43. Elastic MapReduce (EMR) •  Managed, elastic Hadoop cluster •  Integrates with S3 & DynamoDB •  Leverage Hive & Pig analytics scripts Feature   Scalable   Integrated  with  other   services   Comprehensive   Cost  effecFve   Monitoring   Details   Use  as  many  or  as  few  compute  instances  running  Hadoop  as  you  want.  Modify  the  number  of   instances  while  your  job  flow  is  running     Works  seamlessly  with  S3  as  origin  and  output.  Integrates  with  DynamoDB   Supports  languages  such  as  Hive  and  Pig  for  defining  analy&cs,  and  allows  complex  defini&ons  in   Cascading,  Java,  Ruby,  Perl,  Python,  PHP,  R,  or  C++   Works  with  Spot  instance  types   Monitor  job  flows  from  with  the  management  console  
  • 44. Foursquare… …generates a lot of Data Founded in 2009 112M in Venture Capital 33 million users 1.3 million businesses using the service 3.5 billion check-ins 15M+ venues, Terabytes of log data
  • 45. Uses EMR for Evaluation of new features Machine learning Exploratory analysis Daily customer usage reporting Long-term trend analysis
  • 46. Benefits of EMR Ease-of-Use “We have decreased the processing time for urgent data-analysis” Flexibility To deal with changing requirements & dynamically expand reporting clusters Costs “We have reduced our analytics costs by over 50%”
  • 47. Production 1.3 Architecture
  • 48. Production 1.3 Architecture Well-­‐designed,  2  Tier  architecture   Highly  Available  due  to  Mul&ple  Availability  Zone   Load  Balancing  &  Auto-­‐Scaling  for  full  scalability   Sta&c  content  stored  in  durable,  consistent  way   Improved  end-­‐user  experience  through  CDN   Big  Data  analy&cs  built  in  for  con&nuous  op&miza&on   Capable  of  serving  >1m-­‐10M+  users  
  • 49. DEMO   Ge#ng  to  Scale  
  • 50. Thank You aws.amazon.com/start-­‐ups   aws.amazon.com/acFvate/cloudkata