Your SlideShare is downloading. ×
0
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Scaling a Mobile Web App to 100 Million Clients and Beyond (MBL302) | AWS re:Invent 2013

512

Published on

Mobile apps have different service requirements from their desktop and web-based analogs. Bandwidth, client processing, and other considerations can impose significant extra demands on a scalable …

Mobile apps have different service requirements from their desktop and web-based analogs. Bandwidth, client processing, and other considerations can impose significant extra demands on a scalable service. This session is a technical discussion of the challenges Flipboard met while scaling a data-intensive mobile app from 0 to 100 million clients and how they are working on scaling 10x using AWS. At each major step, Flipboard has encountered many challenges. Learn about how they handled those challenges and the evolution of their systems architecture, design choices, and software selection.

Published in: Technology, Travel, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
512
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
26
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Scaling a Mobile Web App to 100 Million Clients and Beyond Joey Parsons @joeyparsons November 14th, 2013 © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc. Friday, November 15, 13
  • 2. Friday, November 15, 13
  • 3. YOUR PERSONAL MAGAZINE Friday, November 15, 13
  • 4. The ultimate way to discover, consume & share content on the mobile, social web Wednesday, June 5, 13 Friday, November 15, 13
  • 5. Friday, November 15, 13
  • 6. Friday, November 15, 13
  • 7. Friday, November 15, 13
  • 8. How are mobile apps different? • WiFi vs Slow connectivity • Variances in bandwidth and global carriers • Taking advantage of the local cache • Control your behavior during latency • Fast devices — significant opportunity for client computation Friday, November 15, 13
  • 9. Prototype Phase: From 0 to 1M users Friday, November 15, 13
  • 10. - Amazon EC2 - Amazon S3 - Amazon RDS Friday, November 15, 13
  • 11. The Initial Launch Night Friday, November 15, 13
  • 12. Things we should have done… • Make sure to prepare for Amazon limits if you need to scale quickly • Make sure your external partners understand the volumes you’ll be accessing them Friday, November 15, 13
  • 13. Challenges • Understanding the scale of our services • Little to no insight into performance • Beginning to build out tooling for Amazon EC2 but still in its infancy • No centralized logging or way of detecting errors Friday, November 15, 13
  • 14. Getting Started: From 1M to 10M Users Friday, November 15, 13
  • 15. - Amazon EC2 - Amazon RDS - Amazon S3 - Amazon CloudFront Friday, November 15, 13
  • 16. Architecture Changes • Different services have different scale profiles — began the shift towards microservices • Image content moved to CloudFront • Moved primary data store to MySQL via Amazon RDS • Home grown bash scripts for deploys • Focus on instrumentation • Logging, Metrics, Monitoring followed suit Friday, November 15, 13
  • 17. SimpleDB for CMDB Host[i-76e33611] name[tsd01] owner[ops] service[OOS] service ami[ami-3f622f56] type[m1.xlarge] loc[us-east-1a] role[flops] subclass[opentsdb] group[0] pool[production] public[50.16.58.220] private[10.60.43.18] Friday, November 15, 13 Amazon Instance ID - Name of the instance Who owns the instance? IS for in service, OOS for out of What AMI was used Type of EC2 instance Region and Availability Zone - Type of role Subclass of role Group of node - Production, staging, dev - Public IP address - Private IP Address
  • 18. Querying our CMDB # fl-inst-describe -r flip -p production -g 0 -s IS -o ops Domain[flipboard.prod.instances] has count[1] hosts meeting criteria ======================================= Servers of role flip ======================================= Host[i-5b8ae323]: name[flip05] owner[ops] service[IS] public[54.226.44.212] private[10.78.167.211] role[flip] group[0] pool[production] subclass[standard] type[c1.xlarge] Friday, November 15, 13
  • 19. The iPhone Launch Night Friday, November 15, 13
  • 20. Scaling Fast: 10M to 100M Users Friday, November 15, 13
  • 21. Storm Kafka Graphite Kibana Friday, November 15, 13
  • 22. Architecture Changes • Heavy focus on instrumentation of all services • Pipeline of batch processing using Hadoop • Pipeline of real-time processing using Storm + Kafka • Keen focus on using appropriately sized EC2 instances • Moving off of bash scripts, moving to puppet Friday, November 15, 13
  • 23. Mobile application instrumentation Friday, November 15, 13
  • 24. Deploy by groups All at once? fl-inst-upgrade -r flip -p production-q … or … By group? fl-inst-upgrade -r flip -p production -g 0 -q Friday, November 15, 13
  • 25. Using CloudWatch metrics for errors Friday, November 15, 13
  • 26. Continued your deploy fl-inst-upgrade -r flip -p production -g 1 -q Friday, November 15, 13
  • 27. Graphite for all metrics Friday, November 15, 13
  • 28. Millions of metrics with Graphite Friday, November 15, 13
  • 29. d3.js + cubism.js Friday, November 15, 13
  • 30. Monitoring via CloudWatch Alarm in PagerDuty Details available in PagerDuty Friday, November 15, 13
  • 31. Lessons Learned • Use Amazon services when possible (Amazon RDS, Amazon Redshift, Amazon Route 53) • Use SSDs where applicable • Understand your scale and your needs going forward and invest in Reserved Instances (3 years!) • But, allow flexibility for changing needs and instance types Friday, November 15, 13
  • 32. Amazon Technologies Used • Amazon CloudFront • Amazon Route 53 • Amazon EC2 • Amazon S3 • Amazon Redshift Friday, November 15, 13 • Amazon RDS • Amazon SimpleDB • Amazon SQS • ElastiCache • Amazon CloudWatch
  • 33. Beyond: From 100M Users to 1B! Friday, November 15, 13
  • 34. What’s next? • Better use of Auto Scaling groups • Predictive analytics — lots of signals • Automated remediation • Heavy focus on using the right instance types for each service • Take advantage of new AWS products Friday, November 15, 13
  • 35. The unknown is exciting … Friday, November 15, 13
  • 36. Questions? Friday, November 15, 13
  • 37. AWS re:Invent 2013 Magazine http://flip.it/NSNEi Friday, November 15, 13
  • 38. Please give us your feedback on this presentation MBL302 As a thank you, we will select prize winners daily for completed surveys! Friday, November 15, 13 Thank You

×