Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

AWS Re:Invent - Optimizing Costs with AWS


Published on

AWS Re:Invent 2012 presentation from Netflix which covers how to optimize cost and usage of your AWS resources. Areas of focus are Autoscaling EC2 instances, batch access of SQS, and improved S3 usage.

Published in: Technology
  • Hi Mark,
    Unfortunately I hadn't checked my slideshare comments since before you posted your comment. If it's too late to present at the forum let me know, otherwise I would be open to ti.
    Are you sure you want to  Yes  No
    Your message goes here
  • Hi Coburn - this is a great presentation. Lots of very interesting patterns and practices related to the important role that software engineers need to play in improving data center resource efficiency. I'd be very interested in having you come present this information or be on a panel at the Green Grid Forum 2013, which will be held on March 5-6, 2013 at the Santa Clara Convention Center in Santa Clara, CA. I look forward to hearing from you!


    Board Member, The Green Grid
    Senior Director of Technology Strategy, Microsoft Sustainability.
    Are you sure you want to  Yes  No
    Your message goes here

AWS Re:Invent - Optimizing Costs with AWS

  1. 1. Optimizing Costs with AWS
  2. 2. Source:
  3. 3. • Rationale and High-level Methodology• AWS resource-specific optimizations• Performance Testing• Results• Q&A
  4. 4. Rationale• Applications operate at massive scale • Across three regions and multiple zones per region• Service oriented architecture • Many moving parts (teams)• Unconstrained deployment capabilities • “Freedom and Responsibility” culture
  5. 5. Rationale, cont.• Improve availability • Avoid saturation of key resources • Dynamically adjust capacity to meet workload demands• Plan for increasing workloads • Less focus on reducing current demand• Maximize efficiency • Balance OLTP and batch demands• “That which is measured improves”
  6. 6. • Asgard framework enables turnkey deployment (Netflix open-sourced) • All engineers have full access • Real-time reservation capacity • Unconstrained ASG size limits
  7. 7. • Birds-eye view of usage• Near real-time data• Open sourcing plans for tool• Decomposes by application
  8. 8. Unused Reservation Instance Hours *2,0001,500 Need to use this1,000 capacity 500 0 Mon Tue Wed Thu Fri Sat * - fictitious volumes
  9. 9. Healthy Thrashing Double-JumpY-axis = number of instances in ASG
  10. 10. Adopted batch deleteRequests/day Started batch send adoption Batch capabilities Adoption complete Time
  11. 11. Legend Github / Techblog Priam Exhibitor Servo and Autoscaling Scripts Apache Cassandra as a Service Zookeeper as a Service Contributions Astyanax Curator HonuTechblog Post Only Cassandra client for Java Zookeeper Patterns Log4j streaming to Hadoop Coming Soon CassJMeter EVCache Circuit Breaker - Hystrix Cassandra test suite Memcached as a Service Robust service pattern Cassandra Multi-region EC2 Eureka / Discovery Asgard - AutoScaleGroup datastore support Service Directory based AWS console Aegisthus Archaius Chaos Monkey Hadoop ETL for Cassandra Dynamics Properties Service Robustness verification Edda Explorers Latency Monkey Queryable config history Governator - Library lifecycle Server-side latency/error Janitor Monkey and dependency injection injection Odin REST Client + mid-tier LB Bakeries and AMI Workflow orchestration Blitz4j - Async logging Configuration REST endpoints Build dynaslaves
  12. 12. Netflix at 2012 re:InventDate/Time Presenter TopicWed 8:30-10:00 Reed Hastings Keynote with Andy JassyWed 1:00-1:45 Coburn Watson Optimizing Costs with AWSWed 2:05-2:55 Kevin McEntee Netflix’s Transcoding TransformationWed 3:25-4:15 Neil Hunt / Yury I. Netflix: Embracing the CloudWed 4:30-5:20 Adrian Cockcroft High Availability Architecture at NetflixThu 10:30-11:20 Jeremy Edberg Rainmakers – Operating CloudsThu 11:35-12:25 Kurt Brown Data Science with Elastic Map Reduce (EMR)Thu 11:35-12:25 Jason Chan Security Panel: Learn from CISOs working with AWSThu 3:00-3:50 Adrian Cockcroft Compute & Networking Masters Customer PanelThu 3:00-3:50 Ruslan M./Gregg U. Optimizing Your Cassandra Database on AWSThu 4:05-4:55 Ariel Tseitlin Intro to Chaos Monkey and the Simian Army
  13. 13. We are sincerely eager tohear your FEEDBACK on thispresentation and on re:Invent. Please fill out an evaluation form when you have a chance. Contact: