Your SlideShare is downloading. ×
0
Aws uk ug #8   not everything that happens in vegas stay in vegas
Aws uk ug #8   not everything that happens in vegas stay in vegas
Aws uk ug #8   not everything that happens in vegas stay in vegas
Aws uk ug #8   not everything that happens in vegas stay in vegas
Aws uk ug #8   not everything that happens in vegas stay in vegas
Aws uk ug #8   not everything that happens in vegas stay in vegas
Aws uk ug #8   not everything that happens in vegas stay in vegas
Aws uk ug #8   not everything that happens in vegas stay in vegas
Aws uk ug #8   not everything that happens in vegas stay in vegas
Aws uk ug #8   not everything that happens in vegas stay in vegas
Aws uk ug #8   not everything that happens in vegas stay in vegas
Aws uk ug #8   not everything that happens in vegas stay in vegas
Aws uk ug #8   not everything that happens in vegas stay in vegas
Aws uk ug #8   not everything that happens in vegas stay in vegas
Aws uk ug #8   not everything that happens in vegas stay in vegas
Aws uk ug #8   not everything that happens in vegas stay in vegas
Aws uk ug #8   not everything that happens in vegas stay in vegas
Aws uk ug #8   not everything that happens in vegas stay in vegas
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Aws uk ug #8 not everything that happens in vegas stay in vegas

400

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
400
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Not everything that happens in Vegas stays in Vegas
  • 2. DevOps or “getting devs to be on call for what they ship” :-)
  • 3. Netflix development Priorities 1. Speed of innovation 2. Availability 3. Running costs a. “It’ll cost what it ends up costing” In practise, they found that holding to the first two ended up costing way less than otherwise expected.
  • 4. Riot Games + League of Legends Cloud == ideal for MMOs. Solve launch issues. ● chef gets used a lot here. ○ talked about their evolution with it, lessons learned ● What sucked? ○ 25 minute bootstrap runs ○ External dependencies (including S3) ○ Duplicating application deployment recipes ● golden masters and immutable servers simplify your life drastically. ● “if you’re doing chef without BerkShelf you’re doing it wrong” ● Make it easy to throw up new things
  • 5. Testing in production Netflix, Riot, Kickstarter - they all do this. At scale. Netflix ● 10s to 100s of code pushes per day ● 1000s to 100,000s of config changes per day ○ they tune their A/B testing constantly Of course, they also have the instrumentation to react to this.
  • 6. How’re other people doing DevOps? Good news - we’re at the “more sophisticated” end of the spectrum. Every “cloud native” was doing this. Things other people did better: ● “Golden master” AMIs ● Immutable instances ● Absolute ownership of vertical slices ● Config-managment (chef/puppet) featured prominently ● Extensive monitoring+logs+visibility == “table stakes” ○ for developers! ● Easy to throw up new things ● Run many small, simple, collaborating things Who? Riot Games, Netflix, change.org, Kickstarter
  • 7. Logging aggregation is important
  • 8. Logging aggregation is important Lots of 3rd party companies are offering centralized logging services, there's a huge appetite for logging and monitoring. ● http://logentries.com/ ● http://www.loggly.com/ ● http://papertrailapp.com/ ● https://www.splunkstorm.com/tour ● http://www.datadoghq.com/ ● DIY - Lumberjacking slides
  • 9. DEMO: Monitoring & Logging https://app.datadoghq.com/infrastructure ● Tag Metrics, awesome Metric discoverability ● Cloud Watch integration ○ I never knew I could see ELB metrics :-) ● Alarms are integrated ● You can template Dashboards https://papertrailapp.com/ ● Can Search, Save Searches, Alerts on searches ● No alert on patterns ● Archive to S3 / Push to Redshift Logging aggregation is FOR DEVELOPERS!!! Saves lots of time when you’re on call.
  • 10. Loggly Session Benefit of logging as a service. ● When your infrastructure is in trouble, you do not want to have your logging analytic system on the same infrastructure. AWS Services that loggly could use: ● Kafka + Storm vs Kinesis ● Elastic Search vs Cloud Search Predictive Analytics using Storm, Hadoop, R and AWS http://www.youtube.com/watch?v=6Sl3eBmDheE
  • 11. Loggly Session ● Provisioned IOPS solve all issues :) ● ELB do not perform with extremely high volume of requests. ● DNS round robin is a very good basic load balancing solution ● Cassandra works very well for application data. ● Cassandra does not work well as a queue system, hard to track order of events. ● Keep the architecture simple.
  • 12. Large Scale Load Testing on AWS
  • 13. Many types of load ● Load testing ○ (running a marathon), predict future load and plan in advance ● Stress testing ○ Break things (figure out limits), mitigation plans ● Resilience test ○ Figure out how many parts of the architecture you can lose and still operate ● Performance test ○ How is latency and throughput changing when the load increase
  • 14. Phase roll out and measure ● Load Testing is necessary but not sufficient. ○ Deploy to alpha cluster. ○ The release cycle is important, phased deployment, one box, monitor and ramp up. ○ Monitor performance and behaviour, look at 99% of the traffic, not at the average. ● Netflix record 1.2 billion metrics per day ○ 5 minutes SLA
  • 15. Gameday
  • 16. Gameday We took part to the AWS Gameday http://www.awsgameday.com/whatisgameday.html Inspired by the 2012 Obama For America DevOps and Amazon.com ops teams ● Build an Autoscaling application ● Exchange administrative IAM credentials with other team ● Break your opponent's systems ● Restore your system ● Lessons learned
  • 17. Who is interested if we wanted to run this? It needs a full day, ~ 6 hours. Weekday? Weekend?
  • 18. Twitter: @petemounce

×