Chicago AWS user group meetup - May 2014 at Cohesive

502 views

Published on

All slides from the May 2014 Meetup. Talks included:
• "Mining crypto currency on AWS spot instance" - Scott VanDenPlas, Engineer at el el see @scottvdp
• "HA for healthcare" - Ryan Koop, Director of Products & Marketing, Cohesive @ryankoop
• "Using AWS for HA at BrightTag" - Matt Kemp, Engineer of Things™ at BrightTag @mattkemp
• So nice, he's talking twice. - Scott VanDenPlas, Engineer at el el see @scottvdp

Join us again June 24 at Mediafly and in July back at Cohesive!

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
502
On SlideShare
0
From Embeds
0
Number of Embeds
14
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Chicago AWS user group meetup - May 2014 at Cohesive

  1. 1. Sponsored by Hosted by Chicago AWS user group - May 2014 ! ! “Using AWS for High Availability” #AWSChicago
  2. 2. Organizer ! Margaret Walker
 CohesiveFT ! ! Tweet: @MargieWalker
 #AWSChicago Sponsored by Hosted by #AWSChicago
  3. 3. Mark your calendars - next AWS user group June 24
 Security in AWS
  4. 4. 6:00 pm Introductions 6:10 pm Lightning Talks "Mining crypto currency on AWS spot instance" - Scott VanDenPlas, Engineer at el el see @scottvdp "HA for healthcare" - Ryan Koop, Director of Products & Marketing, CohesiveFT @ryankoop "Using AWS for HA at BrightTag" - Matt Kemp, Engineer of Things™ at BrightTag @mattkemp So nice, he's talking twice. - ScottVanDenPlas, Engineer at el el see @scottvdp 6:30 pm Q & A 7:00 pm Networking, drinks and pizza Agenda Sponsored by Hosted by #AWSChicago
  5. 5. “Mining crypto currency on AWS spot instance” ! ScottVanDenPlas, Engineer at el el see ! Tweet: @scottvdp
 #AWSChicago ! Sponsored by Hosted by #AWSChicago
  6. 6. AWS Spot Market Arbitrage Or How I Stopped Worrying and Learned to Love the Bid
  7. 7. Scott VanDenPlas ! scott@ elelsee.com
  8. 8. awsofa.info
  9. 9. A (crappy) Primer to Magic Internet Money
  10. 10. You cannot make money doing this. (currently.)
  11. 11. There was a time you could. Five months ago.
  12. 12. g2.2xlarge New generation. Single GPU.! ! CPU 34.09 khash/s! GPU 196.08 khash/s! ! 230.17 khash/s! $0.650 per Hour
  13. 13. cg1.4xlarge Old generation. Dual GPU.! ! CPU 52.51 khash/s! GPU 311.4 khash/s! ! 363.91 khash/s! $2.100 per Hour
  14. 14. Earning Potential. g2.2xl $0.076 hourly. cg1.4xl $0.120 hourly.
  15. 15. Uh… not so much. g2.2xl $0.076 hourly. cg1.4xl $0.120 hourly. ! ! On Demand! g2.2xl $0.650 hourly. cg1.4xl $2.100 hourly.
  16. 16. Spot Instance Pricing Region! ! Availability Zone! ! Account! ! Instance Type! ! Operating System! ! VPC
  17. 17. No amount of money makes it worth it to run Windows.
  18. 18. AMI Defenestration. ! Yep, it is possible.
  19. 19. Proof.
  20. 20. Now I need 3000 of these.
  21. 21. Advice from my lawyer. ! We are not legally laundering money from! our AWS Partner Account.! ! ! I am not implying that ever occurred.
  22. 22. @scottvdp /in/scottvdp
  23. 23. “HA for healthcare” ! Ryan Koop, Director of Products & Marketing, CohesiveFT ! Tweet: @ryankoop
 #AWSChicago Sponsored by Hosted by #AWSChicago
  24. 24. @ryankoop Healthcare HA in AWS AWS User Group May 29, 2014 1
  25. 25. @ryankoop Oh, hello 2 During Business Hours++ Ryan Koop Director of Products & Marketing, Co-founder @ryankoop www.linkedin.com/in/rkoop/ After Hours NAME Ryan Koop CLUB Royal Fox CC - Men LOCAL# 2024 Assoc# 20005661 EFFECTIVE DATE 10/15/2013 SCORES POSTED 12 USGA HDC INDEX 18.9 SCORE HISTORY - MOST RECENT FIRST 1 96*I 98 I 95*I 89*AI 96*AI 6 95*AI 99 H 99 I 99 AI 94*I 11 97 H 96*I 106 A 97 H 95 H 16 97 I 94*H 91*H 96 I 94*H Chicago District Golf Association - www.cdga.org Ryan Koop 2013 GOLD MEMBER
  26. 26. @ryankoop 5/26/14 US-West-1 Single Availability Zone looses power 5/17/14 US-West-2 Increased Launch Error Rates 4/30/14 US-West-2 Connectivity Issues for Single Availability Zone 4/22/14 EU-West-1 Connectivity Issues for Single Availability Zone 4/16/14 EU-West-1 Increases API Error Rates 4/1/14 US-West-1 Connectivity Issues for Single Availability Zone 3/21/14 US-East-1 Increased API Error Rates 3/20/14 US-East-1 Increased API Error Rates 3/20/14 US-West-2 Increased API Error Rates 3/9/14 US-East-1 Connectivity Issues for Single Availability Zone 3 Cloud ≠ Reliability Source:AWS Appstream RSS
  27. 27. @ryankoop AWS SLA - Five 9s? 4 99.95% = ~22min/month Downtime “Region Unavailable” | Burden of Proof | “Demarcation Point” Yo Dawg, we heard you like SLAs So we gave your SLA an SLA!
  28. 28. @ryankoop 5 AWS Data Center | Source: AWS James Hamilton Amazon Perdix | Source: AWS James HamiltonSource:Your Nightmares Source: Your Nightmares You vsThem
  29. 29. @ryankoop Enough of the FUDD 6 Source:Warner Bros.
  30. 30. @ryankoop AWS and HA 7 Region Availability Zone
  31. 31. @ryankoop 8 The H in HA Stands for Hybrid Public A Public B Public Private Public Data Center Source: Chris Swan, CTO CohesiveFT
  32. 32. @ryankoop Hybrid Strategies 9 VPC 2VPC 1 peer PeeredVPCs Common Software Stack public privatepublic private Single Pane of Glass public public Common APIs private Source: Chris Swan, CTO CohesiveFT
  33. 33. @ryankoop Slide Sponsored by: cccccccccccc 10 US Central 1a Customer Data CenterCustomer Remote Office VNS3 1 VNS3 2 VNS3 3 VNS3 Overlay Network Server 1 Server 2 DB 1 DB 2 Server 3 DB 3 Active IPsecTunnel Failover IPsecTunnel Firewall / IPsec Cisco 5505 Firewall / IPsec Cisco 5585 Data Center ServerData Center ServerUser WorkstationUser Workstation Peered Peered US East 1a US West 2b
  34. 34. @ryankoop The future (or now) is loosely coupled 11 Load Balancers Web Servers Load Balancers App Servers Database Cluster
  35. 35. @ryankoop AWS and HIPAA ! • Health InformationTechnology for Economic and Clinical Health (HITECH) Act in 2009 and the 2010 Omnibus rule • Business Associate Agreement - June 18, 2013 • EBS Encryption - May 21, 2014 12 1996 - Privacy, Security, and Breach Notification rules for the storage & transmission of EHI
  36. 36. @ryankoop Shared Responsibility 13 Layer 3 ! Layer 2 ! Layer 1 ! Layer 0 Layer 7 Layer 6 Layer 5 Layer 4 Layer 3 Application Layer Virtual Layer Limit of user access, control and visibility ApplicationOwner CloudOwner Hardware Layer Your HIPAA Compliant App AWS Xen Hypervisor It’s Intel-based but Secret
  37. 37. @ryankoop HIPAATopology 14 IPsec Firewall / IPsec us-east-1c us-east-1d Multi-tenant Overlay Primary Peered VNS3 Manager us-west-2a Multi-tenant Overlay Backup User Workstation NewYork, NY Data Center Server Healthcare Provider 2 Healthcare Provider 1 San Francisco, CASeattle,WA Denver, CO Data Center Server Boston, MA HIPAA App Provier Data Center Data Center Server Data Center Server Healthcare Provider 3 Healthcare Provider N DR Tablet
  38. 38. @ryankoop Zone Failure 15 IPsec us-east-1c us-east-1d Multi-tenant Overlay Primary Peered VNS3 Manager us-west-2a Multi-tenant Overlay Backup X User Workstation NewYork, NY Data Center Server Healthcare Provider 2 Healthcare Provider 1 San Francisco, CASeattle,WA Denver, CO Data Center Server Boston, MA HIPAA App Provier Data Center Data Center Server Data Center Server Healthcare Provider 3 Healthcare Provider N DR Tablet Firewall / IPsec
  39. 39. @ryankoop Regional Failure IPsec us-east-1c us-east-1d Multi-tenant Overlay Primary Peered VNS3 Manager us-west-2a Multi-tenant Overlay Backup X X User Workstation NewYork, NY Data Center Server Healthcare Provider 2 Healthcare Provider 1 San Francisco, CASeattle,WA Denver, CO Data Center Server Boston, MA HIPAA App Provier Data Center Data Center Server Data Center Server Healthcare Provider 3 Healthcare Provider N DR Tablet Firewall / IPsec
  40. 40. @ryankoop Global Failure 17 IPsec US Central Multi-tenant Overlay Cold User Workstation NewYork, NY Data Center Server Healthcare Provider 2 Healthcare Provider 1 San Francisco, CASeattle,WA Denver, CO Data Center Server Boston, MA HIPAA App Provier Data Center Data Center Server Data Center Server Healthcare Provider 3 Healthcare Provider N DR Tablet Firewall / IPsec
  41. 41. @ryankoop ThreeThings for HA 1. Rigorous automation of virtual servers
 2. Rigorous automation of boot time context
 3. Overlay network that quickly, simply differentiates network location from identity 18
  42. 42. @ryankoop ThankYou 19 Questions?
  43. 43. “Using AWS for HA at BrightTag” ! Matt Kemp, Engineer of Things at BrightTag ! Tweet: @mattkemp
 #AWSChicago Sponsored by Hosted by #AWSChicago
  44. 44. Using AWS for HA @ BrightTag Matthew Kemp
  45. 45. Everything Fails Eventually Network splits Instances go down AWS Availability Zones go offline AWS Regions go offline
  46. 46. Cascading Failures Keep failures self contained
  47. 47. Design for Failure Run multiple instances Run in multiple Availability Zones Run in multiple Regions
  48. 48. Redundancy Database Cluster Data Access Service Web Availability Zone A Availability Zone B Region
  49. 49. Local, Local, Local Web haproxy stats Data Access Service Graphite Carbon Region
  50. 50. Zero Downtime Deploys + ++
  51. 51. Instances in 2011 We ran in two regions with ~40 instances One had the minimum of two instances per app The other was only slightly larger
  52. 52. Instances in 2014 We run in four regions with ~600 instances Largest region is ~240 instances Smallest region is ~70 instances
  53. 53. Questions?
  54. 54. Contact Info matt@brighttag.com @mattkemp /in/matthewkemp
  55. 55. “I’ve got 99 problems and capacity is all of them” ! ScottVanDenPlas, Engineer at el el see ! Tweet: @scottvdp
 #AWSChicago ! Sponsored by Hosted by #AWSChicago
  56. 56. Scott ! VanDenPlas! ! scott@ elelsee.com
  57. 57. http://awsofa.info
  58. 58. I’ve got 99 problems and capacity is all of them.
  59. 59. I’ve got 98 problems and capacity is all of them.
  60. 60. http://alive.training
  61. 61. Q & A ! ! Pizza’s almost here! ! ! Sponsored by Hosted by #AWSChicago
  62. 62. Sponsored by Hosted by #AWSChicago

×