Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Technological Accelerants for Organizational Transformation - DVC303 - re:Invent 2017

314 views

Published on

Developers and management can seem at cross purposes when one group looks at technologies and the other looks at organizational issues. Both groups are looking for ways to deliver value faster, leaner, and at less cost. There are technological avenues for accomplishing these goals, including DevOps and serverless architectures. However, these approaches also have organizational implications, as they change the nature and content of communication between teams. In this session, we cover the technology benefits and organizational transformations involved in DevOps and serverless architectures.

This session is part of the re:Invent Developer Community Day, six community-led sessions where AWS enthusiasts share technical insights on trending topics based on first-hand experiences and knowledge shared within local AWS communities.

  • Be the first to comment

  • Be the first to like this

Technological Accelerants for Organizational Transformation - DVC303 - re:Invent 2017

  1. 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS re:INVENT DVC303: Technological Accelerants for Organizational Transformation
  2. 2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Ant Stanley Organizer Serverless London User Group, co-founder JeffConf Paul Duvall CTO Stelligent, AWS Community Hero Ben Kehoe Chief Cloud Robotics Research Scientist, AWS Community Hero
  3. 3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “There is no such thing as a new idea.” Mark Twain
  4. 4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Ephemeralization - “more and more with less and less until eventually you can do everything with nothing” Buckminster Fuller
  5. 5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Everything is getting smaller
  6. 6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Unit of Deployment Entire Machine Operating System Virtual Machine Container Function
  7. 7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Time to Deploy Years Months Days Hours Minutes Seconds
  8. 8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Average Lifespan > 5 Years < 5 Years < 3 Years Days Seconds Entire Machine Operating System Virtual Machine Container Function
  9. 9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Frequency of Deployment Years Months Days Hours Minutes
  10. 10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Scope of Change Entire Platform Operating System Major Version Minor Version Commit
  11. 11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Jidoka – Automation with a human element Invented by Sakichi Toyoda circa 1902
  12. 12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Change Advisory Board Network Security Storage Sysadmin
  13. 13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Network Security Storage Sysadmin
  14. 14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  15. 15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Teams replaced by services Consistent standardized interface API driven Idempotent
  16. 16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Large array of services automated by a small team Focus on delivering value Takes full responsibility for service
  17. 17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  18. 18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  19. 19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “AWS will release a better feature than your horrific workaround” Quinn’s Law, 2017
  20. 20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure” Mel Conway (Conway’s Law, 1968)
  21. 21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  22. 22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  23. 23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  24. 24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “Your AWS Organization structure will replicate your company’s communication structure” Conway’s Law for AWS (2017)
  25. 25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  26. 26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. New (old) challenges
  27. 27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  28. 28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Persephone (FSV304) Creates accounts, sets policy, defines which services are enabled in the account, and runs Lambda functions to ensure continuous compliance. Cloud Custodian Custodian can be used to manage AWS accounts by ensuring real time compliance to security policies (like encryption and access requirements), tag policies, and cost management via garbage collection of unused resources and off-hours resource management. https://github.com/capitalone/cloud-custodian
  29. 29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DevOps Transformation Examples P a u l D u v a l l
  30. 30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. About Me @PaulDuvall AWS Community Hero Continuous Integration @Stelligent
  31. 31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DEPLOYMENT PIPELINE FEEDBACK LOOPDevelopers Customers Based on http://www.slideshare.net/AmazonWebServices/dvo202-devops-at-amazon-a-look-at-our-tools-processes ≈≈≈≈≈ DevOps is About Feedback Build Test Release Plan Monitor
  32. 32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Organizational Cultural Process Tooling Principle: Feedback
  33. 33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Four DevOps Transformation Examples 1 2 3 4 Breaking Down Silos Identifying Process Bottlenecks Making Work Visible Fixing Problems Early https://stelligent.com/dvc303
  34. 34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. App Release Management QA/Testers Production Support DBAs App Developers UI/UX Analyst Build Gate(s) Gate(s) Gate(s) Change Advisory Board Security Gate(s) Problem: How to Breakdown Silos?
  35. 35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Product Security Tooling Spend Management Continuity Management API API API API App Developers UI/UX Analyst Test/QA Infrastructure Build | Test | Deploy | Run | Support Example: Organize Around API-Enabled Services
  36. 36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Example: API-Enabled Services https://github.com/stelligent/infrastructure-api Image from https://aws.amazon.com/blogs/compute/how-to-provision-complex-on-demand-infrastructures-by-using-amazon-api-gateway-and-aws-lambda/ Amazon API Gateway Lambda Function CloudFormation Business-Function Template CloudFormation Stacks 1 2 3 4
  37. 37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Problem: How to Make Work Visible
  38. 38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Example: Amazon CloudWatch Dashboard
  39. 39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Example: Amazon CloudWatch Dashboard CodePipeline CloudWatch Event Rule Lambda Function CloudWatch Metrics CloudWatch Dashboard Public Events Target Put-Metric Get- Statistics CloudWatch Event Rule Lambda Function CloudWatch Scheduled Event CloudWatch Dashboard List-Metrics Trigger Put- Dashoard
  40. 40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Community: Pipeline Dashboard https://github.com/stelligent/pipeline-dashboard
  41. 41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. VALUE WASTE 5 min 15 min ½ week 2 min 2 weeks 15 min 2 weeks 2 hours 1 week 15 min 3 hrs 45 min ½ week 3 min 2 hrs 40 min 6 weeks + 4 hrs Extra 15 minutes to fill out request form Only 15 minutes of 4 hours should be needed to verify Source: Implementing Lean Software Development: From Concept to Cash. Boston: Addison-Wesley Professional; 2006, ISBN: 0321437381 Problem: How to Identify Process Bottlenecks? Request Approve and Prioritize Technical Assessment Code and Test Verify and Fix Depoy Form Sent to Queue To Verification To Operations 1% Efficiency Weekly review of requests means an average wait of ½ week Wait an average of 2 weeks for an architect Wait an average of 2 weeks for developers Biweekly releases means a wait of an average of 1 week for verification Form Sent to Queue Form Sent to Queue
  42. 42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Example: Value-Stream Map as a Pipeline
  43. 43. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Example: CodePipeline in CloudFormation My SNS Tpoic Topic CodeBuild Role Lambda Role CodePipeline Role Bucket Bucket CodeBuild Project CodeBuild Role CodePipeline Parameter EventRule Rule CloudFormati on Role CodePipeline Pipeline
  44. 44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Community: CodePipeline Mocks https://github.com/stelligent/mu https://github.com/stelligent/mock-pipeline
  45. 45. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Problem: How to Fix Problems Early?
  46. 46. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Solution: Stop the Line
  47. 47. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Community: CodePipeline SNS Notifications https://github.com/stelligent/cfn_nag https://github.com/stelligent/devops-essentials/tree/master/samples/ec2
  48. 48. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Additional Resources https://stelligent.com/dvc303 1 2 3 4 Breaking Down Silos Identifying Process Bottlenecks Making Work Visible Fixing Problems Early
  49. 49. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. iRobot B e n K e h o e
  50. 50. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Ben Kehoe • Cloud Robotics Research Scientist at iRobot • Serverless evangelist • AWS Community Hero About me @ben11kehoe
  51. 51. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  52. 52. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Cloud Robotics: Connecting robots to the internet to help them do more and better things
  53. 53. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 2015
  54. 54. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  55. 55. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  56. 56. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  57. 57. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Then
  58. 58. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Then Now
  59. 59. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Then Now Future
  60. 60. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Then Now Future 2015
  61. 61. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Event-driven • Scalable • Lean for device makers • Reverse: AWS Greengrass • Focus IoT + serverless: a natural fit
  62. 62. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS IoT + serverless: an even naturaler fit
  63. 63. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Serverless itself • Event-driven • Scalable • Integrates with AWS ecosystem • Device shadows • Integrates with your process AWS IoT
  64. 64. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Fully serverless production cloud • 2 million connected robots by 2018 • Mostly serverless analytics platform • Basis for future data-powered platform Long story short: success!
  65. 65. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • No provider is immune to problems • Small effects are more common than big outages • More services = blips could be encountered more frequently • This comes with the territory • Set expectations internally • Architecting robustly is key The cloud has weather
  66. 66. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • You only know what the provider tells you • Architecture • Security • Operations • How do they actually do all of the stuff they do? • Many known unknowns and unknown unknowns • Unknown unknown unknowns: what you don’t know that they don’t know they don’t know Visibility
  67. 67. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Heed the warning • Information flow is different in serverless architecture • Organization must change for architecture to succeed Conway's Law https://commons.wikimedia.org/wiki/File:Gospers_glider_gun.gif
  68. 68. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Developers can be platform testers and canaries • Same environment in the platform as production • Orders of magnitude more churn • Exercises the account limits • Tests metrics to determine relevance and meaning • Bonus – Developer activity provides additional visibility into how the platform is currently behaving • Higher volume of deployments in many different AWS accounts means problems found quickly • This can alert us prior to problems hitting prod DevOps
  69. 69. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Servers → serverless is like on-prem → cloud • Easier overall and in most respects • But also includes new challenges • Outsourcing doesn’t mean you do zero work • Being clear about this organizationally is critical DiffOps
  70. 70. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Everything is getting smaller • How DevOps and Serverless affect and improve organizational transformation • Try some the examples: stelligent.com/dvc303 • Become part of the AWS Community Summary and Next Steps
  71. 71. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Q&A

×