Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

DEV332_Using AWS to Achieve Both Autonomy and Governance at 3M

675 views

Published on

There is a constant tension between empowering teams to be agile through autonomy and enforcing governance policies to maintain regulatory compliance. Hear from Nathan Scott, Senior Consultant at AWS and James Martin, Automation Engineering Manager at 3M on how they have achieved both autonomy and governance through self-service automation tools on AWS. Learn how to avoid pitfalls with building the CI/CD team, right sizing and how to address. This session will also feature a demo from Casey Lee, Chief Architect at Stelligent on the tools used to accomplish this for 3M, including AWS Service Catalog, AWS CloudFormation, AWS CodePipeline and Cloud Custodian, an open source tool for managing AWS accounts.

  • Be the first to comment

DEV332_Using AWS to Achieve Both Autonomy and Governance at 3M

  1. 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Using AWS to Achieve Both Autonomy and Governance at 3M N a t h a n S c o t t , S e n i o r C o n s u l t a n t , C l o u d A r c h i t e c t , A W S J a m e s M a r t i n , M a n a g e r , A u t o m a t i o n E n g i n e e r i n g , 3 M C a s e y L e e , C h i e f A r c h i t e c t , S t e l l i g e n t AWS re:INVENT D E V 3 3 2 N o v e m b e r 2 8 , 2 0 1 7
  2. 2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. The problem We have to move FAST, give us access… Individual business and development teams needed the ability to move fast and self-serve to capture market opportunities Not so fast, there are rules… The organization as a whole needs governance to ensure security compliance and minimize risk
  3. 3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What to expect We will hear firsthand from About • How we solved the problem and achieved balance between autonomy and governace at 3M
  4. 4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What to expect Topics • People, process, technology • Governance requirements • Solution approach using: • Continuous delivery • Self-service model • Monitoring of resources
  5. 5. 5 Historical business Our legacy 1983–2011 Helping healthcare organizations get complete and accurate reimbursement and mitigate compliance risks Streamlining and simplifying the process of documenting the patient’s encounter in a hospital Working with hospitals to efficiently access, compile, code, classify, report, store, and exchange health information
  6. 6. 6 Leading in a changing landscape Our present course and future Analyzing the cost, quality, and outcomes data of both patients and populations over time and across the healthcare continuum Ensuring providers capture the full burden of illness of their patients to deliver effective care management and receive accurate and complete payment Measuring performance and effectiveness among payer and provider networks to deliver higher quality outcomes at lower total costs
  7. 7. 73M Confidential. 3M HIS grouper applications 22 states (27 grouper adoptions) through 1983–2006 11 additional states (37 grouper adoptions) 2007–2010 6 additional states (33 grouper adoptions) 2011–Q3 2012 • Industry-recognized expertise in payment methodologies and patient classification • 24 states have adopted APR DRGs for payment, including the eight largest Medicaid programs in the country • The APR DRG adoption by payers typically yields over 75% downstream penetration with providers • Lays a foundation for further payment products 87% of the US population is covered by 3M patient classification systems
  8. 8. 8 Not moving fast enough Lift and shift got us out of the traditional data center, but… Lots of software is getting built with nowhere to go, so it’s time to evolve again.
  9. 9. 9 Development bottlenecks Development time Manual testing Manual QA Manual deployment
  10. 10. 10 Desired bottlenecks Development time Manual testing Manual QA Manual deployment
  11. 11. Deployment pipeline Feedback loop plan monitor build test release Developers Customers Based on slideshare.net/AmazonWebServices/dvo202-devops-at-amazon-a-look-at-our-tools-processes Continuous delivery
  12. 12. 12 The path to continuous delivery
  13. 13. 13 Building the automation team Automation engineering team • Deep knowledge of AWS services • Comfortable talking to other development teams • Understands the complete development lifecycle—from commit to deploy
  14. 14. 14 Choosing the right technology • Focus on the problem at hand • Don’t try to predict the future • Use native AWS services/AWS Lambda/software as a service (Saas) services
  15. 15. 15 Working with security • Gain buy-in early • Security from the start • Security as consumers • Freedom (with guard rails) • Sensitive data
  16. 16. 16 • Find a simple application • Just enough to prove your pipeline • Rinse, repeat The right services and teams
  17. 17. 17 The right services and teams Find the hungry team that • Wants the power • Is willing to do the work • Has a champion • Has the business need
  18. 18. 18 Embed with the AppDev team • Establish success criteria • Works closely with application team • Participates in the team’s sprint cycle • Helps AppDev team consume the pipeline process and tools AppDev team Automation engineering
  19. 19. 19 Establishing a CI/CD process at scale Problems • Complex components • Special snowflakes • Limited governance
  20. 20. 20 The pipeline factory Goals • Reduce barrier to entry • Reduce snowflakes • Reduce setup time • Enforce security controls
  21. 21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Solution Self-serviceContinuous delivery Monitor
  22. 22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Solution Self-serviceContinuous delivery Monitor
  23. 23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Continuous delivery
  24. 24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Continuous delivery
  25. 25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS CodePipeline Continuous delivery
  26. 26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS CodeCommit Continuous delivery
  27. 27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. mavenJob(jobName) { triggers { scm('* * * * *') } rootPOM('pom.xml') goals('clean') goals('compile') goals('pmd:pmd') goals('findbugs:findbugs') goals('package') } Continuous delivery
  28. 28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Jenkins Continuous delivery
  29. 29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Approved? Continuous delivery
  30. 30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS CloudFormation Continuous delivery
  31. 31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS CodeDeploy Continuous delivery
  32. 32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Automated testing Continuous delivery
  33. 33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Blue/green switch Continuous delivery ELB Old ASG New ASG
  34. 34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Solution Self-serviceContinuous delivery Monitor
  35. 35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Self-service
  36. 36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. docs.aws.amazon.com/servicecatalog/latest/adminguide/admin-overview-workflow.html AWS Service Catalog
  37. 37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Self-service
  38. 38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Demo Self-service
  39. 39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Self-service
  40. 40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ProdBuckets: Type: Custom::CloudFormationStack DeletionPolicy: Retain Properties: ServiceToken: arn:aws:... AssumeRole: arn:aws:iam:... TemplateURL: https://s3.amazonaws.com/.../buckets.yml Parameters: TeamName: !Ref TeamName CIAccount: !Ref CIAccount TestAccount: !Ref TestAccount ProdAccount: !Ref ProdAccount Self-service
  41. 41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Self-service
  42. 42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Solution Self-serviceContinuous delivery Monitor
  43. 43. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Monitor—cfn_nag https://github.com/stelligent/cfn_nag
  44. 44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. https://github.com/capitalone/cloud-custodian Monitor—Cloud Custodian
  45. 45. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. - name: s3-global-access mode: type: cloudtrail events: - event: PutBucketACL resource: s3 filters: - type: global-grants allow_website: false actions: - delete-global-grants - remove-website-hosting - type: notify to: - resource-owner Monitor—Cloud Custodian
  46. 46. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. - name: cost-center-absent-ec2 mode: type: periodic schedule: cron(30 * * * ? *) resource: ec2 filters: - and: - tag:Cost Center: absent - tag:Team: absent actions: - stop - type: mark-for-op op: terminate days: 2 Monitor—Cloud Custodian
  47. 47. Pipeline Invokes Governor stack AWSGovernor AWSGovernor TestTypeCatalog Organization Level Tests Policy Service 1. Execute CloudFormation 2. Run AWSGovernor 3. Describe stack resources 4. Get all registered tests 5. Run organization tests 6. Run product tests 7. Report success or failure Pipeline Deploys App Infra/Code Tools Account Security Account Product Level Test 1 2 3 4 5 6 7 Governor Production Account
  48. 48. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Monitor—pipeline dashboard
  49. 49. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Monitor—pipeline dashboard
  50. 50. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ServerlessFunc: Type: 'AWS::Serverless::Function' Properties: Handler: handler.pipeline_event Runtime: python3.6 CodeUri: ../app Events: PipelineEventRule: Type: CloudWatchEvent Properties: Pattern: source: - "aws.codepipeline" detail-type: - "CodePipeline Pipeline Execution State Change" - "CodePipeline Stage Execution State Change" - "CodePipeline Action Execution State Change" Monitor—pipeline dashboard
  51. 51. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. DashboardFunc: Type: 'AWS::Serverless::Function' Properties: Handler: handler.dashboard_event Runtime: python3.6 CodeUri: ../app Events: DashboardEventRule: Type: Schedule Properties: Schedule: "cron(*/5 * * * ? *)" Monitor—pipeline dashboard
  52. 52. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What we learned Autonomy • Achieved through continuous delivery and self- service • All infrastructure defined as code • All deployments done via a pipeline • Pipeline is triggered via commit • Only manual step is approve/reject • Use ServiceCatalog to enable self service
  53. 53. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What we learned Governance • Achieved through self-service and monitoring • CloudFormation all the things • ServiceCatalog for CloudFormation governance • CloudFormation static analysis with cfn_nag • Cloud Custodian to assess and enforce compliance • Monitor pipeline metrics
  54. 54. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Resources https://stelligent.com/dev332
  55. 55. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Thank you!

×