Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Improving Infrastructure Governance on AWS

1,685 views

Published on

Learn how you can achieve a sophisticated level of standardization, configuration compliance, and monitoring using a combination of AWS Service Catalog, AWS Config, and AWS CloudTrail.

Published in: Technology
  • Be the first to comment

Improving Infrastructure Governance on AWS

  1. 1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Bob Griffiths, Solutions Architect Manager September 21st 2016 DevOps on AWS: Improving Infrastructure Governance on AWS
  2. 2. https://secure.flickr.com/photos/mgifford/4525333972 Why are we here today?
  3. 3. Why are we here today? Using cloud based infrastructure changes how to think about governing our infrastructure:  Infrastructure can be provisioned in seconds.. and go away just as quickly!  Development teams expect a higher level of flexibility and self control in interfacing with their infrastructure needs  Being API driven means that the way people provision and manage infrastructure in the cloud has changed compared to on-premises
  4. 4. Why are we here today? That doesn’t mean that our basic governance needs change:  We still need to have some ability to drive best practices/patterns in our organizations  We need to make sure that we’re able to audit and track changes to our infrastructure for both regulation and security purposes  We need to make sure that we understand how resources are related and integrated
  5. 5. What can we do? There are a few areas to focus on that can help us accomplish both the freedom to rapidly provision, manage, and update our infrastructure while meeting our governance needs:  Policy as Code  Infrastructure standardization (via code!)  Self service environments  Logging/Auditing/Reacting to infrastructure change
  6. 6. Policy as Code builds off of infrastructure as code practices by allowing organizations to codify infrastructure and system configurations allowing them to monitor and enforce compliance dynamically and at scale.
  7. 7. Infrastructure as Code is a practice in which infrastructure is provisioned and managed using code and software development techniques, such as version control and continuous integration.
  8. 8. Infrastructure as Code “levels” AWS Resources Operating System and Host Configuration Application Configuration
  9. 9. Infrastructure as Code “levels” AWS Resources Operating System and Host Configuration Application Configuration allOfThis == $Code
  10. 10. Browse and launch AWS ConfigAWS CloudTrail Use and modify Users Admin Putting the AWS Management services together AWS Service Catalog Provision with Tags API calls Configuration checks and reactions to change Troubleshoot and Audit
  11. 11.  Create templates of your infrastructure  CloudFormation provisions AWS resources based on dependency needs  Version control/replicate/update templates like code  Integrates with development, CI/CD, management tools AWS CloudFormation
  12. 12. Template CloudFormation Stack JSON formatted file Parameter definition Resource creation Configuration actions Configured AWS resources Comprehensive service support Service event aware Customizable Framework Stack creation Stack updates Error detection and rollback CloudFormation – Components & Technology
  13. 13. Template File Defining Stack The entire infrastructure can be represented in an AWS CloudFormation template. Many Stacks & Environments from One Template
  14. 14. Template File Defining Stack The entire infrastructure can be represented in an AWS CloudFormation template. Use the version control system of your choice to store and track changes to this template Many Stacks & Environments from One Template Git Perforce SVN …
  15. 15. Template File Defining Stack Git Perforce SVN … Dev Test Prod The entire infrastructure can be represented in an AWS CloudFormation template. Use the version control system of your choice to store and track changes to this template Build out multiple environments, such as for Development, Test, Production and even DR using the same template Many Stacks & Environments from One Template
  16. 16. CloudFormation example use cases: Have “full stack” templates that can be used to stand up common application patterns inside your organization such as a 3-tier application template that:  uses Lambda custom resources to look up appropriate VPC information (VPC ID, Subnets, etc) based on tags  creates an Elastic Beanstalk environment that supports Multi-AZ, AutoScaling, CloudWatch Metrics, and Elastic Load Balancing  contains security controls such as AWS Identity and Access Management (IAM) roles, profiles, and policies, and Security Groups  allows the user to specify the language of their application  allows a user to specify which database they want (SQL or NoSQL) and then creates the appropriate resource
  17. 17. Using Parameters and Conditionals are two key ways to make a single template much more dynamic: "Parameters" : { "Database": { "Type" : "String", "Default" : "RDS", "AllowedValues" : ["RDS", "DynamoDB", "None"], "Description" : "Database to create. Select None if using an existing database.” } }, "Conditions" : { "CreateRDS" : {"Fn::Equals" : [{"Ref" : "Database"}, "RDS"]}, "CreateDynamoDB" : {"Fn::Equals" : [{"Ref" : "Database"}, "DynamoDB"]}, "CreateNone" : {"Fn::Equals" : [{"Ref" : "Database"}, "None"]} }, “Resources” : { ”RDSdb01" : { "Condition" : " CreateRDS ", "Type" : "AWS::RDS::Instance",
  18. 18. https://secure.flickr.com/photos/wscullin/3770015991 Now that we have these templates, what can we do to help simplify our developers’ lives further and increase our infrastructure standardization?
  19. 19.  Customized catalogs of products  Manage products centrally  Personalized, self-service portal  Integrate with existing systems AWS Service Catalog
  20. 20. What is AWS Service Catalog? AWS Service Catalog allows organizations to create and manage catalogs of IT services. It enables users to quickly deploy the approved IT services they need in a self-service manner. Organizations Developers Control Standardization Governance Agility Self-service Time to market
  21. 21. Creates portfolio Adds constraints and grant access 1 4 5 Administrator Portfolio Users Browse Products 6Launch ProductsAWS CloudFormation template Creates product3Authors template2 ProductX ProductY ProductZ 7 Deploys stacks Events Events 8 8 Service Catalog Create custom services and grant access Use a personalized portal to find and launch services
  22. 22. Service Catalog use cases: You can remove the need for developers to understand how all AWS services work. Treat infrastructure provisioning like buying components from a retail site:  provide standardized Service Catalog products around common internal application frameworks/architectural patterns  provide common application component products such as databases, queues, caches, worker tiers, etc  build logging, monitoring, metrics into these stacks  leverage service discovery tools when possible  build in the same best practices across development, staging, production environments with these provided products
  23. 23. We’ve helped solve some of our developer’s access and standardization issues, but how can we now go about auditing changes to our infrastructure? https://www.flickr.com/photos/atoach/7623237104
  24. 24. AWS CloudTrail  Records AWS API calls for your account  Delivers log files of API calls to S3  Delivery typically within 15 minutes of API call  Logs contain detailed information  Log files can be encrypted and have their integrity verified by you
  25. 25. AWS CloudTrail CloudTrail can help you achieve many tasks  Security analysis  Track changes to AWS resources, for example VPC security groups and NACLs  Compliance – log and understand AWS API call history  Prove that you did not:  Use the wrong region  Use services you don’t want  Troubleshoot operational issues – quickly identify the most recent changes to your environment
  26. 26. AWS CloudTrail logs can be delivered cross-account CloudTrail can help you achieve many tasks  Accounts can send their trails to a central account  Central account can then do analytics  Central account can:  Redistribute the trails  Grant access to the trails  Filter and reformat Trails (to meet privacy requirements)
  27. 27. CloudTrail
  28. 28. CloudTrail – Amazon CloudWatch Logs Integration
  29. 29.  Continuous recording  Inventory of AWS resources  New & deleted resources  Configuration change & compliance notifications AWS Config
  30. 30. Relationships Bi-directional map of dependencies automatically assigned Change to a resource propagates to create Configuration Items for related resources
  31. 31. Configuration Item  All configuration attributes  Normalized  Point in time  Captured on configuration change
  32. 32. Component Description Contains Metadata Information about this configuration item Version ID, Configuration item ID, Time when the configuration item was captured, State ID indicating the ordering of the configuration items of a resource, MD5Hash, etc. Common Attributes Resource attributes Resource ID, tags, Resource type. Amazon Resource Name (ARN) Availability Zone, etc. Relationships How the resource is related to other resources associated with the account EBS volume vol-1234567 is attached to an EC2 instance i- a1b2c3d4 Current Configuration Information returned through a call to the Describe or List API of the resource e.g. for EBS Volume State of DeleteOnTermination flag Type of volume. For example, gp2, io1, or standard Related Events The AWS CloudTrail events that are related to the current configuration of the resource AWS CloudTrail event ID Configuration Item
  33. 33. Sample Configuration Item "configurationItemVersion": "1.0", "configurationItemCaptureTime": "2014…", "configurationStateID": “….", "configurationItemStatus": "OK", "resourceId": "vol-ce676ccc", "arn": "arn:aws:us-west-………", "accountId": "12345678910", "availibilityZone": "us-west-2b", "resourceType": "AWS::EC2::Volume", "resourceCreationTime": "2014-02..", "tags": {}, "relationships": [ { "resourceId": "i-344c463d", "resourceType": "AWS::EC2::Instance", "name": "Attached to Instance" } ], "relatedEvents": [ "06c12a39-eb35-11de-ae07-db69edbb1e4", ], Metadata Common Attributes Relationships Related Events
  34. 34. Sample Configuration Item "configuration": { "volumeId": "vol-ce676ccc", "size": 1, "snapshotId": "", "availabilityZone": "us-west-2b", "state": "in-use", "createTime": "2014-02-……", "attachments": [ { "volumeId": "vol-ce676ccc", "instanceId": "i-344c463d", "device": "/dev/sdf", "state": "attached", "attachTime": "2014-03-", "deleteOnTermination": false } ], "tags": [ { "tagName": "environment", "tagValue": "PROD" Configuration
  35. 35. Having this data is one thing, but how can we react to change? https://www.flickr.com/photos/livenature/204420128/
  36. 36. CloudTrail – Amazon CloudWatch Logs Integration
  37. 37. CloudTrail – Amazon CloudWatch Logs Integration Trigger a CloudWatch Alarm based on API call!
  38. 38.  Check configuration changes  Pre-built rules provided by AWS  Custom rules using AWS Lambda  Continuous assessment  Dashboard  Compliance visualization  Identify offending changes  GitHub repo: Community sourced rules AWS Config Rules
  39. 39. NormalizeRecordChanging Resources AWS Config & Config Rules Deliver Stream Snapshot (ex. 2014-11-05) AWS Config APIs Store History Rules
  40. 40. Config Rules
  41. 41. Config Rules
  42. 42. Config Rules GitHub repository
  43. 43. AWS Management Tools Partners
  44. 44. FIN, ACK We’ve seen a quick run through today of the ways you can improve your governance on AWS:  Treat your infrastructure and host configuration as code!  This lends itself to being able to use services like Service Catalog to enable self-service in your organization  Track, trend, and alert on CloudTrail API logs to keep on top of access to your AWS resources  Use Config and Config Rules to understand the relationship between resources and react to policy violations  Putting all this together is what gives you Policy as Code!
  45. 45. Browse and launch AWS ConfigAWS CloudTrail Use and modify Users Admin Putting the AWS Management services together AWS Service Catalog Provision with Tags API calls Configuration checks and reactions to change Troubleshoot and Audit
  46. 46. But wait, there’s more! Resources to learn more:  More on DevOps: https://aws.amazon.com/devops/  AWS Management Services: https://aws.amazon.com/products/management/  AWS CloudFormation  https://aws.amazon.com/cloudformation/  AWS Service Catalog  https://aws.amazon.com/servicecatalog/  AWS CloudTrail  https://aws.amazon.com/cloudtrail/  AWS Config / Config Rules  https://aws.amazon.com/config/  GitHub repo: https://github.com/awslabs/aws-config-rules
  47. 47. Thank you! Happy Deploying!

×