Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

(DVO303) Scaling Infrastructure Operations with AWS


Published on

As the number of developers and size of your infrastructure on AWS grows, timely investments in self-service and monitoring can help you scale operations without being the bottleneck. You can standardize infrastructure configurations for commonly used products to enable your customers to self-serve infrastructure needs for their apps. Once these resources are provisioned, you can easily understand how they are connected to administer them effectively, and monitor changes to configurations and evaluate drift. In this session, we will discuss how you can achieve a sophisticated level of standardization, configuration compliance, and monitoring using a combination of AWS Service Catalog, AWS Config, and AWS CloudTrail.

Published in: Technology

(DVO303) Scaling Infrastructure Operations with AWS

  1. 1. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Prashant Prahlad, Product Manager Abhishek Lal, Product Manager October 2015 DVO303 Scaling Infrastructure Operations with AWS Service Catalog, AWS Config, and AWS CloudTrail
  2. 2. What to Expect from the Session Scale infrastructure administration using standardization Codify your business policies to promote compliance Improve security, ops posture without sacrificing developer productivity Troubleshoot issues in a timely manner
  3. 3. Growth is good 2 devs Few instances 1 app 100s of API actions 3 devs Tens of instances Few services 100s of API actions 10s of devs Several apps and services 1000s API actions 10s of customers Several teams of devs 10s of apps/services 100,000 API actions 100s of customers Experimenting Product launch 6 months 12 months
  4. 4. Growth is good… Enable new users to experiment and make mistakes Various devices access or use the cloud Self-service access to infrastructure Global workforce …..but make good investments early to scale well
  5. 5. Growth is also challenging Several new developers (some new to AWS) Mistakes can be very expensive Keeping developers productive becomes harder Operating and troubleshooting numerous flavors Noisy #slack channel
  6. 6. Traditional options Decentralize and hope: • Self serve, experiment, innovate • Promote agility • Well-intentioned, but dangerous • Compliance subject to interpretation by new users Lock down and approve: • Full control, reduced experimentation • Reduced agility • Scales to number of approvers • Unappealing to developers
  7. 7. Or self serve, self govern at scale….. Goals • Agility • Innovation • Compliance • Risk mitigation • Cost control Culture • DevOps culture • Continuous deployments • Automation • Measurement • Sharing Tooling • Infrastructure-as- code • Self service • Auditing • Change tracking
  8. 8. Browse and launch AWS ConfigAWS CloudTrail Use and modify Users Admin Using AWS management services AWS Service Catalog Provision with Tags API calls Configuration checks Troubleshoot and Audit
  9. 9. What is AWS Service Catalog? AWS Service Catalog allows organizations to create and manage catalogs of IT services. It enables users to quickly deploy the approved IT services they need in a self-service manner. Organizations Developers Control Standardization Governance Agility Self-service Time to market
  10. 10. Creates portfolio Adds constraints and grant access 1 4 5 Administrator Portfolio Users Browse Products 6Launch ProductsAWS CloudFormation template Creates product3Authors template2 ProductX ProductY ProductZ 7 Deploys stacks Events Events 8 8 Service Catalog flow Create custom services and grant access Use a personalized portal to find and launch services
  11. 11. Demo: AWS Service Catalog
  12. 12. Self-service provisioning and standardization • Increase agility with self- service provisioning • Promote standardization and compliance • Tag resources for cost tracking and chargeback AWS Service Catalog
  13. 13. AWS CloudTrail Store/ Archive Troubleshoot Monitor & Alarm You are making API calls... On a growing set of AWS services around the world.. CloudTrail is continuously recording API calls
  14. 14. Use cases enabled by CloudTrail Security analysis Track API calls to AWS resources Troubleshoot operational issues Demonstrate compliance
  15. 15. Look up API calls Look up by user, resource type, API, or resource name
  16. 16. Track user activity and API usage AWS CloudTrail • Complete log of API actions • Answer Who, What, When, Where quickly • Enables faster resolution of issues • Set up alerts on APIs
  17. 17. AWS Config • Get inventory of AWS resources • Create Rules to check recorded configurations • Audit historical configurations • Notifies you when configurations change
  18. 18. NormalizeRecordChanging Resources AWS Config & Config Rules Deliver Stream Snapshot (ex. 2014-11-05) AWS Config APIs Store History Rules
  19. 19. Component Description Contains Metadata Information about this configuration item Version ID, Configuration item ID, Time when the configuration item was captured, State ID indicating the ordering of the configuration items of a resource, MD5Hash, etc. Common Attributes Resource attributes Resource ID, tags, Resource type. Amazon Resource Name (ARN) Availability Zone, etc. Relationships How the resource is related to other resources associated with the account EBS volume vol-1234567 is attached to an EC2 instance i- a1b2c3d4 Current Configuration Information returned through a call to the Describe or List API of the resource e.g. for EBS Volume State of DeleteOnTermination flag Type of volume. For example, gp2, io1, or standard Related Events The AWS CloudTrail events that are related to the current configuration of the resource AWS CloudTrail event ID Configuration Item
  20. 20. Sample Config Item "configurationItemVersion": "1.0", "configurationItemCaptureTime": "2014…", "configurationStateID": “….", "configurationItemStatus": "OK", "resourceId": "vol-ce676ccc", "arn": "arn:aws:us-west-………", "accountId": "12345678910", "availibilityZone": "us-west-2b", "resourceType": "AWS::EC2::Volume", "resourceCreationTime": "2014-02..", "tags": {}, "relatedEvents": [ "06c12a39-eb35-11de-ae07-db69edbb1e4", ], "relationships": [ { "resourceId": "i-344c463d", "resourceType": "AWS::EC2::Instance", "name": "Attached to Instance" } ], "configuration": { "volumeId": "vol-ce676ccc", "size": 1, "snapshotId": "", "availabilityZone": "us-west-2b", "state": "in-use", "createTime": "2014-02-……", "attachments": [ { "volumeId": "vol-ce676ccc", "instanceId": "i-344c463d", "device": "/dev/sdf", "state": "attached", "attachTime": "2014-03-", "deleteOnTermination": false } ], "tags": [ { "tagName": "environment", "tagValue": "PROD" }, { "tagName": "name", "tagValue": "DataVolume1" } ], "volumeType": "standard" } },
  21. 21. Config Rule A rule that will check the validity of configurations recorded • AWS Managed Config Rules Rules defined by AWS and require minimal (or no) configuration to enable. Rules are managed by AWS • Customer Managed Config Rules Rules created in your account, and require authoring or reusing AWS Lambda functions. Rules execute in your account. Report evaluation of {Rule, ResourceType, ResourceID} directly from the rule itself
  22. 22. Why track change events using Config? Security Analysis: Am I safe? Audit Compliance: Where is the evidence? Change Management: What will this change affect? Troubleshooting: What has changed? Discovery: What resources exist?
  23. 23. Demo: Set up and use Config Rules
  24. 24. Track resource inventory and changes AWS Config • Continuous compliance with Config Rules • Set up Config Rules for ideal configurations • Record Configuration changes • Stream change notifications
  25. 25. Browse and launch AWS ConfigAWS CloudTrail Use and modify Users Admin Using AWS management services AWS Service Catalog Provision with Tags API calls Configuration checks Troubleshoot and Audit
  26. 26. Try the AWS management services AWS Service Catalog Self-service Standardization Control AWS Config Config Rules PREVIEW Record Changes Stream notifications Sign up: AWS CloudTrail Track user activity Audit log of API calls Troubleshoot issues
  27. 27. Thank you! Prashant Prahlad, Product Manager Abhishek Lal, Product Manager
  28. 28. Remember to complete your evaluations!