Learn how you can achieve a sophisticated level of standardization, configuration compliance, and monitoring using a combination of AWS Service Catalog, AWS Config, and AWS CloudTrail.
3. Why are we here today?
Using cloud based infrastructure changes how to think about
governing our infrastructure:
Infrastructure can be provisioned in seconds.. and go
away just as quickly!
Development teams expect a higher level of flexibility and
self control in interfacing with their infrastructure needs
Being API driven means that the way people provision and
manage infrastructure in the cloud has changed compared
to on-premises
4. Why are we here today?
That doesn’t mean that our basic governance
needs change:
We still need to have some ability to drive best
practices/patterns in our organizations
We need to make sure that we’re able to audit
and track changes to our infrastructure for both
regulation and security purposes
We need to make sure that we understand
how resources are related and integrated
5. What can we do?
There are a few areas to focus on that can help us accomplish
both the freedom to rapidly provision, manage, and update our
infrastructure while meeting our governance needs:
Policy as Code
Infrastructure standardization (via code!)
Self service environments
Logging/Auditing/Reacting to infrastructure change
6. Policy as Code builds off of
infrastructure as code practices by
allowing organizations to codify
infrastructure and system configurations
allowing them to monitor and enforce
compliance dynamically and at scale.
7. Infrastructure as Code is a practice
in which infrastructure is provisioned
and managed using code and
software development techniques,
such as version control and
continuous integration.
8. Infrastructure as Code “levels”
AWS Resources
Operating System and Host Configuration
Application Configuration
9. Infrastructure as Code “levels”
AWS Resources
Operating System and Host Configuration
Application Configuration
allOfThis == $Code
10. Browse and launch
AWS ConfigAWS CloudTrail
Use and modify
Users Admin
Putting the AWS Management services together
AWS Service Catalog
Provision with Tags
API calls Configuration checks and
reactions to change
Troubleshoot and Audit
11. Create templates of your infrastructure
CloudFormation provisions AWS
resources based on dependency needs
Version control/replicate/update
templates like code
Integrates with development, CI/CD,
management tools
AWS
CloudFormation
12. Template CloudFormation Stack
JSON formatted file
Parameter definition
Resource creation
Configuration actions
Configured AWS resources
Comprehensive service support
Service event aware
Customizable
Framework
Stack creation
Stack updates
Error detection and rollback
CloudFormation – Components & Technology
13. Template File
Defining Stack
The entire infrastructure can be
represented in an AWS
CloudFormation template.
Many Stacks & Environments from One Template
14. Template File
Defining Stack
The entire infrastructure can be
represented in an AWS
CloudFormation template.
Use the version
control system of
your choice to
store and track
changes to this
template
Many Stacks & Environments from One Template
Git
Perforce
SVN
…
15. Template File
Defining Stack
Git
Perforce
SVN
…
Dev
Test
Prod
The entire infrastructure can be
represented in an AWS
CloudFormation template.
Use the version
control system of
your choice to
store and track
changes to this
template
Build out multiple
environments, such
as for Development,
Test, Production and
even DR using the
same template
Many Stacks & Environments from One Template
16. CloudFormation example use cases:
Have “full stack” templates that can be used to stand up common
application patterns inside your organization such as a 3-tier application
template that:
uses Lambda custom resources to look up appropriate VPC
information (VPC ID, Subnets, etc) based on tags
creates an Elastic Beanstalk environment that supports Multi-AZ,
AutoScaling, CloudWatch Metrics, and Elastic Load Balancing
contains security controls such as AWS Identity and Access
Management (IAM) roles, profiles, and policies, and Security Groups
allows the user to specify the language of their application
allows a user to specify which database they want (SQL or NoSQL)
and then creates the appropriate resource
17. Using Parameters and Conditionals are two key ways
to make a single template much more dynamic:
"Parameters" : {
"Database": {
"Type" : "String",
"Default" : "RDS",
"AllowedValues" : ["RDS", "DynamoDB", "None"],
"Description" : "Database to create. Select None if using an existing database.”
}
},
"Conditions" : {
"CreateRDS" : {"Fn::Equals" : [{"Ref" : "Database"}, "RDS"]},
"CreateDynamoDB" : {"Fn::Equals" : [{"Ref" : "Database"}, "DynamoDB"]},
"CreateNone" : {"Fn::Equals" : [{"Ref" : "Database"}, "None"]}
},
“Resources” : {
”RDSdb01" : {
"Condition" : " CreateRDS ",
"Type" : "AWS::RDS::Instance",
19. Customized catalogs of products
Manage products centrally
Personalized, self-service portal
Integrate with existing systems
AWS
Service Catalog
20. What is AWS Service Catalog?
AWS Service Catalog allows organizations to create and manage
catalogs of IT services. It enables users to quickly deploy the approved
IT services they need in a self-service manner.
Organizations Developers
Control
Standardization
Governance
Agility
Self-service
Time to market
21. Creates portfolio
Adds constraints
and grant access
1
4
5
Administrator
Portfolio
Users
Browse Products
6Launch ProductsAWS CloudFormation
template
Creates
product3Authors template2
ProductX ProductY ProductZ
7
Deploys
stacks
Events
Events
8
8
Service Catalog
Create custom
services
and grant access
Use a
personalized
portal to find and
launch services
22. Service Catalog use cases:
You can remove the need for developers to understand how all
AWS services work. Treat infrastructure provisioning like buying
components from a retail site:
provide standardized Service Catalog products around
common internal application frameworks/architectural patterns
provide common application component products such as
databases, queues, caches, worker tiers, etc
build logging, monitoring, metrics into these stacks
leverage service discovery tools when possible
build in the same best practices across development, staging,
production environments with these provided products
23. We’ve helped solve some of our
developer’s access and
standardization issues, but how can
we now go about auditing changes
to our infrastructure?
https://www.flickr.com/photos/atoach/7623237104
24. AWS
CloudTrail
Records AWS API calls for your account
Delivers log files of API calls to S3
Delivery typically within 15 minutes of API call
Logs contain detailed information
Log files can be encrypted and have their
integrity verified by you
25. AWS CloudTrail
CloudTrail can help you achieve many tasks
Security analysis
Track changes to AWS resources, for
example VPC security groups and NACLs
Compliance – log and understand AWS API
call history
Prove that you did not:
Use the wrong region
Use services you don’t want
Troubleshoot operational issues – quickly
identify the most recent changes to your
environment
26. AWS CloudTrail logs can be delivered cross-account
CloudTrail can help you achieve many tasks
Accounts can send their trails to a central
account
Central account can then do analytics
Central account can:
Redistribute the trails
Grant access to the trails
Filter and reformat Trails (to meet
privacy requirements)
30. Relationships
Bi-directional map of dependencies
automatically assigned
Change to a resource propagates
to create Configuration Items for
related resources
31. Configuration Item
All configuration attributes
Normalized
Point in time
Captured on configuration change
32. Component Description Contains
Metadata Information about this configuration
item
Version ID, Configuration item ID,
Time when the configuration item
was captured, State ID indicating
the ordering of the configuration
items of a resource, MD5Hash, etc.
Common Attributes Resource attributes Resource ID, tags, Resource type.
Amazon Resource Name (ARN)
Availability Zone, etc.
Relationships How the resource is related to other
resources associated with the
account
EBS volume vol-1234567 is
attached to an EC2 instance i-
a1b2c3d4
Current Configuration Information returned through a call
to the Describe or List API of the
resource
e.g. for EBS Volume
State of DeleteOnTermination flag
Type of volume. For example, gp2,
io1, or standard
Related Events The AWS CloudTrail events that are
related to the current configuration
of the resource
AWS CloudTrail event ID
Configuration Item
45. FIN, ACK
We’ve seen a quick run through today of the ways you can
improve your governance on AWS:
Treat your infrastructure and host configuration as code!
This lends itself to being able to use services like Service
Catalog to enable self-service in your organization
Track, trend, and alert on CloudTrail API logs to keep on top
of access to your AWS resources
Use Config and Config Rules to understand the relationship
between resources and react to policy violations
Putting all this together is what gives you Policy as Code!
46. Browse and launch
AWS ConfigAWS CloudTrail
Use and modify
Users Admin
Putting the AWS Management services together
AWS Service Catalog
Provision with Tags
API calls Configuration checks and
reactions to change
Troubleshoot and Audit
47. But wait, there’s more!
Resources to learn more:
More on DevOps: https://aws.amazon.com/devops/
AWS Management Services: https://aws.amazon.com/products/management/
AWS CloudFormation
https://aws.amazon.com/cloudformation/
AWS Service Catalog
https://aws.amazon.com/servicecatalog/
AWS CloudTrail
https://aws.amazon.com/cloudtrail/
AWS Config / Config Rules
https://aws.amazon.com/config/
GitHub repo: https://github.com/awslabs/aws-config-rules
So if you look at the components behind Cloudformation. It's starts off with a template.
This is the JSON formatted script file, that deals with things like parameter definition that drive a user driven template, such as name of my databases.
It deals with the resource creation, so the creation of AWS components such as EC2 instances or RDS databases.
And it deals with the configuration actions I wish to apply against this resources, so it might be install software or might be creating an SQS queue for example.
Than that template is deployed into the cloud formation framework. And the framework deals what we call Stack creation, updates and any error detection and rollback required in the creation of a stack.
So a stack is collection of resources that you want to manage together. And the resulting artifact is what we call a Stack of configured AWS services. So this could be in an Elastic Load Balancer and Autosclaing group with EC2 instances and an RDS database.
So the stack is service event aware, the stack creation actions or the changing of that environment can be feed back into Cloudfomration and trigger actions within the CloudFormation tempalte.
And it is also customizable, so once you created a stack you can of course access the underlying resources and change them of modify them as you so which.
Now the error detection and rollback is an interesting point. If at any time in the stack creation a problem is detected, the default behaviour of Cloudformation is to roll-back the creation of all resources and put you back in a constitent known state. So you know if your stack is working or is rolled back and is not.
Notes:
The entire application can be represented in an AWS CloudFormation template.
You can use the version control system of your choice to store and track changes to this template.
You can use the template to quickly build out multiple environments, such as for Development, Test, and Production.
Notes:
The entire application can be represented in an AWS CloudFormation template.
You can use the version control system of your choice to store and track changes to this template.
You can use the template to quickly build out multiple environments, such as for Development, Test, and Production.
Notes:
The entire application can be represented in an AWS CloudFormation template.
You can use the version control system of your choice to store and track changes to this template.
You can use the template to quickly build out multiple environments, such as for Development, Test, and Production.
IT want to control visibility to ensure compliance with business goals and requirements.
In order to be agile, Developers want self service access to their environment to reduce time to market for their Apps.
While these may appear to be in conflict, the AWS Service Catalog allow both IT Admins and Developer to achieve their goals.
Administrators define standardized products that developers can browse and launch in a self-service manner.
Changes, I’d delete this slide; Instead, add a pre-requisite that they know what AWS Service Catalog is.
Let me walk you through the key use case flows in the service catalog
Changes: Changed the title. I’d also run through this quickly as you should assume they know what Service Catalog is.
No setup needed
Configuration item contains all configuration attributes for a given resource at a given point in time, captured on every configuration change
Every change to a resource causes a new configuration item to be created that captures the new configuration of the resource
Who is Threat Stack ?
AWS Advanced Security Partner offering a complete Cloud Security Platform.
The only Continuous Security Monitoring solution that is offering an integrated PLATFORM approach, enabling multiple security functions without the hassle and cost of point solutions
Key Threat Stack customers that are running on AWS include: Acquia, Interactive Intelligence, PagerDuty, Adroll, Ayla Networks
Why is Threat Stack winning in the market?
This is what we are hearing from our customers - 4 of the top reasons we are truly differentiated the reasons they are choosing us over the competition.
Time to detection/value
Deep visibility into who is doing what, where, when
Platform approach; “Single pane of glass”
Compliance for you; assurance for your customers
Where does Threat Stack fit into the AWS Governance Practices ?
Threat Stack seamlessly integrates into AWS Governance Model by enabling continuous monitoring and alerting for AWS Cloud Trail logs
Audit changes against AWS Service against your risk and compliance goals
What is unique about Threat Stack AWS Cloud Trail integration ?
Continuously monitor for changes to AWS resources for changes to security posture using pre-built Rule sets built on AWS and community best practices
Purpose built integration for Ops tools & workflows including:Slack, Pager Duty, VictorOps – designed to help you go faster, securely.