This document provides an overview of a presentation about AWS security best practices. It discusses several methods for hardening an AWS environment including: not using the root account, removing root access keys, auditing IAM policies, enabling multi-factor authentication, implementing a strong password policy, and restricting API access with MFA. It also covers ways to monitor an AWS environment for anomalies using CloudTrail, SNS, Config, and CloudWatch. Specific examples are given around setting up billing alerts with CloudWatch and SNS.
3. @cktricky@cktricky
Things to Mention
• DoubleTree by Hilton at 8901 Business
Park Drive in Austin, TX is great at:
– Selling your room for you without telling you
– Fire alarms
– Murdering puppies and kittens
– Created cancer?
– Created cancer in puppies and kittens?
4. @cktricky@cktricky
Things to Mention
• Ask questions throughout presentation
• There will be no dedicated Q&A – so stick
around after and find me if you want to
chat
• This presentation will cover a lot. Slides
will be available so don’t worry about
minutia.
5. @cktricky@cktricky
Background/About
• Ken Johnson, CTO and Partner at nVisium
• Veteran, US Navy
• I speak about:
– DevOps (In)Security
– Exploiting Web Applications
– Coding and Coding + Security
– Node, Elixir, Python, Ruby, Go
– AWS Security (clearly)
6. @cktricky@cktricky
Background/About
This talk came about because…
– I’m the CTO of a security company and we use
AWS… and it is a challenge
– For some, this is a new challenge, and this is my
opportunity to share
8. @cktricky@cktricky
Our Plan
Our “practical plan”
– Harden – Make it difficult to reach our AWS
environment
– Monitor – If our AWS environment is breached, we
need to know and alert ourselves
– Restore – Have the ability to reconstruct data/configs
after a “hack”
9. @cktricky@cktricky
AWS’s Plan
The AWS Security Fundamentals Course provides
the framework for your plan:
– You are responsible for leveraging the tools AWS
provides to secure your environment (financially)
– Your configuration… that is on you
– https://aws.amazon.com/training/course-
descriptions/security-fundamentals/
10. @cktricky@cktricky
Most Security Checklists
Most AWS security talks and documentation
discuss:
– S3 bucket policies
– Security Group configurations
– SSH Key Management
– Encrypting Data (Volumes, S3 buckets)
11. @cktricky@cktricky
Most Security Checklists
What we’ll mention on the subject:
1. Trusted Advisor – Use it, because it catches a lot of
“low hanging fruit” style issues
2. There are checklists, use them:
– https://media.amazonwebservices.com/AWS_Operational_Che
cklists.pdf
– http://d0.awsstatic.com/whitepapers/compliance/AWS_Auditing
_Security_Checklist.pdf
3. Again let’s reiterate that AWS provides a security
fundamentals course for free (CBT)
12. @cktricky@cktricky
About / Background Recap
Recap:
– We’re not going to cover basic security fundamentals
of unencrypted volumes, security groups, etc.
– We are going to focus on:
• Hardening
• Monitoring
• Recovery
15. @cktricky@cktricky
IAM Hardening Checklist
1. Don’t Use The Root Account!
2. Remove Access Keys for Root Account
3. Audit IAM user policies
4. Multi-Factor Authentication
5. API + MFA
6. Strong Password Policy
17. @cktricky@cktricky
Don’t Use The Root Account
Every AWS environment has a root account
– Root account is the king/god/all-powerful
– Use only when you absolutely must
– When those circumstances arise, notify your team
first
– This is because we will be configuring alerts to notify
our team when the root account is used
18. @cktricky@cktricky
Remove Access Keys for Root Account
Simple steps:
– Disable or delete access keys if they exist:
– Implement verbal/written policy that states “we don’t
create access keys for the root account”
20. @cktricky@cktricky
Audit IAM User Policies
IAM user policy management:
– A single IAM user can have…
• Multiple Managed Policies
• Multiple Inline Policies
• Belong to multiple IAM Groups which…
– Have multiple managed policies
– Have multiple inline policies
21. @cktricky@cktricky
Audit IAM User Policies
Explanation
– Managed Policies: Policies that can be
attached to multiple users, groups, or roles
– Inline Policies: Directly attached to a single
user, group, or role
22. @cktricky@cktricky
Audit IAM User Policies
Tool to inspect each user’s permissions:
– https://gist.github.com/cktricky/257990df2f36aa3a01a
8809777d49f5d
– Will create a CSV file
– Provides you with
• Usernames
• Inline Policies
• Managed Policies
• Groups
26. @cktricky@cktricky
Audit IAM User Policies
Why this is important
– If you house sensitive data, you need to know who
has access
– Permissions should be a need-to-have/know situation
in order to limit damage should creds get stolen
– AWS is a flexible environment that changes – your
permission model might need to change with it
(inventory it)
28. @cktricky@cktricky
MFA
• MFA == 2-Factor Authentication
• If credentials are stolen or guessed, we want a second
layer of protection
• You can use apps or hardware to do this
– Google Authenticator (Apps)
– Gemalto (Hardware)
• Find the full list of MFA devices here:
https://aws.amazon.com/iam/details/mfa/
34. @cktricky@cktricky
MFA
• At this point, its worth mentioning that non-
administrators or those without IAM privileges
cannot enable MFA on their own account
• Why is this a problem? Well, they need to be
able to enable MFA on their own device… not
the administrator’s
• Fortunately, we have a solution!
36. @cktricky@cktricky
MFA
• Okay so that wasn’t the easiest to read, so
here is the link:
http://docs.aws.amazon.com/IAM/latest/Us
erGuide/id_credentials_delegate-
permissions_examples.html#creds-
policies-mfa-console
• Basically this IAM policy allows a user to
manage their *OWN* MFA device
37. @cktricky@cktricky
MFA (for Root Account)
• Need a shared MFA for root? TOTP!
• Recommend using something like
1password for teams, can share the TOTP
code:
https://support.1password.com/guides/mac/totp.html
https://www.youtube.com/watch?v=eZyb-ArMK9g
39. @cktricky@cktricky
API + MFA
API 101
– This is the alternative to interacting with the AWS
environment via the web console
– Typically used for automated tasks
– Automated tasks means “code”. Luckily, developers
never store keys in source, amiright?
– Hypothetically, what would happen if keys were
leaked?
41. @cktricky@cktricky
API + MFA
So that’s the “worst case scenario”, more likely:
– Costs unexpectedly and dramatically increase
– We’ll show examples later but remember, you are
financially responsible for your AWS environment’s
configuration
– Let’s talk about prevention
42. @cktricky@cktricky
API + MFA
• You have the ability to place a restriction where
resources can only be interacted with if the user
has authenticated with MFA
• This helps prevent (ab)use should someone
steal access keys or credentials
43. @cktricky@cktricky
API + MFA
1. At a minimum, apply to administrator & power user
group policies… really any group that can do anything
of importance
45. @cktricky@cktricky
API + MFA
• Truth be told, doing this can be painful at
first
• Things that used to work, might not (via
the API)
• Fortunately, we have some answers for
you
• Firstly, let’s discuss STS or SecurityToken
Service
46. @cktricky@cktricky
API + MFA
• Leverage STS in order to interact with the
AWS API should this MFA restriction be
placed on resources (and it should )
• Example of using STS:
https://gist.github.com/cktricky/127be4e431563a986f0f
51. @cktricky@cktricky
API + MFA
• ElasticBeanstalk does not work with STS. Le
Terrible.
• However, there is a workaround, use
CodePipeline.
• Very simple process to setup but only works
with:
– GitHub
– AWS CodeCommit
– Amazon S3
54. @cktricky@cktricky
Password Policy
• Password policies are important because
historically people do not choose complex
passwords
• MFA should help, but we’re talking about a
layered approach
• Again, making our AWS environment
harder to reach
56. @cktricky@cktricky
Hardening Recap
• Make credentials hard to guess
• Make credentials hard to use if stolen with
MFA
• Audit your accounts and their access
• Root account is King, protect your King
58. @cktricky@cktricky
AWS Monitoring
• Assuming hardening (prevention) has failed,
how would we know?
• Luckily, AWS provides several services which
alert to anomalies
• We will walk through examples of using these
services, but ultimately decide what is right for
you
• Fair warning, some of these services will provide
a lot of noise
59. @cktricky@cktricky
AWS Monitoring
4 important services:
1. CloudTrail – Logs
2. SNS – Notifications
3. Config – Alerts for modifications &
noncompliance
4. CloudWatch – Alerts for specific types of
behavior
70. @cktricky@cktricky
AWS Monitoring (SNS)
• Receive SMS/Email/Slack notifications for
important events
• ^ This is so you get immediate notifications
• You can have multiple subscribers, I’d suggest
you use that functionality
• Basic gist? Receive immediate updates for
things you want to see… immediately ☺
85. @cktricky@cktricky
AWS Monitoring (CloudWatch)
• We can be very particular here about what it is we want
to see
• Some very interesting things you can monitor
• Some examples:
– Billing Alerts (Important for detection of abuse or
mistakes)
– Track Root Account Usage
– Failed login attempts
– Unauthorized Activity
87. @cktricky@cktricky
AWS Monitoring (CloudWatch - Billing)
• Used to prevent abuse or mistakes from costing your
organization money
• Analyze and approximate your monthly spend
• Configure via CloudWatch
• Use SNS for instantaneous alerting
94. @cktricky@cktricky
AWS Monitoring (CloudWatch - Billing)
Exact steps to enable can be found here:
http://docs.aws.amazon.com/awsaccountbilli
ng/latest/aboutv2/free-tier-alarms.html
96. @cktricky@cktricky
AWS Monitoring (CloudWatch – Root Login)
• Remember how I said don’t use the Root
account routinely?
• BUT… if this account is used, you should
know about it
• This is the reason you’ll want to notify
others (who receive SNS alerts) of the fact
you are about to use the account
104. @cktricky@cktricky
AWS Monitoring (CloudWatch – Failed Logins)
• In the event someone is trying to break in,
let’s alert ourselves to this!
• Failed logins typically suggest either
someone forgot their password or…
someone is trying to guess yours
109. @cktricky@cktricky
IAM Unauthorized Activity
• Aws-interrogate tool
• This alarm is the antidote
• Alerts us when someone is trying to
access something in AWS, and does not
have permissions
114. @cktricky@cktricky
AWS Monitoring (CloudWatch) – Filter Patterns
Create your own custom filter patterns, here is a
resource for that:
http://docs.aws.amazon.com/AmazonCloudWatch/latest/De
veloperGuide/FilterAndPatternSyntax.html
129. @cktricky@cktricky
AWS + Splunk
Splunk is a pretty great resource for monitoring
activity
• Two separate plugins:
– Splunk App for AWS
• https://splunkbase.splunk.com/app/1274/
– Splunk Add-On
• https://splunkbase.splunk.com/app/1876/
130. @cktricky@cktricky
AWS + Splunk
• Examples of things you can view:
– Billing
– Topology
– Usage
– IAM Activity
– SSH Key Pair Activity
– User Activity
– Network ACL(s)
– VPC Activity
and a lot more…
134. @cktricky@cktricky
AWS + Splunk
• Splunk will need an AWS account in order
to retrieve data
• Create account(s) for Splunk, grab the
necessary permission policy from here:
http://docs.splunk.com/Documentation/AddOns/r
eleased/AWS/ConfigureAWSpermissions
136. @cktricky@cktricky
AWS + Splunk
• To view things like IAM Activity…
– Subscribe to a cloudtrail log via SNS
– Utilize SQS and subscribe SQS to an SNS
Topic
137. @cktricky@cktricky
AWS Monitoring Recap
• Alert yourself when things change
• This will get noisy, find a way to filter that which is
important
– If it’s a high risk event, send an SMS/Slack/Email
blast
• At a minimum, alert yourself when odd things occur…
like:
– Billing increases past your normal spend
– When somebody authenticates as Root
– When someone has a login failure
– Unauthorized IAM Activity
138. @cktricky@cktricky
AWS Monitoring Recap
• Interesting Quora thread:
– https://www.quora.com/My-AWS-account-was-hacked-and-I-
have-a-50-000-bill-how-can-I-reduce-the-amount-I-need-to-pay
• Highlights from the article:
– AWS has “a review board of sorts” to determine if you should be
refunded
– Bots are scouring GitHub searching for exposed access keys
– One of the more AWS-seasoned responders mentioned doing
part of what we discussed here today to avoid it
– A decent number of the people posting on this thread said “Yes,
happened to me too”
140. @cktricky@cktricky
AWS Restoration & Recovery – Basic Incident
Response (IR)
• Understand who to contact if things go bad
• Understand how to communicate (ex:
“speak only over the phone”)
• Understand what information to parse
• Understand where your backups are
located and how they are secured
141. @cktricky@cktricky
AWS Restoration & Recovery – Basic IR
• Do not USE AWS TO BACKUP YOUR
AWS
• Offsite backups (meaning, off AWS site)
• Common things to back-up:
– Databases/ Snapshots
– S3 Buckets
– EBS Volumes
– CloudFormation Templates
146. @cktricky@cktricky
Recap
• DoubleTree by Hilton at 8901 Business
Park Drive in Austin, TX
– Sells your room
– Loves fire alarms at 5am
– Behind 9/11?
– Can go f**k itself
147. @cktricky@cktricky
Recap
• Makes your environment harder to reach… for
the bad guys
– Limit what stolen or “otherwise obtained”
access keys or credentials could be used to
do
– Prevent them being stolen in the first place
• Alert yourself to anomalies
• Have a plan for if things go bad
• Stay safe out there!