Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

(SEC318) AWS CloudTrail Deep Dive

6,860 views

Published on

"Ever wondered how can you find out which user made a particular API call, when the call was made, and which resources were acted upon? In this session, you will learn how to turn on AWS CloudTrail for hundreds of AWS accounts in all AWS regions to ensure you have full visibility into API activity in all your AWS accounts. We will demonstrate how to use CloudTrail Lookup in the AWS Management Console to troubleshoot operational and security issues and how to use the AWS CLI or SDKs to integrate your applications with CloudTrail.

We will also demonstrate how you can monitor for specific API activity by using Amazon CloudWatch and receive email notifications, when such activity occurs. Using CloudTrail Lookup and CloudWatch Alarms, you can take immediate action to quickly remediate any security or operational issues. We will also share best practices and ready-to-use scripts, and dive deep into new features that help you configure additional layers of security for CloudTrail log files."

Published in: Technology

(SEC318) AWS CloudTrail Deep Dive

  1. 1. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Sivakanth Mundru, Product Manager, AWS CloudTrail October 2015 SEC318 AWS CloudTrail Deep Dive
  2. 2. What to Expect from the Session  Introduction to AWS CloudTrail and use cases  Deep dives on use cases  CloudTrail for multiple AWS accounts  Encryption using KMS New and Log file integrity validation New  AWS Partner solutions integrated with CloudTrail
  3. 3. Introduction to AWS CloudTrail Store/ Archive Troubleshoot Monitor & Alarm You are making API calls... On a growing set of AWS services around the world.. CloudTrail is continuously recording API calls
  4. 4. Use cases enabled by CloudTrail  IT and security administrators can perform security analysis  IT administrators and DevOps engineers can track changes to AWS resources  DevOps engineers can troubleshoot operational issues  IT Auditors can use log files as a compliance aid Security at Scale: Logging in AWS White Paper
  5. 5. CloudTrail Availability & Service Coverage
  6. 6. What can you answer using a CloudTrail event?  Who made the API call?  When was the API call made?  What was the API call?  Which resources were acted up on in the API call?  Where was the API call made from and made to?
  7. 7. What does an event look like? { "eventVersion": "1.01", "userIdentity": { "type": "IAMUser", // Who? "principalId": "AIDAJDPLRKLG7UEXAMPLE", "arn": "arn:aws:iam::123456789012:user/Alice", //Who? "accountId": "123456789012", "accessKeyId": "AKIAIOSFODNN7EXAMPLE", "userName": "Alice", "sessionContext": { "attributes": { "mfaAuthenticated": "false", "creationDate": "2014-03-18T14:29:23Z" } } }, "eventTime": "2014-03-18T14:30:07Z", //When? "eventSource": "cloudtrail.amazonaws.com", "eventName": "StartLogging", //What? "awsRegion": "us-west-2",//Where to? "sourceIPAddress": "72.21.198.64", // Where from? "userAgent": "AWSConsole, aws-sdk-java/1.4.5 Linux/x.xx.fleetxen Java_HotSpot(TM)_64-Bit_Server_VM/xx", "requestParameters": { "name": "Default“ // Which resource? }, // more event details }
  8. 8. Getting Started
  9. 9. Turn on CloudTrail
  10. 10. Turn on CloudTrail
  11. 11. Turn on CloudTrail using AWS CLI Step 1: Create a trail $ aws cloudtrail create-trail --region=eu-central-1 --name ITAuditandOpsTrail --s3-bucket mybucket Step 2: Start logging on the trail $ aws cloudtrail start-logging --region=eu-central-1 --name ITAuditandOpsTrail
  12. 12. CloudTrail log files as compliance aid I need CloudTrail logs for entire month of Jan 2015 Lifecycle Policies
  13. 13. Monitor and Receive Notifications
  14. 14. What type of events should I monitor for?  You can monitor any specific event recorded by CloudTrail and receive notification from CloudWatch  Monitor for security or network related events that are likely to have a high blast radius  Popular examples based on customer feedback 1. Creation, deletion and modification of security groups and VPCs 2. Changes to IAM policies or S3 bucket policies 3. Failed AWS Management Console sign-in events 4. API calls that resulted in authorization failures 5. Launching, terminating, stopping, starting and rebooting EC2 instances  Fully defined and pre-built CloudFormation template to get started
  15. 15. Receive email notifications of specific API activity
  16. 16. How to configure CloudWatch Alarms?  Step 1: Configure CloudTrail to deliver logs to CloudWatch Logs $ aws cloudtrail update-trail --name mytrail --cloud-watch-logs-log-group-arn arn:aws:logs:us-west 2:111111111111:log group:CloudTrail/DefaultLogGroup12345:* --cloud-watch-logs-role-arn arn:aws:iam::111111111111:role/CloudTrail_CloudWatchLog s_Role
  17. 17. How to configure CloudWatch Alarms?  Step 2: Get started with a pre-built CloudFormation template $ aws cloudformation create-stack myCTCWAlarms --template-url CloudFormation_Template -–parameters ParameterKey=Email, ParameterValue=myemail@mydomain.com ParameterKey=LogGroupName, ParameterValue=CloudTrail/DefaultLogGroup12345
  18. 18. What does an email notification look like?
  19. 19. Troubleshoot operational and security issues
  20. 20. Troubleshooting operational and security issues  Look up CloudTrail events related to creation, deletion and modification of AWS resources  Look up events for the last 7 days  Filter events using one of the six different filters  Time range  User name  Resource name  Resource type  Event name  Event ID
  21. 21. Look up events in the CloudTrail console
  22. 22. Event detail view in the CloudTrail console
  23. 23. Look up events using the AWS CLI  List all events for the last 7 days $ aws cloudtrail lookup-events --output json  List all events where user name is root $ aws cloudtrail lookup-events --lookup-attributes AttributeKey=Username, AttributeValue=root --output=json  List all events where the Resource type is EC2 Instance $ aws cloudtrail lookup-events --lookup-attributes AttributeKey=ResourceType, AttributeValue=AWS::EC2::Instance --output=json
  24. 24. CloudTrail – Multiple AWS Accounts
  25. 25. Aggregate log files across multiple accounts in one bucket  You have multiple AWS accounts  You need to aggregate log files for all AWS accounts into one S3 bucket  You can configure CloudTrail to deliver log files for all of your AWS accounts to one S3 bucket  Files will be arranged per account and region in the S3 bucket for easier future access  <bucket_name>/optional_prefix_name/AWSLogs/Account_ID/CloudTrail/region/ YYYY/MM/DD/file_name.json.gz
  26. 26. How to aggregate log files across multiple accounts?  Step 1: Configure the S3 bucket policy //Partial S3 bucket policy shown below "Action": "s3:PutObject", "Resource": "arn:aws:s3:::myBucketName/[optional prefix]/AWSLogs/myAccountID/*", "Condition": { "StringEquals": { "s3:x-amz-acl": "bucket-owner-full-control" } } If you have 3 accounts, add three lines that correspond to those three accounts to the bucket policy "Resource": "arn:aws:s3:::myBucketName/[optional prefix]/AWSLogs/111111111111/*", "Resource": "arn:aws:s3:::myBucketName/[optional prefix]/AWSLogs/222222222222/*", "Resource": "arn:aws:s3:::myBucketName/[optional prefix]/AWSLogs/333333333333/*" Step 2: Turn on CloudTrail for three accounts in all regions
  27. 27. Encrypted CloudTrail log files New
  28. 28. Encrypted CloudTrail log files using SSE-KMS  By default, CloudTrail encrypts log files using S3 server side encryption  Additional layer of security for your log files by encrypting with your KMS key  Application logic for ingesting and processing log files stays the same  S3 will decrypt on your behalf if your credentials have decrypt permissions
  29. 29. Encrypting your log files using SSE KMS Encrypted CloudTrail log files Step 4: S3 GetObject API call Step 5: Decrypted CloudTrail log files Step 1: Create or use an existing KMS Key and apply policy Step 2: Grant decrypt access to log readers Step 3: Specify KMS key to CloudTrail
  30. 30. How to encrypt CloudTrail log files using your KMS Key?  Step 1: Create a KMS key and apply the correct Key Policy $ aws kms create-key $ aws kms put-key-policy --key-id arn:aws:kms:us-west- 2:111111111111:key/example-7ce3-41e9-a4a3- 167example --policy-name default -–policy file://</local/directory>
  31. 31. How to encrypt CloudTrail log files using KMS? Step 2: Grant decrypt access to users, roles or groups that access CloudTrail log files Attach this policy to IAM user Bob or IAM group or role CloudTrail-log-readers { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "kms:Decrypt", "Resource": "arn:aws:kms:us-west- 2::111111111111:key/example-7ce3-41e9-a4a3-167example" } ] }
  32. 32. How to encrypt CloudTrail log files using KMS? Step 3: Update your trail to provide the KMS encryption key to CloudTrail $ aws cloudtrail update-trail --region=us-west-2 --name Trail-Name --kms-key-id arn:aws:kms:us-west- 2:111111111111:key/example-7ce3-41e9- a4a3-167example
  33. 33. Encrypt log files for multiple accounts using one KMS key  Step 1: Update the key policy to allow CloudTrail to use the key for multiple accounts //Partial KMS key policy Action": "kms:GenerateDataKey*", "Resource": "*", "Condition": { "StringLike": { "kms:EncryptionContext:aws:cloudtrail:arn": [ "arn:aws:cloudtrail:*:111111111111:trail/*", "arn:aws:cloudtrail:*:222222222222:trail/*“, "arn:aws:cloudtrail:*:333333333333:trail/*" ] } }
  34. 34. Encrypt log files for multiple accounts using one KMS key Step 2: Update trail in accounts 222222222222 and 33333333333 with the key belonging to account 111111111111 $ aws cloudtrail update-trail --region=us-west-2 --name Trail-account222222222222 --kms-key-id arn:aws:kms:us-west-2::111111111111:key/example-7ce3- 41e9-a4a3-167example $ aws cloudtrail update-trail --region=us-west-2 --name Trail-account333333333333 --kms-key-id arn:aws:kms:us-west-2::111111111111:key/example-7ce3- 41e9-a4a3-167example
  35. 35. Validate the integrity of log files New
  36. 36. CloudTrail log file integrity validation  Validate that a log file has not been changed since CloudTrail delivered the log file to your S3 bucket  Detect whether a log file was deleted or modified or unchanged  Use the tool as an aid in your IT security, audit and compliance processes
  37. 37. CloudTrail log file integrity validation  Enable digest file delivery on your trail aws cloudtrail update-trail --region=us-west-2 --name trail-name --enable-log-file-validation CloudTrail will start delivering digest files on an hourly basis  Digest files contain hash values of log files delivered and are signed by CloudTrail  CloudTrail delivers the log files to the same S3 bucket, but a different folder structure or S3 key map  CloudTrail log files: /AWSLogs/111111111111/CloudTrail/  CloudTrail digest files: /AWSLogs/111111111111/CloudTrail-Digest/
  38. 38. How do I validate the integrity of the log files?  Download the latest AWS CLI or build your own tool  Ensure that you have access to CloudTrail log files stored in S3  If your log files are encrypted, you need decrypt permissions $ aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:us-west-2:111111111111:trail/Trailname --start-time 2015-09-24T00:00:00Z --region=us-west-2
  39. 39. Example 1:Log files are unchanged since CloudTrail delivered $ aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:us-west-2:111111111111:trail/Trailname --start-time 2015-09-24T00:00:00Z --region=us-west-2 Validating log files for trail arn:aws:cloudtrail:us-west- 2:111111111111:trail/Trailname between 2015-09-24T00:00:00Z and 2015-09- 25T18:56:41Z Results requested for 2015-09-24T00:00:00Z to 2015-09-25T18:56:41Z Results found for 2015-09-24T00:30:26Z to 2015-09-25T18:56:41Z: 43/43 digest files valid 31/31 log files valid
  40. 40. Example 2:Log file(s) are deleted since CloudTrail delivered $ aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:us-west-2:111111111111:trail/Trailname --start-time 2015-09-24T00:00:00Z --region=us-west-2 Log file s3://mybucket-CTlogs/AWSLogs/111111111111/CloudTrail/us-west- 2/2015/09/22/111111111111_CloudTrail_us-west- 2_20150922T1720Z_Jy4SwZotr3eTI2FM.json.gz INVALID: not found Results requested for 2015-09-22T00:00:00Z to 2015-09-25T18:42:03Z Results found for 2015-09-22T00:30:26Z to 2015-09-25T18:42:03Z: 43/43 digest files valid 30/31 log files valid, 1/31 log files INVALID
  41. 41. Example 3:Log file(s) are modified since CloudTrail delivered $ aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:us-west-2:111111111111:trail/Trailname --start-time 2015-09-24T00:00:00Z --region=us-west-2 Log file s3://mybucket-CTlogs/AWSLogs/111111111111/CloudTrail/us-west- 2/2015/09/25/111111111111_CloudTrail_us-west- 2_20150925T1845Z_lU58MiCsXyI1U3R1.json.gz INVALID: hash value doesn't match Results requested for 2015-09-24T00:00:00Z to 2015-09-25T21:44:50Z Results found for 2015-09-24T00:30:26Z to 2015-09-25T21:44:50Z: 45/45 digest files valid 35/36 log files valid, 1/36 log files INVALID
  42. 42. Partner Solutions integrated with CloudTrail
  43. 43. AWS Technology Partner solutions integrated with CloudTrail New
  44. 44. AWS Consulting Partner solutions integrated with CloudTrail
  45. 45. Updated Splunk integration for AWS CloudTrail
  46. 46. AWS CloudTrail Key Launches In 2015 Launch Date Feature Description 10/01/2015 Support for SSE KMS encryption & Log file integrity validation 09/01/2015 Support for S3 bucket level API activity 05/29/2015 Support for DynamoDB Table level API activity 05/19/2015 Support for CloudTrail integration with CloudWatch Logs in Northern California 04/09/2015 Support for Lambda, WorkSpaces, and EC2 Container Service 03/12/2015 Support for Lookup Events 03/05/2015 Support for CloudTrail integration with CloudWatch Logs in Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), and EU (Frankfurt)
  47. 47. Related Sessions  SEC314 – AWS Config/Config Rules: Use Config Rules to Improve Governance over Configuration Changes to Your Resources  5:30 PM Thursday Palazzo K  11:30 AM Friday Marcello 4506  DVO 303: Scaling Infrastructure Operations with Service Catalog, CloudTrail and Config  9 00 AM Friday Lido 3001B  SEC403 - Timely Security Alerts and Analytics: Diving into AWS CloudTrail Events by Using Apache Spark on Amazon EMR  10 15 AM Friday Marcello 4506
  48. 48. I would love to see you  Turn on CloudTrail for your accounts  Monitor and alarm for API activity with high blast radius  Use Lookup Events to troubleshoot your operational issues  Provide us feedback on what we should do next
  49. 49. Remember to complete your evaluations!
  50. 50. Thank you!
  51. 51. Questions

×