![AWS Metadata Service in action
Find a vulnerable EC2 Instance and execute this:
[ec2-user@ip-10-XX-XX-234 ~]$ role_name=$( curl -s
http://169.254.169.254/latest/meta-data/iam/security-credentials/ )
[ec2-user@ip-10-XX-XX-234 ~]$ curl -s http://169.254.169.254/latest/meta-
data/iam/security-credentials/${role_name}
{
"Code" : "Success",
"LastUpdated" : "2018-04-23T13:02:26Z",
"Type" : "AWS-HMAC",
"AccessKeyId" : "ASIAR5DV6JBZRSJH2ZZA",
"SecretAccessKey" : "eEuERNfOTHISISEXPIRED/9Ha0YGZ6bd",
"Token" : "FQoDYXTHISISNOTAREALSESSIONTOKENTHISISJUSTATRIBUTEL31gU=",
"Expiration" : "2018-04-23T19:06:48Z"](https://image-private.slidesharecdn.com/capitalonehack-191110000629/85/slide-13-320.jpg?hdnea=acl=/capitalonehack-191110000629/85/slide-13-320.jpg*~exp=1765690240~hmac=dbff11f59c5179d916aac71870e0f5c72167e4c369f311e1a88b2d5aea5d1f89&cb=1573346108)
Capital One Hack - An Analysis
- 1. Chris Farris chrisf@primeharbor.com https://www.chrisfarris.com/ What's inyour Bucket? The Capital One Hack
- 2. Disclaimer ● I donot work for Capital One ● I do not have knowledge of the Hack ● These thoughts are mine and mine alone and do not represent those of my employer, its parent, or SANS.
- 3. Credit ● Some ofthis material is lifted from Eric Johnson's excellent talk: Cloud Security: Attacking The Metadata Service https://pumascan.com/resources/cloud-security-instance-metadata/
- 4. Capital One ● Authorof Cloud Custodian ● 16 Sessions at re:Invent ● 32 Cloud Blog Posts ○ https://medium.com/capital-one-tech/cloud/home
- 5. What we knowfrom the indictment ● Attacker was Paige Thompson ● She worked at AWS from 2015 to 2016 ● Activity occurred from March to July 2019 ● Capital One is not her only victim ○ Victim 2 - State Agency not in Washington state ○ Victim 3 - International Telecom Conglomerate ○ Victim 4 - Public Research University
- 6. What we knowfrom the indictment ● 30 other victims are alleged ● Her primary goal was Crypto Mining ● Data was exfiltrated outside of AWS ○ IPredator and TOR were leveraged ● Capital One didn't detect it. ○ They got a responsible disclosure
- 7. What we knowfrom the indictment ● Compromised Credentials came from an AWS Role "*****-WAF-Role" ● Compromised Credentials were used to ListBuckets ○ Then to Sync the data ● Data was stored in GitHub ○ 700 "folders or buckets" ○ 100,000,000 records were involved
- 8. What we canguess from the indictment ● Credentials were pulled from Metadata service ○ Multiple references to a Role ○ "obtained security credentials for an account known as ****-WAF-Role" ○ SSRF was potentially involved
- 9.
- 10. What we don'tknow ● What was the firewall misconfiguration? ● That a "firewall" was even involved ○ Role says WAF ○ Does firewall mean security group? Happened to Tesla ● How the heck GuardDuty didn't detect it?
- 11. Some Background onAWS Security
- 12. How IAM Works IAMRole for CoolApp Lambda Function B Writes to S3 Bucket "Sid":"AllowCoolAppToReadWriteObjectData", "Action":"s3:*", "Effect":"Allow", "Resource":"*" S3 teambucket EC2 Instances Reads From S3 Bucket
- 13. AWS Metadata Servicein action Find a vulnerable EC2 Instance and execute this: [ec2-user@ip-10-XX-XX-234 ~]$ role_name=$( curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/ ) [ec2-user@ip-10-XX-XX-234 ~]$ curl -s http://169.254.169.254/latest/meta- data/iam/security-credentials/${role_name} { "Code" : "Success", "LastUpdated" : "2018-04-23T13:02:26Z", "Type" : "AWS-HMAC", "AccessKeyId" : "ASIAR5DV6JBZRSJH2ZZA", "SecretAccessKey" : "eEuERNfOTHISISEXPIRED/9Ha0YGZ6bd", "Token" : "FQoDYXTHISISNOTAREALSESSIONTOKENTHISISJUSTATRIBUTEL31gU=", "Expiration" : "2018-04-23T19:06:48Z"
- 14. And how theycan be used Patton:~ chris$ export AWS_ACCESS_KEY_ID=ASIAR5DV6JBZRSJH2ZZA Patton:~ chris$ export AWS_SECRET_ACCESS_KEY=eEuERNfOTHISISEXPIRED/9Ha0YGZ6bd Patton:~ chris$ export AWS_SESSION_TOKEN=FQoDYXTHISISNOTAREALSESSIONTOKENTHISISJUSTATRIBUTEL31gU= Patton:~ chris$ aws s3 ls 2018-03-03 08:15:24 p0wnage-this-is-your-bucket
- 15. Andres Riancho, 2014:Pivoting in Amazon Clouds
- 16. GuardDuty AWS ML appliedto - CloudTrail - VPC Flowlogs - Known threat actors UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration Credentials that were created exclusively for an EC2 instance through an instance launch role are being used from an external IP address.
- 17. Will Bengston -Netflix Information Security: Preventing Credential Compromise in AWS (2018)
- 18. What we knewbefore the attack ● AWS Metadata service was subject to SSRF ● GuardDuty doesn't detect credentials used inside another AWS customer's VPC ● GuardDuty wasn't universally available
- 19. What we cantheorize happened ● Attacker finds SSRF in Capital One server ● Leverages SSRF to extract Session Creds ● Capital One's server had the S3ReadOnly or S3FullAccess policy attached ● Session Creds were used to enumerate all the buckets ● Exfil occurred outside watch of GuardDuty
- 20. Unanswered Questions ● Howthe hell did GuardDuty miss this? ● What was running on the server that was compromised? ○ Is that vulnerability still out there?
- 21.
- 22. Primer on SharedResponsibility AWS Foundation Services Compute Storage Database Networking AWS Global Infrastructure Regions Availability Zones Edge Locations Client-side Data Encryption Server-side Data Encryption Network Traffic Protection Platform, Applications, Identity & Access Management Operating System, Network, and Firewall Configuration Customer applications & content You
- 23. How Capital OneScrewed up ● Overly permissive ****-WAF-Role ● Application had an SSRF (or RCE) ● Kept 100m credit card applications in a bucket ○ Probably didn't have a sufficient BucketPolicy ● Didn't detect it / relied on GuardDuty
- 24. Bucket Policy ● Securityconditions on a bucket ○ Usually these are screwed up and make the bucket public ● Bucket Policy could have made the bucket more secure ○ Require all access from a VPCEndpoint (not a TOR node) ○ Limit access to specific IP Addresses ○ Deny access except to explicit roles
- 25. Where AWS Failedits customers ● Metadata issues have been known for awhile ● GuardDuty failed to detect ○ AWS opened a region (Stockholm) and didn't support GuardDuty for six months
- 26.
- 27. Scott Piper -Feature Request: Provide a way to better protect metadata service (2018)
- 28. And now Congressis involved
- 29.
Editor's Notes
- #4 A few disclaimers - I have no relationship with Capital One nor do I have any inside knowledge of the hack The opinions reflected here have nothing to do with my employer or SANS
- #5 Some of the slides and material is inspired by Eric Johnson's blog post and slides from his talk for SANS
- #7 FakeNews Debunk #1 - This wasn't an inside job Attacker worked for AWS in 2015 and 2016 The attack was in 2019 Capital One is currently the only named victim.
- #8 But 30 more victims are alleged. Her primary goal was to Crypto Mine in victim accounts Her discovery of the personal financial info was by luck What is interesting is that Capital One didn't detect this data exfil or credential compromise
- #9 This was not a public bucket She compromised credentials and accessed a bucket as if she was Capital One Per the indictment she enumerated over 700 buckets in the Capital One account and got over 100m records
- #10 Now we move slightly into speculation There were multiple references to a Role and obtaining credentials From this, the general speculation was a Server Side Request Forgery
- #11 AWS mostly admits as much
- #12 There are still several unanswered questions What was this firewall misconfiguration? Everyone says "firewall" no one is saying WAF Was a firewall even involved, or by firewall did they mean a security group? Tesla got popped from a permissive security group allowing access to k8s But GuardDuty has detections for this right?
- #13 Lets dive into some of the technology involved here....
- #14 IAM is the service that grants permissions to AWS services.
- #15 The AWS Metadata Service is part of the AWS Hypervisor and allows the guest OS to get information about itself and its environment The IP address 169.254.169.254 exists only to the guest. The AWS CLI and SDKs know to get temporary credentials by reaching to the Metadata service on that IP. But as you can see you can get it via CURL
- #16 This is how an attacker can leverage those creds
- #17 This is not a new vector. Andres Riancho did a presentation in 2014 on this topic.
- #18 At re:Invent 2017, AWS Released GuardDuty. It leverages ML magic to analize cloudtrail, VPC flowlogs and DNS queries One of the detections is a finding when Instance Profile credentials are used outside of AWS. Note the "external IP address". It does not mean external to your account or external to your VPC. Which is why....
- #19 Will Bengston from Netflix create this post in 2018. It outlined how Netflix was addressing the flaws in the GuardDuty's logic for the InstanceCredentialExfiltration
- #20 Before the compromise we knew that Metadata was subject to SSRF We knew GuardDuty wasn't sufficient to protect against exfil inside AWS But the attacker was using TOR and IPredetor We also new GuardDuty wasn't available in the new Stockholm region during the time of the attack.
- #21 At this point we move into speculation Attacker discovered an SSRF in a CapOne server She get some creds. From those creds she discovers she has broad access to S3 She enumerates all the buckets and finds a jackpot Somehow the exfiltration occurred outside the watch of GuardDuty
- #23 How about a little finger pointing
- #24 Shared Responsibility model says that Capital One was responsible for their Security In the Cloud.
- #25 What were Capital One's failures? Overly permissive WAF Role was one. They had the SSRF, and that was clearly on them to have detected and resolved. Leaving 100m old credit applications in an active AWS account was probably the wrong move. Archiving that data where no one had access would have helped They didn't detect it, or they relied solely on GuardDuty to detect.
- #26 What could they have done with that data Had a bucket policy that limited access to only a specific VPC or IP Range
- #27 What about security OF the Cloud? SSRF against Metadata has been known for awhile. GCP and Azure have mitigations against it AWS opened a region and didn't Deploy all the proper security controls And didn't allow their customers to opt-out of using that region till the controls were made available. Fundamentally, AWS sells these security services that we rely on, but we cannot actually rely on them.
- #28 AWS deflects Tries to sell Macie - ha! "several sub-systems deep in the tech stack"?
- #29 Scott Piper opened the feature request to fix the Metadata service in August of 2018
- #30 Of course out politicians have to get their mugs in front of any issues. Here is a letter to the Federal Trade Commission calling for an investigation into AWS