Hunting for the secrets in
a cloud forest
Paweł Rzepa
CONFidence, 4th June 2018
• Senior Security Consultant in SecuRing
• Pentesting
• Consultancy in cloud security
• Blog: https://medium.com/@rzepsky
• GitHub: https://github.com/xep624/
• Twitter: @Rzepsky
#whoami
The goal of this presentation is to show how
access keys may leak from your company
regardless service provider you use (AWS, Azure,
GCP etc.) and to discuss reliable
countermeasures.
TL;DR
Rzepsky
• Passwords vs keys
• Key leaks from storage containers
• Key leaks via compromised accounts
• Key leaks from web apps
• Key leaks over 3rd parties
• How entropy can help?
• Let’s hunt with DumpsterDiver!
• Countermeasures
Presentation plan
Rzepsky
Passwords vs Keys
Access key ID =
AKIAJIS2NP37SW1AYBH
A
Secret access key =
nTRcofv3N9ls6MqFhsR8lx
Qp+aNfoDv+2lXzv9nT
Login = admin
Password = Dupa.8
VS
Rzepsky
Passwords vs Keys
Source:	https://www.blackhat.com/docs/us-16/materials/us-16-Simon-Access-Keys-Will-Kill-You-Before-You-Kill-The-Password.pdf
Rzepsky
• Passwords vs keys
• Key leaks from storage containers
• Key leaks via compromised accounts
• Key leaks from web apps
• Key leaks over 3rd parties
• How entropy can help?
• Let’s hunt with DumpsterDiver!
• Countermeasures
Presentation plan
Rzepsky
Wrong access control
Sometimes	all	you	
have	to	do	is…
just asking!
Rzepsky
Low hanging fruits:
*.db
*.sql
*.config
*backup*
Define a target
Rzepsky
https:/github.com/securing/BucketScanner/
https://[bucketname].s3.amazonaws.com
https://[aws_endpoint].amazonaws.com/[bucket_name]/
For example: https://chicagodb.s3.amazonaws.com/
Source: https://www.upguard.com/breaches/cloud-leak-chicago-voters
Let’s find open buckets
Rzepsky
Querying Google, Yahoo, VirusTotal, Certificate
Transparency Logs etc. (e.g. Sublist3r, amass, bucket-stream)
Querying 3rd parties
Rzepsky
Querying a domain in archive.org
Wayback Machine
Rzepsky
• Found 24652 buckets
• 5241 (21%) of them has public READ access
• And amongst them...
Results
Rzepsky
Rzepsky
Some of them are quite interesting…
Rzepsky
Some of them are quite interesting…
Rzepsky
• There is no groups like “Any authenticated Azure user” (thanks Microsoft!)
• You have to discover 2 variables instead of 1 (consider only Full public read access):
http://[storage account name].blob.core.windows.net/[container
name]?restype=container&comp=list
What about Azure?
Rzepsky
• Passwords vs keys
• Key leaks from storage containers
• Key leaks via compromised accounts
• Key leaks from web apps
• Key leaks over 3rd parties
• How entropy can help?
• Let’s hunt with DumpsterDiver!
• Countermeasures
Presentation plan
Rzepsky
Leaks via
compromised
accounts
Overwrite
trusted	file
Customer	downloads
Term_of_use.pdf
During my	research I‘ve
found 1365	(6%	of tested
buckets)	buckets which
allow for writing (and	
overwriting)	arbitrary file
Rzepsky
• Numerous ways of infecting employee’s computer
• Leaks via:
• Local config files, tools etc.
• ~/.aws/credentials
---------------------------------------------------------------------------------------------
• Enforcing MFA is a must!!! à https://bit.ly/2oYKBmf
• Remember about the principle of least privilege (e.g. Repokid
may help you à https://bit.ly/2kUT3Bq)
Leaks via compromised accounts
Rzepsky
• Passwords vs keys
• Key leaks from storage containers
• Key leaks via compromised accounts
• Key leaks from web apps
• Key leaks over 3rd parties
• How entropy can help?
• Let’s hunt with DumpsterDiver!
• Countermeasures
Presentation plan
Rzepsky
Key leaks via web apps
2.	I’m	authenticated	
user,	pls gimme keys
API
3.	Upload	a	file	directly	
to	the	bucket
1.	I	want	to	
upload	a	file
Rzepsky
Key leaks via metadata
Rzepsky
Some vulns can be much more dangerous in cloud:
§ CWE-200: Information Exposure
§ CWE-441: Unintended Proxy or Intermediary
§ CWE-611: XXE
§ CWE-918: SSRF
…because any of them may reveal your metadata!!!
Old vulns gain new life
Rzepsky
Unintended	proxy	example
• Data about your instance:
• Accessible only from within the instance
itself via link:
http://169.254.169.254/latest/meta-
data/
What is “meta-data”
Rzepsky
• Usually, automated tools fail in detecting such leaks
• But penetration tests are remedium
How to catch such leaks?
Rzepsky
• Passwords vs keys
• Key leaks from storage containers
• Key leaks via compromised accounts
• Key leaks from web apps
• Key leaks over 3rd parties
• How entropy can help?
• Let’s hunt with DumpsterDiver!
• Countermeasures
Presentation plan
Rzepsky
Key leaks over 3rd parties
GitHub
Pastebin
Forums
etc…
Rzepsky
Key leaks over 3rd parties
Rzepsky
You don’t have
to use GitHub to
see your keys
there…
Story	details:	https://www.olindata.com/en/blog/2017/04/spending-100k-usd-45-days-amazon-
web-services
Rzepsky
• Before releasing any repo – just scan it:
• TruffleHog (https://github.com/dxa4481/truffleHog)
• git-secrets (https://github.com/awslabs/git-secrets)
• Add it to continuous integration process
Catch git leaks!
Rzepsky
What about
creating my own
scanner, which
can catch a leak
in any kind of
file?
Rzepsky
• Passwords vs keys
• Key leaks from storage containers
• Key leaks via compromised accounts
• Key leaks from web apps
• Key leaks over 3rd parties
• How entropy can help?
• Let’s hunt with DumpsterDiver!
• Countermeasures
Presentation plan
Rzepsky
Manual search is ineffective - PoC
Define	your	
target
Specify	target’s	
characteristics
Locate	the	
target
Find a Pepsi
on a next slide…
Rzepsky
Find the Pepsi bottle!
Rzepsky
• They have fixed length
• All chars from Base64 charset
• They are random = they have high entropy
AWS_SECRET_ACCESS_KEY =
2r9pAuQxUFAqtrWhEy4G4WiVx5iJ74Hja5AWgHq9
Shared_Key =
M3mmbjOlIZr11OZoULqUWyFA1EpOdZAEcmaC64E/Ft9
MRfDEYE7qDJm+9ezGQY15==
Specify keys characteristics
Rzepsky
The entropy = disorder
HIGH ENTROPY LOW ENTROPY
Source:	http://awesomenator.com/fun/rearranging-the-world-chaos-vs-order/
Rzepsky
Entropy: how to count it?
P( ) = 1
low
entropy
P( ) = 0,75
P( ) = 0,25
medium
entropy
P( ) = 0,5
P( ) = 0,5
high
entropy
Rzepsky
Entropy: how to count it?
Source:	https://en.wikipedia.org/wiki/Claude_Shannon#/media/File:ClaudeShannon_MFO3807.jpg
Rzepsky
Shannon entropy in practice
• Hash
404e554d243c1a11d13c96b60129504a31b0abd has 3.57 entropy
• Long string
“ ChuckNorriscountedtoinfinitytwentytwice” has 3.81 entropy
“Where_are_my_keys?!¯_(ツ)_/¯” contains characters out of Base64
• AWS secret key
2r9pAuQxUFAstrWhEy4G4WiVx5iJ74Hja5AWgHq9 has 4.67 entropy
Interesting fact: AWS	secret	key	has always	entropy	>	4.3
Rzepsky
• Passwords vs keys
• Key leaks from storage containers
• Key leaks via compromised accounts
• Key leaks from web apps
• Key leaks over 3rd parties
• How entropy can help?
• Let’s hunt with DumpsterDiver!
• Countermeasures
Presentation plan
Rzepsky
Let’s hunt with DumpsterDiver!
Rzepsky
DumpsterDiver – main features
• It’s open-source!
• It uses Shannon Entropy to find
private keys
• It searches inside compressed
archives (e.g. zip, tar.gz etc.)
• It searches through the git
repositories
• It supports advanced search using
simple rules
Rzepsky
https://github.com/securing/DumpsterDiver
• Triggers if it finds
“aws_secret_access_key”
• Triggers if it finds 10 emails in
.db or .sql file
• Triggers if it finds any of the
pattern: *pass*, *haslo*, *key*
Advanced search - allows for
creating additional rules
Rzepsky
• Scanning big volumes of data is time consuming L
• DumpsterDiver will quickly tell you if you just got an
access to a treasure J
Use case scenario 1: for pentesters/researchers
Rzepsky
Use case scenario 2:
create quasi cloud
data leak prevention
system
Rzepsky
Use case scenario 3: up to you! Feedback,
suggestions, ideas
and/or
contributors ARE
MORE THAN
WELCOME!!!
Rzepsky
• Passwords vs keys
• Key leaks from storage containers
• Key leaks via compromised accounts
• Key leaks from web apps
• Key leaks over 3rd parties
• How entropy can help?
• Let’s hunt with DumpsterDiver!
• Countermeasures
Presentation plan
Rzepsky
• Set proper access control to your resources
• Encrypt files at rest
• Allow access only from VPN
• Enforce using MFA
• Create a process of verifying stored data (DumpsterDiver)
• Test your environment
Countermeasures
Rzepsky
Extras: hunt the keys (legally)
https://www.securing.biz/krkanalytica
Rzepsky
Thank you,
pawel.rzepa@securing.pl
@Rzepsky

Hunting for the secrets in a cloud forest

  • 1.
    Hunting for thesecrets in a cloud forest Paweł Rzepa CONFidence, 4th June 2018
  • 2.
    • Senior SecurityConsultant in SecuRing • Pentesting • Consultancy in cloud security • Blog: https://medium.com/@rzepsky • GitHub: https://github.com/xep624/ • Twitter: @Rzepsky #whoami
  • 3.
    The goal ofthis presentation is to show how access keys may leak from your company regardless service provider you use (AWS, Azure, GCP etc.) and to discuss reliable countermeasures. TL;DR Rzepsky
  • 4.
    • Passwords vskeys • Key leaks from storage containers • Key leaks via compromised accounts • Key leaks from web apps • Key leaks over 3rd parties • How entropy can help? • Let’s hunt with DumpsterDiver! • Countermeasures Presentation plan Rzepsky
  • 5.
    Passwords vs Keys Accesskey ID = AKIAJIS2NP37SW1AYBH A Secret access key = nTRcofv3N9ls6MqFhsR8lx Qp+aNfoDv+2lXzv9nT Login = admin Password = Dupa.8 VS Rzepsky
  • 6.
  • 7.
    • Passwords vskeys • Key leaks from storage containers • Key leaks via compromised accounts • Key leaks from web apps • Key leaks over 3rd parties • How entropy can help? • Let’s hunt with DumpsterDiver! • Countermeasures Presentation plan Rzepsky
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
    Querying Google, Yahoo,VirusTotal, Certificate Transparency Logs etc. (e.g. Sublist3r, amass, bucket-stream) Querying 3rd parties Rzepsky
  • 13.
    Querying a domainin archive.org Wayback Machine Rzepsky
  • 14.
    • Found 24652buckets • 5241 (21%) of them has public READ access • And amongst them... Results Rzepsky
  • 15.
  • 16.
    Some of themare quite interesting… Rzepsky
  • 17.
    Some of themare quite interesting… Rzepsky
  • 18.
    • There isno groups like “Any authenticated Azure user” (thanks Microsoft!) • You have to discover 2 variables instead of 1 (consider only Full public read access): http://[storage account name].blob.core.windows.net/[container name]?restype=container&comp=list What about Azure? Rzepsky
  • 19.
    • Passwords vskeys • Key leaks from storage containers • Key leaks via compromised accounts • Key leaks from web apps • Key leaks over 3rd parties • How entropy can help? • Let’s hunt with DumpsterDiver! • Countermeasures Presentation plan Rzepsky
  • 20.
    Leaks via compromised accounts Overwrite trusted file Customer downloads Term_of_use.pdf During my researchI‘ve found 1365 (6% of tested buckets) buckets which allow for writing (and overwriting) arbitrary file Rzepsky
  • 21.
    • Numerous waysof infecting employee’s computer • Leaks via: • Local config files, tools etc. • ~/.aws/credentials --------------------------------------------------------------------------------------------- • Enforcing MFA is a must!!! à https://bit.ly/2oYKBmf • Remember about the principle of least privilege (e.g. Repokid may help you à https://bit.ly/2kUT3Bq) Leaks via compromised accounts Rzepsky
  • 22.
    • Passwords vskeys • Key leaks from storage containers • Key leaks via compromised accounts • Key leaks from web apps • Key leaks over 3rd parties • How entropy can help? • Let’s hunt with DumpsterDiver! • Countermeasures Presentation plan Rzepsky
  • 23.
    Key leaks viaweb apps 2. I’m authenticated user, pls gimme keys API 3. Upload a file directly to the bucket 1. I want to upload a file Rzepsky
  • 24.
    Key leaks viametadata Rzepsky
  • 25.
    Some vulns canbe much more dangerous in cloud: § CWE-200: Information Exposure § CWE-441: Unintended Proxy or Intermediary § CWE-611: XXE § CWE-918: SSRF …because any of them may reveal your metadata!!! Old vulns gain new life Rzepsky
  • 26.
  • 27.
    • Data aboutyour instance: • Accessible only from within the instance itself via link: http://169.254.169.254/latest/meta- data/ What is “meta-data” Rzepsky
  • 28.
    • Usually, automatedtools fail in detecting such leaks • But penetration tests are remedium How to catch such leaks? Rzepsky
  • 29.
    • Passwords vskeys • Key leaks from storage containers • Key leaks via compromised accounts • Key leaks from web apps • Key leaks over 3rd parties • How entropy can help? • Let’s hunt with DumpsterDiver! • Countermeasures Presentation plan Rzepsky
  • 30.
    Key leaks over3rd parties GitHub Pastebin Forums etc… Rzepsky
  • 31.
    Key leaks over3rd parties Rzepsky
  • 32.
    You don’t have touse GitHub to see your keys there… Story details: https://www.olindata.com/en/blog/2017/04/spending-100k-usd-45-days-amazon- web-services Rzepsky
  • 33.
    • Before releasingany repo – just scan it: • TruffleHog (https://github.com/dxa4481/truffleHog) • git-secrets (https://github.com/awslabs/git-secrets) • Add it to continuous integration process Catch git leaks! Rzepsky
  • 34.
    What about creating myown scanner, which can catch a leak in any kind of file? Rzepsky
  • 35.
    • Passwords vskeys • Key leaks from storage containers • Key leaks via compromised accounts • Key leaks from web apps • Key leaks over 3rd parties • How entropy can help? • Let’s hunt with DumpsterDiver! • Countermeasures Presentation plan Rzepsky
  • 36.
    Manual search isineffective - PoC Define your target Specify target’s characteristics Locate the target Find a Pepsi on a next slide… Rzepsky
  • 37.
    Find the Pepsibottle! Rzepsky
  • 38.
    • They havefixed length • All chars from Base64 charset • They are random = they have high entropy AWS_SECRET_ACCESS_KEY = 2r9pAuQxUFAqtrWhEy4G4WiVx5iJ74Hja5AWgHq9 Shared_Key = M3mmbjOlIZr11OZoULqUWyFA1EpOdZAEcmaC64E/Ft9 MRfDEYE7qDJm+9ezGQY15== Specify keys characteristics Rzepsky
  • 39.
    The entropy =disorder HIGH ENTROPY LOW ENTROPY Source: http://awesomenator.com/fun/rearranging-the-world-chaos-vs-order/ Rzepsky
  • 40.
    Entropy: how tocount it? P( ) = 1 low entropy P( ) = 0,75 P( ) = 0,25 medium entropy P( ) = 0,5 P( ) = 0,5 high entropy Rzepsky
  • 41.
    Entropy: how tocount it? Source: https://en.wikipedia.org/wiki/Claude_Shannon#/media/File:ClaudeShannon_MFO3807.jpg Rzepsky
  • 42.
    Shannon entropy inpractice • Hash 404e554d243c1a11d13c96b60129504a31b0abd has 3.57 entropy • Long string “ ChuckNorriscountedtoinfinitytwentytwice” has 3.81 entropy “Where_are_my_keys?!¯_(ツ)_/¯” contains characters out of Base64 • AWS secret key 2r9pAuQxUFAstrWhEy4G4WiVx5iJ74Hja5AWgHq9 has 4.67 entropy Interesting fact: AWS secret key has always entropy > 4.3 Rzepsky
  • 43.
    • Passwords vskeys • Key leaks from storage containers • Key leaks via compromised accounts • Key leaks from web apps • Key leaks over 3rd parties • How entropy can help? • Let’s hunt with DumpsterDiver! • Countermeasures Presentation plan Rzepsky
  • 44.
    Let’s hunt withDumpsterDiver! Rzepsky
  • 45.
    DumpsterDiver – mainfeatures • It’s open-source! • It uses Shannon Entropy to find private keys • It searches inside compressed archives (e.g. zip, tar.gz etc.) • It searches through the git repositories • It supports advanced search using simple rules Rzepsky
  • 46.
  • 47.
    • Triggers ifit finds “aws_secret_access_key” • Triggers if it finds 10 emails in .db or .sql file • Triggers if it finds any of the pattern: *pass*, *haslo*, *key* Advanced search - allows for creating additional rules Rzepsky
  • 48.
    • Scanning bigvolumes of data is time consuming L • DumpsterDiver will quickly tell you if you just got an access to a treasure J Use case scenario 1: for pentesters/researchers Rzepsky
  • 49.
    Use case scenario2: create quasi cloud data leak prevention system Rzepsky
  • 50.
    Use case scenario3: up to you! Feedback, suggestions, ideas and/or contributors ARE MORE THAN WELCOME!!! Rzepsky
  • 51.
    • Passwords vskeys • Key leaks from storage containers • Key leaks via compromised accounts • Key leaks from web apps • Key leaks over 3rd parties • How entropy can help? • Let’s hunt with DumpsterDiver! • Countermeasures Presentation plan Rzepsky
  • 52.
    • Set properaccess control to your resources • Encrypt files at rest • Allow access only from VPN • Enforce using MFA • Create a process of verifying stored data (DumpsterDiver) • Test your environment Countermeasures Rzepsky
  • 53.
    Extras: hunt thekeys (legally) https://www.securing.biz/krkanalytica Rzepsky
  • 54.