Crypto Miners in the Cloud
BSides Vancouver
Teri Radichel
2nd Sight Lab
@teriradichel
Agenda
• Crypto Mining + Cryptojacking
• Some malicious examples…
• How they [may have] happened
• Indicators of compromise
• What you can do about it
Crypto Mining
Cryptojacking Using YOUR resources without your permission to
mine cryptocurrency.
Types of Coin
• Bitcoin – massive mining rig, high fees
• Litecoin – lower transaction fees
• Monero – anonymity, mine on CPUs
• Many More… zcash, dash…
Case Study:
Double Crypto
• Ransomware and Cryptomining software
• Had to get past the ransomware
• Trying to figure out how it was deployed
• Found the cryptomining software
Mining in the Cloud
Attacker uploads or
downloads crypto
mining software to
your virtual machines
running in the cloud
First Steps
• Logs: VPC Flow logs, CloudTrail, etc.
• Turn on termination protection
• Take snapshot of EBS Volume(s)
• Create image, memory capture
• Shared copies out to separate account
• Quarantine via security group rules
• Created a bastion host to access the instances
VPN
connection
VPC subnet NACL limits from
responder on remote access ports
and return ephemeral ports
SG1 security group limits access inbound
from responder. Outbound to SG2
Bastion Host Quarantine Infected Host
Dedicated
Machine
SG2 security group limits access
inbound from SG2 - no outbound
Quarantine infected instances
Application subnet - access from incident
responder subnet on remote access port
and ephemeral return ports
Bypass the
Ransomware
• Figured out how to get to […] in UI
• Open Windows Explorer
• Recursively make files readable from root
• Woot! (Some things still inaccessible…)
• Navigate and open cmd.exe
• Usually C:WindowsSystem32cmd.exe
Services
• A bunch of strange services
• Names ~ random four letters
• Paths including domain names and IPs
• Many stopped (service timeout?)
• Using BITS to download files
Network
• Lots of inbound traffic port 389
• Outbound traffic to strange IPs
• RDP brute force traffic and successful connect
• Host firewall opened up by malware
• (^ The problem with endpoint protection)
Suspect Files
• Logs missing for window of time
• Indicates likely time of incident
• Check for changes in that window
• Found suspicious files
• Same files referenced in services
• Googled it…
https://github.com/xmrig/xmrig
IOCs +
Prevention
• Monitor creation of new services
• URLs and IP addresses in service paths
• Watch for or prevent this Powershell command
transferStart-BitsTransfer-Source
http://Server1.TrustedDomain.com/File1.zip-
Destination c:tempdownloads-Asynchronous
• Or this BitsAdmin command line tool
bitsadmin /transfer myDownloadJob /download
/priority normal http://downloadsrv/10mb.zip
c:10mb.zip
• Spikes in CPU, costs in cloud accounts
• Deletion of logs, new users, changes to firewall
https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/US_AlarmAtThresholdEC2.html
Monitor and Respond to CPU Spikes
Limit Access from Internet: Bastion Host
• Limit Internet Access
• Bastion Host
• VPN to Bastion
• Non-admin Access
• MFA
• Alert on failures
Case Study:
Cloud Hosted
JavaScript
• LA Times serving up crypto mining code
• JavaScript on web site
• Leveraged web site visitor CPUs
• Discovered by Troy Mursch:@bad_packets
• World writeable S3 bucket : - /
Mining in the Browser
Attacker inserts
JavaScript into
legitimate web
site to mine crypto
Unsuspecting visitor
downloads page
which runs mining
code on his or her
device in browser
“While it's possible to run the miner without
informing your users, we strongly advise
against it.You know this. Long term goodwill
of your users is much more important than
any short term profits.”
…right….
Click Here…
• Set up an account on Coin Hive
• Grabbed the script
• Stuck it on a page in my bucket
• Asked everyone on social media to click
• Two weeks before BSides Vancouver…
• Results?
https://2ndsightlab.com/cryptomining.html
Not trying to
be sneaky!
IOCs +
Prevention
• No world writeable buckets
• S3 bucket policies and code in source control
• Automated deployment with security checks
• Monitor configurations
• Monitor traffic: IDS/IPS/DNS logs
• CPU spikes on end user machines (+sound)
• Note: GuardDuty doesn’t help in this case
Vigilance
Required
Summary
• Attackers are “borrowing” your resources
• Applications and machines might slow down
• Rogue resources may be injected in the cloud
• If unintended cryptominers, what else installed?
• In the browser? Legitimate? You decide.
Thank you!
Teri Radichel
2nd Sight Lab
@teriradichel

Crypto Miners in the Cloud

  • 1.
    Crypto Miners inthe Cloud BSides Vancouver Teri Radichel 2nd Sight Lab @teriradichel
  • 2.
    Agenda • Crypto Mining+ Cryptojacking • Some malicious examples… • How they [may have] happened • Indicators of compromise • What you can do about it
  • 3.
  • 4.
    Cryptojacking Using YOURresources without your permission to mine cryptocurrency.
  • 5.
    Types of Coin •Bitcoin – massive mining rig, high fees • Litecoin – lower transaction fees • Monero – anonymity, mine on CPUs • Many More… zcash, dash…
  • 6.
    Case Study: Double Crypto •Ransomware and Cryptomining software • Had to get past the ransomware • Trying to figure out how it was deployed • Found the cryptomining software
  • 7.
    Mining in theCloud Attacker uploads or downloads crypto mining software to your virtual machines running in the cloud
  • 8.
    First Steps • Logs:VPC Flow logs, CloudTrail, etc. • Turn on termination protection • Take snapshot of EBS Volume(s) • Create image, memory capture • Shared copies out to separate account • Quarantine via security group rules • Created a bastion host to access the instances
  • 11.
    VPN connection VPC subnet NACLlimits from responder on remote access ports and return ephemeral ports SG1 security group limits access inbound from responder. Outbound to SG2 Bastion Host Quarantine Infected Host Dedicated Machine SG2 security group limits access inbound from SG2 - no outbound Quarantine infected instances Application subnet - access from incident responder subnet on remote access port and ephemeral return ports
  • 12.
    Bypass the Ransomware • Figuredout how to get to […] in UI • Open Windows Explorer • Recursively make files readable from root • Woot! (Some things still inaccessible…) • Navigate and open cmd.exe • Usually C:WindowsSystem32cmd.exe
  • 13.
    Services • A bunchof strange services • Names ~ random four letters • Paths including domain names and IPs • Many stopped (service timeout?) • Using BITS to download files
  • 14.
    Network • Lots ofinbound traffic port 389 • Outbound traffic to strange IPs • RDP brute force traffic and successful connect • Host firewall opened up by malware • (^ The problem with endpoint protection)
  • 15.
    Suspect Files • Logsmissing for window of time • Indicates likely time of incident • Check for changes in that window • Found suspicious files • Same files referenced in services • Googled it…
  • 16.
  • 17.
    IOCs + Prevention • Monitorcreation of new services • URLs and IP addresses in service paths • Watch for or prevent this Powershell command transferStart-BitsTransfer-Source http://Server1.TrustedDomain.com/File1.zip- Destination c:tempdownloads-Asynchronous • Or this BitsAdmin command line tool bitsadmin /transfer myDownloadJob /download /priority normal http://downloadsrv/10mb.zip c:10mb.zip • Spikes in CPU, costs in cloud accounts • Deletion of logs, new users, changes to firewall
  • 18.
  • 19.
    Limit Access fromInternet: Bastion Host • Limit Internet Access • Bastion Host • VPN to Bastion • Non-admin Access • MFA • Alert on failures
  • 22.
    Case Study: Cloud Hosted JavaScript •LA Times serving up crypto mining code • JavaScript on web site • Leveraged web site visitor CPUs • Discovered by Troy Mursch:@bad_packets • World writeable S3 bucket : - /
  • 23.
    Mining in theBrowser Attacker inserts JavaScript into legitimate web site to mine crypto Unsuspecting visitor downloads page which runs mining code on his or her device in browser
  • 26.
    “While it's possibleto run the miner without informing your users, we strongly advise against it.You know this. Long term goodwill of your users is much more important than any short term profits.” …right….
  • 28.
    Click Here… • Setup an account on Coin Hive • Grabbed the script • Stuck it on a page in my bucket • Asked everyone on social media to click • Two weeks before BSides Vancouver… • Results? https://2ndsightlab.com/cryptomining.html
  • 29.
  • 31.
    IOCs + Prevention • Noworld writeable buckets • S3 bucket policies and code in source control • Automated deployment with security checks • Monitor configurations • Monitor traffic: IDS/IPS/DNS logs • CPU spikes on end user machines (+sound) • Note: GuardDuty doesn’t help in this case
  • 35.
  • 36.
    Summary • Attackers are“borrowing” your resources • Applications and machines might slow down • Rogue resources may be injected in the cloud • If unintended cryptominers, what else installed? • In the browser? Legitimate? You decide.
  • 37.
    Thank you! Teri Radichel 2ndSight Lab @teriradichel

Editor's Notes

  • #3 The purpose of this presentation is to raise awareness that your resources may be participating in crypto mining without your knowledge. This attack is becoming more and more prevalent. I’ll be talking about two different incidents where this occurred to explain how these attacks are occurring and how you might protect yourself.
  • #4 Crypto miners use compute resources to validate crypto currency transactions in hopes of getting crypto coins in return. I read a book on bitcoin mining but haven’t done it outside of what I demo in this presentation (for reasons that are not the subject of this presentation) however this definition made me chuckle. The first reaction I had to the activity of crypto mining is that it sounds like brute force password cracking. As it turns out now the cost of electricity in some locations is too high to allow crypto mining to be profitable so some people are using crypto mining specific clouds or executing crypto mining in other countries where the costs are lower.
  • #5 Cryptojacking is use of your resources to mine cryptocurrencies for someone else without your permission. This could be on a cell phones and IOT devices, servers, end user machines, or virtual machines in the cloud. In addition to compute resources, organizations need to be aware of the use of electricity since that is a resource that is making mining cost prohibitive. I heard about companies hosting mining rigs in office space where all the tenants split the electric bill evenly. Dr. Johnannes Ullrich of SANS Institute mentioned in a mailing list I am part of that in some organizations employees are bring mining rigs into the office and leveraging their employer’s electricity.
  • #6 Bitcoins require GPUs or specially designed hardware to mine coins. Companies are staring to sell mining rigs – you can even find them on Amazon.com. As a side note for anyone who is thinking about mining bitcoins on AWS that the type of GPUs available in Amazon AWS Cloud our not the best type of equipment for crypto mining. As mentioned it is similar to password cracking which is is best done on a GPU optimized for integer calculations, not NVIDIA GPUs available on AWS which are better for floating point calculations. The transaction fees around Bitcoin are making it somewhat unusable for every day transactions. Many people are using it as an investment option. Some people trust it more because it has the longest chain. Litecoin – has lower transaction fees. Some on the Dark Web trying to switch to Litecoin because Bitcoin is becoming too cost prohibitive due to transaction fees. Monero – After Wannacry it became apparent that Bitcoin is not quite as anonymous as everyone initially though. All the world was watching the wallets where the bitcoins from Wannacry were deposited because upon transfer out of the wallet was a point where the attackers might be identified. As a result some people began moving to Monero, a currency that provides more anonymity. This is the currency that was used in the attacks we will talk about today. Other – There are may other types of currencies. Malware Bytes has a blog post mentioning zcash and dash which also focus on hiding identities involved in transactions.
  • #7 I call this case double crypto because it started out as ransomware but also discovered the host had a crypto miner installed.
  • #8 In this case the attacker pushed a malware library to an instance in AWS.
  • #9 Please…turn on all your logs before the incident occurs but if you have an incident turn on your logs immediately and look for malicious traffic. Turn on termination protection so you don’t accidentally delete an instance you are investigating. In AWS take EBS volume and image snapshots. You’ll likely want to take memory captures as well. In my case I shared the backups out to a separate account so they could not accidentally be deleted in the account in which I was working. Immediately quarantine the instance using networking rules and access the instance with a bastion host, preferably over a VPN to prevent any further access to or from the Internet.
  • #10 How to turn on termination protection on AWS.
  • #12 A possible set up to quarantine and access an infected instance.
  • #13 When I logged in I needed to get past the ransomware. Run as… would not work, nor search. No applications were available. I ended up right clicking on the task bar to get to a dialog box in either task manager or services that would get me to the […] option which then allowed me to open Windows Explorer. I only saw about three files but just clicking on the root and recursively making everything readable showed me the rest of the files, and that point I could get to cmd.exe and run commands to inventory a lot of details about the host.
  • #14 Querying the services produced obviously problematic services created by the attacker. This was not a sophisticated attack. Service names had 4 letter names and there were many different stopped and started services. Some were using BITS to download files. BITS is used for Windows updates so is typically running on most Windows machines.
  • #15 Network traffic indicated a C2 command channel and a lot of traffic on port 389. The traffic on 389 is likely a red herring – and I didn’t have time to look into this further before I lost access to the host. Just a note that you probably don’t want to expose port 389 to the Internet unless you really mean it, and in any case make sure your traffic on port 389 is legitimate. Logs indicated a number of RDP brute force attempts that sped up right before the suspected point at which the host was compromised. The host firewall was opened up (which is one of the issues with endpoint security – a piece of malware with admin privileges can turn it off).
  • #16 Logs were missing for a block of time. Correlating that with the timing of the brute force attacks and creation of new users indicated the likely time period when the host was initially compromised. Searching for other changes around that time period produced a list of files which aligned with files downloaded by the rogue services. One of the files looked interesting so I Googled it.
  • #17 The file turned out to be XMRig which is an open source crypto miner written in C++. At the point I didn’t find anything else written about this particular library being used for this purpose but F5 Networks subsequently wrote about it: https://f5.com/labs/articles/threat-intelligence/malware/xmrig-miner-now-targeting-oracle-weblogic-and-jenkins-servers-to-mine-Monero
  • #18 Some potential indicators of compromise and things that can be monitored to find crypto miners (and other malware) in your environment: Monitor for creation of new services – especially those with suspect names. Look for URLs and IP addresses in the path for the service executable. Watch out for use of BITS to download files if you are not intentionally doing so. Powershell: https://msdn.microsoft.com/en-us/library/windows/desktop/ee663885(v=vs.85).aspx Cmd line: https://msdn.microsoft.com/en-us/library/windows/desktop/aa362813(v=vs.85).aspx Watch for spikes in CPU or increased costs for particular instances in cloud accounts. Alert on unexpected deletion of logs, creation of new users, and changes to firewall rules.
  • #19 In AWS you can use CloudWatch to monitor CPU usage on EC2 instances. In addition to monitoring you could send alerts and programmatically take other actions on those instances in these cases.
  • #20 Use a bastion host and require users to connect to a VPN to prevent access to your cloud instances from the Internet. Take a look at how much traffic hits your AWS instances the minute it comes online to see why this is important.
  • #21 At AWS re:Invent in 2017 Amazon released a security service called GuardDuty. It is agentless – nothing to install, no ports to open. It uses various logs in your account such as DNS logs and VPC Flow Logs, but you don’t have to turn on and pay for VPC Flow logs to use GuardDuty. It has a threat list but you can also import your won custom threat lists. You can turn it on with one button click. It sends an alert for certain suspicious activity but does not block anything.
  • #22 Here’s a list of some of the alerts offered by GuardDuty. As you can see a few of them relate to the incident just presented: RDP brute force attacks and potential crypto miners. One of the main use cases they presented was alerting on potential crypto mining activity which hints that this may be happening a lot on AWS resources without customers’ knowledge. If you haven’t turned this on, highly recommended that you do and look for crypto mining activity.
  • #23 The second case will take a look at a different type of hijacking compute resources by leveraging web site visitor CPUs. I was curious about the case of the LA Times serving up crypto mining software to their visitors and wanted to know how this worked so looked into it. In this case an attacker was able to upload crypto mining software that operates in JavaScript downloaded from their web site to unsuspecting web visitors. The root cause of this incident was another S3 bucket configuration error. In this case the S3 bucket was word writable so anyone could upload code to the web site.
  • #24 If you were not aware, S3 buckets can be used to host public web sites. In this case the S3 bucket is supposed to be world-readable, but not writeable! This picture shows how the crypto mining code was inserted into a particular web page and then uploaded to an S3 bucket. Then a web visitor came to the web site and downloaded the page, at which point the crypto mining software runs in his or her browser.
  • #25 I have an S3 bucket. In fact at this time that is where I host my web site which is as of this presentation, a whole two pages (I just started my business). I set out to find out how hard it is to run a crypto miner in JavaScript which led me to Coinhive. Coinhive is actually a legitimate looking service that states what it is doing on the home page – it allows you to monetize your business by leveraging the CPU of your application users or web site visitors.
  • #26 Coinhive offers a number of options for incorporating this mining software into your application or web site. Some of these options can be transparent to the user so they are not aware that crypto mining is occurring while they use your application. For the purposes of this talk, I selected the JavaScript miner because JavaScript was the mechanism used in the LA Times incident.
  • #27 Along the way I saw this little note…which is somewhat amusing.
  • #28 As it turns out it took me about five minutes, maybe less, to insert a crypto miner onto my web page. This is it. Just create a site key in your Coinhive account, copy it into this code, and add it to your web page.
  • #29 After setting up my crypto mining web page in preparation for this talk, I posted it on social media and asked everyone to click it to help fund my trip to the conference to give this presentation. : ) I give a warning on my home page and everywhere I post this that I haven’t actually de-obfuscated the JavaScript or inspected it so click on it at your own risk. But hey, don’t worry if you don’t want to click it…I couldn’t even get my family to click on it.
  • #30 Note that I am NOT trying to be sneaky in this case. I used the default version of this JavaScript which pops up a warning that tells you what is going on. It will use up to 70% CPU on a laptop or desktop and won’t run on a mobile device according to the statements on the Coinhive web site.
  • #31 So what happened? Well, I’m not getting rich off this any time soon. In order to get a payout you have to get up to .5 XMR and as you can see I’m a ways off. Coinhive pays out 70% of anything mined so they are taking almost a third cut of any mined profits. It was interesting when I first published the link that the numbers were spinning but then it just stopped. I suspect this was from bots hitting the link when I posted it on social media.
  • #32 So how can this attack be detected and prevented? First of all secure your S3 buckets. https://www.secplicity.org/2017/10/13/s3-bucket-security-acls-policies/ Automate and monitor deployments, and post deployment https://www.slideshare.net/TeriRadichel/locking-down-your-cloud Use IDS, IPS, DNS logs to look for suspicious traffic Look for CPU spikes on end-user machines. Tell your users to report unexpectedly loud and long fan noise. Note that GuardDuty doesn’t help in this case.
  • #33 As you can see the change in CPU is pretty blatant. Check out the Google Chrome Helper at 577.1 in the % CPU column…
  • #34 Malwarebytes talks about why they block Coinhive. But hey, what if you want to contribute to me speaking at conferences by contributing a little CPU at my web site? They also allow you to unblock it if you prefer.
  • #35 Someone in the SANS Reverse Engineering Malware community (SEC610 https://www.sans.org/course/reverse-engineering-malware-malware-analysis-tools-techniques) posted this example code using BRO to detect certain types of miners but…
  • #36 Using signatures to detect minors is going to prove to be pretty challenging. A search for crypto miner github on Google reveals numerous crypto miners in differet languages that could be deployed on your resources. It seems to me that CPU usage spikes might be easier to track down but a combination of methods is probably ideal.
  • #37 If you haven’t yet investigated your environment for crypto miners, I would suggest you do it now…
  • #38 Follow me on Twitter to see what I’m working on and share research!