CodeOne SF 2018 "Are you deploying and operating with security in mind?"

Are You Deploying and Operating
with Security in Mind?
Steve Poole @spoole167
Daniel Bryant @danielbryantuk

Containers: Expectations versus reality
22/10/2018 @danielbryantuk
“DevOps”

tl;dr
• Using containers does not make you invulnerable from attack.
• Practical mitigation of risk needs you to understand how you are open
to attack.
• Practical mitigation requires you to pay attention to how you develop
software –and how you behave
• Adding metadata to containers is vital for understanding your software
supply chain and current state of the world in production
• Testing system quality attributes ("NFRs") within an automated build
pipeline is table stakes for today's threat landscape.

@danielbryantuk
Independent Tech Consultant, Product Architect at Datawire
Architecture, DevOps, Java, microservices, cloud, containers
@spoole167
Developer Advocate, IBM
Java (pre 1.0), DevOps, security, cloud, containers

The economics of crime – and
why you’re culpable

In 2016 Cybercrime was
estimated to be worth
450 Billion Dollars
@spoole167
Cybercrime is the most profitable type of crime
In 2016 The illicit drug trade
was estimated to be worth
435 Billion Dollars

Cybercrime is the most profitable type of crime
• Guess which one has the least risk to the criminal ?
• Guess which is growing the fastest ?
• Guess which one is the hardest to prosecute ?
• Guess which one is predicted to reach 2100 Billion Dollars by 2019?
• Guess which one is predicted to reach 6000 Billion Dollars by 2021?
@spoole167

0
1000
2000
3000
4000
5000
6000
2013 2014 2015 2016 2017 2018 2019 2020 2021
Cybercrime Drug trade

That’s about $600 for every
person on the planet
In the US it’s about $8000
each

Don’t agree?
“The bad guys prey on the weak, vulnerable and ignorant”
That’s you
@spoole167

Ever googled for:
“very trusting trust manager”
“Getting Java to accept all certs over HTTPS”
“How to Trust Any SSL Certificate”
“Disable Certificate Validation in Java”
@spoole167

TrustManager[] trustAllCerts = new TrustManager[]{
new X509TrustManager() {
public X509Certificate[] getAcceptedIssuers() {
return null;
}
public void checkClientTrusted(
X509Certificate[] certs, String authType) {
}
public void checkServerTrusted(
X509Certificate[] certs, String authType) {
}
public boolean isClientTrusted( X509Certificate[] cert) {
return true;
}
public boolean isServerTrusted( X509Certificate[] cert) {
return true;
}
}}
Ever written
something
like this?
@spoole167

We’ve all done something like that
@spoole167

We do it all the time
@spoole167

The whole world does it
How bad can it be?
@spoole167

The whole world does it
Github search “implements TrustManager” ….
@spoole167

We’ve found 72,609 code results
AlwaysValidTrustManager
TrustAllServersWrappingTrustManager
A very friendly, accepting trust
manager factory. Allows anything
through. all kind of certificates are
accepted and trusted.
A very trusting trust manager that
accepts anything
// Install the all-trusting trust
manager
OverTrustingTrustProvider
AllTrustingSecurityManagerPlugin.java
AcceptingTrustManagerFactory.java
AllTrustingCertHttpRequester.java

And.. given how important using https
correctly is…

Why do we turn it off?
curl –insecure
wget --no-check-certificate
sudo apt-get --allow-unauthenticated

For reasonable reasons?
• “The server I access is self-signed”
• “I want to access multiple servers “
Unexpectedly?
• “I thought I was using the tool correctly”
• “I didn’t realize what the default setting was”
• “I trusted the tool to do the right thing”
Maliciously?
• “Someone changed the script and I don’t know why”

Attackers use vulnerabilities to get into your system
And go through your system to another (and repeat)
To steal your data To change your data To crash your systems Use your compute power

if (multiWrapper.hasErrors()) {
if(validation!=null){
validation.addActionError( … );
}
}
This is a major vulnerability

So is this. Just two characters

Vulnerabilities are almost always simple
there are no smoking guns
Exploits are chaining together vulnerabilities
https://www.flickr.com/photos/84744710@N06/

Apache struts 2 - the Equifax affair
Attackers used Remote Code Execution to steal data over
a long period
CVE-2017-5638 The Jakarta Multipart parser in Apache Struts 2 2.3.x before 2.3.32 and
2.5.x before 2.5.10.1 has incorrect exception handling and error-
message generation during file-upload attempts, which allows remote
attackers to execute arbitrary commands via a crafted Content-Type,
Content-Disposition, or Content-Length HTTP header, as exploited in the
wild in March 2017 with a Content-Type header containing a #cmd=
string.

Content-Type: %{(#_='multipart/form-data').
(#_memberAccess=@ognl.OgnlContext@DEFAULT_MEMBER_ACCESS)
.
(@java.lang.Runtime@getRuntime().exec('curl
localhost:8000'))}
https://dzone.com/articles/will-it-pwn-cve-2017-5638-remote-code-
execution-in
Apache Struts
OGNL
If type contains “multipart/form-data’
try to parse it as form data
This fails and as part of the building an error message the
OGNL is evaluated…

Large Scale Exploit
March 9
First ‘whoami’ is run
AprMar May Jun Jul Aug Sept
March 7
Apache Struts releases
updated version to thwart
vulnerability
CVE-2017-5638
July 29
Breach is
discovered by
Equifax.
Sept 7
A new strutcs RCE
vulnerability is
announced and fixed.
CVE-2017-9805
Probe Crisis Management
March 5 vulnerability and fixed
bit.ly/devoxx-sec-workshop

Time to EXPLOIT?Source: Adapted from IBM X-Force / Analysis by Gartner Research (September 2016)
Year of Date Reported
2006 2007 2008 2009 2010 2011 2012 2013 2104 2015
10
20
30
40
50
0
AverageDaystoExploit
Average
45
15
2017

The bad news
• Vulnerabilities are the gateway into the server
Systems with vulnerabilities are easily found
Everyone has vulnerabilities
2 days from publication to attack
Exploits are easily found
• Developers don’t think about security
• Application design thinking suffers
• Development processes etc suffer

The truth is
• Most of the time you’re attacked just
because your more vulnerable and
weak than the next guy
• Keeping up to date with vulnerability
patches keeps you lower down the list
and hence safer
• Once they get into your system – they
spend a long time looking around and
finding all your weaknesses

Container security

Container technology 101
• OS-level virtualisation
• Just a bunch of (Linux) processes
• cgroups, namespaces, rootfs
• Package and execute software

How do you view containers?

Sorry, it’s more like…

Container Runtime Security

Container runtime security 101
https://kubernetes.io/docs/tasks/configure-pod-container/security-context/
https://www.infoq.com/news/2016/08/secure-docker-microservices

Privileges and Capabilities

Container Supply Chain

Lesson learned: Metadata is valuable
• Application metadata
• Version / GIT SHA
• Build metadata
• Build date
• Image name
• Vendor
• Quality metadata
• QA control, signed binaries, ephemeral support
• Security profiles (AppArmor), Security audited etc

Metadata - Adding Labels at build time
• Docker Labels
• Add key/value data to image

Metadata - Adding Labels at build time
• Microscaling Systems’ Makefile
• Labelling automated builds on
DockerHub (h/t Ross Fairbanks)
• Create file ‘/hooks/build’
• label-schema.org
• microbadger.com

Metadata - Adding Labels at runtime
$ docker run -d --label
uk.co.danielbryant.lbname=frontdoor nginx
• Can ’docker commit’, but creates new image
• Not possible to update running container
• Docker Proposal: Update labels #21721

External registry with metadata support

Grafeas + Kritis

Testing in a Java Build Pipeline

Testing security in the build pipeline
• PMD /Findsecbugs
• OWASP Dependency check
• DockerHub Scanner / CoreOS Clair
• Bdd-security (OWASP ZAP) / Arachni
• Gauntlt / Serverspec
• Docker Bench / KubeBench

Security Visibility: Basic (Java) Code Scanning

Dependency Scanning
www.owasp.org/index.php/OWASP_Dependency_Check

Static Image Scanning
github.com/arminc/clair-scanner

Delaying NFRs to the ‘Last Responsible Moment’
Newsflash!
Sometimes the
last responsible moment
is up-front
Modern platforms/architectures
don’t necessarily make this easier

Bonus: Secure runtime comms

Google’s BeyondCorp: Zero Trust Networks
cloud.google.com/beyondcorp/

Securing communications
• Mutial TLS (mTLS)
• Intentions
• Policy-based controls

mTLS: Securing communications

Summary

In summary
• Using containers does not make you invulnerable from attack. Used
wisely containers can substantially reduce your exposure. Used poorly
containers substantially increase your risk.
• Practical mitigation of risk needs you to understand how you are
open to attack. Practical mitigation requires you to pay attention to
how you develop software –and how you behave
• Adding metadata to containers is vital for understanding your
software supply chain and current state of the world in production
• Testing system quality attributes ("NFRs") within an automated build
pipeline is table stakes for today's threat landscape.

Thanks for listening…
Twitter: @spoole167
Email:
Twitter: @danielbryantuk
Email: daniel.bryant@tai-dev.co.uk

CodeOne SF 2018 "Are you deploying and operating with security in mind?"

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to CodeOne SF 2018 "Are you deploying and operating with security in mind?"

Similar to CodeOne SF 2018 "Are you deploying and operating with security in mind?" (20)

More from Daniel Bryant

More from Daniel Bryant (20)

Recently uploaded

Recently uploaded (20)

CodeOne SF 2018 "Are you deploying and operating with security in mind?"