APPSEC &
MICROSERVICES
Sam Newman
Velocity 2016
@samnewman#velocityconf
@samnewman#velocityconf
Sam Newman
Building
Microservices
DESIGNING FINE-GRAINED SYSTEMS
@samnewman#velocityconf
Microservices Can Make
Everything Worse
@samnewman#velocityconf
@samnewman#velocityconfhttps://www.flickr.com/photos/seattlemunicipalarchives/4058808950
@samnewman#velocityconf https://www.flickr.com/photos/theseanster93/485390997/
@samnewman#velocityconf
http://map.norsecorp.com/
@samnewman#velocityconf
@samnewman#velocityconf
@samnewman#velocityconf
Accounts
Returns
Invoicing
Shipping
Inventory
Customer
Service
@samnewman#velocityconf
Accounts
Returns
Invoicing
Shipping
Inventory
Customer
Service
Small Independently Deployable
services that work together, modelled
around a business domain
https://www.flickr.com/photos/wwworks/2607036664/
https://www.flickr.com/photos/lkowen/15803718243/
@samnewman#velocityconf
@samnewman#velocityconf
@samnewman#velocityconf
@samnewman#velocityconf
@samnewman#velocityconf
@samnewman#velocityconf
Prevention
@samnewman#velocityconf
Prevention Detection
@samnewman#velocityconf
Prevention Detection
Response
@samnewman#velocityconf
Prevention Detection
ResponseRecovery
@samnewman#velocityconf
Prevention Detection
ResponseRecovery
@samnewman#velocityconf
Prevention Detection
ResponseRecovery
@samnewman#velocityconf https://www.flickr.com/photos/adulau/15680439035/
@samnewman#velocityconf https://www.flickr.com/photos/duanestorey/469163789/
@samnewman#velocityconf
https://www.schneier.com/paper-attacktrees-ddj-ft.html
@samnewman#velocityconf
Open Safe
@samnewman#velocityconf
Open Safe
Pick Lock Learn Combo Cut Open
@samnewman#velocityconf
Open Safe
Pick Lock Learn Combo Cut Open
Find Written
Combo
Get Combo from
the target
@samnewman#velocityconf
Open Safe
Pick Lock Learn Combo Cut Open
Find Written
Combo
Get Combo from
the target
Blackmail Threaten Bribe
@samnewman#velocityconf
Open Safe
Pick Lock Learn Combo Cut Open
Find Written
Combo
Get Combo from
the target
Blackmail Threaten Bribe
Impossible
Impossible Impossible
Possible
Possible
Possible
@samnewman#velocityconf
Open Safe
Pick Lock Learn Combo Cut Open
Find Written
Combo
Get Combo from
the target
Blackmail Threaten Bribe
@samnewman#velocityconf
Open Safe
Pick Lock Learn Combo Cut Open
Find Written
Combo
Get Combo from
the target
Blackmail Threaten Bribe
$$$$
$$$$ $$$$
$$
$$
$
@samnewman#velocityconf
Catalog
service
Music
Web Shop
Recommend
service
Royalty
Payment
Gateway
Mobile
app
Web
browsers
User
service
@samnewman#velocityconf
Catalog
service
Music
Web Shop
Recommend
service
Royalty
Payment
Gateway
Mobile
app
Web
browsers
User
service
Transport Security
@samnewman#velocityconf
HTTPS Everywhere!
BENEFITS OF HTTPS?
BENEFITS OF HTTPS?
▫︎Server guarantees!
BENEFITS OF HTTPS?
▫︎Server guarantees!
▫︎Payload not manipulated…
BENEFITS OF HTTPS?
▫︎Server guarantees!
▫︎Payload not manipulated…
▫︎…but no client guarantee and…
BENEFITS OF HTTPS?
▫︎Server guarantees!
▫︎Payload not manipulated…
▫︎…but no client guarantee and…
▫︎…certificates can be a pain
@samnewman#velocityconf
https://letsencrypt.org/
@samnewman#velocityconf
Catalog
service
Music
Web Shop
Recommend
service
Royalty
Payment
Gateway
Mobile
app
Web
browsers
User
service
CLIENT-SIDE CERTIFICATES?
CLIENT-SIDE CERTIFICATES?
▫︎Client guarantees!
CLIENT-SIDE CERTIFICATES?
▫︎Client guarantees!
▫︎…but a PITA to manage….
@samnewman#velocityconf
http://techblog.netflix.com/2015/09/introducing-lemur.html
@samnewman#velocityconf
Catalog
service
Music
Web Shop
Recommend
service
Royalty
Payment
Gateway
Mobile
app
Web
browsers
User
service
@samnewman#velocityconf
Auth?
@samnewman#velocityconf
Auth?
Authentication
@samnewman#velocityconf
Auth?
Authentication Authorisation
@samnewman#velocityconf
Catalog
service
Music
Web Shop
Recommend
service
Royalty
Payment
Gateway
Mobile
app
Web
browsers
User
service
Web
browsers
@samnewman#velocityconf
Catalog
service
Music
Web Shop
Recommend
service
Royalty
Payment
Gateway
Mobile
app
Web
browsers
User
service
Web
browsers
Form AuthOAuth
@samnewman#velocityconf
Catalog
service
Music
Web Shop
Recommend
service
Royalty
Payment
Gateway
Mobile
app
Web
browsers
User
service
Web
browsers
Form AuthOAuth
PERIMETER SECURITY!
@samnewman#velocityconf
Catalog
service
Music
Web Shop
Recommend
service
Royalty
Payment
Gateway
Mobile
app
Web
browsers
User
service
Web
browsers
Form AuthOAuth
PERIMETER SECURITY!
User
service
@samnewman#velocityconf
Music
Web Shop
User
service
User
service
Implicit Trust?
@samnewman#velocityconf
Catalog
service
Music
Web Shop
Recommend
service
Mobile
app
Web
browsers
User
service
Web
browsers
User
service
@samnewman#velocityconf
Catalog
service
Music
Web Shop
Recommend
service
Mobile
app
Web
browsers
User
service
Web
browsers
User
service
Asking As Bob
@samnewman#velocityconf
Catalog
service
Music
Web Shop
Recommend
service
Mobile
app
Web
browsers
User
service
Web
browsers
User
service
Asking As Bob
Can I see
Alice’s Data?
@samnewman#velocityconf https://www.flickr.com/photos/lundyd/14481829564/
Confused
Deputy
Problem!
@samnewman#velocityconf
Music
Web Shop
Web
browsers
User
service
@samnewman#velocityconf
Music
Web Shop
Web
browsers
User
service
@samnewman#velocityconf
Music
Web Shop
Web
browsers
User
service
@samnewman#velocityconf
Music
Web Shop
Web
browsers
User
service
{
"id": "402ndj39",
"name": “Alice Alison"
}
@samnewman#velocityconf
Music
Web Shop
Web
browsers
User
service
{
"id": "402ndj39",
"name": “Alice Alison"
}
@samnewman#velocityconf
Music
Web Shop
Web
browsers
User
service
{
"id": "402ndj39",
"name": “Alice Alison"
}
@samnewman#velocityconf
Data At Rest?
@samnewman#velocityconf
Catalog
service
Music
Web Shop
Recommend
service
Royalty
Payment
Gateway
Mobile
app
Web
browsers
User
service
User
service
@samnewman#velocityconf
Encryption!
@samnewman#velocityconf https://www.flickr.com/photos/aigle_dore/2781302649
@samnewman#velocityconf
Plain Text?
@samnewman#velocityconf
@samnewman#velocityconf
“In the API server secret data is stored as plaintext in etcd"
http://kubernetes.io/docs/user-guide/secrets/#security-properties
@samnewman#velocityconf
Secure Vaults
@samnewman#velocityconf
@samnewman#velocityconf
@samnewman#velocityconf
Aside: Docker
@samnewman#velocityconf
http://www.banyanops.com/blog/analyzing-docker-hub/
@samnewman#velocityconf
@samnewman#velocityconf
@samnewman#velocityconf
S/M TestsBuild Large Tests Production
@samnewman#velocityconf
S/M TestsBuild Large Tests Production
Security?
@samnewman#velocityconf
S/M TestsBuild Large Tests Production
Security?
OWASP ZAP Attack Proxy
Static Analysers
@samnewman#velocityconf https://www.microsoft.com/en-us/sdl/
@samnewman#velocityconf
https://medium.com/built-to-adapt/the-three-r-s-of-enterprise-security-
rotate-repave-and-repair-f64f6d6ba29d
@samnewman#velocityconf
“At or near the top of security concerns in the
datacenter is something called an Advanced
Persistent Threat (APT). An APT gains
unauthorized access to a network and can stay
hidden for a long period of time. Its goal is
usually to steal, corrupt, or ransom data.”
- Justin Smith, Pivotal
@samnewman#velocityconf
Rotate: Short-lived Credentials
@samnewman#velocityconf
Rotate: Short-lived Credentials
Repair: Patch Your Stuff
@samnewman#velocityconf
Rotate: Short-lived Credentials
Repave: Burn It Down!
Repair: Patch Your Stuff
@samnewman#velocityconf
http://www.theregister.co.uk/2014/06/18/code_spaces_destroyed/
@samnewman#velocityconf
https://github.com/michenriksen/gitrob
@samnewman#velocityconf
(don’t forget to limit
credential scope too)
@samnewman#velocityconf
Prevention Detection
ResponseRecovery
@samnewman#velocityconf
Prevention Detection
ResponseRecovery
@samnewman#velocityconf
https://www.qualys.com/research/top10/
@samnewman#velocityconf
http://www.extremetech.com/computing/190959-shellshock-a-deadly-new-vulnerability-that-could-lay-waste-to-the-internet
@samnewman#velocityconf
@samnewman#velocityconf
Repair: Patch Your Stuff
@samnewman#velocityconf
https://www.modsecurity.org/
@samnewman#velocityconf
Catalog
service
Music
Web Shop
Recommend
service
Royalty
service
Mobile
app
Web
browsers
User
service
@samnewman#velocityconf
Catalog
service
Music
Web Shop
Recommend
service
Royalty
service
Mobile
app
Web
browsers
User
service
PERIMETER SECURITY!
@samnewman#velocityconf
Catalog
service
Music
Web Shop
Recommend
service
Royalty
service
Mobile
app
Web
browsers
User
service
PERIMETER SECURITY!
PERIMETER SECURITY!
@samnewman#velocityconf
Catalog
service
Music
Web Shop
Recommend
service
Royalty
service
Mobile
app
Web
browsers
User
service
PERIMETER SECURITY!
PERIMETER SECURITY!
PERIMETERSECURITY!
@samnewman#velocityconf
Polyglot = more stuff to track!
@samnewman#velocityconf
https://www.npmjs.com/package/npm-check
@samnewman#velocityconf
@samnewman#velocityconf
b4a2f5ga2
4335egad3
ab2d56be3
847ea3dbe
@samnewman#velocityconf
b4a2f5ga2
4335egad3
ab2d56be3
847ea3dbe !!!
!!!
@samnewman#velocityconf
b4a2f5ga2
4335egad3
ab2d56be3
847ea3dbe
847ea3dbe
847ea3dbe
847ea3dbe
4335egad3
4335egad3
4335egad3
4335egad3
4335egad3
4335egad3
4335egad3
4335egad3
4335egad3
4335egad3
4335egad3
847ea3dbe
!!!
!!!
@samnewman#velocityconf
https://github.com/coreos/clair
@samnewman#velocityconf
Repair: Patch Your Stuff
@samnewman#velocityconf
Repair: Patch Your Stuff
Automate it
@samnewman#velocityconf
Repair: Patch Your Stuff
Automate it
Do It A Lot
@samnewman#velocityconf
Repair: Patch Your Stuff
Automate it
Do It A Lot
And Check Your Work
@samnewman#velocityconf
@samnewman#velocityconf
Polyglot = more things to break?
@samnewman#velocityconf
Prevention Detection
ResponseRecovery
@samnewman#velocityconf
Prevention Detection
ResponseRecovery
@samnewman#velocityconf
@samnewman#velocityconf
@samnewman#velocityconf
@samnewman#velocityconf http://krebsonsecurity.com/tag/target-data-breach/
@samnewman#velocityconf
Comms
@samnewman#velocityconf
@samnewman#velocityconf
@samnewman#velocityconf
https://en.wikipedia.org/wiki/Chicago_Tylenol_murders
@samnewman#velocityconf
@samnewman#velocityconf
@samnewman#velocityconf
Customer
@samnewman#velocityconf
Customer
@samnewman#velocityconf
Prevention Detection
ResponseRecovery
@samnewman#velocityconf
Prevention Detection
ResponseRecovery
@samnewman#velocityconf
Backups
@samnewman#velocityconf
@samnewman#velocityconf
Repave: Burn It Down!
@samnewman#velocityconf
Phoenix Servers
@samnewman#velocityconf
Phoenix Servers
Immutable Servers
@samnewman#velocityconf
Phoenix Servers
Immutable Servers
= repave on every release
@samnewman#velocityconf
Why not repave automatically when
you apply a patch?
@samnewman#velocityconf
RepaveBackups
@samnewman#velocityconf
Harder with microservices?
RepaveBackups
@samnewman#velocityconf
Harder with microservices?
RepaveBackups
AUTOMATE ALL THE THINGS
@samnewman#velocityconf
Post Mortems
@samnewman#velocityconf
http://www.smh.com.au/digital-life/mobiles/telstra-outage-manager-connected-customers-to-faulty-node-in-embarrassing-
error-20160209-gmpn7f.html
@samnewman#velocityconf
"[The employee responsible] didn't follow
procedures and clearly that's not a good thing
but I wouldn't want to pre-empt the proper
investigation and we'll figure out what the right
response is when we've had a chance to dig into
the detail."
- Australian Financial Review
http://www.afr.com/business/telecommunications/telstra-mobile-network-down-across-
australia-reports-20160209-gmpaty
@samnewman#velocityconf
http://samnewman.io/blog/2016/02/10/telstra_outage/
@samnewman#velocityconf
https://vimeo.com/102167635
@samnewman#velocityconf
“Finding the root cause of a
failure is like finding a root
cause of a success.”
http://www.kitchensoap.com/2012/02/10/each-necessary-but-only-jointly-sufficient/
John Allspaw
@samnewman#velocityconf
http://www.smh.com.au/technology/technology-news/telstra-free-data-guy-clocks-up-almost-
a-terabyte-of-downloads-20160404-gnxu14.html
@samnewman#velocityconf
Don’t forget to review your old
post-mortems too…
@samnewman#velocityconf
Don’t forget to review your old
post-mortems too…
…and the resulting action plans!
@samnewman#velocityconf
Prevention Detection
ResponseRecovery
@samnewman#velocityconf
Sam Newman
Building
Microservices
DESIGNING FINE-GRAINED SYSTEMS
http://buildingmicroservices.com/
@samnewman#velocityconf
Sam Newman
Building
Microservices
DESIGNING FINE-GRAINED SYSTEMS
http://buildingmicroservices.com/
http://samnewman.io/
@samnewman#velocityconf
Sam Newman
Building
Microservices
DESIGNING FINE-GRAINED SYSTEMS
http://buildingmicroservices.com/
http://magpietalkshow.com/
http://samnewman.io/
@samnewman#velocityconf
Wednesday 22nd
Sam Newman
Building
Microservices
DESIGNING FINE-GRAINED SYSTEMS
Signing
5.45pm
@ Oreilly Booth
@samnewman
snewman@thoughtworks.com
THANKS!

AppSec & Microservices - Velocity 2016