3. “There are known knowns. These
are things we know that we know.
!
There are known unknowns. That
is to say, there are things that we
know we don't know.
!
But there are also unknown
unknowns. There are things we
don't know we don't know.”
!
- Satan Donald Rumsfeld
uncertainties
4. the dream of the
cloud
automation!
scalability!
flexibility!
speed!
17. legacy unknowns
1. what devices are on my network?
2. what are those devices really doing? what applications
are installed, and what network activity is really
happening?
3. where does my important data live and who has
access to it?
18. the cloud world
1. what devices are on my network?
2. what are those devices really doing? what applications
are installed, and what network activity is really
happening?
3. where does my important data live and who has
access to it?
19. the cloud world
1. what devices are on my network?
2. what are those devices really doing? what applications
are installed, and what network activity is really
happening?
3. where does my important data live and who has
access to it?
+ everything is online and now there’s a documented API
for it!
20. shiny new single points
of failure
CI
Config Management
Your ops guy with the SSL keys to push CM changes
AWS and other IaaS/SaaS APIs
21. shiny new single points
of failure
CI
Config Management
Your ops guy with the SSL keys to push CM changes
AWS and other IaaS/SaaS APIs
who watches the watchmen?
42. Security + DevOps = ?
•add security protections and automate
them
•continuously verify your protections with
automation
•add security monitoring but automate it
•detect intrusions and automate response
•score yourself on your security posture and
automate the grading
43. Security + DevOps = ?
•add security protections and automate
them
•continuously verify your protections with
automation
•add security monitoring but automate it
•detect intrusions and automate response
•score yourself on your security posture and
automate the grading
50. questions to answer
“is that guy running commands he shouldn’t be?”
(e.g. why is anyone except chef user MAYBE running
gcc on a prod system)
!
“are accounts logging in from non-standard locations?”
!
“are there anomalies in my traffic?”
!
“did some process suddenly start making outbound
connections?”
51. step 1: toolbox
process auditing linux audit
network flow libnetfilter_conntrack
logins wtmp/audit/pam_loginuid
52. !
collect this data from every EC2 host you launch
+
store and correlate in a big data backend
+
build an audit log
+
analyze for suspicious behaviors and insights
61. THIS ONE WEIRD TRICK!
!
enable rate limiting or it could ‘crash’ your
box
!
always be listening (or same)
!
…relatively stable otherwise ;)
auditctl -b 1000 -r 15000 # 1000 buffers, 15000 eps max
69. # last
jandre pts/1 dev.threatstack. Sun May 4 11:20 - 01:37
(14:17)
jandre pts/0 dev.threatstack. Sun May 4 11:16 still logged
in
!
# in json format
!
{ type: ‘USER_PROCESS',
pid: 777,
line: ‘pts/1',
id: 52,
user: 'jandre',
host: ‘dev.threatstack',
exit_status: { termination: 0, code: 0 },
timestamp: Tue May 06 2014 03:50:03 GMT-0700 (PDT),
address: ’10.0.0.10’ }
}
!
!
wtmp
70. # if pam is built with audit support…
!
type=USER_AUTH msg=audit(1234877011.791:7731): user
pid=26127 uid=0 1
auid=4294967295 ses=4294967295
msg='op=PAM:authentication acct="root" exe="/usr/sbin/
sshd"
(hostname=jupiter.example.com, addr=192.168.2.100,
terminal=ssh res=success)'
plus audit
71. !
# /etc/pam.d/login, sshd, wherever you care
!
/etc/pam.d$ grep loginuid *
!
login:session required pam_loginuid.so
sshd:session required pam_loginuid.so
!
# now you get in /var/log/audit/audit.log:
!
type=LOGIN msg=audit(1234877011.799:7734): login pid=26125
uid=0
old auid=4294967295 new auid=0 old ses=4294967295 new ses=1172
!
# cat /proc/<login or sshd pid>/sessionid
!
1172
add pam_loginuid
76. questions to answer
who is using my AWS credentials, and from where?
!
who is engaging anomalous or suspicious API activity
that could indicate a breach?
!
have my credentials been compromised?
!
are IAM credentials assigned with appropriate
permissions?
77. how?
cloud trail to an s3 bucket for audit logging
AWS APIs for auditing IAM permissions
!
collect periodically and store using IAM role with limited
permissions
group by user account, activity, location
alert by new or suspicious behaviors
