2. EHLO
• Scott Behrens
• Application Security
Engineer at Netflix
• Likes automation
• Torn between breaking
and making
• Loves research and open
source
3. What We Will Cover
• Challenges of a modern infrastructure
• Proactive security as a solution
• Examination of mature security program
• How to get started
• Conclusion
4. Terminology Primer
• Define technology terms:
• Application
• Instance
• Autoscaling groups (ASGs) / Load balancers (ELBs)
• AMI
• SecurityGroups
• Regions / Availability Zones (AZs)
5. Netflix Primer
• 100's of Developers
• Over 1,000 applications
• Hundreds of production pushes a day
• Over 50k instances
• No Security Gates!
6. Continuous Deployment
• Can be fully automated
• Immutable Platform
• Looks like:
• git -> Create deb (Jenkins) -> AMI Snapshot -> Deploy
7. Potential Pitfalls
• Multiple concurrent code-bases (A/B testing,
regional functionality, etc.)
• New applications brought online/old
applications retired constantly
• Insecure third party dependencies
8. More Pitfalls
• How do we identify and catalog assets (IP?, DNS
Name?, Application?, ELB?...)
• How do we profile and provide baseline security to new
applications?
• How do we monitor security policy changes and
configurations in AWS
• Monitoring the web for credential dumps, hacktivism,
sensitive data exposure?
9. The List Goes On...
• Many of these problems are not unique to
Netflix/the cloud
• Some are unique or can be solved in different
ways
• We have coined our approach to these and other
problems as...
10. Proactive Security
• Dictionary.com
• Proactive - "serving to prepare for, intervene in,
or control an expected occurrence or situation,
especially a negative or difficult one;
anticipatory"
11. Proactive Security
• Security controls should be:
• Integrated, automated, scalable, adaptive,
actionable, and intelligent
• Time is limited
• End goal: Less babysitting, more time for harder
problems
12. Proactive Security
• Find problems early and address them
• Know your weaknesses and work to improve them
• Monitor for anomalies and be prepared to respond
• Collect meaningful data and use it to improve
• Simplify make security the easy path
• Reevaluate your approach
• Share what you learn with others
13. Find
• Identify issues early
• Old hat:
• Static asset lists, fileshare with old pentest
reports
problems early and address them
14. Find - New Way
• Define what an asset is:
• Application
• ELB
• FQDN
• IP
15. Find - New Way
• Have an intelligent way to collect/
track assets and vulns
Monterey
16. Monterey
• Provides way to automatically define and scan
assets
• Soon to be open sourced
• Asset groups as application name:
• Contain any ELBs or FQDN
• Do not contain instance
17. Monterey
• Monterey queries AWS for new assets and
changes
• Monterey runs "Monklets" for things like
vulnerability scanning
• Store vulns centrally in S3
19. Monitor
• Monitor, detect, and respond to security issues
• Old hat:
• Reactionary
• New hat:
• Less reactionary aka. more proactive
• Automatic
• Intelligent
for anomalies and be prepared to respond
20. Simian Army
• aka The Monkeys!
• Open source
• Proactively wreck your
environment to simulate
outages
• Take down apps, instances,
ELBs, even entire regions
• Ensures developers write
resilient code
21. "Dirty Laundry" Project
• Look for assets unintentionally exposed
• Leverages Monterey for assets
• Uses Scumblr for actioning findings
• Uses Sketchy for collecting status codes,
generating screenshots, and text scrapes
• Both Sketchy/Scumblr are open source
22. Scumblr
• Intelligent automation platform aka.
our swiss army knife
• Monitor things by using plugins
• Credential Dumps
• Hacktivism
• Brand Reputation
• Full Disclosure
• Leverages Sketchy for screen shots and
text scrapes
• Custom Workflows
24. Speedbump
• Mechanism to detect attacks and enforce security policies
automatically
• You define what you want to monitor/filter
• Security policies can be time delays, blocking, routing, etc.
• A WAF, proxy and firewall on steroids because it has
application intelligence (business logic)
• Ensnare
25. Security Monkey
• Open Source
• Monitor events/changes within
AWS
• Changes to IAM users
• Security groups
• Policy changes
• Notify when things change
26. Have you noticed a trend?
• A more proactive approach
• Identify something interesting automatically
• Notify automatically
• Provide workflows for more complex
remediations
27. Know
• None of us have unlimited time or resources--we all have
weaknesses
• Work to automatically identify blindspots
• Leverage tools and information where useful
• Old hat:
• Spreadsheet containing static list of apps and manually
created risk rankings
• Not scalable when we are talking 100's or 1000's of apps
your weaknesses and work to improve them
28. Penguin Shortbread
• Catalog assets
• Measure attributes
• Calculate risk
• Will be open sourced if we can
make it more generic (TBD)
29. Case Study: Monklet Your Way
To Infinity
• Shellshock
• Vulnerable vs exploitable?
• Some commercial scanners only hit root of
webpage
• What about other scripts?
30. Case Study: Monklet Your Way
To Infinity
• Monterey Monklet!
• Spiders a provided asset (gevented)
• Checks each page for exploitability of Shellshock
• Stores results in S3 centrally
• Deploy N instances of Monklet to cover 100's of
apps
31. Collect
• Collect data from intelligent systems
• Actually use it to:
• Drive initiatives
• Find what is/isn't working
meaningful data and use it to improve
33. Simplify
• If it's easy developers will do it
• Make security an enabler and not a burden
• Encourage developers to come to you
• Also: v0.1 doesn't need to be perfect
make security the easy path
34. Immutable Base AMI
• Make your platform consistent
• For non-cloud users: system image/template
• Conformity Monkey for AWS users (open source)
35. Danger! Danger!
• Applications often use legacy/outdated dependencies
• Make it clear to developers they are using dangerous stuff
• Netflix leverages an API based on OWASP Dependency
Checker
• API will eventually be open sourced
• Provides information into our Dependency tool
• Work in progress...
37. FindSecBugs
• Allow users to opt-in to static analysis
• FindSecBugs is lightweight and pretty useful (for
Java)
• They can still deploy regardless but no one likes
seeing red
38. Reevaluate
• Environment is always changing--what works today
may be ineffective tomorrow
• Developers are agile, which means we must be too
• Be willing to start simple, knowing the initial
solution is temporary (Don't over-engineer!)
• Use the data you've collected to determine when
an approach is no longer sufficient
your approach
39. Share
• We (security professionals) are all working to solve
similar problems
• Sharing tools and information make all of our lives
easier
• Allows improvement through the collective
consciousness of the industry
• Prevents duplicate effort
what you learn with others
42. Proactive Security: Level 2
• Strategize security based on asset risk. Ex:
1. Sensitive assets: Examine thoroughly
2. Frequently used components: Secure by
default
3. Everything else: Find a good baseline level
of due diligence
43. Proactive Security: Level 3
• Identify weak links
• Architectural: Poor network segmentation
• Cultural: Developers don't understand XSS
• Specific: "Application X" relies on an outdated
version of Struts
• Determine how to measure the weaknesses and put in
place a plan to address or manage
44. Proactive Security: Level 4
• Monitor, alert, and gather more intel
• Detect anomalies in security relevant functionality:
Login successes/failures, password resets,
authorization failures...
• Dashboards are fun, but intelligent alerting is better
• "Effective Approaches to Web Application
Security" by Zane Lackey
45. Proactive Security: All Levels
• Continually drive improvements into your tools
and processes
• Be flexible and ready to adapt
• Share what you've learned and done!
48. Thanks!
• Contact Us:
• sbehrens@netflix.com
• @helloarbit
• OSS: http://netflix.github.io/
• Want to be part of the team? We're hiring!
• Questions?
