THE JOY OF PROACTIVE SECURITY
SCOTT BEHRENS && ANDY HOERNECKE
EHLO
• Scott Behrens
• Application Security
Engineer at Netflix
• Likes automation
• Torn between breaking
and making
• Loves research and open
source
HELO
• Andy Hoernecke
• Application Security
Engineer at Netflix
• App Sec
• Data Visualization
What We Will Cover
• Challenges of a modern infrastructure
• Proactive security as a solution
• Examination of mature security program
• How to get started
• Conclusion
Terminology Primer
• Define technology terms:
• Application
• Instance
• Autoscaling groups (ASGs) / Load balancers (ELBs)
• AMI
• SecurityGroups
• Regions / Availability Zones (AZs)
Netflix Primer
• 100's of Developers
• Over 1,000 applications
• Hundreds of production pushes a day
• Over 50k instances
• No Security Gates!
Continuous Deployment
• Can be fully automated
• Immutable Platform
• Looks like:
• git -> Create deb (Jenkins) -> AMI Snapshot -> Deploy
Potential Pitfalls
• Multiple concurrent code-bases (A/B testing,
regional functionality, etc.)
• New applications brought online/old
applications retired constantly
• Insecure third party dependencies
More Pitfalls
• How do we identify and catalog assets (IP?, DNS
Name?, Application?, ELB?...)
• How do we profile and provide baseline security to new
applications?
• How do we monitor security policy changes and
configurations in AWS
• Monitoring the web for credential dumps, hacktivism,
sensitive data exposure?
The List Goes On...
• Many of these problems are not unique to
Netflix/the cloud
• Some are unique or can be solved in different
ways
• We have coined our approach to these and other
problems as...
Proactive Security
• Dictionary.com
• Proactive - "serving to prepare for, intervene in,
or control an expected occurrence or situation,
especially a negative or difficult one;
anticipatory"
Proactive Security
• Security controls should be:
• Integrated, automated, scalable, adaptive,
actionable, and intelligent
• Time is limited
• End goal: Less babysitting, more time for harder
problems
Proactive Security
• Find problems early and address them
• Know your weaknesses and work to improve them
• Monitor for anomalies and be prepared to respond
• Collect meaningful data and use it to improve
• Simplify make security the easy path
• Reevaluate your approach
• Share what you learn with others
Find
• Identify issues early
• Old hat:
• Static asset lists, fileshare with old pentest
reports
problems early and address them
Find - New Way
• Define what an asset is:
• Application
• ELB
• FQDN
• IP
Find - New Way
• Have an intelligent way to collect/
track assets and vulns
Monterey
Monterey
• Provides way to automatically define and scan
assets
• Soon to be open sourced
• Asset groups as application name:
• Contain any ELBs or FQDN
• Do not contain instance
Monterey
• Monterey queries AWS for new assets and
changes
• Monterey runs "Monklets" for things like
vulnerability scanning
• Store vulns centrally in S3
Monterey Demo
Monitor
• Monitor, detect, and respond to security issues
• Old hat:
• Reactionary
• New hat:
• Less reactionary aka. more proactive
• Automatic
• Intelligent
for anomalies and be prepared to respond
Simian Army
• aka The Monkeys!
• Open source
• Proactively wreck your
environment to simulate
outages
• Take down apps, instances,
ELBs, even entire regions
• Ensures developers write
resilient code
"Dirty Laundry" Project
• Look for assets unintentionally exposed
• Leverages Monterey for assets
• Uses Scumblr for actioning findings
• Uses Sketchy for collecting status codes,
generating screenshots, and text scrapes
• Both Sketchy/Scumblr are open source
Scumblr
• Intelligent automation platform aka.
our swiss army knife
• Monitor things by using plugins
• Credential Dumps
• Hacktivism
• Brand Reputation
• Full Disclosure
• Leverages Sketchy for screen shots and
text scrapes
• Custom Workflows
Dirty Laundry Demo
Speedbump
• Mechanism to detect attacks and enforce security policies
automatically
• You define what you want to monitor/filter
• Security policies can be time delays, blocking, routing, etc.
• A WAF, proxy and firewall on steroids because it has
application intelligence (business logic)
• Ensnare
Security Monkey
• Open Source
• Monitor events/changes within
AWS
• Changes to IAM users
• Security groups
• Policy changes
• Notify when things change
Have you noticed a trend?
• A more proactive approach
• Identify something interesting automatically
• Notify automatically
• Provide workflows for more complex
remediations
Know
• None of us have unlimited time or resources--we all have
weaknesses
• Work to automatically identify blindspots
• Leverage tools and information where useful
• Old hat:
• Spreadsheet containing static list of apps and manually
created risk rankings
• Not scalable when we are talking 100's or 1000's of apps
your weaknesses and work to improve them
Penguin Shortbread
• Catalog assets
• Measure attributes
• Calculate risk
• Will be open sourced if we can
make it more generic (TBD)
Penguin Shortbread
Demo
Case Study: Monklet Your Way
To Infinity
• Shellshock
• Vulnerable vs exploitable?
• Some commercial scanners only hit root of
webpage
• What about other scripts?
Case Study: Monklet Your Way
To Infinity
• Monterey Monklet!
• Spiders a provided asset (gevented)
• Checks each page for exploitability of Shellshock
• Stores results in S3 centrally
• Deploy N instances of Monklet to cover 100's of
apps
Collect
• Collect data from intelligent systems
• Actually use it to:
• Drive initiatives
• Find what is/isn't working
meaningful data and use it to improve
Example
Simplify
• If it's easy developers will do it
• Make security an enabler and not a burden
• Encourage developers to come to you
• Also: v0.1 doesn't need to be perfect
make security the easy path
Immutable Base AMI
• Make your platform consistent
• For non-cloud users: system image/template
• Conformity Monkey for AWS users (open source)
Danger! Danger!
• Applications often use legacy/outdated dependencies
• Make it clear to developers they are using dangerous stuff
• Netflix leverages an API based on OWASP Dependency
Checker
• API will eventually be open sourced
• Provides information into our Dependency tool
• Work in progress...
Danger! Danger!
FindSecBugs
• Allow users to opt-in to static analysis
• FindSecBugs is lightweight and pretty useful (for
Java)
• They can still deploy regardless but no one likes
seeing red
Reevaluate
• Environment is always changing--what works today
may be ineffective tomorrow
• Developers are agile, which means we must be too
• Be willing to start simple, knowing the initial
solution is temporary (Don't over-engineer!)
• Use the data you've collected to determine when
an approach is no longer sufficient
your approach
Share
• We (security professionals) are all working to solve
similar problems
• Sharing tools and information make all of our lives
easier
• Allows improvement through the collective
consciousness of the industry
• Prevents duplicate effort
what you learn with others
http://netflix.github.io/
(More to come!)
Proactive Security: Level 1
• Create a list of assets
• Rank by sensitivity
• Automate
Proactive Security: Level 2
• Strategize security based on asset risk. Ex:
1. Sensitive assets: Examine thoroughly
2. Frequently used components: Secure by
default
3. Everything else: Find a good baseline level
of due diligence
Proactive Security: Level 3
• Identify weak links
• Architectural: Poor network segmentation
• Cultural: Developers don't understand XSS
• Specific: "Application X" relies on an outdated
version of Struts
• Determine how to measure the weaknesses and put in
place a plan to address or manage
Proactive Security: Level 4
• Monitor, alert, and gather more intel
• Detect anomalies in security relevant functionality:
Login successes/failures, password resets,
authorization failures...
• Dashboards are fun, but intelligent alerting is better
• "Effective Approaches to Web Application
Security" by Zane Lackey
Proactive Security: All Levels
• Continually drive improvements into your tools
and processes
• Be flexible and ready to adapt
• Share what you've learned and done!
Works Cited
• http://www.netbraintech.com/products/feature-
guide/images/asset-report-step-5-2.png
• http://media.amazonwebservices.com/
architecturecenter/AWS_ac_ra_web_01.pdf
• http://www.slideshare.net/zanelackey/effective-
approaches-to-web-application-security
Tools
• Netflix OSS

https://netflix.github.io/
• OWASP Dependency Check

https://www.owasp.org/index.php/
OWASP_Dependency_Check
• Ensnare

https://github.com/ahoernecke/ensnare
• FindSecBugs

https://github.com/h3xstream/find-sec-bugs
Thanks!
• Contact Us:
• sbehrens@netflix.com
• ahoernecke@netflix.com
• OSS: http://netflix.github.io/
• Want to be part of the team? We're hiring!
• Questions?

The Joy of Proactive Security

  • 1.
    THE JOY OFPROACTIVE SECURITY SCOTT BEHRENS && ANDY HOERNECKE
  • 2.
    EHLO • Scott Behrens •Application Security Engineer at Netflix • Likes automation • Torn between breaking and making • Loves research and open source
  • 3.
    HELO • Andy Hoernecke •Application Security Engineer at Netflix • App Sec • Data Visualization
  • 4.
    What We WillCover • Challenges of a modern infrastructure • Proactive security as a solution • Examination of mature security program • How to get started • Conclusion
  • 5.
    Terminology Primer • Definetechnology terms: • Application • Instance • Autoscaling groups (ASGs) / Load balancers (ELBs) • AMI • SecurityGroups • Regions / Availability Zones (AZs)
  • 6.
    Netflix Primer • 100'sof Developers • Over 1,000 applications • Hundreds of production pushes a day • Over 50k instances • No Security Gates!
  • 7.
    Continuous Deployment • Canbe fully automated • Immutable Platform • Looks like: • git -> Create deb (Jenkins) -> AMI Snapshot -> Deploy
  • 8.
    Potential Pitfalls • Multipleconcurrent code-bases (A/B testing, regional functionality, etc.) • New applications brought online/old applications retired constantly • Insecure third party dependencies
  • 9.
    More Pitfalls • Howdo we identify and catalog assets (IP?, DNS Name?, Application?, ELB?...) • How do we profile and provide baseline security to new applications? • How do we monitor security policy changes and configurations in AWS • Monitoring the web for credential dumps, hacktivism, sensitive data exposure?
  • 10.
    The List GoesOn... • Many of these problems are not unique to Netflix/the cloud • Some are unique or can be solved in different ways • We have coined our approach to these and other problems as...
  • 11.
    Proactive Security • Dictionary.com •Proactive - "serving to prepare for, intervene in, or control an expected occurrence or situation, especially a negative or difficult one; anticipatory"
  • 12.
    Proactive Security • Securitycontrols should be: • Integrated, automated, scalable, adaptive, actionable, and intelligent • Time is limited • End goal: Less babysitting, more time for harder problems
  • 13.
    Proactive Security • Findproblems early and address them • Know your weaknesses and work to improve them • Monitor for anomalies and be prepared to respond • Collect meaningful data and use it to improve • Simplify make security the easy path • Reevaluate your approach • Share what you learn with others
  • 14.
    Find • Identify issuesearly • Old hat: • Static asset lists, fileshare with old pentest reports problems early and address them
  • 15.
    Find - NewWay • Define what an asset is: • Application • ELB • FQDN • IP
  • 16.
    Find - NewWay • Have an intelligent way to collect/ track assets and vulns Monterey
  • 17.
    Monterey • Provides wayto automatically define and scan assets • Soon to be open sourced • Asset groups as application name: • Contain any ELBs or FQDN • Do not contain instance
  • 18.
    Monterey • Monterey queriesAWS for new assets and changes • Monterey runs "Monklets" for things like vulnerability scanning • Store vulns centrally in S3
  • 19.
  • 20.
    Monitor • Monitor, detect,and respond to security issues • Old hat: • Reactionary • New hat: • Less reactionary aka. more proactive • Automatic • Intelligent for anomalies and be prepared to respond
  • 21.
    Simian Army • akaThe Monkeys! • Open source • Proactively wreck your environment to simulate outages • Take down apps, instances, ELBs, even entire regions • Ensures developers write resilient code
  • 22.
    "Dirty Laundry" Project •Look for assets unintentionally exposed • Leverages Monterey for assets • Uses Scumblr for actioning findings • Uses Sketchy for collecting status codes, generating screenshots, and text scrapes • Both Sketchy/Scumblr are open source
  • 23.
    Scumblr • Intelligent automationplatform aka. our swiss army knife • Monitor things by using plugins • Credential Dumps • Hacktivism • Brand Reputation • Full Disclosure • Leverages Sketchy for screen shots and text scrapes • Custom Workflows
  • 24.
  • 25.
    Speedbump • Mechanism todetect attacks and enforce security policies automatically • You define what you want to monitor/filter • Security policies can be time delays, blocking, routing, etc. • A WAF, proxy and firewall on steroids because it has application intelligence (business logic) • Ensnare
  • 26.
    Security Monkey • OpenSource • Monitor events/changes within AWS • Changes to IAM users • Security groups • Policy changes • Notify when things change
  • 27.
    Have you noticeda trend? • A more proactive approach • Identify something interesting automatically • Notify automatically • Provide workflows for more complex remediations
  • 28.
    Know • None ofus have unlimited time or resources--we all have weaknesses • Work to automatically identify blindspots • Leverage tools and information where useful • Old hat: • Spreadsheet containing static list of apps and manually created risk rankings • Not scalable when we are talking 100's or 1000's of apps your weaknesses and work to improve them
  • 29.
    Penguin Shortbread • Catalogassets • Measure attributes • Calculate risk • Will be open sourced if we can make it more generic (TBD)
  • 30.
  • 31.
    Case Study: MonkletYour Way To Infinity • Shellshock • Vulnerable vs exploitable? • Some commercial scanners only hit root of webpage • What about other scripts?
  • 32.
    Case Study: MonkletYour Way To Infinity • Monterey Monklet! • Spiders a provided asset (gevented) • Checks each page for exploitability of Shellshock • Stores results in S3 centrally • Deploy N instances of Monklet to cover 100's of apps
  • 33.
    Collect • Collect datafrom intelligent systems • Actually use it to: • Drive initiatives • Find what is/isn't working meaningful data and use it to improve
  • 34.
  • 35.
    Simplify • If it'seasy developers will do it • Make security an enabler and not a burden • Encourage developers to come to you • Also: v0.1 doesn't need to be perfect make security the easy path
  • 36.
    Immutable Base AMI •Make your platform consistent • For non-cloud users: system image/template • Conformity Monkey for AWS users (open source)
  • 37.
    Danger! Danger! • Applicationsoften use legacy/outdated dependencies • Make it clear to developers they are using dangerous stuff • Netflix leverages an API based on OWASP Dependency Checker • API will eventually be open sourced • Provides information into our Dependency tool • Work in progress...
  • 38.
  • 39.
    FindSecBugs • Allow usersto opt-in to static analysis • FindSecBugs is lightweight and pretty useful (for Java) • They can still deploy regardless but no one likes seeing red
  • 40.
    Reevaluate • Environment isalways changing--what works today may be ineffective tomorrow • Developers are agile, which means we must be too • Be willing to start simple, knowing the initial solution is temporary (Don't over-engineer!) • Use the data you've collected to determine when an approach is no longer sufficient your approach
  • 41.
    Share • We (securityprofessionals) are all working to solve similar problems • Sharing tools and information make all of our lives easier • Allows improvement through the collective consciousness of the industry • Prevents duplicate effort what you learn with others
  • 42.
  • 43.
    Proactive Security: Level1 • Create a list of assets • Rank by sensitivity • Automate
  • 44.
    Proactive Security: Level2 • Strategize security based on asset risk. Ex: 1. Sensitive assets: Examine thoroughly 2. Frequently used components: Secure by default 3. Everything else: Find a good baseline level of due diligence
  • 45.
    Proactive Security: Level3 • Identify weak links • Architectural: Poor network segmentation • Cultural: Developers don't understand XSS • Specific: "Application X" relies on an outdated version of Struts • Determine how to measure the weaknesses and put in place a plan to address or manage
  • 46.
    Proactive Security: Level4 • Monitor, alert, and gather more intel • Detect anomalies in security relevant functionality: Login successes/failures, password resets, authorization failures... • Dashboards are fun, but intelligent alerting is better • "Effective Approaches to Web Application Security" by Zane Lackey
  • 47.
    Proactive Security: AllLevels • Continually drive improvements into your tools and processes • Be flexible and ready to adapt • Share what you've learned and done!
  • 48.
    Works Cited • http://www.netbraintech.com/products/feature- guide/images/asset-report-step-5-2.png •http://media.amazonwebservices.com/ architecturecenter/AWS_ac_ra_web_01.pdf • http://www.slideshare.net/zanelackey/effective- approaches-to-web-application-security
  • 49.
    Tools • Netflix OSS
 https://netflix.github.io/ •OWASP Dependency Check
 https://www.owasp.org/index.php/ OWASP_Dependency_Check • Ensnare
 https://github.com/ahoernecke/ensnare • FindSecBugs
 https://github.com/h3xstream/find-sec-bugs
  • 50.
    Thanks! • Contact Us: •sbehrens@netflix.com • ahoernecke@netflix.com • OSS: http://netflix.github.io/ • Want to be part of the team? We're hiring! • Questions?