A little over a year ago I made the transition from external security consultant to internal offensive security engineer at Facebook. I went from a full time breaker to part time fixer. This talk is aimed at providing lessons learned and documenting the mindset changes I've made over the last year that I feel can be used by the industry as a whole. I've broken the lessons learned into three primary buckets; Red, Blue, and Purple and the talk will hopefully bring value to anyone working in their respective bucket or assist in their creation/continuing of purple teaming at their company.
3. Overview
1. Provide lessons learned over the last year
2. $PreviousCompany’s approach to Red Teaming
3. Our take on Purple Teaming
4. Examples in Red, Blue, and Purple Buckets
9. Red Bucket
• Groundhog day every week, after multiple years of doing this and giving
advice really wasn’t seeing people get any better.
• Spent a bit of time wondering why only 1 or 2 clients were really getting
better while the rest were only marginally getting better.
• Is it me and my advice? (More on this soon) Are clients just lazy?
Something else?
Why Chris made the job change
12. Red Bucket
• Consultants aren’t always invested in making a company better
o They provide what the client wants/perceive as valuable
o Most clients don’t know what is valuable
o Interesting/Michael Bay Explosions not always valuable
• Exhausted usefulness of outside pentest entities
• Need internal knowledge to craft more interesting attacks
Why an Internal Red Team?
13. Red Bucket
• “Find vulnerable paths to achieve an objective” --> Pentest
• IMPACT driven
• Deliver a (researched) solution to system owners
o Don’t just drop off a report of “oh that sucks to the system owner”
• Example:
o Problem: Jenkins didn’t require auth, leads to RCE
o Solution: Research the various ways Jenkins does auth, test locally, write
up details instructions to fix, task affected system owners to fix + link to
how to fix
o Enjoy seeing vuln go away… :-)
Pentesting at $PreviousCompany
14. Red Bucket
• Real life training event for IR team(s)
o Post Exploit, Persistence, Lateral Movement more important than initial
hook
o Use “assist” to avoid initial detection (if necessary)
o “Exercise our ability to respond to an incident and find broken tools/
processes” --> Red Team,
o “Understand where are detection mechanisms are strong/weak” -->
Detection Focused Red Team.
o Red Team switches to Blue Team after objectives achieved
Red Teaming at $PreviousCompany
15. Red Bucket
• $PreviousCompany’s Public Red Team Exercises
o Vampire
o Loopback
o Shire
https://medium.com/@magoo/red-teams-6faa8d95f602
https://threatpost.com/how-facebook-prepared-be-hacked-030813/77602/
http://arstechnica.com/security/2013/02/at-facebook-zero-day-exploits-backdoor-code-bring-war-games-drill-to-life/
Red Teaming at $PreviousCompany
16.
17. Blue Bucket
• Strive to be seen as helping not creating work
• Encourage “same team” mentality
• Same manager for blue and red teams
• Partnered with senior blue team member (@cmccsec)
o Learned from each other
o Highest impact to date
o Develop empathy for what the other side deals with:
q ex. Implant deployer failed
q ex. Common lateral movement got us caught
q ex. Feel the despair as blue team takes out our implants
How to work effectively with your IR/SOC/CERT teams
18. Blue Bucket
• IR teams need someone with attacker knowledge/methodology
o “More offense in your defense”
o What can I do with these credentials?
o Where can I REALLY go with this level of access? vs where you THINK I
can go
o Knowledge of current attacker methodologies
o Ability to test vs. believing assumptions or info from another team
How to work effectively with your IR/SOC/CERT teams (2)
19. Blue Bucket
• Pentesters are interested in IR as
long as they don’t have to make a
career change to learn it
• Responders are interested in
offense as long as they don’t have
to make a career change to learn it
Working with IR/SOC/CERT teams (3)
Image from: http://blog.workisnotajob.com/post/3129795739/new-things-and-new-thoughts-important
20. Blue Bucket
• Most red teamers have limited access to enterprise defensive tools
o No visibility into what actions were detected or not
o What actions/events causes an alert to fire
o Red team within IR fills this gap & allows both teams to learn
• Blue teamers sometimes don’t have time to fully understand new attacks
o Know what a golden ticket is, but never made/used one/understand
risks or artifacts it leaves behind
Writing alerts to catch yourself makes you a better pentester (1)
21. Blue Bucket
• Tons of tools leave tons of artifacts if you are equipped to look for them
• MSF creates a random 16 character hostname during smb_login requests
o Random just became not so random :-(
How can I modify my tools to avoid public or internal signatures?
23. Blue Bucket
• Fix-->Modify metasploit to hardcoded or different value
MSF 16 character smb_login example
24. Blue Bucket
• Do I need a tool when I can do it via command line?
o Most tools/scripts leave artifacts
• Force you to up your game by fixing low hanging vulns & alerting on the
easy to catch stuff
• Stay up to date with latest techniques for lateral movement or persistence
o Then write rules for them
o Then modify the tools not to be detected :-)
Writing alerts to catch yourself makes you a better pentester (2)
25. Blue Bucket
• My biggest gripe as a consultant was rarely getting feedback on what the
client saw
o Did you see me and let me go because it was a Pentest? Or not see me
at all?
q Now I make a W.A.G. in your report
o Actions on detection often not clear in ROE
o The answer drives recommendations and remediation
• Validate your tools and techniques against other “professionals”
• Steal their good ideas :-)
Incident Response can be fun?!?!
26. Blue Bucket
• I was tasked to come up with actual solutions to issues I found on pentests
• Took me MUCH longer to research and test a fix than break it
• Developed quite a bit of Empathy for receivers of my report
• People don’t have time to act on generic findings/recommendations
o **We** give pretty bad recommendations (more on this soon)
• Delivering a detailed/REAL fix with the problem goes far
Fixing vulnerabilities you find is hard
27.
28. Purple Bucket
• Recommendations are usually given in a one size fits all format
o Company size, culture, etc are generally not considered
o Best practice vs what would actually work for the specific environment
o ex: “Utilize application whitelisting”
o ex: “Segment your network”
We give bad recommendations and wonder why clients don't fix things
31. Purple Bucket
• Keys to this:
o Show impact of the vuln
q “Access to X gave me Y information”
q “Compromising X allowed me to bypass 2fac requirements”
q Attackers don’t pop shells for fun, only pentesters do
• Attackers have specific objectives (Impact)
o Remove all the fluff...Get full point across in Executive Summary
q Lead with the “So What?”
o Technicals details in later sections
Making pentesting a desired activity and not just creating work for
others -- high value / low friction (1)
32. Purple Bucket
• Keys to this:
o Work through solutions and don’t overhype the problem
o Provide actionable solutions, with steps, preferably tested in our
environment, with the recommendations.
q Example: Don’t say “use iptables to restrict access”
q Instead: Test some iptables rules then provide the iptables rules so
they can cut and paste
q Difficult for external consultants to provide this level of detail
o Retest and retest until it is fixed
Making pentesting a desired activity and not just creating work for
others -- high value / low friction (2)
33. Purple Bucket
Typical Red Team vs Blue Team relationship...
http://fellows-house.deviantart.com/art/Red-vs-Blue-195109671
34. Purple Bucket
• Chris sits with the Incident Response Team
• Most other companies Red Team vs Blue Team is VERY adversarial
o One of the primary goals with the role was to avoid this relationship
o Ex: Blue Team not wanting to share detection/defense strategy
q Focused (wrongly) on catching the Red Team
q Sharing strategy forces Red Team to get creative; just like real
attackers will
q Iterate and make both teams improve
Where the Red Team sits @ $PreviousCompany and why (1)
35. Purple Bucket
• Purple team reports can include both Red and Blue narrative
o Significantly more valuable than “How I pwned all your stuff” narrative
o Highlight to leadership the value of the IR team, show wins with new
initiatives, gear, training, etc
o Identify logging gaps, identify technology overlap
Where the Red Team sits @ $PreviousCompany and why (2)
36. Purple Bucket
• More complicated the environment the longer it takes to achieve goals
• Internal Red Team can test fixes against the environment
• You can construct a Red Team exercise to force IR team to test/refine
specific skills:
o How people react under fire
o Forensics (Disk, Memory, Mobile)
o Malware analysis
o Identify logging gaps
o External relationships (comms, legal, network, IT, etc)
o Blue Team OPSEC
Pros/cons of internal vs external red teams (1)
37. Purple Bucket
• It’s really fun to watch someone that didn’t conduct the attack...deconstruct
the attack
o What artifacts did I leave behind?
o How good/bad was my OPSEC?
o Did my pentester tricks work?
o Small things that red team may consider insignificant can be all the
breadcrumbs the IR team needs
• Ability to suppress alerts in order not to start the exercise early
o Use “the assist” to get us past initial access vector
Pros/cons of internal vs external red teams (2)
38. Purple Bucket
• Outside teams aren’t conditioned to the network
o They won’t automatically ASSUME network conditions
• Can find vulns while messing with unknown systems
o Not following the normal flow or use cases
o This happened during a previous red team (internal system)
• Outside team can bring in skills the inside team doesn’t have
o Previous outside team found a privilege escalation exploit
o Would have been VERY difficult for me to find and exploit that bug
Pros/cons of internal vs external red teams (3)
39. Purple Bucket
• Internal team gets to:
o Work with Blue team during containment, eradication, recovery and post-
incident (fixing) phases
o Be Included in planning and execution of the response
o See everything that goes into an effective response
o Capture gaps in response
q ex. IR missed the backup implant
Pros/cons of internal vs external red teams (4)
40. Purple Bucket
• Fully internal -- No external Contractors
• Partnered with senior Blue Team member
• Took things I found pentesting…chained together story for the exercise
• “Create internal havoc” attackers
Overview of a Recent $PreviousCompany Red Team Exercise
60. Wrapup
● Purple Teaming is bleeding edge
○ There is no right or wrong way of doing it
● Forming strong bonds between Red and Blue Teams IS possible
○ We have proven this makes both team better/stronger
● We need more offense in our defense
○ And defense in our offense...Purple teaming is a way to do this
● Try it and see
○ Share your results