This is the slide deck that I used at MITRE ResilienCyCon 2022 when discussing how I approach cyber resiliency in my talk.

1. Accidental Resiliency
Small- and Medium-Sized Business
Evolution by Necessity
by Art Ocain, Vice President of Service Delivery at Airiam
ResilienCyCon
2022

2. art@airiam:~$ whoami
Information Security & Incident Response
• Lead Incident Response
• Sys admin background with
detours through network
admin and cloud admin
• Cybersecurity enthusiast &
evangelist

3. Key Points
Develop
Automation
• Redeployment Scripts
(“Shift-Left” Automation)
• Backup Remediation
• Focus on the constraint
• IT Ops → CI/CD
• Automated incident
creation
• Continuous monitoring
and correlation
• Continuous vulnerability
management & patching
• Continuous pentesting
Patching is a
Resilience Test
• Techniques:
• Redundancy
• Analytic Monitoring
• Dynamic Positioning
• Adaptive Response
(Dynamic
Reconfiguration)
Shorten
Feedback Loops
• Enable rapid feedback
loops between
• Incident Response (IR)
• Security Operations
(SOC)
• Network Operations
(NOC)
• IT Operations (Server,
Endpoint, Cloud, IAM,
etc.)
• Development / DevOps
• Risk Management

4. Front Line Observations
All companies
experience cyber
attack
Most companies will
experience
ransomware
Most companies
have recovery
problems

5. Incidents-a-Plenty
Manufacturing Company
• DoD Contracts, CMMC
• Only wanted “proactive
management & security”
on their ERP server
• Total investment in IT
and Security only
~$250/month

6. Incidents-a-Plenty
Manufacturing Company
• Attacked several times
• Ransomware twice in 1 year
• ERP downtime was minimal
• All other systems needed to
be completely rebuilt from
scratch EACH TIME

7. Incidents-a-Plenty
Manufacturing Company
• Each ransomware attack:
• ~1 month recovery time
• ~$1M lost revenue
• ~$.5M cyber legal,
forensics, and recovery
costs
Vivian: Put an image on this side
with stock image of someone
who is broke /
Empty pockets turned out

8. Back to the Drawing Board
Change Client
Perspectives
• Pain alone is not enough for
people to change bad clients
Create Resilient
Solutions
• We need to create resilience even
when the client is in their own way
Use Incidents
As R&D
• Use every failure we see on an
incident to improve our solution

9. Back to the Drawing Board
Security
software and
tooling cost $.
We can’t do
everything for
everyone for
free.
Client budget is the constraint.

10. Incidents-a-Plenty
SaaS Software Company
• Called on Friday
afternoon
• 1000s of servers
• 100% encrypted with
ransomware
• No uncorrupted backups

11. Incidents-a-Plenty
SaaS Software Company
• No segmentation
• No monitoring
• No security automation
• No detection and response capability
• Unpatched Windows Servers w/ IIS

12. Incidents-a-Plenty
SaaS Software Company
• Created containment to protect recovery
• Prioritized systems, focusing on
critical assets first
MITRE|CREF Navigator™
→Constrain
→→Segmentation
MITRE|CREF Navigator™
→Strategic Design Principle
→→Focus on common critical assets

13. Incidents-a-Plenty
SaaS Software Company
• We scripted:
• eradication, recovery and
redeployment
• Client was back up and running on
Monday
MITRE|CREF Navigator™
→Reconstitute
→→Adaptive Response

14. Preparation
Detection &
Analysis
Containment,
Eradication
& Recovery
Post-Incident
Activity
Incident Response vs Cyber Resilience Engineering Framework
NIST SP 800-61 r2 NIST SP 800-160 v2 r1

15. Back to the Drawing Board
Evolution
Problem Identified
• No segmentation
• No monitoring
• No security
automation
• No detection &
response capability
• Unpatched servers
Solution
• Microsegmentation
• Full SIEM monitoring
• DevSecOps
management system
• Deployed EDR and
XDR
• Daily patching
Evolution for our
MSP Clients
• Microsegmentation
• Full SIEM monitoring
• DevSecOps
management system
• Deployed EDR and
XDR
• Daily patching

16. Back to the Drawing Board
Evolution
Evolution for our
MSP Clients
• Microsegmentation
• Full SIEM monitoring
• DevSecOps
management system
• Deployed EDR and
XDR
• Daily patching
MITRE|CREF Navigator™
→Constrain
→→Segmentation
MITRE|CREF Navigator™
→Constrain & Understand
→→Analytic Monitoring
MITRE|CREF Navigator™
→Prevent or Avoid, Prepare & Re-Architect
→→Coordinated Protection

17. Back to the Drawing Board
• Automation:
• workstation/server/network maintenance
• deployment (Terraform, Puppet, Microsoft WDS)
• backup fixes (auto-remediation)
• Pushed to all of our clients:
• vulnerability management
• security patches and fixes: WE PATCH DAILY!
• our XDR platform
Formed Dev{Sec}Ops Team
Created
immutable
air-gapped
backup
solution
Created
dedicated
Incident
Response
(IR) Team

18. Incident
Response
Managed IT
Operations
Managed
Cybersecurity
Insights
Resilience Journey
• Rapid Feedback Loops - As a
managed service provider
(MSP) for small businesses,
we adapted our IT and
Cybersecurity processes
using what we have learned
from incidents.

19. Incident
Response
Managed IT
Operations
Managed
Cybersecurity
Insights
Resilience Journey
Vulnerable Systems –
Business is afraid to patch
critical systems or does not
have patch plans and
automation.

20. PATCHING = RESILIENCE TESTING

21. Back to the Drawing Board
• Focus on the constraint
• IT Ops → CI/CD
• Continuous vulnerability
management & patching
• Continuous pentesting
Shift to DevOps

22. Back to the Drawing Board
Mindset Evolution
Be A Good MSP
Previous Desired State:
• keep clients happy
• give stellar service
• keep them fast, stable,
and productive on new
technology
Be a Great Resilience
Service Provider
New Desired State:
• manage our
heterogenous clients
as using our
automation
• be able to rapidly
redeploy a client’s
environment
anywhere, anytime

24. Incident
Response
Managed IT
Operations
Managed
Cybersecurity
Insights
The VEEAM Story and Threat Actors:
• Run “kill scripts” on target
hosts to disable VSS, delete
VSS snapshots
• Delete/encrypt backups
• Steal credentials for SAN
and delete SAN snapshots
• Encrypt VMware ESXi hosts
• Attack DR sites and cloud
resources over VPN from
production network

25. VLAN 101
I’m simple, but
flat and easy to
attack.

26. The VEEAM Story: Veeam (on-premises backup software and
infrastructure) was being deleted or encrypted in ransomware
attacks.
• Outcome: All servers, backup infrastructure, and storage is
accessible to attacker and encrypted/deleted in ransomware
attack. VLAN 101
I’m simple, but
flat and easy
to attack.

27. VLAN 101 VLAN 102
1 3
2 4
5
6
7
8
PWR
FAN
ALM
STS
HA
TMP
MGT CONSOLE USB
9 11 13 15
10 12 14 16
17
18
HA1
HA2
PA-3060
Firewall: Block inter-VLAN traffic for 101
and 102.
Hole punched
into VLAN for
admin access.
MITRE|CREF Navigator™
→Re-Architect
→→Predefined Segmentation

28. The VEEAM Story:
• We disjoined Veeam Backup
Server from domain (so
attacker cannot access it by
compromising Active
Directory).
• We segmented the network.
• Outcome: Improved
restorability, but
administrative access to
VLAN102 can still be
compromised if attacker
compromises admin PC.
VLAN 101 VLAN 102
1 3
2 4
5
6
7
8
PWR
FAN
ALM
STS
HA
TMP
MGT CONSOLE USB
9 11 13 15
10 12 14 16
17
18
HA1
HA2
PA-3060
Firewall: Block inter-VLAN
traffic for 101 and 102.
Hole
punched
into VLAN
for admin
access.
MITRE|CREF Navigator™
→Re-Architect
→→Predefined Segmentation

29. VLAN 101 VLAN 102
1 3
2 4
5
6
7
8
PWR
FAN
ALM
STS
HA
TMP
MGT CONSOLE USB
9 11 13 15
10 12 14 16
17
18
HA1
HA2
PA-3060
Firewall: Block inter-VLAN traffic for
101 and 102.
AWS
Azure
MITRE|CREF Navigator™
→Reconstitute → Redundancy
→→Protected Backup & Restore

30. The VEEAM Story:
• We added immutable
cloud storage for
Veeam scale sets.
• Outcome: If on-
premises
infrastructure is
completely
compromised, we still
have a recovery path.
VLAN 101 VLAN 102
1 3
2 4
5
6
7
8
PWR
FAN
ALM
STS
HA
TMP
MGT CONSOLE USB
9 11 13 15
10 12 14 16
17
18
HA1
HA2
PA-3060
Firewall: Block inter-VLAN
traffic for 101 and 102.
AWS
Azure
MITRE|CREF Navigator™
→Reconstitute → Redundancy
→→Protected Backup & Restore

31. VLAN 101 VLAN 102
1 3
2 4
5
6
7
8
PWR
FAN
ALM
STS
HA
TMP
MGT CONSOLE USB
9 11 13 15
10 12 14 16
17
18
HA1
HA2
PA-3060
Firewall: Block inter-VLAN traffic
for 101 and 102.
AWS
Azure
DRaaS
Cloud
Credential &
Key Vault
MITRE|CREF Navigator™
→Reconstitute → Redundancy
→→Protected Backup & Restore

32. The VEEAM Story:
• We used a key vault for
AWS and Azure storage
keys and associated creds
to prevent credential
theft. (Admins don’t keep
the passwords/keys.)
• We added alternate, slower
immutable cloud backup of
critical VMs as a failsafe.
• Outcome: Protection of
backup resources and
recovery paths.
VLAN 101 VLAN 102
1 3
2 4
5
6
7
8
PWR
FAN
ALM
STS
HA
TMP
MGT CONSOLE USB
9 11 13 15
10 12 14 16
17
18
HA1
HA2
PA-3060
Firewall: Block inter-VLAN
traffic for 101 and 102.
AWS
Azure
DRaaS
Cloud
Credential &
Key Vault
MITRE|CREF Navigator™
→Reconstitute → Redundancy
→→Protected Backup & Restore

33. Wrapping Up
The Problem
Threat actors are
focused on SMB/SME
companies.

34. What would the
world be like if:
Enterprises could
recover quickly from
ransomware attack

35. What would the
world be like if
Medium-sized
business could
recover quickly from
ransomware attack

36. What would the
world be like if
Small business could
recover quickly from
ransomware attack