Why cant organizations fix all vulnerabilities? Why is it so difficult? Let's find it out
Presentation given at BSides Ottawa on Nov 29, 2019 by Dennis Chaupis and Ivan Perez
2. Who are we?
BSides Ottawa 2019 | Coexisting with vulnerabilities 2
Dennis Chaupis
CISSP, CRISC, CTPRP
Senior Manager
Deloitte – Risk Advisory - Cyber Security
Dennis is leading the Vulnerability Management &
Penetration testing practice in Toronto from where
he supports engagements across Canada and
globally. He focuses on helping organizations
identifying, managing, and remediating
vulnerabilities that could lead to a business impact.
Dennis has also worked for a major Canadian bank
in its Operational Risk Management group.
Opinions and views are my own and not necessarily
my current or past employers.
3. Who are we?
BSides Ottawa 2019 | Coexisting with vulnerabilities 3
Ivan Perez
OSCP
Consultant
Deloitte – Risk Advisory - Cyber Security
Ivan is a consultant with Deloitte’s Cyber Risk
Services practice and a holder of Offensive Security
Certified Professional (OSCP) certificate with
experience in threat intelligence, vulnerability
assessments, and penetration testing. Since joining
Deloitte, Ivan has worked on offensive
engagements with clients in numerous sectors
including the financial, shared services, energy,
public, and academic sectors. He also likes sea
turtles.
Opinions and views are my own and not necessarily
my current or past employers.
4. “Coexisting with vulnerabilities”……… what?
Pentester:
• How many times have you performed a recurring
pentest and have found the exact same
vulnerability over and over again?
BSides Ottawa 2019 | Coexisting with vulnerabilities 4
SOC:
• How many times you have either identified a
vulnerability as part of your VA scan, log review, or
reported by an “intel” source and when you created
a ticket so its resolved……. Ticket gets closed but
nothing has been fixed?
Architect / Application Team:
• “We are going to create the new banking platform,
we are going to be revolutionary with this; however,
we are still allowing insecure communications
to the mainframe so….. We will just deal with
it”.
6. BSides Ottawa 2019 | Coexisting with vulnerabilities 6
Case study 7
What Apps look like in real life 10
Three key roles 11
Examples 13
What to do? 19
Food for thought 22
Contents
7. Case Study
BSides Ottawa 2019 | Coexisting with vulnerabilities 7
192.168.CBA.XY1 – 192.168.CBA.XY5 -> Internal IP’s
• Unpatched software
• Unsupported software
• Public shares
• Weak SSH configuration
• Self signed /default certificates in use
• Unauthenticated remote support access
• More… more… more….
200.101.ABC.XYZ -> External IP
• Insecure communication protocols (FTP, Telnet, Remote Mgmt)
• Default web application server pages
• Management console accessible over the Internet
www.MyNextGenBank.com
• Use of TLS 1.0 or weak ciphers
• Error messages allow user enumeration
• Weak password recovery process via PVQs
• XSS / SQL Injection
• Default pages / error messages
8. And the question is…
BSides Ottawa 2019 | Coexisting with vulnerabilities 8
9. Case Study
BSides Ottawa 2019 | Coexisting with vulnerabilities 9
192.168.CBA.XY1 – 192.168.CBA.XY5 -> Internal IP’s
• Unpatched software
• Unsupported software
• Public shares
• Weak SSH configuration
• Self signed /default certificates in use
• Unauthenticated remote support access
• More… more… more….
200.101.ABC.XYZ -> External IP
• Insecure communication protocols (FTP, Telnet, Remote Mgmt)
• Default web application server pages
• Management console accessible over the Internet
www.MyNextGenBank.com
• Use of TLS 1.0 or weak ciphers
• Error messages allow user enumeration
• Weak password recovery process via PVQs
• XSS / SQL Injection
• Default pages / error messages
10. Stakeholder What they “see”
Business (App Owner)
What apps look like in “real life”
BSides Ottawa 2019 | Coexisting with vulnerabilities 10
F5 LB
App Srv
Farm 1
App Srv
Farm 2
ESXi
…
DB
3rd Party
App Y
API Z
AWS/Azure/GCP
OS
Apps DB Agents
Servers Some “Cloud” DatabaseMicro services
CIO / CTO
App Team (e.g. Architect, QA
Team)
Server Owner Most commonly seen as… “The Admin”. This is the person who is the responsible for server
management, configuration, patching. Usually this individual does not have “full visibility”
of what the app exactly does (nor wants to); but this is the person who can help you or
*destroy you*.
“The admin” also goes by: “Server guy”, “DBA”
11. Three key roles
BSides Ottawa 2019 | Coexisting with vulnerabilities 11
CTO
• Focused on new technologies
and how the organization
“keeps up” with them.
• Drives how technology is
provided to clients.
CISO
• Responsible for the overall
Information Security / Cyber
Security of the organization.
• Usually reports to the CIO
• Asset management
• Server / Infra build
• Patch Management
• Software Development
• …
• VA Scans
• Pentest, RTO, Threat Hunting
• Secure Software Development
• …
• “We need cloud functionality
for A,B,C or E”
• “Client’s need to click-and-call”
• …
Vs Vs
CIO
• Most commonly focused on
internal tasks/operations.
• Crucial for IT management
and operations.
12. We won’t be able to patch them all
“Coexisting with
vulnerabilities” is
not the same as
“ignoring
vulnerabilities” nor
an excuse for not
remediating.
Coexisting with vulnerabilities is the natural “path” that
organizations follow to focus in their business.
• This is the part where most technical people do not want to
spend their time at…. It is “someone else’s job”
• Other terms like “risk”, “impact”, “compensating control”,
“risk treatment”, “residual risk” come into play.
• Must not be the rule for everything the organization does
not want to remediate.
• This is the part where senior management will spend most of
their time: Trying to find the best way possible to deal with
vulnerabilities and threats. On the other hand, this is the
part where most technical people, struggle the most,
because they do not understand why something “this
simple” cannot be patched.
What can we do?
BSides Ottawa 2019 | Coexisting with vulnerabilities 12
13. Example 1
BSides Ottawa 2019 | Coexisting with vulnerabilities 13
Mainframe
HQ
LaptopsPhones Mobile
3rd Parties
• Vendors
• Partners
• Alliances
Application A is one of our banking
applications and can only be
accessed via terminal using IBM’s
implementation of Telnet.
14. Example 1: What does it mean?
It means that all the communication that is being
exchanged between the client and the mainframe
is being transmitted in plain text.
BSides Ottawa 2019 | Coexisting with vulnerabilities 14
Mainframe
Pentester: “Consider replacing insecure
protocol with its secure version”
Easy there Mr. Security….. Can’t do
that. Can’t just “enable SSH”…
- Who will manage the keys?
- What type of keys are we going to
use?
- Do we know if the application support
it?
- Does the business know this will cost
them $1 Million to start?
- Are there other applications impacted
if we just “switch” to secure protocol?
- Will we just shut down current port
and replace with secure one? Or is
there a rollout plan?
- Do we need to change anything in
application or leave as-is?
- Does this impact NW usage?
Application delay?
15. Example 2
BSides Ottawa 2019 | Coexisting with vulnerabilities 15
HQ
Laptops
VoIP
Printers
LaptopsVoIP Printers
Laptops
VoIP
Printers
Company A has recently deployed their “state of the art” NAC solution. Now they can
control who can connect to the LAN network. They felt very confident that no
unauthorized user can access their LAN network just by plugging in to the network port.
During a pentest, the tester was able to bypass the NAC solution by impersonating a
trusted device. The tester cloned the MAC address of a VoIP device and…… done!
Network access was achieved
16. Example 2: What does it mean?
It means that the current NAC solution did not
consider the “weakest link” in its design and
can be bypassed.
BSides Ottawa 2019 | Coexisting with vulnerabilities 16
Pentester: “Consider using certificates
for authentication”
Easy there Mr. Security….. Can’t do
that. Can’t just “install certificates”…
- Who will manage the certificates?
- Are we buying certificates?
- Are we deploying our own CA?
- Who is going to pay for it?
- Are we sure our devices will support
it?
- Can we use certificates in VoIP or
printers? Any other devices we are
forgetting?
- Do we have a centralized asset
database that we can trust?
Laptops
VoIP
Printers
17. Example 3
BSides Ottawa 2019 | Coexisting with vulnerabilities 17
Company X launched their ecommerce platform developed by a word class vendor a year
ago, it was supposed to use the latest and greatest technologies.
During a pentest, the tester found that it was possible to downgrade the encryption
to something much less secure (i.e. SSLv3 – POODLE attack).
LaptopsPhones Mobile
Backend
Frontend
Clients
18. Example 3: What does it mean?
It means that the current TLS/SSL configuration
allows a client to use insecure TLS/SSL
algorithms and protocols.
BSides Ottawa 2019 | Coexisting with vulnerabilities 18
Pentester: “Disable support for weak
protocols and encryption algorithms”
Easy there Mr. Security….. Can’t do
that. Can’t just “disable weak
protocols”…
- Have we checked if the application is
supporting it?
- Is it an infrastructure or application
issue? Or both?
- Was it well defined when creating the
requirements for the application?
- What about compatibility with legacy
applications?
19. So… what to do?
BSides Ottawa 2019 | Coexisting with vulnerabilities 19
Pentester: Other than reporting it and providing as
much technical information as possible……. Not
much more.
Server Owner / Admin: Provide as much clarity and
clarification on what can be done and what cannot
be done; moreover, be willing to help the non-
technical people to understand the technical
aspects of the vulnerabilities and/or technology
constraints of remediation activities.
Risk Professional: Try to be as technical as
possible. This will allow you understand the actual
vulnerability and to assess what the right
remediation alternative.
CXO: Try to listen to the whole story when possible.
In the end is your responsibility and accountability
if something goes wrong.
20. Back to the study case…
What and how would YOU patch this now?
BSides Ottawa 2019 | Coexisting with vulnerabilities 20
192.168.CBA.XY1 – 192.168.CBA.XY5 -> Internal IP’s
• Unpatched software
• Unsupported software
• Public shares
• Weak SSH configuration
• Self signed /default certificates in use
• Unauthenticated remote support access
• More… more… more….
200.101.ABC.XYZ -> External IP
• Insecure communication protocols (FTP, Telnet, Remote Mgmt)
• Default web application server pages
• Management console accessible over the Internet
www.MyNextGenBank.com
• Use of TLS 1.0 or weak ciphers
• Error messages allow user enumeration
• Weak password recovery process via PVQs
• XSS / SQL Injection
• Default pages / error messages
21. Back to the study case…
What and how would YOU patch this now?
BSides Ottawa 2019 | Coexisting with vulnerabilities 21
192.168.CBA.XY1 – 192.168.CBA.XY5 -> Int. IP’s
• Unpatched software
• Unsupported software
• Public shares
• Weak SSH configuration
• Self signed /default certificates in use
• Unauthenticated remote support access
• More… more… more….
200.101.ABC.XYZ -> External IP
• Insecure communication protocols (FTP, Telnet, Remote
Mgmt)
• Default web application server pages
• Management console accessible over the Internet
www.MyNextGenBank.com
• Use of TLS 1.0 or weak ciphers
• Error messages allow user enumeration
• Weak password recovery process via PVQs
• XSS / SQL Injection
• Default pages / error messages
One of many options could be:
• Get the detailed findings
• Question the existence of the
vulnerabilities. For example: “Why is TLS
1.0 enabled… is it a server built issue or
application issue?” instead of “No it is not
and it’s been working like that without
any issues”.
• Try to see the “end-to-end”.
• Have all the relevant parties in the same
discussion. Do not work in silos.
• Do not start remediating before thinking
all the possible outcomes.
• Do you make all the changes at the same
time or gradually? Are we “that much”
exposed?
22. Food for thought
• Not all vulnerabilities must be “patched; however, all vulnerabilities
must be treated.
• Avoid thinking that “hackers do not come after us… they go after the
big companies”.
• Create a remediation strategy that meets both risk exposure and risk
appetite.
• If you are an admin, do not just fix what was reported. Most likely you
might know of other components that are also affected by the same
vulnerability.
• Do not just assign a risk rating (L/M/H) or a score (e.g. CVSS). You
must consider value of the asset and business impact.
• Crown Jewels: Identify the assets that matter the most first.
BSides Ottawa 2019 | Coexisting with vulnerabilities 22