Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Salus H4D 2021 Lessons Learned

business model, business model canvas, mission model, mission model canvas, customer development, hacking for defense, H4D, lean launchpad, lean startup, stanford, startup, steve blank, pete newell, bmnt, entrepreneurship, I-Corps, Security, NSIN, NSA, cybersecurity, Joe Felter

  • Be the first to comment

Salus H4D 2021 Lessons Learned

  1. Current Problem Scope: Sub-optimal maintenance window scheduling delays the remediation of vulnerabilities at the DLA, weakening the agency’s cybersecurity. The tradeoff between operational uptime requirements and the security benefits of frequent patching isn’t quantitatively understood. Original Problem Scope: Cybersecurity analysts need a tool to more quickly remediate vulnerabilities in DLA systems in order to keep their network secure. Team Salus Support Team Sponsor: Shane Williams, DLA Information System Security Manager Business Mentor: Richard Tippitt, Defense Innovation Unit (DIU), Product Specialist Defense Mentor: LTC Jim Wiese, Hoover Institution, National Security Affairs Fellow 94 Total Interviews Noah Frick MBA, Strategy/Product Shreyas Parab BS Biocomputation, Product Kyla Guru BS Compsci/IR, Cyber Expert Henry Person MS MS&E, Industry Expert Michael Wornow PhD Compsci, AI Expert Sponsor Organization Defense Logistics Agency (DLA)
  2. The Problem 2 The DLA provides critical logistics to the Department of Defense and across the federal government. Cyber attacks present an existential risk to a critical node that helps maintain readiness Example Vulnerability Breakdown: ● Critical 27,000 ● High 22,000 ● Medium 87,000 ● Low 17,00 19 critical business applications running on thousands of servers across several different hosted environments
  3. Discovering the problem 3 10 9 8 7 6 5 4 3 2 Week 1 We learned about the problem sponsor and the current vulnerability management process, and set off on our first hypotheses.
  4. At first, we thought it was all about detecting... 4 “We need an AI-powered malware detector based on cutting-edge research.” Team Salus DLA Sponsor “Taking a step back, a tool that simply scanned and ranked vulnerabilities might be super helpful!”
  5. “Vulnerability scanning actually does a pretty good job at detecting known vulnerabilities, but we have to know what assets to scan.” Enterprise Vulnerability Scanner 5
  6. Then, we thought it was asset management! 6 “We need an AI-powered malware detector based on cutting-edge research.” Team Salus DLA Sponsor “Taking a step back, a tool that simply scanned and ranked vulnerabilities might be super helpful!” Team Salus “Wait, the DLA doesn’t even know what computers are on their network; let’s fix that!”
  7. Then, we thought it was asset management! 7 “We need an AI-powered malware detector based on cutting-edge research.” Team Salus DLA Sponsor “Taking a step back, a tool that simply scanned and ranked vulnerabilities might be super helpful!” Team Salus “Wait, the DLA doesn’t even know what computers are on their network; let’s fix that!” DLA Cyber Tools Team “Hold on, we already built an internal tool that solves that problem.”
  8. So, we focused on learning about process Scanning Patch Testing Patch Deployment Patch Validation
  9. And we learned... Requires an initial coordination process to test the patch... ...and then an additional coordination process to deploy the patch into production!
  10. Focus on Patching 10 10 9 8 7 6 5 4 3 2 Week 1 We doubled down where we thought we could make a difference.
  11. We realized we needed to update our Beneficiaries 11 J61 J62 J64 J6 Vulnerability Managers and Information System Security Managers Application Programs Infrastructure Programs Audit vulnerabilities, track patching progress Own the software and hardware which are affected by patches...Coordinate and implement patches! Information Technology Division
  12. We realized we needed to update our Beneficiaries 12 J61 J62 J64 J6 Vulnerability Managers and Information System Security Managers Application Programs Infrastructure Programs and System Administrators “All I can do is ask nicely” “I care about patching, but it’s hard to coordinate with [infrastructure programs]” “We don’t want to annoy the applications, all they care about is uptime” Information Technology Division
  13. “The problem pretty much always boils down to a lack of understanding across all involved parties regarding what will happen when we install this patch.” - @VA_Network_Nerd “Imagine Stanford grad students coming to reddit for help...” - @geezer1492 We found more validation in alternative sources... 13
  14. And challenged common sense... 14 “We only schedule our maintenance windows in the middle of the payroll period ” J62 Application Program Manager “Nope! We just rely on common sense” “That makes sense. Do you look at any usage data that validates that belief?” J62 Application Program Manager
  15. How is scheduling currently conducted? 15 J62 J64 ● Change Management Meetings ● Ticketing System or Emails ● Static Calendars “We want to be patching more on our terms. Our frustration is we have no say in the matter” J62 Application Program Manager “I need to be a little gun-shy with updates, because I’ve gotten blowback from applications” J64 Windows Patch Technician
  16. Smart Maintenance Window Scheduler Patch 1 Patch 2 Patch 3 Application 1 x x Application 2 x x Application 3 x CCRI Exposur e Time Unremediated Patch 3 CRITICAL 2 5 Patch 2 MEDIUM 1 30 Patch 1 MEDIUM 2 10 1 2 3 2 Application 1&2 Patch 2 (Reason: Patch 2 has longer un-remediated time) 1 Application 2 Patch 3&1 (Patch 3 is CRITICAL, Optimal Time for Patch 1) 3 Application 3 Patch 3 (Reason: Optimal time for Patch 3) Click Here To Schedule w/ CM
  17. And found validation presenting our ideas 17 “I really like how you’re thinking about this from a logistics point of view...right now, we’re [patching] blindly” - DARPA Cyber Researcher “Determining when maintenance windows should be, now THAT sounds helpful” - Industry Cybersecurity Professional
  18. But our elation was short-lived…
  19. Reality Checks 19 10 9 8 7 6 5 4 3 2 Week 1 “I like some of your ideas, but it’s clear to me you have a vastly oversimplified understanding of this stuff” - CSO Cybersecurity Firm
  20. Mixed feedback... Refuting Validating “I’m not sure if it is possible.” - DLA Enterprise Infrastructure Director 20
  21. Mixed feedback... Refuting Validating “I’m not sure if it is possible.” - DLA Enterprise Infrastructure Director “Scheduling is definitely something that needs to be considered.” - DLA Enterprise Infrastructure Director 21
  22. Mixed feedback... Refuting Validating “I’m not sure if it is possible.” - DLA Enterprise Infrastructure Director “It needs to be optimized for the customer.” - DLA Enterprise Infrastructure Director “Scheduling is definitely something that needs to be considered.” - DLA Enterprise Infrastructure Director 22
  23. Mixed feedback... Refuting Validating “I’m not sure if it is possible.” - DLA Enterprise Infrastructure Director “It needs to be optimized for the customer.” - DLA Enterprise Infrastructure Director “Scheduling is definitely something that needs to be considered.” - DLA Enterprise Infrastructure Director “Maybe we need to be willing to accept impacts to customers and business to improve our security.” - DLA Enterprise Infrastructure Director 23
  24. Mixed feedback... Refuting Validating “I’m not sure if it is possible.” - DLA Enterprise Infrastructure Director “It needs to be optimized for the customer.” - DLA Enterprise Infrastructure Director “This is too simplified.” - CSO, Cybersecurity Vendor “Scheduling is definitely something that needs to be considered.” - DLA Enterprise Infrastructure Director “Maybe we need to be willing to accept impacts to customers and business to improve our security.” - DLA Enterprise Infrastructure Director 24
  25. Mixed feedback... Refuting Validating “I’m not sure if it is possible.” - DLA Enterprise Infrastructure Director “It needs to be optimized for the customer.” - DLA Enterprise Infrastructure Director “This is too simplified.” - CSO, Cybersecurity Vendor “Scheduling is definitely something that needs to be considered.” - DLA Enterprise Infrastructure Director “Maybe we need to be willing to accept impacts to customers and business to improve our security.” - DLA Enterprise Infrastructure Director “I like your ideas of algorithm recommendations, and patching more frequently is the right mindset.” - CSO, Cybersecurity Vendor 25
  26. We continued testing our MVP and receive mixed feedback... Refuting Validating “I’m not sure if it is possible.” - DLA Enterprise Infrastructure Director “It needs to be optimized for the customer.” - DLA Enterprise Infrastructure Director “This is too simplified.” - CSO, Cybersecurity Vendor “We don’t have enough changes for backlogs.” - Stanford ISO “Scheduling is definitely something that needs to be considered.” - DLA Enterprise Infrastructure Director “Maybe we need to be willing to accept impacts to customers and business to improve our security.” - DLA Enterprise Infrastructure Director “I like your ideas of algorithm recommendations, and patching more frequently is the right mindset.” - CSO, Cybersecurity Vendor 26
  27. Mixed feedback... Refuting Validating “I’m not sure if it is possible.” - DLA Enterprise Infrastructure Director “It needs to be optimized for the customer.” - DLA Enterprise Infrastructure Director “This is too simplified.” - CSO, Cybersecurity Vendor “We don’t have enough changes for backlogs.” - Stanford ISO “Scheduling is definitely something that needs to be considered.” - DLA Enterprise Infrastructure Director “Maybe we need to be willing to accept impacts to customers and business to improve our security.” - DLA Enterprise Infrastructure Director “I like your ideas of algorithm recommendations, and patching more frequently is the right mindset.” - CSO, Cybersecurity Vendor “We sometimes have large patch backlogs that are from patches not being implemented in previous months.” - Stanford VM 27
  28. And struggled to find a champion... 28 “I like your ideas, they seem very interesting!” DLA Chief of Application Support “It sounds interesting, and I’d love to help you in your research.” ...in other words… No... “Great! Would you be interested in writing a requirement for us?” DLA Chief of Application Support
  29. 29 “I think there’s an opportunity in the space you’re looking at, but it has to do with how you’re pitching it. It’s a really tough sell to ask decision makers to invest in security, which is a cost-sucker and not a value-driver” - University of San Diego Cyber Researcher Decision-makers need to be convinced that patching more frequently will BOTH minimally impact business AND tangibly improve security TO more efficiently allocate limited resources A key learning!
  30. 30 So we made an information sheet... Salus monitors the vulnerability state of your organization’s cyber assets and recommend more dynamic, smarter, and less disruptive maintenance windows 1) Decrease your risk exposure 2) Minimize impact to business operations 3) Allow for better allocation of limited IT resources.
  31. 31 “This sounds great, if you could prove to me that it’s feasible.” - DLA Deputy Director of Strategic Business Operations
  32. How can we do this? 32 10 9 8 7 6 5 4 3 2 Week 1 Focus on Key Activities, Partners, Deployment Options
  33. 33 Data Collection Model Simulation Academic and Risk Research We focused on activities that could prove our feasibility
  34. We searched for commercial proxies Multinational Non-Tech Companies Large, Decentralized Universities Agile companies who have modern tech stack, low technical debt and mostly built cybersecurity features within past 5 years 34
  35. Department of Defense Stanford + National Labs Enterprise Customers And weighed several possible routes to deployment... ITCR with DLA SBIR / AFWERX grant DoD Integrator SaaS vendor (ServiceNow, SAP) Proof-of-concept on Stanford network CRADA for data Research collab (LBNL, Sandia) Open market
  36. But finally learned that ServiceNOW is taking over the world! 36 “Your optimized scheduling idea could likely be implemented on the ServiceNOW development engine!” DLA Program Manager overseeing ServiceNOW implementation
  37. What did we learn? What’s next? 37 10 9 8 7 6 5 4 3 2 Week 1 Reflection and Summer Plans
  38. 38
  39. 39
  40. Final Recommendations for DLA 40 ● Implement ServiceNOW for Change Management with top- down emphasis ● Include ServiceNOW integration expectation in contracts with service providers ● Recognize that ServiceNOW does not provide insights into those tradeoffs with real data or risk analysis 1) Develop a NOW platform business application internally 2) Team Salus
  41. 41 Day 1: We broadly wanted to help analysts remediate cyber vulnerabilities. Day 70: We aim to help program managers, infrastructure owners, and change managers better schedule their maintenance downtime for patching. Main Lesson: We often mistook curiosity and interest as strong validation. We didn’t ask “Would you buy?” often enough. Key Takeaway: Patch management is a surprisingly time- consuming, error-prone process, and we’re confident there is significant room for improvement in the space. Our understanding of business impact requires additional legwork. Looking back, we learned a lot...
  42. Team Salus Noah Frick MBA, Strategy/Product Shreyas Parab BS Biocomputation, Product Kyla Guru BS Compsci/IR, Cyber Expert Henry Person MS MS&E, Industry Expert Michael Wornow PhD Compsci, AI Expert Improving cyber security by optimizing the vulnerability patching process Team Salus will continue to research and prove the feasibility of our optimization ideas and use of data and risk analysis in scheduling maintenance. If you or anyone you know would be interested in improving their organization’s cyber security posture, reach out to us at teamsalus.h4d@gmail.com

    Be the first to comment

  • MatthewKemph

    Jun. 3, 2021

business model, business model canvas, mission model, mission model canvas, customer development, hacking for defense, H4D, lean launchpad, lean startup, stanford, startup, steve blank, pete newell, bmnt, entrepreneurship, I-Corps, Security, NSIN, NSA, cybersecurity, Joe Felter

Views

Total views

12,103

On Slideshare

0

From embeds

0

Number of embeds

11,798

Actions

Downloads

5

Shares

0

Comments

0

Likes

1

×