Vendors, and Risk, and Tigers, and Bears, Oh My: How to Create a Vendor Review Process From the Ground Up
•Download as PPTX, PDF•
3 likes•243 views
ShmooCon 2020 You’ve just been tasked with creating a vendor review management process at your company, but what does that even mean, and how are you going to do this? Do you need to buy a lot of expensive GRC software and hire an army of compliance staffers? This talk will explain what a vendor review process is and walk through setting one up at your company, using nothing more complicated than email, text files, and maybe some Slack and Google Forms.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Vendors, and Risk, and Tigers, and Bears, Oh My: How to Create a Vendor Review Process From the Ground Up
Similar to Vendors, and Risk, and Tigers, and Bears, Oh My: How to Create a Vendor Review Process From the Ground Up (20)
More from Wendy Knox Everette
More from Wendy Knox Everette (11)
Recently uploaded
Recently uploaded (20)
Vendors, and Risk, and Tigers, and Bears, Oh My: How to Create a Vendor Review Process From the Ground Up
- 1. Vendors, and Risk, and Tigers, and Bears, Oh My: How to create a vendor review process from the ground up Wendy Knox Everette @wendyck ShmooCon 2020
- 2. Who am I? Wendy Knox Everette @wendyck Senior Security Advisor, Leviathan Security Group. I am a lawyer. I am very much not your lawyer.
- 3. What in the world is a vendor review? #WoCInTech
- 4. At a high level, this is the process of trying to ensure that partners we give trusted access or data to will take reasonable care of that access or data.
- 5. Why are these a thing now?
- 6. • aka tools hosted outside your network. SAAS…. aka tools hosted outside your network
- 7. And we’re generally more aware of security & privacy issues
- 8. Who is this talk for?
- 9. So what does it involve?
- 10. Let's Begin
- 11. What are we trying to accomplish? Ensuring that our company’s data and systems stay secure
- 12. What else are we trying to accomplish? Capturing a fuller picture of the risk we carry
- 14. Things we aren't going to talk about: Vulnerability management
- 15. Goals Short term • Set up a way to track the external services and tools we use • Do an initial risk triage Long term • Better understand and track the risk we take on from using third party tools and services
- 16. So how do I set up a vendor review program?
- 17. Understand the major steps 1. Intake 2. Gather information 3. Evaluate 4. Document the decision 5. Set up the accepted services 6. Iterate & Improve
- 18. 1. Intake
- 19. Getting buy-in
- 20. To start our review process…. We need employees to give us information
- 21. Who is using this tool and what for?
- 26. Talking to the vendor What are we asking for?
- 28. Advantages of asking for a standard questionnaire: many companies have this already filled out
- 29. Cloud Security Alliance Consensus Assessment Initiative Questionnaire - or “CAIQ” https://cloudsecurityalliance.org/
- 33. SOC 2 Type 1 and Type 2 SOC 2 reports assess a company on at least three of five “Trust Factors” • Security • Availability • Processing Integrity • Confidentiality • Privacy A Type 1 looks at a control set and asserts that the control set, if it operated, would fulfill the requirements. A Type 2 is over a time period (3 months, 6 months, 1 year) and asserts that the controls DID operate during that time period. https://www.aicpa.org/interestareas/frc/assuranceadvisoryservices/sorhome.html
- 34. Advantage of creating your own you can ask exactly about what you care about!
- 36. My top questions for a place that builds software 1. Could you briefly explain your SDLC and change management? 2. Do run any security tests on code as part of your deployment process? 3. Are there reviews or any approvals of the code committed by your SDE contractors before they go live? 4. Does your dev/staging environment hold live customer data?
- 37. My top questions – Access Control 1. Do you do access control reviews, particularly for access to any production/cloud environments or your source control repos?
- 38. My top questions – Monitoring/Incident Response 1. What sort of alerting and monitoring do you do, particularly around availability and security? 2. Do you have an Incident Response playbook or any plans?
- 39. Reaching out Sending the questionnaire to the vendor
- 40. Status Tracking
- 41. 3. Evaluate
- 42. What should we consider when deciding what information we’d need to evaluate a service? • what are its touchpoints • into our network or • on our website or • with our data?
- 43. Taking a step back: we should classify our data
- 44. Some data your company has is really sensitive: customer PII, your IP or special sauce, your employee’s payroll or HR data, PHI, etc.
- 45. Some data your company has is less sensitive: for instance, information that is also publicly available on your website
- 48. What’s the worst thing that could happen from an availability or security perspective?
- 49. How are we reviewing this material anyway?
- 50. Some ideas to guide the review process What threats to our company’s data are we concerned about? What business processes will depend on this new tool, and what happens if it goes down?
- 53. Attack can be completed with common skills Are there tools that automate the exploit, and if so, what tools are they? • A Metasploit plugin or similar likely means that it is relatively easy to perform the attack. • XSS or XSRF or SQLi would generally be considered easy to complete with common skills, although some chained exploits or particularly uncommon XSS attacks may be considered skilled attacks. If an attack normally classified as “YES” for this question is instead determined to require advanced skills, the risk assessor should document this in the Jira ticket. For example, “XSS attack requires novel technique not in common use” or similar.
- 54. Attack can be completed without significant resources • Resources should be interpreted as requiring the attacker to invest time and research into the attack. Must they acquire a particular type of account? • Can the attack be completed from anywhere on the Internet (for example exploiting an XSS flaw on an unauthenticated web page) or does it require authentication or a position on an internal network? • Does it require breaking into a physical data center or document storage facility? The investment of time and research to acquire this sort of access should be considered significant resources.
- 55. The Asset is undefended • Are there mitigating controls? These may be corrective controls (a VM that is re-set if it strays from a baseline); defense in depth (asset can be breached but the data is encrypted, and the key is not reachable) or similar protections. • A ransomware attack against an end user on a laptop would not be considered an attack against an undefended computer if there are protections in the email to flag suspicious emails; if the user must click past a warning to run Office macros; if the laptop is fully backed up, and so forth. • Are there detective controls such as logging and alerting in place which would trip on this attack? Are the people who receive these logs aware of the type of activity that would be consistent with an attack of this kind?
- 56. The vulnerability is always present in the asset • Is the vulnerability something inherent in the activity or asset, such as a business need to take in a large quantity of PII and store it? Or can it be eliminated, such as redacting or blurring PII? • Is the asset vulnerable only during certain time periods, such as during an intake process? • Is the item a legacy system that is lacking an upgrade path?
- 57. What am I worried about?
- 58. Are there mitigating controls we might want to put in place?
- 59. Do we need to follow up on any concerns?
- 60. who should be involved in our vendor review process?
- 61. Legal • you’ll need them to sign NDAs, and review service contracts
- 62. Signing NDAs
- 63. IT
- 65. Finance
- 69. Keep track of your vendor acceptances & rejections • Revisiting a previously rejected vendor • Compliance needs
- 70. Communicate status • The original requestor • Legal and other internal stakeholders • The accepted vendor
- 71. What can you automate?
- 72. Make a master list of all your onboarded vendors
- 73. 5. Setting up your service once it's accepted
- 74. Accounts
- 75. Contacts Do we have an enterprise sales rep? How do we report suspected security incidents?
- 76. SLAs • Are you tracking these? • How do you contact the vendor if one is exceeded?
- 77. 6. Iterate & Improve: Running a program
- 78. Iterate on your intake process
- 79. Stay organized • Review pipeline and individual request statuses • History of vendors reviewed, accepted, rejected • Updating risk registers • Who is the internal owner of the service? Do you know if they leave the company?
- 80. How is your questionnaire working for you?
- 81. Annual Reviews • For compliance reasons • To find un-used services or tools
- 82. Finding data creep • Would you find it if a department started sending more data types to a service than was originally approved?
- 83. Accounts on 3rd party tools and websites •Updating access – do your access control reviews and offboarding processes handle third party accounts? •What happens if the person with admin access on a third party leaves your company?
- 84. What process improvements can you make?
- 85. Congratulations, you can now manage your risk!
- 86. Thank you #WOCInTech for the photos https://www.flickr.com/photos/wocintechchat/page1
- 87. Resources • CAIQ: https://cloudsecurityalliance.org/artifacts/consensus-assessments- initiative-questionnaire-v3-0-1/ • VSA: https://www.vendorsecurityalliance.org/ • SIG: https://sharedassessments.org/sig/ • VSAQ: https://vsaq-demo.withgoogle.com/ • How to do vendor reviews, someone else's writeup: https://www.getkisi.com/blog/carrying-out-vendor-security-assessments • Dropbox: https://blogs.dropbox.com/tech/2019/03/towards-better- vendor-security-assessments/ Thank you! @wendyck
Editor's Notes
- You’ve just been tasked with creating a vendor review management process at your company! What in the world does that even involve?
- Vendor review management programs are designed to review tools and services used by an enterprise, with the goal of making an informed decision about whether the risk of using a particular tool is worth taking on. Third party risk can be a blind spot at some companies, and many organizations aren’t sure how to deal with it even if they are aware of it.
- Over the last few years, vendor security reviews have been moving from somewhat of a niche thing that only organizations in niche, regulated, areas do (I started doing them for financial firms) to something that more and more organizations are taking on TWO MIN
- One reason why third party risk has been getting more attention is the increasing number of SAAS tools. SAAS: software as a service your company’s data is no longer just in your server closet, and there are other organizations that you need to trust in order to run an enterprise now. Many tools are hosted elsewhere now, and so a lot of sensitive data leaves the networks that your company controls.
- Combine this with a general increase in the awareness of security best practices and some other trends, and all of a sudden we have a lot of security teams being asked to implement some form of vendor review.
- The audience for this talk is a security engineer at a smaller company who either has been getting worried about all the tools they see being used within their company, with access to lots of sensitive data, or one who has been asked to stand up a risk management and vendor review program to meet some compliance need. Really, this talk is for me, circa 2016, when I started doing these and couldn't find anything about third party risk management that made sense to me.
- Do you need to buy a lot of expensive GRC software and hire an army of compliance staffers? No. GRC --> Governance Risk Compliance There’s not a lot out there about the DIY approach and breaking things down for understaffed teams.
- 5 min I’ve tried to structure this talk so that it would give a lot of operational takeaways that will help practitioners. So instead of recommending a lot of expensive GRC software, this talk tries to focus on tools-agnostic approaches and simple solutions that teams likely already have access to, like shared drives and Slack. I’m also going to talk about how to cooperate with other teams at your company to get this program off the ground and running.
- Hopefully: ensuring that our company’s data and systems stay secure. Maybe we also need to generate compliance artifacts- do we need to show that we’ve done these reviews for a SOC II, HITRUST, or other audit?
- Vendor reviews are a big part of capturing and cataloging the third party risk taken on by an enterprise. Cataloging who has our sensitive data or who has privileged access in our systems helps give a fuller picture of the overall risk our organization carries.
- The third party libraries that your development teams depend on could be seen as part of your overall vendor review process. However, that introduces a lot of complexity, and is more often own directly by the development teams, so I’m going to set that aside. Just note that the libraries that they use should be reviewed, and you’ll want to make sure you’re regularly patching them.
- Instead, we’re going to focus on three goals with this talk. First, let’s figure out a way to do a security review of external tools & services that our company relies on. Next, let’s do an initial risk triage of all the vendors that we’re already using. Long term, we’ll want to set up a program to track and assess our overall third party risk profile.
- 7. min We’ll begin with the idea that we need a way to assess third party risk when an employee comes to us and says “Hey, I’d like to use this cool new service!” – how do we start?
- There are six big steps to our review process, and we’ll go through them one by one and explore what they’ll entail.
- First –intake. This is the process by which you have employees tell you about new services they will want to evaluate. When you start, this might be more of a discovery process, as you'll have to decide if you're going to evaluate all the vendors and tools you already have. We’ll go through this as if we have a single new service to review.
- Before we getting into the specifics of running the program, remember that we need buy-in from management, developers, marketing teams: users of the systems that we want to review. Without their cooperation and active interest, you’re going to be shuffling pointless paperwork. We need some way to partner with them, without being seen as “the security team always says no” or “they’re just a roadblock to getting work done.” The way we overcome this is to focus on being a partner organization, remembering that our co-workers are subject matter experts in their domains, and communicating business risk, not specific vulns or hacks.
- Usually we’re going to need the employee to kick off this process, by letting us know what service they want to begin using.
- 10 min? From our perspective, we have a few things we need to learn from them. For instance - What do they expect to use this for, and what are the failure cases? How does the service interact with our accounts or customers? One way that we can run this intake process is with a web form that we can direct users to. We’ll go through an example one to see what it might look like.
- * What does your project do? * How will the product be used? * Does the vendor have a TOS? * Does the vendor have a privacy policy?
- * What service are you planning to use? * When do you hope to have this in place? * URL to product * What alternatives did you look at * Why is this one best? * What will it cost?
- * What classes of information will it process? * Does it require a service account connection to other systems? * Does it support SAML/Single Sign On?
- 13 min? Now that we have information from the employee, we need to pivot to getting some information about the security and privacy of the tool or service. Here you'll reach out to the vendor for that information.
- Do your users already have a sales contact? Many time they will come to you having already opened a dialog with the vendor. If so, it’s very helpful to leverage that channel when you talk to the vendor. We know we want to ask “how secure is your service?” – but how do we get the information we need to make that assessment?
- Security Questionnaires are the primary way people gather information about vendors under consideration. They're a set of privacy and security questions for the company being considered to fill out. There are several standard ones in the industry , and we’ll go through a few of them first.
- The standard ones are very comprehensive, and it can be quick turn around since many larger orgs have these available to send to you right away.
- The Cloud Security Alliance’s Consensus Assessment Initiative Questionnaire - it has a little something for everybody: A little policy, a little governance, some infrastructure information and basic control responses. the CAIQ, mapped to many of the most popular (NIST, ISO, FedRAMP, ENISA, etc.) works for many customer industry verticals . For many companies, if you aren't subscribing to their enterprise level, the most they will give you is a standard questionnaire, like the CAIQ
- Google VSAQ: Created by Google, open source (hosted on GitHub), and it's basically an essay question about your systems and your network. These kind of feel like you’re back in school where you’re not quite sure how much you should write.
- Vendor Security Alliance Questionnaire: Confusingly, also called VSAQ, because fuck it, I guess. Created by lots of well-meaning companies, including Uber and Palantir, and---I swear to god---marketed in conjunction with a company that will collect your VSAQ responses AND MARKET BASED ON YOUR CONFIDENTIAL INFORMATION. I made the mistake of letting browser autofill put in my work number for this once and got months of phone calls from them.
- Standardized Information Gathering (SIG) is a set of questionnaires- light or medium. All of them are awful, but they tend to be used by people who are over 50. First expose to vendor security Qs, doing this for finance firms
- This isn’t a questionnaire, but you’ll often see third parties offer you a SOC report. SOC stands for “System and Organization” controls, and a SOC 2 reports assess a company on at least three of five “Trust Factors” Security Availability Processing Integrity Confidentiality Privacy There are two type of SOC 2 reports, Type 1 and Type 2. One looks at a point in time, and the other looks at how controls operated over the audit period.
- You can also create your own questionnaire. But note that if you’re not subscribing to the enterprise tier of a service, the vendor may not be willing to fill out your questionnaire. Or they may not have staff who are used to filling out these questionnaires and aren’t sure what to answer. Also note, that often these are filled in by sales people, so you should think about how you word your questions – could a non-technical person successfully look up the answer, or understand the context? Sometimes they will offer to answer just a few questions for you, if you ask.
- 18 min? This is an example of a vendor security questionnaire that focuses on the internal audit controls at a vendor
- If we’re looking at a SAAS platform that is built by a smaller company, these are some of my smell test questions that try to get a handle on how mature they are and how much risk we’d be taking on by entrusting some of our data to this organization.
- Asking about access control reviews is important if they’ll have access to sensitive data, such as customer data or personal information of your employees.
- I like to ask about their monitoring & incident response programs to get a handle on how likely I think it is that they might even notice if they lose our data.
- Now that we have some idea of the information we want to gather, we have to go get it. Often having the requestor reach out initially is helpful, to let them know that we’ll be asking some questions about the service. Have a template that you fill in to send this out, it makes things go much faster.
- Track where you are with each vendor so that you don’t waste time re-creating steps or forget to follow up: either through a Slack channel, a whiteboard list, or some other way. Many of these reviews take a lot of going back-and-forth initially to get the material you need (see word salad from sales people in vendor Q forms)
- 20 min? Now you have to analyze all the information you gathered. Here’s where we’ll do the risk assessment part of this process.
- Does the service run javascript on our company website? Do you send it data? Does it have access to a DB that you operate?
- Do you have a data classification policy already? Does your legal department? Remember when we said that we’d ask about access control reviews and practices at the vendor if we’re sending highly confidential data like customer data or personal information of our employees to the vendor? We should have some agreed-upon definition internally of what our sensitive data types are. This helps us understand what level of risk we should assign to the vendor, which informs the level of scrutiny we’re going to apply during our vendor review.
- Some of the data our company has are highly confidential, like our source code, or employee Social Security Numbers, or customer content we might store. Maybe we hold health data (PHI).
- Publicly available information may be less sensitive regarding disclosure, but we may be granting the tool the ability to speak on our behalf
- https://skift.com/2019/08/19/delta-sues-chatbot-vendor-faulted-for-data-breach
- This is a vendor that doesn’t have good monitoring and detection.
- The big question we can keep asking ourselves is, what's the worst thing that could happen if this tool has the access it wants, or if this vendor loses the type of data we want to give them?
- 25 minutes There should be a documented process that you use to assess each vendor, based on the initial level of risk that we think we’d be taking on based on the data we’ll send to them, what level of access they have, and where the touchpoints to our company. For instance, if you’ll send customer data to a vendor, you may require that they have a written IR plan and run Incident Response exercises, and that they be able to explain what sort of monitoring for data exfiltration they have in place. You might get that information from a security questionnaire they filled out, or by looking for certain controls in their SOC 2 report.
- We want to think about how to get a handle on the risk that this vendor introduces. You can use a variety of tools to do this; one that we’ve found can help guide conversations well is Binary Risk Assessment.
- There’s a white paper on this website that explains the whole process, and also a little web app that can guide you through an assessment.
- This is the binary workcard on the website. As you say yes/no, it updates the risk/likelihood/impact. It can be hard to come to agreement on the yes/no for some of these answers, so sometimes we like to come up with "guard rails" or assumptions to guide us
- These are some sample guidelines we’ve created for filling out the Binary Risk Assessment that help keep everyone on the same page. Common skills/metaspolit plugins
- Significant investment of resources - is this drive-by exploitation/mass scanning?
- What mitigating controls are in place?
- Is the asset always vulnerable? 28 minutes?
- think about what your biggest worries are with this service, and make sure that you have enough information here to make a reasonably informed decision. Is this a javascript library we’re putting on our homepage? Let’s ask about change management controls so make sure they don’t break our homepage. Is this a payroll tool that gives their employees access to our employees bank account numbers? We should ask about their background check process.
- Now is a great time to document them. For instance, we might find that a service poses a high risk to our enterprise – it has to hold very sensitive data, but they seem to be immature and not take security seriously. However, our business needs to partner with this company for a critical business purpose. Once we’ve communicated the risk to the stakeholders and company management, and they’ve accepted this risk, we should think about what compensating controls we can put in place. Is there any form of monitoring we can institute that catch a some malicious activity or data leakage as soon as it happens?
- Some companies will get into endless loops of asking for follow ups to their questionnaires. And it can be really frustrating to send out your questions and get back vague, hand-wavy non-answers. If you’re in a highly regulated industry, you may have an obligation to dig into a lot of the details of how your data is being stored or processed.
- 30 min? You’ll want to coordinate with some other departments within your company in order to run a successful review process.
- The Legal team is tasked with reviewing contracts like NDAs, or Master Service Agreements. Often they will also want to do a Privacy review, or make sure a Data Protection Agreement is signed.
- set up a process with your legal team to review and sign these, since most vendors won’t give you a SOC II report without one in place. What to think about: how much time do they need to review? Can they authorize you to sign some of them?
- IT will usually need to onboard the services you approve, and they may be the ones to integrate a tool into your Single Sign On.
- Many times we see sales and marketing teams coming to us with the most asks for new tools, so it will make sense to reach out to them early and get them onboard with your process. They’re interested in protecting the company’s reputation, so you share many of the same end goals.
- Finance will need to set up payments for services, and they may want to review service contracts.
- They can also be a great team to partner with as you launch your program. Finance can also alert you to new charges they see, such as on corporate cards, which might mean a new vendor or service to review. You should figure out a way to communicate approved services to them, so that they can alert you if they see charges for unapproved services or tools.
- While we aren’t going to dive into the use of third party libraries by development teams, engineering teams do often use other third party tools you’ll want to review. Some teams use code review tools, or github plugins. They may want to onboard an AB test tool, or some performance analysis tools. These will run on your website, but you may or may not end up hosting the code.
- 33 min? Especially if you were asked to do this process for a SOC 2 or similar audit: document everything that you do. Take meeting notes, or use slack channels you can archive. Or create Jira tickets; whatever works for your company's culture.
- Create some form of a tracking system that can help you list approved or rejected vendors. Ideally it will also have a way to help you track in-process reviews, especially if you have to wait for legal or other external sign offs, or if some vendors are slow in returning the security questionnaire you sent out. This is tracking can be done with an excel spreadsheet, or jira, or within Slack. If you’re doing this for a SOC audit, you’ll want to be able to show that you considered the security risk of any vendor you onboarded.
- There are a lot of stakeholders who will want to know where the review process is. Especially when you first launch the program, the initial requestor may be expecting to begin using a new tool quickly, and may be very impatient while we perform the security risk assessment.
- To help track status and communicate it to stake holders, think about what sort of tools you might be able to leverage. This is an example of aa slackbot that we use at a company to track our signoffs. If you invoke the slackbot in the channel, it will tell you what signoffs its received already, and we can tell it which ones we’re submitting.
- As you vet the third parties, make a master list of all the approved and onboarded vendors, as well as the internal company owner. You may want to also track your internal risk rating of them, or some other metric that alerts you to what sort of data they hold or how key they are to the company’s operations. 34 min?
- 35 min? After the approval, there are some onboarding steps that may largely be owned by IT or other groups
- do they use SSO? Who will be an admin? And all kinds of other fun account management things like required password rotations, account recovery, etc.
- The third party contacts can go into your master spreadsheet of vendors, along side the internal company owner. Sometimes there are different security and availability contacts at the company, and youll want to be able to track that distinction. Otherplaces may give you a single account rep, and you would usually work with them on anything that comes up.
- Availability might be owned by another group at your company, but many times it will fall to you to be the point person. Do you have any visibility into the availability and performance of these services? How would you be alerted if defined SLAs are exceeded? What do you do then?
- 40 min After a few iterations, you'll get the hang of it. So this section is some tips that we've picked up from running these programs at small companies.
- this can be key to getting buy-in from the rest of the company. Would you like the process started by slack? Can you create a Google Form for your co-workers to fill out with their new requests?
- Update your tracking spreadsheet - you should have one Source of Truth that has all your vendors/services/tools/other 3rd party integrations. How you structure this is up to you, because it should make sense to your team.
- Reviewing the questions you ask: if you made your own questionnaire, or if you have a checklist of the items you look for in a CAIQ or a SOC II report, don’t just let it stagnate. Learn from past issues, from news reports, etc
- Do users skip your intake form because it’s too unwieldy and asks for information they don’t have yet? Do deadlines for contract renewals catch you unaware and cause scrambles? Both of these are examples of pain points that should point towards places where we can try to improve our processes.