Open Source Compliance Automation Capability Map

Capability Map
OC Tooling Reference Workgroup - v1.5.5
V1.5.5 by Open Chain Tooling Workgroup, July, 20th 2022
v1.5.4 by Open Chain Tooling Workgroup, July, 6th 2022
v1.5.3 by Open Chain Tooling Workgroup, June, 22nd 2022
v1.5.2 by Open Chain Tooling Workgroup, June, 8th 2022
v1.5.0 by Open Chain Tooling Workgroup, May 11th 2022
v1.4.0 by Open Chain Tooling Workgroup, 30.3.22
v1.3.2 by Dr. Peter Ellsiepen (ESA) & Jan Thielscher (TrustSource)

Licensed under CC-BY-SA-4.0
Open Source Compliance Capability Model (v1.5.5)
Changelog
Version Date by Comments/Changes
1.2 3.12.19 Jan, Peter Initial draft
1.3 6.12.19 Jan Rename Case Data => Situation Data, delete „Compliance Artefacts“ as capability, change Mission of Snippet scanner
1.3.1 11.1.21 Jan Review spelling, add some Readme‘s in the surrounding, review & harmonize definitions
1.3.2. 11.1.21 Jan Added a few samples for capability mapping
1.4.0 30.3.22 Tooling WG
Reviewed Capabilities Package Crawler, Scanners (Binary, Source and Container) as well as License & Copyright Scanner,
added CI/CD rule enforcement
1.4.1 13.4.22 Tooling WG Reviewed changes, extended Snippet-Scanning,
1.5.0 11.5.22 Tooling WG Split Case Data into Case Data Analyzer & Collector Capabilities, re-arranged overview slide
1.5.2 8.6.22 Tooling WG Reviewed Legal Solver, Policies & Rules, 3rd party component data
1.5.3 22.6.22 Tooling WG Reviewed License Repositiry, Compliance Artefact Generator and Approval flow
1.5.4 6.7.22 Tooling WG Reviewed User & Role Management, Audit Log, started with Reporting & Analytics
1.5.5 20.7.22 Tooling WG Finalized Reporting & Analytics and reviewed Tool Orchestrator
PLEASE NOTE:
To keep an overview of working state, we mark the agreed capabilities with this symbol
2

Traceability of data sources, decisions and configs as a General Requirment
We need to provide the general requirement, that all decisions, data and sources need to be tracible, so that it always is possible to track
why and on what basis a decision has been made. This involves:
• Provide all information available under which a certain decision is made and that point in time
• Track changes and their originators
• Archive sources / binaries that are used in a solution
• Link notice files and other documentation with sources/binaries
• Document decisions and choices made
3

ToolChain Capabilities - Overview
4
Tool Orchestrator
Reporting and Analytics
Case Data Collector (Situation, Inputs, Status)
Policies & Rules
Approval Flow
(WFE)
Compliance
Artefact Generator
Snippet &
Similarity Scanner
(forensics)
License, Copyright
& Authors Scanner
License Repository
(license facts,
rights obligations)
Package Crawler
Compliance
Artefacts
Legal Solver
(determine
obligations)
COTS Management
User & Role Management
Package Source
Archive
Audit Log
Package Metadata
Repository
Data Flow Data Sink
CI/CD OSG Rule
Enforcement
EXCLUSION:
At this point in time
the model is not
addressing Security
or Export regulations
Dependency Analyzer
Source Container Binary
Input Condition
Management
Case Data Data Analysis
1
2 3
17
18
19
4
5
16
15
14
6
8
9
10
7
11
12
20
13
Control Flow

ToolChain Capabilities - Package Crawler/Finder
Mission
• Research information on (new) components such as locate the repository, current and
former versions, project homepage and viability information
Responsibilities
• Collect and provide accurate information about the component
• Alert, if component can’t be matched/found
Tasks
• Scan package managers for new packages or versions of packages
• Collect package data
• Transfer data into package repository
Input • Component descriptor or component name
Output
• Component Information, such as: source repository url, version history, branches, commit
count, stars, last commit date, etc.
Comments
=> Distinguish between component loader & assessment or just cralwer for information
5

ToolChain Capabilities - Dependency Analyzer (Source)
Mission
• Provide composition analysis of software to be built from these sources
Responsibilities
• Determine all packages and dependencies (incl. transitive) used to build the software
• Determine the way of linking of dependencies
Tasks
• Integrate with build process (CI/CD)
• Determine composition (_complete_ Bill of Materials)
• Provide output for further analysis, e.g. as SPDX
• Provide link between scanned source and BoM information, e.g. Commit ID
Input • Build description, e.g. POM or requirements.txt
Output • Bill of Materials (BoM) for particular build
Comments
Analysis and dependency resolution is highly language specific. Thus a language specific
implementation might be required
Discussion: Would it make sense to declare a task or responsibility to stop CI/CD in sit of
violation?
6

ToolChain Capabilities - Dependency Analyzer (Binary)
Mission
• Provide composition analysis of a software binary
Responsibilities • Determine all packages and dependencies used within this binary
Tasks
• Download binary (if required)
• Unpack binary
• Assess content and determine used packages/components
• Collect information and assemble Bill of Materials
• Provide Bill of Materials (e.g. as SPDX)
• Provide link between BoM and scanned artefact, e.g. binary repo ID
• Hash to identify the binary scanned should be generated and archived
Input • Binary or link to binary location
Output
• Bill of Materials (BoM) for particular binary
• Status of processing (e.g. errors, inclompleteness, failures in processing)
Comments
7

ToolChain Capabilities - Depdendency Analyzer (Container)
Mission • Provide composition analysis of a container
Responsibilities
• Determine all packages and dependencies used within this container
Tasks
• Download container (if necessary)
• Assess container content/structure and determine used packages/components
• Collect information and assemble Bill of Materials
• Provide Bill of Materials (e.g. as SPDX)
• Provide link between BoM and scanned container, e.g. Repo + image ID + tag
▪ Hash to identify the scanned container should be generated and archived
Input • Container or link to container location
Output
• Bill of Materials (BoM) for particular container
Comments
8

ToolChain Capabilities - License, Copyright & Authors Scanner
Mission
• Precise scanning of sources to determine exact situation for proper compliance
declarations
Responsibility • Ensure completeness and correctness of compliance information
Tasks
• Identify & gather copyright statements
• Identify & gather authors
• Identify & gather effective licenses (e.g. license identifier & if available license text)
• Identify & gather changes and / or additions to license terms
Input • Repository or file(s) to scan
Output
• List of effective and declared licenses with links into code
• List of changed licenses with links into code
• List of copyright statements with links into code
• List of author information with links into code
Comments
• TODO: Clarify granularity required to differentiate between author, commiter and
copyright holder
9

ToolChain Capabilities – (CI/CD) OSG Rule Enforcement
Mission • Ensure only compliant artifacts will leave the automated tool chain
Responsibilities • Break build, deployment or packaging as long as compliance violations exist
Tasks
• Verify compliance state
• Interrupt automated build/deployment processing in case of violations
• Log event and causes
• Alert
Input • Automation event
Output
• „Confirmation“ or „break“ event – or any sort of recording of required action
• Log entry
Comments
• The key of this is to ensure that no non-compliant artifact will leave the process. It must
not be CI/CD driven, but it should ensure that a check happens
1
0
OSG = Open Source Governance
Data Flow
Data Flow

ToolChain Capabilities – Input Condition Management
Mission
• Determine that all copyright holders of commits finally grant rights and will not claim
back
Responsibilities
• Prevent code from entering the repository without the commiter having agreed to the
terms seeked by repo-owner
Tasks
• Link confirmation into Pull-request
• Provide sort of proof that code commited to repo went through this process
• Log event and confirmations of commiters
Input • Automation event
Output
• „Confirmation“ or „break“ event
• Log entry
Comments
• One option could be to apply CLA-Assistant by SAP
1
1

ToolChain Capabilities - Snippet & Similarity Scanner
Mission • Identify pieces of original code (source, object, binary) by comparing against known codebase
Responsibility
• Ensure code is free from copyright infringements due to copying routines or third party code
• Discover re-use of code
• Determine modification of identified code
Tasks
• Scan files for copies
• Scan sources for known snippets
• Provide scan results including references to copies/identified origin (e.g. earliest known appearance)
Input
• Repository or file(s) to scan
• Comparison basis (known data sets)
Output
• List of potential infringements with links to potential matches (e.g. in existing OSS)
• Weighting/ordering of potential matches
Comments
• Snippet Scanning (e.g. plagiarism check), similarity scanning (rough check) and delta analysis (identify
change) serve different purposes
• While similarity analysis gives indication that something might require further analysis, Snippet
scanning delivers proof of re-use
• Similarity analysis also allows delta analysis to be performed
12

ToolChain Capabilities - Package Metadata Repository
Mission • Collect package information and clearing metadata on packages
Responsibility • Single point of truth for package information
Tasks
• Store package metadata and quality verification status (of that metadata concenring
completeness and correctness)
• Support composition analysis (verification of dependency analysis)
• Provide search capabilities to identify existing packages
• Support authentication/authorization to ensure responsible data handling/editing
Input
• Package identifier (e.g. purl) + already identified metadata
• Package metadata
Output
• Package metadata, including package type (e.g. OSS, COTS, internal) and completion/
verification status of associated metadata
• Containment structures (consists of)
• Dependency structures (depends on)
• Optional: relate known vulnerability information (not OSC specific, but a good place)
Comments
• Archive should be provided by archive capability. Tools supporting both functions in one
are not limited by the capabilities beeing separate.
1
3

ToolChain Capabilities - Case Data Collector
Mission
• Provide bracket for all compliance relevant information that is not directly related to source of a
product / distribution item
Responsibility • Ensure completeness of case documentation
Tasks
• Collect all product specific information, including package change & linkage status
(via history)
• Follow the release cycle of a particular product, e.g. approvals
• Build canvas for reporting and analysis of a given composition & in a given situation
• Versioning of analysis results to map with input situations
Input
• Business context (business model, distribution, external contractual obligations, etc.)
• Software Bill of Materials (SBOM) + Component meta data (see Package Metadata Repo)
• External components, e.g. runtime environments, middleware or resources (as part of solution)
• Type of delivery/distribution (binary, source (oss), source (proprietary & oss), source (proprietary, oss ,
COTS and combinations of these)
• Participants / Stakeholders (audience)
• Approval Feedback
Output
• Status Overview
• History of events and changes to context and meta data
Comments
1
4

ToolChain Capabilities – Case Data Analyzer
Mission • Interpret all collected case data in given context and determine deltas
Responsibility • Identify obligations, violations and warnings
Tasks
• Check for completeness of information
• Identify missing information (e.g. missing Copyright information)
• Determine rights and obligations, compare with requirements from business context
Input
• Case Data (see 13. ToolChain Capabilities - Case Data (Structure of Solution...)
• Policy & Rules
• Legal interpretation
Output • Analysis result for further processing
Comments
• Review after re-draw of model
15

ToolChain Capabilities - Policies & Rules
Mission • Capturing the Organisation specific interpretation of its obligations, objectives & goals
Responsibility • Represent the rules derived from organisations legal understanding
Tasks
• Rules how to treat specific legal circumstances, e.g. commercial aspects, trade secrets or IP protection
requirements, etc.
• Translate human readable policies to machine readable instructions/rules
(as input input for analysis)
• Document / Track changes in project specific allow- lists or deny-lists (licenses, components,
frameworks, etc.)
• Allow managing groups of projects with consistent policies & rules
• Optional: Store open source policy for reference
Input
• Legal requirements for particular application scenarios
• Definition allow- and deny-lists
• Project specific rules and policies (e.g. versions, OpenSSF Score, specific components, viability, etc.)
Output • History of changes
Comments
16

ToolChain Capabilities – Management of 3rd party provided Components
Mission • Manage Commercial-Off-The-Shelf (COTS) and infrastructure (open source or COTS) packages of a
solution
Responsibility
• Allow tracking 3rd party components concerning vulnerability and compliance
• Collect and provide meta data for 3rd party or infrastructure packages
Tasks
• Store package metadata or 3rd party components and quality verification status (of that metadata
concenring completeness and correctness)
• Store information about 3rd party/private commercial conditions (license information)
• Allow to assemble reports like SOUP-lists
• Optional: Review 3rd party assemblies for known vulnerabilities
Input
• Package data and metadata (if known)
• Binary scan information (BoM)
Output
• Package data and metadata (updated)
• License information about 3rd party components
Comments
• PLEASE NOTE: For full compliance a storage for 3rd party sources/binaries should be available and
referenceable
• PLEASE NOTE: Commercial Licenses may have different aspects involved like termination by time /
renewable
• SOUP lists will require additional meta information, which is not in the scope of open source
components
17

ToolChain Capabilities - Legal Solver
Mission
• Determine legal rights and obligations resulting from the usage of the listed packages within the
project context
Responsibility
• Provide compliance requirements: obligations and violations (missing rights)
• Verify license compatibility under given circumstances
Tasks
• Assess license information from all packages (recent BoMs, infrastructure and 3rd party) and
circumstances of use (business model, licensing amibition, IP protection requirements)
• Determine license obligations and potential violations
Input
• Composition analysis of all project related packages, their status (binding and modification status),
and licenses
• Legal circumstances and requirements of the project
Output
• List of legal obligations and missing rights (if) by package and mitigation hints
• Information on license in-compatibility (yes, no, why?)
Comments
• Independent from package status the analysis results may vary depending on changes in the
circumstances. Thus analysis results should be versioned to allow allocation to related circumstances.
• How to handle jurisdiction specific decisions? Would this be the place to put the information?
18

ToolChain Capabilities - License Repository
Mission • Capture and archive legal information & interpretation about licenses
Responsibility • Manage and provide legal information about known licenses
Tasks
• Capture & Update all license information including derived requirements and exceptions
• Provide reference for original license texts
• Provide environment to allow license analysis
• Track changes in license interpretation
• Manage classification and tagging
Input • License data + interpretations
Output • License data (updated) machine readable format
Comments
• Could be combined with legal solver, but we decided to provide as separate capability.
A solver requires the repository, but the solver also could be a human worker.
• How to represent different jurisdictions (e.g. case law UK / US)?
=> probably overdone, stay with most restrictive interpretation to prevent failure
19

ToolChain Capabilities - Compliance Artefact Generator
Mission • Support provisioning of compliance documentation
Responsibility • Ensure legally compliant documentation
Tasks
• Generate documentation according to requirements
• Support Compliance Managers in completing the documentation
• Assemble documentation parts, e.g. written offer, license texts, copyrights, modification
statement, etc.
• Link documentation with objects (version management / binary links)
• Provide documentation in machine readable export formats, e.g. JSON, SPDX, CyDX,
etc.
Input
• List of versioned packages to be documented (BoMs) and their meta data
• Legal requirements with respect to particular circumstances
Output
• Stub with all documentation requirements
• Pre-assembled stub with all existing information (e.g. from repositories)
• Identified TODOs for missing bits
Comments
20

ToolChain Capabilities - Approval Flow
Mission • Ensure that the outgoing documentation fits the purpose
Responsibility • Provide approval flow appropriate for audit
Tasks
• Track all legally relevant changes to products and packages
• Identify authors of change
• Provide compliance status and overview
• Allow to approve or reject an approval request
• Document/archive all decisions (auditing)
• Support for different roles / instances of approval flows
Input • Artifacts to be approved and approval type (e.g. security, compliance, etc.)
Output
• State of compliance analysis for approval request
• Approval / Rejection documentation
Comments
• The approval by a dedicated, skilled resource (Compliance Manager) combined with the
automation support for all prior steps reduces the need for Compliance Managers
• Could be used for other objects, e.g. completeness of list of packages, etc.
21

ToolChain Capabilities - User & Role Management
Mission • Provide role based authorization
Responsibility
• Authenticate users
• Manage and/or map roles and authorizations
• Assign users to roles
Tasks
• Identify users (Login, oAuth, MFA)
• Manage roles and related authorizations (permissions assigned to roles)
• Manage programmatical access (e.g. API keys)
Input
• Users
• Roles
Output • Authenticated user and associated roles (e.g. via access token)
Comments • Agreement that these „infrastructural capabilities“ should be added and described
22
TODO: Provide support for infrastructural services to other capabilities

ToolChain Capabilities - Audit Log
Mission • Maintain log of changes and user actions (create accountability)
Responsibility
• Ensure traceability of configuration changes
• Ensure tracing and archiving of all user actions/decisions for auditing purposes
Tasks
• Track user activity and changes in settings, especially legal settings
• Track and archive user decisions and related context to enable auditing
• Confirmation of completeness (e.g. by project owner)
• Derive configuration status at a certain point in history
Input • User actions / events
Output
• History of changes with actors
• History of changes, configurations and decisions that lead to a particular compliance
artefact (e.g. version number of scanner, scan config, etc.)
Comments
23

ToolChain Capabilities - Reporting & Analytics
Mission • Visualize current work status, todos, efforts spent and success of compliance initiative
Responsibility
• Provide insights into state of portfolio
• Create overview of workload and help to assign priorities
• Measure compliance related activity
Tasks
• Collect data from different capabilities to allow reporting
• Report design
Input • Report specific data required
Output
• Reports (human AND machine readable format)
• Transparency
Comments
• Specific reports should be defined on org level
• See Todo Group for potential KPI ideas , e.g. scans/period, num of products scanned,
number of issues found , etc.
24

ToolChain Capabilities - Tool Orchestrator
Mission • Co-ordinate overall compliance workflow(s)
Responsibility
• Arrange combination of tools to cope with compliance challenge
• Handle handover between capabilities
Tasks • Trigger events
Input • Events
Output
• Events
Comments
• Depending on the degree of process automation the orchestrator may be a combination
of event driven rule engine or a ticket system
25

Open Questions for further discussions
1. How to capture policies & rules in a form that allows automation/repetition? (from Rules & polices)
• What constitutes a policy? = document (statement of intent, limits, ownership…)
• What makes a rule ? Allow / Deny a User or Group to execute an action
2. Defined list of use cases that should be covered (check at Todo Group)
i. Product/Solution compliance (create the output)
ii. Handling an inquiry (internal/external)
iii. Running an audit
iv. Maintain / update compliance documentation
v. Finding specific components across the portfolio
vi. Pre-analysis of potentially useful components (or contributions)
vii. Verifying 3rd party components (COTS)
viii. Showing progress in compliance (visualizing metrics)
ix. Maintain proper functionality of tooling chain
x. Update license list / interpretation & handling consequences of it
26

Tool Orchestrator
ToolChain Capabilities (v1.3.1) – Mapping of Tools (example BANG)
Dependency
Analyzer (Source)
Dependency
Analyzer (Binary)
Dependency
Analyzer
(Container)
Case Data (Situation, Inputs, Status)
Policies & Rules
Approval Flow
(WFE)
Compliance
Artefact Generator
Snippet Scanner
(forensics)
License, Copyright
& Authors Scanner
License Repository
(license facts, rights
obligations)
Package Crawler
Compliance
Artefacts
Legal Solver
(determine
obligations)
COTS Management
Package Source
Archive
Audit Log
Package Data
Repository
2
7
Data Flow Data Sink
BANG

Tool Orchestrator
ToolChain Capabilities (v1.3.1) – Mapping of Tools (example Software Heritage)
Dependency
Analyzer (Source)
Dependency
Analyzer (Binary)
Dependency
Analyzer
(Container)
Policies & Rules
Approval Flow
(WFE)
Compliance
Artefact Generator
Snippet Scanner
(forensics)
License, Copyright
& Authors Scanner
License Repository
obligations)
Package Crawler
Compliance
Artefacts
Legal Solver
(determine
obligations)
COTS Management
Package Source
Archive
Audit Log
Package Data
Repository
2
8
Data Flow Data Sink

Tool Orchestrator
ToolChain Capabilities (v1.3.1) – Mapping of Tools (example TERN)
Dependency
Analyzer (Source)
Dependency
Analyzer (Binary)
Dependency
Analyzer
(Container)
Policies & Rules
Approval Flow
(WFE)
Compliance
Artefact Generator
Snippet Scanner
(forensics)
License, Copyright
& Authors Scanner
License Repository
obligations)
Package Crawler
Compliance
Artefacts
Legal Solver
(determine
obligations)
COTS Management
Package Source
Archive
Audit Log
Package Data
Repository
2
9
Data Flow Data Sink

Tool Orchestrator
ToolChain Capabilities (v1.3.1) – Mapping of Tools (example ClearlyDefined)
Dependency
Analyzer (Source)
Dependency
Analyzer (Binary)
Policies & Rules
Approval Flow
(WFE)
Compliance
Artefact Generator
Snippet Scanner
(forensics)
License, Copyright
& Authors Scanner
License Repository
obligations)
Package Crawler
Compliance
Artefacts
Legal Solver
(determine
obligations)
COTS Management
Package Source
Archive
Audit Log
Package Data
Repository
3
0
Data Flow Data Sink
Dependency
Analyzer
(Container)

Tool Orchestrator
ToolChain Capabilities (v1.3.1) – Mapping of Tools (example TrustSource Scanners)
Dependency
Analyzer (Source)
Dependency
Analyzer (Binary)
Dependency
Analyzer
(Container)
Policies & Rules
Approval Flow
(WFE)
Compliance
Artefact Generator
Snippet Scanner
(forensics)
License, Copyright
& Authors Scanner
License Repository
obligations)
Package Crawler
Compliance
Artefacts
Legal Solver
(determine
obligations)
COTS Management
Package Source
Archive
Audit Log
Package Data
Repository
3
1
Data Flow Data Sink
DeepScan

Tool Orchestrator
ToolChain Capabilities (v1.3.1) – Mapping of Tools (example SCANOSS)
Dependency
Analyzer (Source)
Dependency
Analyzer (Binary)
Dependency
Analyzer
(Container)
Policies & Rules
Approval Flow
(WFE)
Compliance
Artefact Generator
License, Copyright
& Authors Scanner
License Repository
obligations)
Compliance
Artefacts
COTS Management
Package Source
Archive
Audit Log
Package Data
Repository
3
2
Data
Flow
Data Sink
Snippet Scanner
(forensics)
Legal Solver
(determine
obligations)
Package Crawler

Open Source Compliance Automation Capability Map

Recommended

Recommended

More Related Content

Similar to Open Source Compliance Automation Capability Map

Similar to Open Source Compliance Automation Capability Map (20)

More from Shane Coughlan

More from Shane Coughlan (20)

Recently uploaded

Recently uploaded (20)

Open Source Compliance Automation Capability Map