Webinar recording available at the end of the slide deck. Heather Meeker, partner at O'Melveny & Myers LLP and Philippe Ombredanne, founder at nexB Inc. discussed the latest open source software identification tools available for use in your compliance process. Agenda - Key Elements of a Policy for use of OSS - Overview of OSS Identification - Survey of open source and commercial tools for OSS Identification - Rightsizing your OSS Identification Process and Tools. If you are interested in open source scanning and open source compliance products, please visit http://www.nexb.com/, see also https://www.youtube.com/user/DejaCode/ for other webinar recordings.

1. Rightsizing Open Source Software Identification
September 9, 2015 – Silicon Valley, CA

2. Agenda
Introduction
Key Elements of a Policy for use of OSS
OSS Identification: Why, What, How ?
Survey of open source and commercial tools for OSS Identification
Rightsizing your OSS Identification Process and Tools

3. Open Source Policies
Most companies today have written open source policies
• Decrease risk of compliance claims
• Set policy for transactions with vendors, customers
• Manage unintended consequences to IP (for contributions)
TACTICS
STRATEGY

4. LET’S TALK ABOUT TACTICS
Policies mostly deal with compliance
• Substantive policy: license stop/go lists avoid engineering problems
(for GPL/LGPL)
• Procedural policy: keeping track of open source use
Compliance is mostly an information/process problem
• Open source circumvents business processes
• Most companies try to automate the management of information
• Most people managing this process in the trenches find the array of
tools confusing

5. Substance of Open Source Policies on Compliance
Use Cases
• Modification
• Internal use v. Distribution
o Product deployment
o Development tools
o Internal business use
Licenses
• Most companies triage by license, others by
project

6. Go Caution Stop
You do not need Legal approval to use code
licensed under these licenses for all use cases.
You must obtain Legal approval to
distribute code licensed under
these licenses.
You do not need Legal approval to
use code licensed under these
licenses internally.
You must obtain Legal
approval to use any code
licensed under these
licenses.
•Public Domain
•BSD
•MIT
•Apache 1.1 or 2.0 (not 1.0)
•Artistic License
•PHP License
•Python Software Foundation License
•zlib/libpng License
•Boost Software License
•OpenSSL/SSLeay License
•WTFPL
•CC0 (public domain dedication)
•Creative Commons Attribution-Only (“CC
BY”)
•Unlicense
•Any software licensed under a choice of
multiple licenses that include any of the
above.
GPL 2.0 (only when a standalone
process)
LGPL 2.1 (only when dynamically
linked)
Mozilla Public License 1.1 or 2.0
(“MPL”)
CDDL
CPL or IBM
Eclipse Public License
GPL 2.0 + plus exception (such as
linking, Classpath, or FOSS
exception)
Apache 1.0
GPL 3.0
LGPL 3.0
Affero GPL v1 or v3
Sleepycat
GNU Documentation License
Creative Commons
ShareAlike (“CC BY-SA”)
Open Software License
Academic Free License
Example – Stop/Go/Caution List

7. Why do you need to identify OSS Components?
Legal: You need to know the license and usage in order to comply
with the license terms
• Attribution
• Redistribution of source code
Business: You need to know the origin to assess the business
risks related to
• Quality
• Security
• Support (community or commercial)
• Project viability

8. OSS Identification
Create an inventory of OSS software components used in a
product, application or system where:
• A component can be a product, package, library, directory of code
files, a file or a snippet in a file
• The granularity of a component in an inventory is based on level of
detail you need to document origin and license
Categorize by primary usage:
• Development tools - e.g. compilers, editors, test tools
• Development code - components that are used to build some
version of a product (current, prior or future release)
• Deployed code - the subset of Development code
deployed/distributed for a particular product release

9. OSS Identification
For each component determine:
• The origin: who wrote it, who holds the copyright, where can it be
downloaded from?
• The license: what are the license terms (high level categorization
and specific licensing terms)
• The version being used
• How it is used:
o Modified or as-is?
o Standalone or in combination with other proprietary code or OSS code?

10. How do you perform OSS Identification?
Overall Process = Combination of Tools + (human) review
• Some identification can be fully automated
• Most requires interpretation
Goals
• Create a reasonably detailed Inventory of OSS components in a
codebase
• Identify and resolve license issues relative to your policies
• Prepare for outbound compliance
Primary Techniques
• Scanning - extract information directly from codebase files
• Matching - compare codebase files to a repository of OSS files

11. How does Scanning work?
Detect and discover “evidence” and “assertions” in the code
• Copyright statements
• License notices, mentions or texts
• Software package “manifests” (Java Jar, RubyGem, RPM, etc.)
• Email, URL, author and person names
• Other origin and license clues found in the code
• Data may be in source code, text files or binaries
Techniques
• Copyright “grammar”
• License text search and matching
• Specific “parsers” to extract package metadata, emails, etc.

12. How does Scanning work? [2]
High confidence for identification with scanning
• Evidence in the code itself is primary
• Will not detect two cases:
o Copyright or license notices intentionally removed (rare)
o Snippets where developer did not provide comments or clues about origin of the
snippet
Interpretation typically required to:
• Conclude overall license at component level when files have
different notices
• Research clues
The quality of identification depends on who interprets the data -
can often be performed by a developer who understands the
codebase

13. How does Matching work?
Find similarities between your code and an index of OSS code
• If your code is similar it “may” share a similar origin
Multiple levels
• Whole package, library or archive
• Files or snippet(s) of a few lines of code
Techniques
• File comparison, similar to comparing two documents (aka redlining
or blacklining)
• Create and compare digital “fingerprints”
o Single fingerprint for an entire package or archive or file
o Set of fingerprints for a set of files
o Set of snippet fingerprints within a file - e.g. every 10 lines

14. How does Matching work? [2]
All matches except an exact or string match at the package or
library level typically requires interpretation of the matching
pattern
The finer the matching (e.g.. snippets), the higher the risk of
irrelevant matches (false matches)
Code can be generated by tools, leading to fake similarities
Snippet matches typically require extensive (and costly) human
review
• Commonly re-used code may have hundreds of matches (or more)
• Snippet matches need to be evaluated in context of the type of code
and the domain to determine most likely match

15. How does Matching work? [3]
More coarse-grained matching has a risk of missing some
detection (false positives)
The quality of identification depends on:
• Who interprets the matches
• The quality of the origin and license data in the OSS repository used
to create the OSS index. Bigger is not always better.

16. Completing the OSS Identification work
Expert technical and legal review is needed to confirm, validate (or
invalidate) the data from tools
• The level of effort is related to the technique used
Scanning and Component Matching require less review
• Scanning is about evidence
• Component Matching is also conclusive on its own
Snippet Matching require more review - More false matches and
larger volume of possible matches
Policy should balance risk and effort
• More frequent Scanning and Component Matching
• Less frequent Snippet Matching. Eventually not needed with trained
developer teams

17. Finding and Resolving Issues
Finding Issues
• An origin or license that does match your policy for a certain usage
context should be an issue trigger
• Non-compliance with license terms (missing attribution,
redistribution, etc..)
Resolving Issues
• Legal advisors need to research legal or policy implications of an
issue
• Use development team ticket system or similar to track each issue
to resolution

18. Rightsizing your OSS ID Process and Tools
Assess risk and define policies at appropriate level(s)
• Company-level
• Product-line or product-level
• Factor in business and technology domains
Evaluate your Risk Profile for use of OSS - considering:
• Internal vs. External Use
• Business or Consumer Customers or both
• Product Deployment modes - Distribution, SaaS, Resellers/OEMs
• Software Domains - Linux, Codecs
• Patent considerations

19. Rightsizing your OSS ID Process and Tools
Define the scope of data you need for/from OSS Identification
• License
• Knowing only the license may be sufficient for internal use
• Provenance
• Knowing the copyright holder is needed for Attribution
• Knowing details about the project is recommended for tracking
bugs, evaluating level of community support, etc.
• Usage
• Knowing where and how you use a component is needed for
compliance with Copyleft licenses
• Also helps you manage other risks – e.g. for a security alert such
as Heartbleed for OpenSSL you need to know quickly where you
a component version

20. Rightsizing your OSS ID – Scope
LICENSE
PROVENANCE
LICENSE
PROVENANCE
USAGE
PROVENANCE
LICENSE
Business criticality
Exposure
High
Low High
Your primary focus should be:
1
2 4
3

21. Business criticality
Exposure
High
Low High
Your primary focus should be:
1
2 4
3
IT mgt
systems
Desktop
Office suites
IT
Security
Financial
systems
Consumer
web site mobile
consumer
products
server-side
management
products
development
tools
CRM
systems
Rightsizing your OSS ID – Scope examples

22. Rightsizing your OSS ID Process and Tools
Define level of depth (precision) needed for OSS Identification
• The cost of OSS Identification is directly proportional to the depth of
analysis
• Easier to identify larger components – e.g. libraries
• Progressively harder to identify smaller components – e.g. files
and snippets
• The value of an OSS component is also typically proportional to size
• Product-level OSS projects or libraries provide significant
functional value
• Files or snippets from an OSS project may be useful, but
typically offer less functional value
o Depth of analysis may be variable across OSS ID processes
• More frequent Library-level analysis
• Less frequent File/Snippet-level analysis

23. Rightsizing your OSS ID – Depth
VALUE RISK
Product
Libraries
Files /
snippets
Files /
snippets
Libraries
Files /
snippets
COST
50%
50%
There is more
value in reusing a
whole open source
product or a library
Reporting reuse of
whole products is
straightforward
Risk of non-
compliance decreases
with files and snippets
Reusing single files
and snippets of code
is tactical
Detecting and reviewing
borrowed code snippets
requires a significant effort
Aligning cost with value

24. Insource or outsource OSS Identification process and tools
• Internal staff may be able to handle basic OSS Identification
• Engage external experts (retainer-style?)
• Plan for periodic audits ⇒ real audits, not OSS ID
Legal Role
• Define and communicate policies
• “Enforce” policies and guide issue resolution
Developer Role (or QA or Release Management)
• First level OSS Identification
• Maintain Software Inventory
Augment with outside resources where needed
Distributing the OSS ID workload

25. OSS tools for OSS Identification
Linux Foundation Tools
• http://www.linuxfoundation.org/programs/legal/compliance/tools
• SPDX Tools - https://github.com/spdx-tools
FOSSology
• http://www.fossology.org/projects/fossology
• Moving to Linux Foundation soon
nexB Projects
• AboutCode - https://github.com/dejacode/about-code-tool
• ScanCode - https://github.com/nexB/scancode-toolkit

26. • Free and Open Source license and copyright scanner (Commercial
Support available)
• Command line tool with interactive HTML reports or JSON.
• Available on GitHub at: https://github.com/nexB/scancode-toolkit/
Demo

27. Commercial tools for OSS Identification
Commercial
• Black Duck Protex
• Palamida
• Protecode
• OpenLogic (Rogue Wave) – SaaS based on FOSSology
• White Source - SaaS based on Ninka

28. Other online resources for OSS compliance
• LF Open Compliance Program -
http://www.linuxfoundation.org/programs/legal/compliance
• Open Source Initiative - http://opensource.org/
• Software Package Data Exchange - https://spdx.org/
• TLDRLegal - https://tldrlegal.com/
• DejaCode (public site) - https://enterprise.dejacode.com/

29. 29
Thank You!
• To receive CLE credit for your participation in today’s
program, please send an email to ikim@omm.com with
the following:
–Your name
–Bar jurisdiction
–Bar number

30. Questions

31. About nexB Inc.
Business is software component management
• Current focus on open source governance and compliance
• Primary product is an enterprise system for tracking all software
components in your products.
We offer
• DejaCode™- Open Data Platform for Managing Open Source -
http://www.dejacode.com/
• Open Source Scanning & Attribution Generation Tools -
https://github.com/nexB
• Open Source Software Expert Audit Services -
http://www.nexb.com/services.html

32. Contact
O’Melveny & Myers - http://www.omm.com/
• Heather Meeker - hmeeker@omm.com +1 650 473 2635
• Subscribe to news and events alert at
http://heathermeeker.squarespace.com/
nexB Inc. - http://www.nexb.com/
• Philippe Ombredanne - pombredanne@nexB.com +1 650 799 0949