One of the major focuses of the\u00a0ClearlyDefined Technical Roadmap\u00a0is the improvement in the quality of license data. As such, we are excited to announce the release of\u00a0ClearlyDefined v2.0\u00a0which adds over 2,000 new well-known licenses it can identify. You can see the complete list of new non-SPDX licenses in\u00a0ScanCode LicenseDB.<\/p>\n
A little historical background, when Clearly Defined was first created, it was initially decided to limit the reported licenses to only those on the\u00a0SPDX License List. As teams worked with the Clearly Defined data, it became clear that additional license discovery is important to give users a fuller picture of the projects they depend on. In previous releases of ClearlyDefined, licenses not on the\u00a0SPDX License List\u00a0were represented in the definition as\u00a0NOASSERTION\u00a0or\u00a0OTHER. (See the breakdown of licenses in\u00a0The most popular licenses for each language in 2023.)The\u00a0v2.0 release of ClearlyDefined\u00a0includes an update of\u00a0ScanCode to v32\u00a0and the support of LicenseRefs to identify non-SPDX licenses. The license in the definition will now be a LicenseRef with prefix\u00a0LicenseRef-scancode-\u00a0if ScanCode identifies a non-SPDX license. This improves the license coverage in the ClearlyDefined definitions and consumers ability to accurately construct license compliance policies.<\/p>\n
ClearlyDefined identifies licenses in definitions using SPDX expressions. The\u00a0SPDX specification\u00a0has a way to include non-SPDX licenses in license expressions.<\/p>\n
A license expression could be a single license identifier found on the\u00a0SPDX License List; a user defined license reference denoted by the LicenseRef-[idString]; a license identifier combined with an SPDX exception; or some combination of license identifiers, license references and exceptions constructed using a small set of defined operators (e.g., AND, OR, WITH and +)<\/p>\n
\u2014\u00a0excerpt from\u00a0SPDX Annexes: SPDX license expressions<\/p>\n
Example change of a definition:<\/p>\n
CoordinatesLicense BEFORELicense AFTERnpm\/npmjs\/@alexa-games\/sfb-story-debugger\/2.1.0NOASSERTIONLicenseRef-.amazon.com.-AmznSL-1.0<\/p>\n
Note: ClearlyDefined v2.0 also includes an update to ScanCode v32.<\/p>\n
What does this mean for definitions?<\/p>\n
This section includes a simplified description of what happens when you request a definition from ClearlyDefined. These examples only refer to the ScanCode tool. Other tools are run as well and are handled in similar ways.<\/p>\n
When the definition already exists<\/p>\n
Any request for a definition through the\u00a0\/definitions\u00a0API makes a couple of checks before returning the definition:<\/p>\n
If the definition exists, it checks whether the definition was created using the latest version of the ClearlyDefined service.<\/p>\n
If yes, it returns the definition as is.<\/p>\n
If not, it recomputes the definition using the existing raw results from the tools run during the previous harvest for the existing definition. In this case, the tool version will be earlier than ScanCode v32.<\/p>\n
NOTE: ClearlyDefined does not support LicenseRefs from ScanCode prior to v32. For earlier versions of ScanCode, ClearlyDefined stores any LicenseRefs as\u00a0NOASSERTION. In some cases, you may see\u00a0OTHER\u00a0when the definition was curated.<\/p>\n
When the definition does not exist<\/p>\n
If the definition does not exist:<\/p>\n
It will send a\u00a0harvest\u00a0request which will run the latest version of all the tools and produce raw results.<\/p>\n
From these raw results, it will compute a definition which might include a LicenseRef.<\/p>\n
What does it mean if I still see\u00a0NOASSERTION?<\/p>\n
If you see\u00a0NOASSERTION\u00a0in the license expression, you can check the definition to determine the version of ScanCode in the\u00a0\u201cdescribed\u201d: \u201ctools\u201d\u00a0section.<\/p>\n
If ScanCode is a version earlier than v32, you can submit a\u00a0harvest\u00a0API request. This will run any tools for which ClearlyDefined now supports a later version. Once the tools complete, the definition will be recomputed based on the new results.<\/p>\n
In some cases, even when the results are from ScanCode v32, you may still see\u00a0NOASSERTION. Reharvesting when the ScanCode version is already v32 will not change the definition.<\/p>\n
What does this mean for tools?<\/p>\n
When adding ScanCode licenses to allow\/deny lists, note the\u00a0ScanCode LicenseDB\u00a0lists licenses without the LicenseRef prefix. All LicenseRefs coming from ScanCode will start with\u00a0LicenseRef-scancode-.<\/p>\n
Tools using an Allow List<\/p>\n
A recomputed definition may change the license to include a LicenseRef that you want to allow. All new LicenseRefs that are acceptable will need to be added to your allow list. We are taking the approach of adding them as they appear in flagged package-version licenses. An alternative is to review the\u00a0ScanCode LicenseDB\u00a0to proactively add LicenseRefs to your allow list.<\/p>\n
Tools using a Deny List<\/p>\n
Deny lists need to be exhaustive to prevent a new license from being allowed by default. It is recommended that you review the\u00a0ScanCode LicenseDB\u00a0to determine if there are LicenseRefs you want to add to the deny list.<\/p>\n
Note: The\u00a0SPDX License List\u00a0also changes over time. A periodic review to maintain the Deny list is always a good idea.<\/p>\n
Providing Feedback<\/p>\n
As with any major version change, there can be unexpected behavior. You can reach out with questions, feedback, or requests. Find how to get in touch with us in the\u00a0Get Involved\u00a0doc.<\/p>\n
If you have comments or questions on the actual LicenseRefs, you should reach out to ScanCode License DB maintainers.<\/p>\n
Acknowledgements<\/p>\n
A huge thank you to the contributing developers and their organizations for supporting the work of ClearlyDefined.<\/p>\n
In alphabetical order, contributors were\u2026<\/p>\n
ajhenry (GitHub)<\/p>\n
brifl (Microsoft)<\/p>\n
elrayle (GitHub)<\/p>\n
jeff-luszcz (GitHub)<\/p>\n
ljones140 (GitHub)<\/p>\n
lumaxis (GitHub)<\/p>\n
mpcen (Microsoft)<\/p>\n
nickvidal (Open Source Initiative)<\/p>\n
qtomlinson (SAP)<\/p>\n
RomanIakovlev (GitHub)<\/p>\n
yashkohli88 (SAP)<\/p>\n
See something you\u2019d like ClearlyDefined to do or could do better? If you have resources to help out, we have work to be done to further improve data quality, performance, and sustainability. We\u2019d love to hear from you.<\/p>\n
References<\/p>\n
SPDX License List<\/p>\n
SPDX Specifications<\/p>\n
SPDX Annexes: SPDX license expressions<\/p>\n
ScanCode LicenseDB<\/p>\n
ScanCode v32.3 release notes<\/p>\n
ClearlyDefined Service API v2.0 release notesI<\/p>\n
ClearlyDefined Crawler\/Harvester v2.0 release notes<\/p>\n
Beyond SPDX: expanding licenses identified by ClearlyDefined
\n
\n
\nSource: opensource.org
<\/p>\n","protected":false},"excerpt":{"rendered":"
One of the major focuses of the\u00a0ClearlyDefined Technical Roadmap\u00a0is the improvement in the quality of license data. As such, we […]<\/p>\n","protected":false},"author":73,"featured_media":0,"comment_status":"false","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-14010","post","type-post","status-publish","format-standard","hentry","category-mp"],"_links":{"self":[{"href":"https:\/\/plus.maciejpiasecki.info\/index.php\/wp-json\/wp\/v2\/posts\/14010","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/plus.maciejpiasecki.info\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/plus.maciejpiasecki.info\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/plus.maciejpiasecki.info\/index.php\/wp-json\/wp\/v2\/users\/73"}],"replies":[{"embeddable":true,"href":"https:\/\/plus.maciejpiasecki.info\/index.php\/wp-json\/wp\/v2\/comments?post=14010"}],"version-history":[{"count":1,"href":"https:\/\/plus.maciejpiasecki.info\/index.php\/wp-json\/wp\/v2\/posts\/14010\/revisions"}],"predecessor-version":[{"id":14011,"href":"https:\/\/plus.maciejpiasecki.info\/index.php\/wp-json\/wp\/v2\/posts\/14010\/revisions\/14011"}],"wp:attachment":[{"href":"https:\/\/plus.maciejpiasecki.info\/index.php\/wp-json\/wp\/v2\/media?parent=14010"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/plus.maciejpiasecki.info\/index.php\/wp-json\/wp\/v2\/categories?post=14010"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/plus.maciejpiasecki.info\/index.php\/wp-json\/wp\/v2\/tags?post=14010"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}