Repository navigation
feat(ecosystem): Add Red Hat Lightwell ecosystem support (version logic + PURL) - #6142
Conversation
Registers the Red Hat Lightwell ecosystem (Red Hat Lightwell:Maven / :PyPI). go/osv/ecosystem: redHatLightwellFactory delegates version semantics to the base ecosystem named by the :suffix (Maven/PyPI); an unsupported suffix yields a nil ecosystem (reported as not found). go/purl: generator delegates to the base ecosystem's registered PURL generator; no reverse parser is registered since pkg:maven/pkg:pypi types belong to the upstream ecosystems. Bumps the osv-schema bindings to the version defining EcosystemRedHatLightwell (ossf/osv-schema#571). Tests cover Maven/PyPI delegation and unsupported/missing suffixes. Companion to the source_test.yaml import (google#6139) and tracking issue google#6105.
|
@tobyhawker @another-rex can you please take a look at this PR? |
michaelkedar
left a comment
michaelkedar
left a comment
There was a problem hiding this comment.
Hi, thanks for your patience.
One thing, though
There was a problem hiding this comment.
I have a bit of a concern regarding the purl types you're using here.
Your PURLs map indistinguishably to the packages on their default repositories.
e.g. Red Hat Lightwell:Maven/org.apache.commons:commons-lang3 and Maven/org.apache.commons:commons-lang3 both map to the same pkg:maven/org.apache.commons/commons-lang3 which seems incorrect - the patches don't exist on Maven central (or on PyPI) so it's hard to say it's the same package.
I guess the more correct thing would be to include the repository_url qualifier?
Regardless, the purl field is optional, and the Red Hat Lightwell already includes the purl on the records, so this generator might not be necessary
There was a problem hiding this comment.
@michaelkedar thanks for your feedback. Intentionally our purl does not contain version and repository_url qualifier to allow scanners indirect downstream to upstream package matching. This is one of old known problems I described in #6105 (comment). We have dedicated ecosystem and we include additional metadata for traceability like repository_url in the database_specific section.
There was a problem hiding this comment.
I think I am just a bit confused on how these records are intended to be used. The OSV API does not match directly on PURLs (it translates them to OSV ecosystem / package names before matching), so the PURLs on the records aren't really used for anything on our side.
The scanner workflow you describe seems unclear to me. I understand that you want to be able to match both vulnerabilities from regular Maven GHSA records and use Lightwell records to remove "false positives" from users, but I don't really understand the workflow you envision for this, how scanners are supposed to reconcile this data, and how the OSV.dev API / exports help with this. (tbh, I don't really think the osv-schema is expressive enough to support this use-case well)
There was a problem hiding this comment.
Though, these concerns don't really matter for this PR, since I don't believe this PURL generation code will run if the data source is already providing PURLs (which Lightwell is). So I guess I can just approve this and get it merged after the deployment tomorrow (as long as the merge conflict is resovled).
There was a problem hiding this comment.
@michaelkedar thanks for approving this PR.
Let me also walk back my earlier reply a bit: saying the bare PURL is there "to allow scanners indirect downstream→upstream matching" was too much of a shortcut and not precise about the mechanism. Let me restate it properly.
Our Lightwell data-usage flow queries osv.dev by {name, ecosystem} against Red Hat Lightwell:Maven / :PyPI and relies on the API ordering versions correctly within that ecosystem: < fixed → affected, >= fixed → empty = not affected. That delegation to the base Maven/PyPI semantics is exactly what this PR adds. Without it, any version-scoped query against the Lightwell ecosystem (e.g. "is my installed 6.0.3.rhlw-00001 build fixed?") can't be evaluated at all.
On schema expressiveness and the full fix: I fully agree with you that it isn't there yet for the complete use-case. The permanent piece -> routing a repository_url-qualified PURL (pkg:maven/...?repository_url=…/lightwell/…) to Red Hat Lightwell:Maven automatically, so an ordinary scanner doesn't have to detect the .rhlw- suffix and re-query osv.dev, it's the harder problem I'm tracking in #6105. This PR is the foundation that makes the interim consumer-side flow actually work on the API. Happy to keep the permanent-routing design discussion in #6105.
There was a problem hiding this comment.
@michaelkedar as I can see @syncrou pushed new updates to fix merge conflicts and now we need your approval once again.
syncrou
changed the title
|
/gcbrun |
…6139) ## What Adds **project-lightwell/lightwell-osv** as a new OSV data source under the **Red Hat Lightwell** ecosystem (`db_prefix: RHLW-`), via `source_test.yaml` (test import first). Tracking issue: #6105. ## Records - Published at https://github.com/project-lightwell/lightwell-osv/tree/main/advisories (44 records today, growing). - **Single ecosystem** — `Red Hat Lightwell:Maven` / `Red Hat Lightwell:PyPI`; the plain `Maven`/`PyPI` dual entries were dropped per the maintainer guidance in #6105 (they misrepresent ecosystem semantics for `.rhlw` versions that don't exist upstream). - **Opaque IDs** — `RHLW-YYYY-<16 hex>` (MurmurHash3-128), per ossf/osv-schema#547. - OSV schema 1.9.0; the `Red Hat Lightwell` ecosystem was registered via ossf/osv-schema#571 (shipped in v1.9.1). ## Config notes - `type: 0` (GIT), `directory_path: advisories`, `ignore_patterns` restricts processing to `RHLW-` files. - `accepted_ecosystems: ['Red Hat Lightwell']` (base name; records carry the `:Maven`/`:PyPI` suffix — matching the AlmaLinux pattern). - `strict_validation: False` for this initial test import, so records import and any issues surface via `importfindings` rather than hard-rejecting. ## Follow-up PR A separate PR adds the new-ecosystem **PURL mappings** (`go/purl`) and **version logic** (`go/osv/ecosystem`) for full API version querying, per the new-source checklist #6142
…6207) ## Summary Promotes the **Red Hat Lightwell** OSV data source from `source_test.yaml` to the production `source.yaml`, turning on ingestion of Lightwell remediated-backport advisories into osv.dev. This is the follow-up to the two prerequisite PRs that have now merged: - #6142 — Red Hat Lightwell ecosystem support (version logic + PURL) - #6139 — Red Hat Lightwell data source added to `source_test.yaml` (test import) The `lightwell` source entry here is **byte-identical** to the one merged in the test import. It pulls from `https://github.com/project-lightwell/lightwell-osv` (`advisories/`), ingests only records matching `^RHLW-`, and accepts the `Red Hat Lightwell` ecosystem. ## Content readiness The `lightwell-osv` repo content has been verified ahead of this promotion: - All published advisories use the opaque `RHLW-YYYY-<hex>` ID and OSV 1.9.0 schema. - No internal/non-public URLs in any record's `references`. - `ignore_patterns: ['^(?!RHLW-).*$']` means any non-`RHLW-` file is skipped regardless. Opening as a **draft** for a final review before we flip production ingestion on. Context: #6105
What
Adds OSV ecosystem support for Red Hat Lightwell (
Red Hat Lightwell:Maven,Red Hat Lightwell:PyPI) so API version queries resolve for the data source in #6139.Tracking issue: #6105.
Changes
go/osv/ecosystem—redHatLightwellFactorydelegates version semantics to the base ecosystem named by the:suffix (Maven→Maven,PyPI→PyPI). Red Hat Lightwell republishes upstream artifacts at backport versions (.rhlw-NNNNN/+rhlw.N) that sort within the base ecosystem's version space, so base comparison is correct. An unsupported suffix yields a nil ecosystem (reported as "not found").go/purl— registers a PURL generator that delegates to the base ecosystem's generator. No reverse parser is registered:pkg:maven/pkg:pypitypes are already owned by the upstream Maven/PyPI parsers (the registry panics on collision), so Lightwell PURL→ecosystem resolution is intentionally left to explicit ecosystem queries.EcosystemRedHatLightwell(Add RHLW prefix for Red Hat Lightwell ossf/osv-schema#571).Related
source_test.yamlimport for the Red Hat Lightwell data source.Red Hat Lightwell:<eco>affected entries, opaqueRHLW-YYYY-<hash>IDs).