ci: add matrix test workflow for debian slim images in .gitea #15

Merged

Philipp merged 9 commits from feat-test-matrix-workflow into main 2026-09-15 00:26:58 +02:00

Conversation 0 Commits 9 Files Changed 11

+500 -22

Philipp commented 2026-09-14 03:05:16 +02:00

Owner

Copy link

Copy source

Overview

This PR enhances the action's robustness, testability, and CI workflow by introducing an automated multi-platform matrix integration test suite, dedicated unit tests, automatic OS distribution detection and resilient fallback for in-build SBOMs, detailed trade-off documentation, and workflow artifact archiving.

Key Changes

1. Dedicated CI Unit Tests Job & Fixtures

• Added native Node.js unit tests in test/cve-diff.test.js using node:test and node:assert/strict (no external dependencies).
• Added fixture test datasets in test/fixtures/ covering multi-architecture reports, set-based diffs, and empty/zero-result scans.
• Added a dedicated unit-test job in .gitea/workflows/test.yaml that runs fast unit tests in ~8s before starting heavier container image scans (needs: unit-test).

2. Distro Detection & Resilient Fallback for In-Build SBOMs

• Distro Detection: Third-party in-build SBOM generators (Docker Scout / BuildKit / Syft) omit OS qualifiers from SPDX packages, which previously caused Trivy to match 0 CVEs. The action now parses package PURLs (e.g. pkg:deb/debian/apt@...os_name=debian&os_version=12) to detect the distribution and passes --distro <family>/<version> to trivy sbom.
• Resilient Fallback: If an extracted in-build SBOM still yields 0 CVE findings or fails to scan, the action automatically falls back to generating an SBOM directly with Trivy (trivy image --platform <platform> --format spdx-json).
• Audit Copies: Preserves raw extracted in-build SBOMs (inbuild-sbom-*.spdx.json) alongside final scanned SBOMs (sbom-*.spdx.json) for auditing.

3. Strategy Trade-offs & Detailed Documentation

• Documented the fidelity vs. speed trade-offs in action.yaml and README.md:
  • Direct Trivy Scanning (use-inbuild-sbom: false - Default & Recommended): Inspects full rootfs and /var/lib/dpkg/status mapping binary packages to source packages for maximum vulnerability fidelity (~225 CVEs on debian:bookworm-slim).
  • Built-in Attestation (use-inbuild-sbom: true): Skips layer downloads for high CI speed (~5x faster, 1m vs 5m), cataloging binary-package manifests (~104 CVEs on debian:bookworm-slim).
• Added informative logging in action.yaml diagnostics to display the active scanning strategy.

4. CI Artifact Archiving

• Added an actions/upload-artifact@v4 step in .gitea/workflows/test.yaml with if: always() to archive sboms/, cve-reports/, cve-delta.md, and cve-summary.md on every matrix run.

5. Engine & Script Fixes

• Fixed a bash syntax error in action.yaml where grep -o ... | wc -l || echo 0 under pipefail produced "0\n0", masking zero-vulnerability warnings.
• Fixed cve-diff.js to gracefully accept reports where Trivy omits the Results array on 0 findings (data.Results = data.Results || []).
• Escaped pipe characters (\|) in Markdown table cells to preserve table formatting.

## Overview This PR enhances the action's robustness, testability, and CI workflow by introducing an automated multi-platform matrix integration test suite, dedicated unit tests, automatic OS distribution detection and resilient fallback for in-build SBOMs, detailed trade-off documentation, and workflow artifact archiving. ## Key Changes ### 1. Dedicated CI Unit Tests Job & Fixtures - Added native Node.js unit tests in `test/cve-diff.test.js` using `node:test` and `node:assert/strict` (no external dependencies). - Added fixture test datasets in `test/fixtures/` covering multi-architecture reports, set-based diffs, and empty/zero-result scans. - Added a dedicated `unit-test` job in `.gitea/workflows/test.yaml` that runs fast unit tests in ~8s before starting heavier container image scans (`needs: unit-test`). ### 2. Distro Detection & Resilient Fallback for In-Build SBOMs - **Distro Detection**: Third-party in-build SBOM generators (Docker Scout / BuildKit / Syft) omit OS qualifiers from SPDX packages, which previously caused Trivy to match 0 CVEs. The action now parses package PURLs (e.g. `pkg:deb/debian/apt@...os_name=debian&os_version=12`) to detect the distribution and passes `--distro <family>/<version>` to `trivy sbom`. - **Resilient Fallback**: If an extracted in-build SBOM still yields 0 CVE findings or fails to scan, the action automatically falls back to generating an SBOM directly with Trivy (`trivy image --platform <platform> --format spdx-json`). - **Audit Copies**: Preserves raw extracted in-build SBOMs (`inbuild-sbom-*.spdx.json`) alongside final scanned SBOMs (`sbom-*.spdx.json`) for auditing. ### 3. Strategy Trade-offs & Detailed Documentation - Documented the fidelity vs. speed trade-offs in `action.yaml` and `README.md`: - **Direct Trivy Scanning (`use-inbuild-sbom: false` - Default & Recommended)**: Inspects full rootfs and `/var/lib/dpkg/status` mapping binary packages to source packages for maximum vulnerability fidelity (~225 CVEs on `debian:bookworm-slim`). - **Built-in Attestation (`use-inbuild-sbom: true`)**: Skips layer downloads for high CI speed (~5x faster, 1m vs 5m), cataloging binary-package manifests (~104 CVEs on `debian:bookworm-slim`). - Added informative logging in `action.yaml` diagnostics to display the active scanning strategy. ### 4. CI Artifact Archiving - Added an `actions/upload-artifact@v4` step in `.gitea/workflows/test.yaml` with `if: always()` to archive `sboms/`, `cve-reports/`, `cve-delta.md`, and `cve-summary.md` on every matrix run. ### 5. Engine & Script Fixes - Fixed a bash syntax error in `action.yaml` where `grep -o ... | wc -l || echo 0` under `pipefail` produced `"0\n0"`, masking zero-vulnerability warnings. - Fixed `cve-diff.js` to gracefully accept reports where Trivy omits the `Results` array on 0 findings (`data.Results = data.Results || []`). - Escaped pipe characters (`\|`) in Markdown table cells to preserve table formatting.

Philipp added 1 commit 2026-09-14 03:05:16 +02:00

ci: add matrix test workflow for debian slim images in .gitea 44c4b5a304

Philipp added 1 commit 2026-09-14 03:10:12 +02:00

style: format test workflow to comply with pretty-format-yaml 40209bbeed

Philipp added 1 commit 2026-09-14 22:28:17 +02:00

test: add unit tests for cve-diff and fallback for in-build sbom scans d89fb25d7b

Philipp added 1 commit 2026-09-14 22:29:46 +02:00

ci: move unit tests into dedicated unit-test job f31be5eef6

Philipp added 1 commit 2026-09-14 22:34:33 +02:00

docs: update README with in-build fallback and unit test details 790c1aa3de

Philipp added 1 commit 2026-09-14 23:26:35 +02:00

feat: detect distro from in-build SBOM PURLs and pass --distro to trivy 3b9043a746

Philipp added 1 commit 2026-09-14 23:37:33 +02:00

fix: robustly parse distro from in-build SBOM package PURLs 64747ec466

Philipp added 1 commit 2026-09-15 00:16:23 +02:00

docs: document trade-offs and differences between in-build and direct scanning def42c37a8

Philipp added 1 commit 2026-09-15 00:17:08 +02:00

Merge branch 'main' into feat-test-matrix-workflow c98b31c23a

Philipp scheduled this pull request to auto merge when all checks succeed 2026-09-15 00:17:16 +02:00

Philipp merged commit 0ccfd5f512 into main 2026-09-15 00:26:58 +02:00

Philipp deleted branch feat-test-matrix-workflow 2026-09-15 00:26:58 +02:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Assignees

Clear assignees

AutoReview (LLMReview) Renovate (Renovate Bot) runner (Act Runner) Philipp (Philipp Horstenkamp)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Philipp/cve-differential#15