Skip to content

Five defects the curacoach v1.0.0-qa10 report surfaced #61

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
grafele merged 1 commit into from
Aug 28, 2026
Merged

Five defects the curacoach v1.0.0-qa10 report surfaced #61

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions soup-discovery/IMPLEMENTATION.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,8 @@ dependency_currency.max_behind
.major / .minor / .patch | 0 / 1 / unlimited | `unlimited` patch is refused in TR-03161 scope.
dependency_currency.stale_after| 12m| —
dependency_currency.
stale_exempt_publishers| dart.dev, flutter.dev| Publishers whose staleness is answered by the process rather than per product: an SDK-pinned package releases on the platform cadence and can never look current. Keyed on the registry-*verified* publisher, which today only pub.dev exposes — an npm `author` string is free text and never earns it. Answers staleness only; being behind the update limit still owes an upgrade. Adding a publisher is a widening and needs `dependency_currency.reason`.
dependency_currency.
obsolescence_may_be_accepted| false | `true` is refused in TR-03161 scope.
production_release.tag_pattern| ^v?\d+\\.\d+\\.\d+$ | Selects the document level only.
onboarded
Expand Down Expand Up @@ -178,6 +180,9 @@ quickbird:soup:direct-without-record
(+ :direct-without-record-name) | A component the manifests mark as a direct choice, with no SOUP record, chosen, shipped, never approved. Count in the metadata, one named entry per component.
quickbird:currency:latest
:status / :detail | The latest available version next to the shipped one, per component: current, behind, stale, stale-and-behind, or unknown with the reason.
quickbird:currency:publisher
:stale-exempt | The registry-verified publisher (pub only), and the process-default reason where one answers this component's staleness. The row stays in section 4 of the report either way — the staleness is a fact — but an exempt row is unshaded and carries the reason instead of "No decision recorded."
quickbird:vuln:fix | `available`, `prerelease-only`, `none-published` or `unknown`. `prerelease-only` means upstream has a fix but only as an alpha/rc, which a released product cannot adopt: the work is to track the stable release, not to bump. Separate from `none-published`, where no fix exists at all and the answer is a compensating control or a VEX statement.

The scanner version is pinned, currently syft 1.51.0, and never `latest`: the component list must not change because a scanner updated itself between two runs of the same commit.

Expand Down
22 changes: 22 additions & 0 deletions soup-discovery/policy-defaults.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,28 @@ dependency_currency:
# answer — there is nothing to upgrade to. 12 months matches the analysis period the
# SOUP records already use in grq-3 ("Is maintained and support is available").
stale_after: 12m
# Staleness answered by the process instead of per product.
#
# A package published by the platform vendor releases on the platform cadence, not on its
# own: package:collection and url_launcher are pinned by the Dart SDK constraint and can
# never look current against a 12-month window. Without this every Flutter product carries
# the same handful of rubber-stamp decisions at every release, which is the outcome
# WI-006-09 exists to remove.
#
# Keyed on the *verified* publisher. pub.dev proves domain ownership before it shows one,
# so this is a fact read from the registry. npm has no equivalent — its `author` field is
# free text set by whoever publishes — so no npm package is exempt here and one that is
# stale still needs a reason in its SOUP record.
#
# Deliberately narrow. google.dev is not on this list: it publishes a grab-bag rather than
# the SDK, and visibility_detector (last release 2023) is exactly the finding a product
# should still have to answer.
#
# Exempts staleness only. A package from a listed publisher that is behind the update
# limit is still behind and still owes an upgrade.
stale_exempt_publishers:
- dart.dev
- flutter.dev
# Written explicitly, not left implicit: validate-policy.sh's TR-03161 check already
# treats an absent key the same as false, so this changes nothing about what is allowed.
# It exists so policy.effective.json shows the value in force instead of being silent
Expand Down
20 changes: 18 additions & 2 deletions soup-discovery/scripts/backstop-report.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,12 +81,19 @@ def _gh_json(args, timeout=60):
return None


def production_deploys(repo):
def production_deploys(repo, not_after=None):
"""Production deployment timestamps with a tag ref, newest first.

Returns (dates, basis, environments_seen). `dates` is None only if the records could not be
read — which is not the same as a product that never deployed and must not read as one.

`not_after` bounds the lookup to the report clock. Without it the deployment history was
read live whatever `--now` said, so a report dated T could claim a cadence that only
holds because of a deployment made after T — and the same evidence directory produced
different answers on different days, which is the property a frozen clock exists to
remove. In a live run `not_after` is the moment the run started, so nothing is excluded
that the run could have known about.

Filtered server-side by environment. Paginating the whole deployment history does not work at
this scale: one product has ~25,000 records across its environments and the unfiltered walk timed
out, reporting a product with 1059 production deploys as unknown. Filtered, the same answer
Expand Down Expand Up @@ -121,6 +128,15 @@ def production_deploys(repo):
prod_rows.extend(got)
suffix = ""

if not_after is not None:
kept = [x for x in prod_rows
if (t := parse_ts(x.get("at"))) is not None and t <= not_after]
excluded = len(prod_rows) - len(kept)
prod_rows = kept
if excluded:
suffix += (f" ({excluded} record(s) after {not_after.date().isoformat()} excluded — "
f"a report cannot rest on a deployment made after the date it states)")

if not prod_rows:
return ([], f"no deployment records at all{suffix}"
if not envs else f"no production deployment records{suffix}", envs)
Expand Down Expand Up @@ -351,7 +367,7 @@ def main():
"detail": "the run record carries no repo, so releases cannot be read"})
else:
iv = parse_interval_days(interval)
dates, basis, envs = production_deploys(repo)
dates, basis, envs = production_deploys(repo, not_after=now)
if dates is None:
cadence.append({"product": product, "declared": interval, "status": "unknown",
"detail": f"could not read the deployment records for {repo}. "
Expand Down
60 changes: 57 additions & 3 deletions soup-discovery/scripts/check-currency.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,26 @@ def fetch(url, timeout=20, accept=None):
return json.loads(r.read().decode("utf-8"))


def pub_publisher(name):
"""pub.dev verified publisher domain for a package, or None.

A second request, which nothing else here needs — the package document does not carry
the publisher. Worth the round trip because it is the only *verified* publisher identity
any of these registries exposes: pub.dev proves domain ownership before it will show
one. npm has no equivalent, its `author` is free text set by whoever publishes, which is
why the staleness exemption keys on this field and not on `supplier`.

A failed lookup returns None and the package simply is not exempt, so the failure
direction is to keep reporting a finding rather than to silence one.
"""
try:
d = fetch(f"https://pub.dev/api/packages/{name}/publisher")
return (d or {}).get("publisherId") or None
except (urllib.error.URLError, urllib.error.HTTPError, KeyError,
json.JSONDecodeError, TimeoutError):
return None


def latest_version(purl, meta=None):
"""(version, published_iso, error). Returns None rather than guessing.

Expand Down Expand Up @@ -145,9 +165,16 @@ def latest_version(purl, meta=None):
if eco == "pub":
full = fetch(REGISTRY["pub"].format(name=name))
d = full["latest"]
if meta is not None and full.get("isDiscontinued"):
meta["deprecated"] = "discontinued" + (
f" (replaced by {full.get('replacedBy')})" if full.get("replacedBy") else "")
if meta is not None:
if full.get("isDiscontinued"):
meta["deprecated"] = "discontinued" + (
f" (replaced by {full.get('replacedBy')})" if full.get("replacedBy") else "")
who = pub_publisher(name)
if who:
meta["publisher"] = who
# pub components carried no supplier at all until now; the verified
# publisher is the best answer the registry has to that question.
meta.setdefault("supplier", who)
return d["version"], d.get("published"), None
if eco == "golang":
d = fetch(REGISTRY["golang"].format(name=name))
Expand Down Expand Up @@ -238,6 +265,10 @@ def annotate_bom(path, notes):
extra.append({"name": "quickbird:currency:detail", "value": n["detail"]})
if n.get("deprecated"):
extra.append({"name": "quickbird:currency:deprecated", "value": n["deprecated"][:200]})
if n.get("publisher"):
extra.append({"name": "quickbird:currency:publisher", "value": str(n["publisher"])})
if n.get("stale_exempt"):
extra.append({"name": "quickbird:currency:stale-exempt", "value": n["stale_exempt"]})
# supplier and license go into the CycloneDX standard fields — that is where every
# other consumer expects them; nothing is overwritten that the scanner already knew
if n.get("supplier") and not c.get("supplier"):
Expand Down Expand Up @@ -310,6 +341,12 @@ def limit(key, default):
# not by a run — the default is unlimited, so no default-configured product could show it.
max_patch = limit("patch", "unlimited")
stale_days = parse_window(cur_policy.get("stale_after", "12m"))
# Publishers whose staleness is answered by the process rather than per product. See
# policy-defaults.yml for what belongs in here and why it is keyed on the verified
# publisher rather than on a supplier string.
exempt_publishers = {str(x).strip().lower()
for x in (cur_policy.get("stale_exempt_publishers") or [])
if str(x).strip()}
if args.now:
now = datetime.fromisoformat(str(args.now).replace("Z", "+00:00"))
if now.tzinfo is None:
Expand Down Expand Up @@ -407,6 +444,15 @@ def check(c):
else "update-available" if b != (0, 0, 0) else "current")
note = {"status": status, "latest": latest}
note.update({k: v for k, v in meta.items()})
pub_id = (meta.get("publisher") or "").lower()
# `not meta.get("deprecated")` because a discontinued package is a finding whatever
# its publisher: the exemption answers "upstream is quiet", not "upstream is gone".
exempt = bool(is_stale and not over and not meta.get("deprecated")
and pub_id and pub_id in exempt_publishers)
if exempt:
note["stale_exempt"] = (
f"Accepted by process default: {meta['publisher']} releases on the "
f"platform cadence and is pinned by the SDK constraint.")
if meta.get("deprecated"):
note["status"] = "deprecated"
note["detail"] = f"declared deprecated by the registry: {meta['deprecated'][:120]}"
Expand Down Expand Up @@ -444,6 +490,13 @@ def check(c):
entry["finding"] = "behind"
entry["action"] = f"upgrade to {latest}"

# A publisher exemption answers staleness and nothing else. A platform vendor
# ships versions behind the update limit like anyone else, and that finding
# keeps its upgrade — which is why `not over` is part of the test above.
if exempt:
entry["justified"] = True
entry["exempt_publisher"] = meta["publisher"]
entry["reason"] = note["stale_exempt"]
if name in reasons:
entry["justified"] = True
entry["reason"] = reasons[name]
Expand Down Expand Up @@ -487,6 +540,7 @@ def check(c):
"stale_with_no_upgrade": len([r for r in flagged
if r["finding"] == "upstream-stale-and-we-are-current"]),
"justified": len(justified),
"stale_exempt_by_publisher": len([r for r in justified if r.get("exempt_publisher")]),
"stale_images": len(stale_images),
"unknown": len(unknown),
},
Expand Down
48 changes: 43 additions & 5 deletions soup-discovery/scripts/group-remediation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@
import re
import subprocess
import sys
from urllib.parse import unquote
from collections import defaultdict
from datetime import datetime, timezone

Expand Down Expand Up @@ -140,10 +141,30 @@ def purl_type(purl):


def purl_name(purl):
"""Readable package name: the last path segment before the version."""
body = (purl or "").split("?")[0].split("@")[0]
seg = [s for s in body.replace("pkg:", "", 1).split("/") if s]
return seg[-1] if len(seg) > 1 else (seg[0] if seg else "?")
"""Readable package name, namespace included.

The last path segment on its own is not a name. `pkg:npm/%40nestjs/core` and
`pkg:npm/%40sigstore/core` both reduce to "core", which produced three unrelated
actions all titled "upgrade core" in one report — and because the unit key below is
built from this name, two scoped packages sharing a last segment inside the same
artifact would collapse into a single action carrying one of the two names.

Splitting on the *first* `@` was the second half of the same defect: an unencoded
scoped purl (`pkg:npm/@nestjs/core@11.1.14`) lost everything after `pkg:npm/` and the
function answered "npm". The version separator is the last `@`, and only when what
follows it is not another path segment.
"""
s = (purl or "").split("?")[0].split("#")[0]
if s.startswith("pkg:"):
s = s[4:]
at = s.rfind("@")
if at > 0 and "/" not in s[at:]:
s = s[:at]
seg = [unquote(x) for x in s.split("/") if x]
if len(seg) < 2:
return seg[0] if seg else "?"
# seg[0] is the purl type; everything after it is namespace + name.
return "/".join(seg[1:])


def main():
Expand Down Expand Up @@ -222,6 +243,17 @@ def main():
action = (f"no upgrade path in {artifact.replace('quickbird:artifact:', '')} — "
f"the advisory publishes no fixed version, so this needs a "
f"compensating control or a VEX statement, not a bump")
elif fx == "prerelease-only":
# Keyed per package, unlike no-upgrade-path above, because what is being
# waited on differs per package: multer's stable 3.0.0 and babel's 8.0.0 are
# unrelated releases on unrelated schedules, and one action covering both
# could only ever be half closed.
key = ("no-stable-upgrade-path", artifact, purl_name(c["purl"]))
action = (f"no stable upgrade for {purl_name(c['purl'])} in "
f"{artifact.replace('quickbird:artifact:', '')} — the only fixed "
f"version published is a prerelease, which a released product "
f"cannot adopt; track the stable release, add a compensating "
f"control, or record a VEX statement")
elif ptype in OS_PKG_TYPES:
key = ("base-image-bump", artifact)
action = (f"bump the base image of "
Expand All @@ -234,14 +266,16 @@ def main():
u = units.setdefault(key, {
"kind": key[0], "artifact": artifact, "action": action,
"findings": [], "components": set(), "fix_status": set(),
"no_fix": set(),
"no_fix": set(), "no_stable_fix": set(),
})
u["findings"].append(fid)
u["components"].add(f"{c['name']}@{c['version']}")
u["fix_status"].add(fx)
member_of[fid].add(key)
if fx == "none-published":
u["no_fix"].add(fid)
if fx == "prerelease-only":
u["no_stable_fix"].add(fid)

by_track = {f["id"]: f for f in doc.get("findings", [])}
out_units = []
Expand Down Expand Up @@ -283,6 +317,10 @@ def earliest(field):
# Even a bump may not clear these: the advisory publishes no fixed version. Carried
# on the unit so it is visible without splitting the action in two.
"findings_without_published_fix": len(u["no_fix"]),
# A fix exists upstream but only as a prerelease. Separate from the line above
# because the answer differs: this one is waiting on a release date, not on a
# compensating control.
"findings_without_stable_fix": len(u["no_stable_fix"]),
"mitigation_due": earliest("mitigation_due"),
"remediation_due": earliest("remediation_due"),
"fix_status": sorted(u["fix_status"]),
Expand Down
Loading