test(tools): cover PR metadata error paths

[chaofengw-nv]

Background

The PR metadata validator's CLI error handling and successful script entry point lacked direct regression coverage. Add checks for malformed event JSON, missing pull-request metadata or draft state, and a null PR body.

Exit Criteria

• Each invalid event returns exit code 1 and reports the expected diagnostic; a null body also emits the required GitHub error annotation.
• Running the script with complete metadata exits successfully and prints the success message.
• Existing tests continue to pass, with full statement and branch coverage for tools/pr_metadata.py.

Implementation

Add four parameterized invalid-event cases and one successful script-entry case in tools/tests/test_pr_metadata.py. The tests use temporary event files, capture CLI output, and restore sys.argv through pytest's monkeypatch fixture. Production code, existing tests, coverage exclusions, dependencies, APIs, ABI, and artifact formats are unchanged.

Change categories

• Model or runtime behavior
• Public API
• ABI
• Bundle or artifact format
• Dependencies
• Documentation only
• CI or developer tooling

Validation

Commands and Results

• PYTHONPATH=. python3 -m coverage run --branch --source=tools.pr_metadata -m pytest -q tools/tests/test_pr_metadata.py: 17 passed, including five added cases.
• python3 -m coverage report -m: 109/109 statements and 34/34 branches covered, both 100%. Baseline coverage was 98/109 statements (89.9%) and 26/34 branches (76.5%); exclusions remain zero.
• python3 -m ruff check --config ruff.toml tools/tests/test_pr_metadata.py: passed.
• PYTHONPATH=core/builder:apps/benchmark:. python3 -m tools.model_ci validate: passed.
• PYTHONPATH=core/builder:apps/benchmark:. python3 tools/test_impact.py --validate: passed.
• git diff --check github/main...HEAD: passed.

Hardware, Environment, and Revisions

CPU-only Linux x86_64; Python 3.12.3, pytest 9.0.3, PyYAML 6.0.3, Ruff 0.15.16, and coverage.py 7.6.10. Tested head: 5449a773e1fa251ceea5ede50787803468541ea8; upstream base: 9b083a7fdac56f9d0f14084e61e2aa9dc0dc8816. Model, checkpoint, dataset, CUDA, TensorRT, and precision revisions are not applicable to this developer-tool test change.

Not Run / Remaining Gaps

The full repository suite and remote CI were not run as part of local validation; GitHub CI will report its results separately. GPU execution, model parity, performance, and native C++ coverage are outside this CPU validator test scope. Full line and branch coverage does not claim coverage of every possible event payload.

Contributor Self-Review

• I have completed a self-review of this change.

Manually reviewed the exact diff, public CLI assertions, fixture cleanup, unchanged existing tests and production source, and matching commit-author/DCO identity. No blocking findings.

Notes For Future Readers

Keep the assertions on exit status, diagnostic output, and GitHub annotations: these are the validator's observable CI behavior. The coverage increase requires no new exclusions or dependency changes. Review starts with the five added cases in the test file.

Risk level

• Low
• Medium
• High

The change adds isolated CPU tests without altering validator behavior or model execution.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Repository: NVIDIA/TensorRT-Model-Connect/.coderabbit.yaml
• Review profile: CHILL
• Plan: Enterprise
• Run ID: c402e846-3971-4770-8032-af8d0ac55ffa

📥 Commits

Reviewing files that changed from the base of the PR and between 9b083a7 and 5449a77.

📒 Files selected for processing (1)

• tools/tests/test_pr_metadata.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

📝 Summary

Summary

Added five regression cases in tools/tests/test_pr_metadata.py. Four parameterized cases check malformed JSON, missing pull-request metadata, missing draft state, and a null PR body. They verify failure status and diagnostics, including the GitHub error annotation for the null body. One case runs tools/pr_metadata.py as a script and checks its successful exit and completion message.

Production code, public APIs, dependencies, and artifact formats are unchanged.

Architecture impact

• Family ownership: The change is confined to tooling tests. It does not add model-family implementation or validation dependencies.
• Shared surfaces: No shared production surface changed.
• Dependency direction: The tests import the existing tools.pr_metadata module and execute its script entry point. No new dependency direction is evident.
• Affected consumers: The tests exercise behavior used by the PR metadata workflow. No other consumer changes are reported.
• Blast radius: No material blast-radius question is evident in the inspected files. The full repository suite and remote CI were not run, according to the supplied objectives.

Review outcome: PASS — no standards or spec violation was found in the reviewed evidence. This is not proof of correctness. Review finding counts are unavailable because no current review findings were supplied.

The supplied objectives report 17 targeted tests passing and full statement and branch coverage for tools.pr_metadata. They also report passing Ruff, model CI validation, test-impact validation, and git diff --check. The full repository suite and remote CI were not run.

Walkthrough

The test suite now checks successful execution of tools/pr_metadata.py as a script and failure responses for malformed JSON, missing pull-request metadata, missing draft state, and invalid body metadata.

Changes

Metadata validator tests

Layer / File(s)	Summary
Script entry point and failure cases tools/tests/test_pr_metadata.py	Tests verify successful script execution and its completion message. Parameterized cases verify failure status and expected error output for invalid event metadata. The invalid-body case also checks for a GitHub error annotation.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

Suggested reviewers: yifeif-nv

Merge Risk: ⚪ Minimal · up to 5449a

The added PR metadata tests are ready to merge after normal checks.

🚥 Pre-merge checks | ✅ 8 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 1 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (8 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the main change: adding tests for PR metadata error paths.
Description check	✅ Passed	The description covers the background, exit criteria, implementation, change category, validation commands and results, environment, remaining gaps, self-review, notes, and risk. It is complete and al…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Family Ownership Boundary	✅ Passed	PASS. The pull request changes only tools/tests/test_pr_metadata.py. The added dependencies are standard-library runpy and sys, plus the existing test dependency pytest; the tests continue to …
Shared Semantic Neutrality	✅ Passed	PASS — The PR changes only tools/tests/test_pr_metadata.py, adding imports and regression tests for existing PR metadata CLI behavior. The diff contains no model-specific configuration, topology, te…
Benchmark Validation Integrity	✅ Passed	PASS: The pull request changes only tools/tests/test_pr_metadata.py. It adds tests for PR metadata CLI success and error paths. It does not change benchmark, performance, reference, metric, workload…
Shared Change Blast Radius	✅ Passed	The check is not applicable. The pull request changes only tools/tests/test_pr_metadata.py and adds regression tests. tools/pr_metadata.py, its workflow, and the pull-request template are unchange…

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

_{Comment @coderabbitai help to get the list of available commands.}