Skip to content

feat(cli): add --json output for agent readers - #324

Closed
Jamie-BitFlight wants to merge 27 commits into
mainfrom
claude/skilllint-json-flag
Closed

Jamie-BitFlight wants to merge 27 commits into
mainfrom
claude/skilllint-json-flag

Conversation

@Jamie-BitFlight

Copy link
Copy Markdown
Contributor

Stacked on #312. The base is claude/pr-289-rebase-conflicts-xuyr0w, so the diff shows only this change (14 commits). Retarget to main after #312 merges.

What this adds

An additive --json option on the commands agents run. With --json, stdout is exactly one line of compact JSON built from frozen Pydantic models (responses.py, Rich-free), diagnostics go to stderr, and no Rich-rendered output reaches either stream.

  • Commands: check (including --tokens-only), rules, rule, the six docs commands, and --version at the root. The flag is per command. A root --json is valid only with --version.
  • Exit codes: results (0 or 1) are JSON on stdout. Usage errors (2) are plain text on stderr with empty stdout. Under --json, check with no paths or a bad path no longer prints help to stdout and exits 2.
  • check JSON carries an omitted block (passed_files, passed_validators, info_issues, retrieve_with) that names the flags restoring what was left out.
  • docs sections, docs section and docs verify report file_exists, so a missing file differs from an empty one.
  • --record keeps working. With --json the file is written first, then the JSON carries record_path (absolute, resolved). A failed write exits 2 with nothing on stdout.
  • A non-UTF-8 path prints as ? under --json, the same way the text path shows it.
  • Rich is declared as an explicit runtime dependency (rich>=15.0.0). The built wheel imports it in cli_docs, output, plugin_validator, record_export and reporting.

What does not change

Without --json, stdout, stderr and exit codes stay byte-identical to the base (2edc05d), apart from two differences accepted up front:

  • --help screens gain one --json option line.
  • A mistyped option such as check --jsn now ends with (Possible options: --json). This is Click's suggestion for the new option, and a test pins it.

action.yml, .pre-commit-hooks.yaml and other callers are not edited, because nothing in the repository passes --json. --no-color, --show-summary and the reporters are untouched.

Evidence

Test-first. f97babe adds 266 golden outputs (106 cases, captured from the real executable in nine environment profiles), a test that replays them, 125 strict-xfail --json probes, and a parity test between the text and JSON visibility rules. The goldens are unchanged since that commit. Each later commit removes the xfail markers it satisfies.

Reported by the implementing agent, run on Linux in a worktree:

  • uv run prek run --all-files passes.
  • uv run pytest: 2,450 passed, 14 skipped.
  • A built wheel has Requires-Dist: rich>=15.0.0 and scripts/assert_installed_artifact.py --wheel exits 0.

An independent review ran the base and this branch side by side over about 30 default-path invocations. It found no stdout or exit-code difference. It found the mistyped-option suggestion above and the non-UTF-8 crash, which are fixed in 0fb71c0 and a5fbaff.

I ran --version --json, check <invalid SKILL.md> --json and check --json by hand. They gave one JSON line, the failed validator with its issues and an omitted block, and exit 2, as expected.

Not verified

  • No GitHub workflow has run on this branch yet.
  • The sdist, macOS and Windows are untested. The non-UTF-8 test needs a file system that accepts such names.
  • check --json is not measured on the 1,000-skill fixture.
  • docs verify <directory> --json still ends in an IsADirectoryError traceback, exit 1, which is the existing behaviour tracked in docs sections crashes with a traceback when given a directory #315.

Known follow-ups, not in this PR

🤖 Generated with Claude Code

https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ


Generated by Claude Code

claude and others added 21 commits September 30, 2026 13:41
`check --help` wrapped and truncated the platform choices line in a
narrow terminal (40 columns cut "--platform" and the choices list).
Follow the daily-releases scripts' Typer pattern: set
context_settings={"terminal_width": 800} and rich_markup_mode=None on
the root app so Click renders plain, unwrapped help. Rich help sizes
itself from TERMINAL_WIDTH/COLUMNS and ignores terminal_width, so both
settings are needed. Tighten the platform help test to assert the full
choices line under COLUMNS=40.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ
Complete the Typer app pattern used by the daily-releases,
receiving-pr-reviews and create-merge-request-changelog skill scripts
in claude_skills: pretty_exceptions_enable=False alongside the fixed
terminal_width and rich_markup_mode=None, so tracebacks are plain and
not wrapped to the terminal width either.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ
State why terminal_width is fixed, how Click uses it
(typer/_click/formatting.py), and that 800 matches the Typer apps in
claude_skills, per the repository's no-invented-constraints rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ
Address review findings on the fixed-width help change:

- Click re-wraps help paragraphs, so Args:/Raises: docstring sections ran
  together on one line once rich_markup_mode=None applied. Mark those
  paragraphs with Click's \b no-rewrap marker (D301 noqa: ruff's autofix
  would add an r prefix and silently break the marker).
- The root app's rich_markup_mode=None governs every sub-app, so
  docs_app's rich_markup_mode="rich" and the rich_help_panel arguments
  did nothing. Help output is byte-identical across all 11 screens
  without them; remove them.
- State the real default help width, including its 50-column floor
  (typer/_click/formatting.py), in the terminal_width comment.
- Test that every help screen (root, check, rule, rules, docs, docs
  fetch) renders identically at 40 and 200 columns, which also covers
  the docs sub-app inheriting terminal_width, and that each docstring
  section keeps its own lines.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ
…ich off

Typer's CliRunner forces the help width to 80 columns while it runs, so
the width tests passed even with terminal_width removed (mutation check:
0 of 6 screens failed). Call the app directly with COLUMNS set so the
real width path runs; dropping terminal_width and re-enabling Rich now
each fail 7 tests.

Also state rich_markup_mode=None on docs_app instead of leaving the
Typer default ("rich") and relying on the root app to override it. The
root app governs every sub-app, so output is unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ
Replace fixed-width help reflow with complete-content Typer formatting.
Measure Rich tables and panels from content, preserve raw docs and literal
diagnostic values, and size SVG recordings from their captured lines.

Verify long fields, nested Markdown, row context, narrow terminals, and
TERM=dumb with forced color. Full suite: 1936 passed, 14 skipped.
Step 0 of the additive --json migration, tests only. No product source changes.

- Golden baselines of every command's default output (stdout, stderr,
  merged 2>&1, exit code, written files), captured from the real
  skilllint executable under baselines/. Core cases also run under five
  environment profiles, three widths and two pty runs.
- test_default_output_unchanged.py replays them. Only the --json option
  line in --help screens is tolerated, and only as one inserted block
  with nothing removed or changed.
- strict-xfail probes for the --json contract, each naming the migration
  step that flips it and failing today with "No such option: --json".
- the text/JSON visibility parity test, red until skilllint.responses
  exists.
- cli_probe.py: subprocess harness with pinned environment, pty raw
  mode and the two-line shallow-clone warning filter.
- pyproject.toml: add the tests directory to ty's extra-paths, so the
  type gate resolves the sibling helper modules the way pytest does.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ
The built wheel imports rich at runtime (cli_docs, output, plugin_validator,
record_export, reporting) but its METADATA listed no Requires-Dist: rich; it
arrived only through typer. Move it from the dev group to the project
dependencies with scripts/uvu.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ
Add skilllint.responses: frozen, extra-forbid Pydantic models for every
planned --json response, pure builders that read the same typed objects the
reporters read (FileResults, RuleEntry, the vendor_cache results), and
emit_response, the single stdout writer. The module imports no Rich.

vendor_cache gains find_section, which returns the matched section and its
text; read_section now delegates to it with unchanged behaviour.

No command uses the module yet. Schema snapshots, Hypothesis round trips and
builder tests cover it; the check visibility parity test no longer expects
the module to be missing.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ
make_recording_console gains an optional file argument, default stdout. A
buffer lets the --json path render for --record without anything reaching
the terminal. Tests show the exported HTML and SVG are byte-equal for
stdout and for a buffer, for the check, rules and rule renderings.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ
fetch, fetch-authorities, latest, sections, section and verify print one
compact JSON line on stdout with --json. Cache status moves from a stderr
line into the status field, fetch-authorities reports every URL, and
sections, section and verify gain a JSON-only file_exists so a missing file
is distinguishable from an empty one. Results exit 0 or 1; no Rich output is
created on this path.

cli_json holds the shared --json option and the emit-and-exit helper. The
docs probes in test_cli_json_contract no longer expect failure.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ
rules and rule print one compact JSON line with --json. With --record the
usual rendering goes into an in-memory buffer, the file is written first and
only then the response names it as an absolute, resolved record_path. A file
that cannot be written exits 2 with one plain stderr line and nothing on
stdout; an unknown rule exits 1 with a response and writes no file, as on the
text path.

The rules table and its footer move into _show_rules_report so the text and
the recorded --json rendering share one body. The rules and rule probes in
test_cli_json_contract no longer expect failure.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ
check prints one compact JSON line with --json, including --tokens-only.
Files, validators and issues are listed by the rule the text reporters use
and what is left out is counted in omitted, with the flags that restore it.
fixes lists what --fix applied. With --record the usual rendering goes into a
buffer through the same report_results the text path calls, the file is
written first, and the response then names it as record_path.

Under --json the two paths that print help on stdout (no paths, a path that
does not exist) print nothing there and exit 2: no paths raises Click's own
missing-argument error, a bad path keeps its stderr lines. Exit-2 runs still
write the --record file, as on the text path.

scan_runtime.run_validation_loop keeps its name and signature and now calls
collect_validation_results and report_results, a verbatim split of its
body. Counting tokens moves into _count_body_tokens, and main's argument
checks into _require_usable_paths, which keeps main under the complexity
limit. The check probes no longer expect failure. The tokens-only batch probe's
oracle now reads the bare integers that explicit file arguments print.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ
skilllint --version --json prints {command, name, version} as one JSON line,
in either option order and with -V. At the root --json is valid only together
with --version; on its own, or before a subcommand, it is a usage error (exit
2, stderr only) so it can never be a silent no-op. The remaining probes in
test_cli_json_contract no longer expect failure, and the xfail machinery is
removed with the last of them.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ
record_path lived on a shared base class, so it was the first key of the
four responses that carry it. Declare it last on each so the line starts with
the command that produced it, as every other response does. The key set and
the schema snapshots are unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ
Add --json to the usage guide's output section and to the agent skill's scan
step: one compact JSON line on stdout, exit status 0 and 1 as results, usage
errors on stderr only, record_path with --record, and the root option valid
only with --version.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ
ctx.fail gives the same stderr and exit status as Click's MissingParameter
(compared on the real CLI: byte-identical stderr, exit 2, empty stdout), so
the private typer._click import and the file-wide PLC2701 ignore are not
needed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ
…ows them

A path or argument that is not valid UTF-8 reaches Python as a lone
surrogate, and model_dump_json() raised PydanticSerializationError on it:
exit 1 with an empty stdout, indistinguishable from a failed validation. The
default output prints such a path with ? because stdout replaces what it
cannot encode. emit_response now builds the same compact line with json and
applies that replacement, so the JSON carries sk?/SKILL.md and the command
exits with its real status. Output for valid UTF-8 is byte-equal to
model_dump_json(), which a test pins. Tests cover check, --fix,
--tokens-only, docs file arguments, a docs heading query and an unknown rule id.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ
Click's close-match hint can now name --json for a mistyped option on check
and rules (No such option: --jsn (Possible options: --json)). The user
accepted this stderr change; no golden covers it. A test pins exactly that
line, with exit 2 and an empty stdout, and docs/usage.md notes it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ
docs sections, section and verify report file_exists under --json, the only
way to tell a missing file from an empty one, since the text output treats
both the same.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K5rAHJfvZyEEaUQghV7hCQ
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@coderabbitai

coderabbitai Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 7eccd29f-375c-49b8-bda8-4770fb06a540
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copy link
Copy Markdown
Contributor Author

Applied the follow-up review improvements on this branch:

  • --show-progress --json now exposes passed validator records instead of reporting an omitted count with no restoring flag.
  • JSON serialization now preserves POSIX surrogateescape path identity via JSON escapes; distinct non-UTF-8 filenames no longer collapse to the same ? path.
  • Added a collision/round-trip regression test for distinct non-UTF-8 paths.
  • docs sections|section|verify <directory> --json now classifies the directory as a usage error (exit 2, stderr, empty stdout) instead of leaking an IsADirectoryError traceback.

I am not merging this PR yet. Its head is still not mergeable with current main, and GitHub has not produced fresh workflow runs for the amended head. Current main has also moved through #321/#322/#323/#329, including overlapping changes in plugin_validator.py, test_cli.py, pyproject.toml, and uv.lock. Those need a real reconciliation/rebase rather than forcing the stale 310-file branch over current architecture.

Copy link
Copy Markdown
Contributor Author

Superseded by #330, rebuilt from current main with the reviewed JSON-contract fixes and without the stale stacked/baseline history. Closing this PR so review and CI continue on the clean replacement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants