test: exterminate test-law violations; fix aliased/set model commands - #200
Merged
Merged
Conversation
…dels Drop the ad-hoc _FakeSummaryStats/_FakeProjectFailureInfo classes (ban-test-fakes) and feed u.Cli.summary/project_failure real frozen m.Tests models that satisfy p.Cli.SummaryStats/ProjectFailureInfo. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- test_constants: delete tests that restate owned constants/defaults as literals or assert how authority tuples are derived; keep invariants (map coverage, glyph distinctness, template interpolation, classifiers). - test_base: delete isinstance-of-just-built tautology and tests of the test service scaffolding. - cli_service part_03: assert shared flags are preserved relative to the pre-invocation state instead of restating settings defaults. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…doubles The prompt tests subclassed FlextCliPrompts and overwrote private state (_input_reader, _password_reader, _test_env_override) and overrode _log (ban-test-private-access), coupling them to internals. - FlextCliPromptsSupport: input_reader/password_reader become public, dump-excluded DI ports (stdin/getpass by default); the test-only _test_env_override hook is removed. - conftest: one make_prompts fixture builds the real FlextCliPrompts with scripted ports; capture/failing-log doubles and their protocols deleted. - Deleted tests that asserted logging side effects or pinned test-env logging (behaviour outside the public result contract). - Hardcoded error strings replaced by the owning c.Cli constants. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The model_command tests read the generated signature via inspect.signature (construction, not behaviour). Rewritten to register the command in a real app and invoke it with arguments, they exposed two product defects the signature tests hid: - aliased fields: Typer passes options under the field name, so model_validate rejected them; validate with by_name=True. - set/frozenset fields resolved to bare set/frozenset, which Typer rejects at invocation (Type not yet supported); they now resolve to a repeated list option the request model validates into a set. Mapping-typed test fields that no Typer command can accept, their constants, and the FrameworkOption test protocol are deleted. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- test_protocols: deleted; it asserted how protocols are declared (runtime_checkable, namespace identity, x is x) using stub classes, and none of those protocols is checked at runtime by the product. - public contracts: drop inspect.signature probing, the isinstance of the returned component model, and the restated 'available' literal. - model_commands: drop callable(cmd) tautology (invocation is covered). - streamed process: prove no leaked threads by live-thread snapshot instead of matching the internal pump thread name; drop the isinstance of a just-built deadline model. - version/constants: drop arbitrary length bounds, determinism tautology, duplicated semver checks, and flext-core regex tests. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- prompt password tests take their secrets from scripted_password_pair instead of literals (ruff S106). - annotation invocation cases expect the directly constructed model, so set/frozenset cases type-check (pyrefly bad-assignment). - make fmt output adopted (including _json/_navigate.py wrapping). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
marlon-costa-dc
added a commit
that referenced
this pull request
Sep 26, 2026
…options; fail-loud prompt printing (#201) ## Summary Resolves the known residual from #200: a model field typed as a mapping became a `dict` Typer option, which Typer rejects at invocation, so the mapping branch of `u.Cli.field_default` never produced a usable command. Decision (Pydantic law: owned payloads are Pydantic two-way; JSON enters through Pydantic validation): a field without a native CLI form — a mapping, a nested model, or a collection/union that carries one — is exposed as **one string option carrying JSON**, validated by Pydantic into the field's declared type. ## Design - `u.Cli.is_json_option(annotation)` (new, `_options_parts/…part_01`) is the single decision owner. It strips aliases and `Annotated` (`u.Cli.unwrap_annotation`, shared with the resolver), resolves subscripted type-alias origins (`t.MappingKV[...]` → `Mapping`), and flags `Mapping` subclasses, `m.BaseModel` subclasses, and generics/unions carrying them. - `u.Cli.resolve_typer_annotation` returns `str` for those fields (the old `dict` branch is gone). - `u.Cli.framework_build_parameter(..., json_annotation=...)` sets the Typer `parser` and the `JSON` metavar. The parser builds `TypeAdapter(<declared type>)` **only at parse time** and runs `validate_json`; a Pydantic error becomes `typer.BadParameter` chained `from` the `ValidationError` (Click's parser hook otherwise drops a `ValueError`'s text). `cli.execute_app` returns it as `e.fail_validation` with the Pydantic cause; `cli.invoke_app` exits 2 before the handler runs. - `u.Cli.field_default` renders a JSON option's default (settings first, then model default/factory, including nested model instances) as JSON text through `u.to_json`; the old per-entry mapping normalization and the now-unused `t.Cli.MutableDefaultMapping` alias are removed. - Help appends `c.Cli.CLI_JSON_OPTION_HELP` to the field description and shows the `JSON` metavar and the JSON default. ## Fail-loud fix `FlextCliPromptsSupport._print_message` caught a logger failure and returned a formatted failed Result. The catch is removed so the failure propagates with its cause; the per-caller error templates that only fed that branch are removed from `prompts.py`. ## Tests (behavioural, public surface, no mocks) New `tests/unit/test_model_command_json_options.py` registers `cli.model_command` in a real app: - valid JSON mapping reaches the model; nested model and `list[model]` options reach the model; - omitted JSON options deliver model defaults, or settings values when a settings model is given; - malformed JSON (`json_invalid`) and wrong types (`int_parsing`) fail: `invoke_app` not succeeded and handler not called; `execute_app` is a failed Result carrying the Pydantic error type; - `--help` renders the JSON options with the `JSON` metavar. `test_options_public_cov` updated to the new contract (mapping resolves to `str`; mapping default is JSON text that decodes to the settings value). ## Validation (lane worktree `/home/marlonsc/flext-work/v8-cli-mapping/flext-cli`) - `make gen` ×2: exit 0 both, identical diff (fixed point). - `make check`: exit 0, `Total: 1 Success: 1 Failed: 0` (lint, pyrefly, mypy, pyright, silent-failure, security, … OK; duplication/codemod/boundary/namespace/runtime-census are SUSPENDED by the check owner under flext-itpd1.3). - `make test-full`: exit 0, incremental `78 passed`, full `1202 passed`, 0 failures. ## Residuals (not changed here) - `u.Cli.build_model_command` (`_utilities/model_commands.py` `Builder`) is a second model-command path that passes raw field annotations to Typer, so it still cannot handle mapping fields; it duplicates `cli.model_command`. - `u.Cli.field_default` still catches `c.EXC_VALIDATION_TYPE_VALUE` and maps a non-CLI default to `None`. - Other `CLI_SAFE_EXCEPTIONS` catches in `_prompts_support.py` (`_run_or_fail`) were not flagged and remain. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Fixes model commands with mapping fields — previously a `dict` Typer option that Typer rejects at invocation — and makes prompt printing fail loud. A model field without a native CLI form (a mapping, a nested model, or a collection/union carrying one) now becomes one string option with the `JSON` metavar. Pydantic validates the text into the field's declared type at parse time. - Malformed JSON or a schema mismatch is a usage error, and `execute_app` returns it with the Pydantic cause. - Defaults (settings first, then the model) render as JSON text; help shows the format marker and never builds an adapter. - `_print_message` no longer swallows logger failures — they propagate with their cause, and the per-caller error templates are removed. <sup>Written for commit dbcdb0e. Summary will update on new commits.</sup> <a href="https://cubic.dev/pr/flext-sh/flext-cli/pull/201?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
![https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source](https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source){kind=link}
![https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img](https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img){kind=link}
![https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a](https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a){kind=link}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Exterminates test-law violations in flext-cli (operator orders 2026-09-26): tests verify behaviour through public facades, never how code is built. Coverage floors are released; this PR does not add coverage for its own sake.
Test-law counts (inventory rules from flext-infra PR #875, same
sg scaninvocation astest-law-inventory.md)The sweep went past the inventory and also removed module-shape, signature-introspection, internal-name and owned-value assertions.
Deleted / rewritten tests
test_output_cov:_FakeSummaryStats/_FakeProjectFailureInforeplaced by frozenm.Tests.SummaryStats/m.Tests.ProjectFailureInfomodels.test_constants: deleted tests that restated owned constants and defaults as literals or checked how authority tuples are derived. Kept the invariants: map coverage, template interpolation, glyph distinctness, and the classifiers.test_base: deleted the isinstance-of-just-built tautology and the tests of test scaffolding.cli_service part_03: shared flags are now compared with their state before the invocation, not with literal defaults._input_reader,_password_readerand_test_env_overrideand overrode_log. Tests now build the realFlextCliPromptswith scripted input ports. Deleted the tests that asserted logging side effects. Hardcoded error strings now use thec.Cliconstants, and password tests take their secrets from the fixture.test_options_cov:inspect.signatureprobing is replaced by registering the command in a real app and callingcli.invoke_app. The rewritten tests cover alias, custom declarations, bool toggle, every annotation, a missing required option, the settings default and the model defaults.test_protocols: deleted. It checkedruntime_checkable, namespace identity andx is xwith stub classes, and the product never checks those protocols at runtime.callable()and determinism tautologies, arbitrary length bounds, and duplicate semver tests (including tests of the flext-core regex). Thread-leak checks now take a live-thread snapshot instead of matching the internal pump thread name.Product fixes (found by the behavioural rewrites; the signature tests passed while these were broken)
services/_cli_parts/flextclicli_part_01.py(_ModelCommand.__call__): Typer passes options under the field name, so an aliased field failed validation and the command exited non-zero. It now callsmodel_validate(kwargs, by_name=True)._utilities/_options_parts/flextcliutilitiesoptions_part_01.py(resolve_typer_annotation):set/frozensetfields resolved to a bareset/frozenset, and Typer rejects those at invocation withRuntimeError: Type not yet supported. They now resolve to a repeatedlist[...]option, which the request model validates into a set.services/_prompts_support.py:input_reader/password_readerare now public, dump-excluded DI ports (stdin and getpass by default). The test-only_test_env_overridehook is removed.Validation (lane worktree)
make gen×2: exit 0, clean tree both times (fixed point).make check: exit 0,Total: 1 Success: 1 Failed: 0.make test: exit 0.make test-full: exit 0,1195 passed. The baseline before this change was1265 passed; the difference is the tests deleted here.Known residual (not changed here)
dict, and Typer rejects it at invocation (Type not yet supported). It fails loudly, not silently, but it means the mapping branch ofu.Cli.field_defaultnever produces a command that can be invoked. This needs a product decision.🤖 Generated with Claude Code
Summary by cubic
Rewrites the
flext-clitests to verify behavior through public facades instead of implementation details, clearing the 10 test-law violations plus signature-probing, tautology, and constant-restating assertions. The behavioral rewrites exposed two product bugs the old signature-based tests hid; fixing them makes aliased and set-typed model commands invocable.Bug Fixes
_ModelCommand.__call__now callsmodel_validate(kwargs, by_name=True)so aliased fields pass validation (Typer passes options under the field name).set/frozensetfields now resolve to a repeatedlistoption, which the request model validates into a set; Typer previously rejected the bare set type at invocation.Refactors
input_reader/password_readerbecame public, dump-excluded DI ports onFlextCliPromptsSupport; the test-only_test_env_overridehook and the prompt subclass doubles were removed, and prompt tests now inject scripted readers into the real service.test_protocols.pyand assertions that restated constants, introspected signatures, or matched the internal pump thread name; streamed-process tests now rely on live-thread snapshots.Known residual: a mapping-typed model field still resolves to
dictand Typer rejects it loudly at invocation, so the mapping branch ofu.Cli.field_defaultcannot produce an invocable command; a product decision is needed.Written for commit cf528cf. Summary will update on new commits.