Skip to content

fix(cli): fail loud at the remaining CLI_SAFE_EXCEPTIONS sites; timing-free process tests - #203

Merged
marlon-costa-dc merged 2 commits into
0.12.0-devfrom
fix/cli-fail-loud-remaining
Sep 26, 2026
Merged

marlon-costa-dc merged 2 commits into
0.12.0-devfrom
fix/cli-fail-loud-remaining

Conversation

@marlon-costa-dc

@marlon-costa-dc marlon-costa-dc commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Follow-up to #202. It resolves two rule violations: the remaining catch-and-normalize sites (fail loud), and tests that measured host timing instead of behaviour.

1. Remaining CLI_SAFE_EXCEPTIONS catches

This is the same violation class #202 removed from prompts. CLI_SAFE_EXCEPTIONS has no consumers left, so it is deleted.

Site Before After
u.Cli.cmd_validate_settings (_utilities/cmd.py) ValueError/TypeError/KeyError became r.fail("Settings validation failed: …") No catch. Any exception escapes with its cause.
u.Cli.tables_resolve_config (_utilities/tables.py) Every listed type became r.fail with message only Only the model's ValidationError (the declared outcome) becomes r.fail(…, exception=exc). Everything else escapes.
u.Cli.tables_render (_utilities/tables.py) Renderer errors became r.fail_op("Table formatting", …) No catch. Renderer exceptions escape.
u.Cli.process_mapping (_utilities/validation.py) skip mode swallowed exceptions into a debug log and collect aggregated them Removed together with t.Cli.MappingProcessor. Nothing in the workspace calls it.
cli.apply_to_config (services/cli_params.py) Every listed type became r.fail with message only Only a CliParamsConfig ValidationError becomes r.fail(…, exception=exc). params_apply failures remain owner r.fails. Everything else escapes.

Constants removed because nothing uses them now: CLI_SAFE_EXCEPTIONS, ERR_SETTINGS_VALIDATION_FAILED, OUTPUT_TABLE_FORMATTING_OPERATION.

Consumers (grep of every member in the workspace)

  • process_mapping, MappingProcessor and CLI_SAFE_EXCEPTIONS have no consumers outside flext-cli.
  • cmd_validate_settings, tables_resolve_config, tables_render and apply_to_config are called only inside flext-cli (services/cmd.py, services/tables.py, _utilities/formatters.py, services/_cli_parts/flextclicli_part_02.py).
  • Other repos match only unrelated homonyms (u.Web.validate_settings, flext-core _validate_settings) or generated architecture docs.

2. Timing-sensitive runtime-process tests

  • test_completion_before_wake_clear_returns_promptly is now …_ends_monitoring.
    • The trace holds the monitor at wake.clear() until the waiter has set process_done and wake. It waits without a timeout; the old 2s cap is gone.
    • A monitor that lost that wake-up would sleep until its deadline and report timed_out. So not outcome.timed_out is the proof. The elapsed < 2 assertion is removed.
  • test_normal_root_exit_leaves_no_descendant:
    • The descendant writes its pid and process group to a pipe, then blocks on an Event.
    • The root does a blocking readline, records it, and exits.
    • The proof is that the pid and process group no longer exist when run_to_file returns.
    • Removed: file polling, the heartbeat, sleep(0.15), and the < 5s assertion.
    • Interpreters start with -I -S.
  • No timeout was raised.

Local gates (worktree fix/cli-fail-loud-remaining)

  • make gen twice: exit 0 both times, and the diff hash is the same after each run (fixed point).
  • make check: exit 0 (Total: 1 Success: 1).
  • make test-full:
    • Run 1: exit 2, 1 failed, 1192 passed, at load average ~25. The failure was test_file_tools_yaml.py::test_single_source_returns_requested_model, blocked in os.fsync for 36s during atomic_write_text_file (host I/O stall). It is not a test changed here. Both rewritten tests passed in that run.
    • Runs 2 and 3: exit 0 each, 1193 passed, at load average ~20. Both rewritten tests passed.

🤖 Generated with Claude Code


Summary by cubic

Follow-up to #202: makes the remaining exception-handling sites fail loud and rewrites two runtime-process tests so they assert behavior, not the host clock. Exceptions these call paths used to normalize into generic failures now escape with their cause; only the Pydantic ValidationErrors the callers declare as outcomes are still reported as failures, and callers that relied on the old normalized messages now see the underlying exception.

Fail loud at remaining catch sites

  • cmd_validate_settings and tables_render no longer catch anything; renderer errors now propagate instead of becoming r.fail messages.
  • tables_resolve_config and apply_to_config only report the model's ValidationError as a failure and now attach it via exception=exc; params_apply failures remain the r.fail results their owners produce.
  • Removes process_mapping and MappingProcessor; their skip mode swallowed exceptions into a debug log and collect mode aggregated them, and nothing in the workspace calls them.
  • Deletes the now-unused CLI_SAFE_EXCEPTIONS, ERR_SETTINGS_VALIDATION_FAILED, and OUTPUT_TABLE_FORMATTING_OPERATION constants.

Timing-free runtime-process tests

  • The completion test holds the monitor at its first wake.clear() until the root waiter has set process_done and wake, so a lost notification would surface as timed_out; the elapsed < 2 assertion, the child sleep, and the 2s wait caps are gone.
  • The descendant test has the child announce its pid and process group over a pipe and then block forever; the root reads the line and exits, and the test asserts the pid and process group no longer exist when run_to_file returns. File polling, the heartbeat, sleep(0.15), and the < 5s assertion are removed.
  • Both scenarios start interpreters with -I -S.

Written for commit 8280536. Summary will update on new commits.

Review in cubic

Marlon Costa and others added 2 commits September 26, 2026 10:07
Remove every remaining catch-and-normalize over CLI_SAFE_EXCEPTIONS
(ValueError, TypeError, KeyError), the same violation class #202 removed
from prompts, and delete the constant now that nothing uses it.

- u.Cli.cmd_validate_settings: no catch. The structure probe raises none of
  those types; any exception it raises now escapes with its cause.
- u.Cli.tables_resolve_config: only the model's ValidationError is the
  declared failure outcome; the result carries it (exception=exc). Anything
  else escapes.
- u.Cli.tables_render: renderer exceptions escape instead of becoming
  r.fail("Table formatting ...").
- cli.apply_to_config: only a CliParamsConfig ValidationError is a declared
  failure and the result carries it; params_apply failures stay r.fail from
  their owners; every other exception escapes.
- u.Cli.process_mapping: removed with t.Cli.MappingProcessor. Its skip mode
  swallowed exceptions into a debug log and its collect mode aggregated
  them; no member of the workspace calls it.
- Removed the constants only the removed catches used:
  CLI_SAFE_EXCEPTIONS, ERR_SETTINGS_VALIDATION_FAILED,
  OUTPUT_TABLE_FORMATTING_OPERATION.

Tests that only asserted process_mapping's normalization were removed. The
tables and params tests now assert the failure carries the ValidationError.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Both runtime-process tests asserted wall-clock thresholds and failed under
host load (3.09s against a 2s limit, and the 10s pytest timeout, at load
~9.8). The thresholds measured the host, not the behaviour.

- test_completion_before_wake_clear_ends_monitoring (was ..._returns_promptly):
  the trace holds the monitor at its first wake.clear() until the root
  waiter has set process_done and wake (unbounded Event.wait, no 2s cap).
  A monitor that lost that notification would sleep to its deadline and
  report timed_out, so `not outcome.timed_out` is the proof. The elapsed <2s
  assertion is gone and the child only prints (no sleep), started with
  -I -S.
- test_normal_root_exit_leaves_no_descendant: the descendant announces its
  pid and process group over a pipe and blocks on an Event forever; the root
  does a blocking readline, records it, and exits. No file polling, no
  heartbeat, no sleep(0.15), no elapsed <5s check. The proof is that the
  descendant pid and its process group no longer exist when run_to_file
  returns. Interpreters start with -I -S so the scenario does not pay
  site-packages startup.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

@marlon-costa-dc
marlon-costa-dc merged commit 0f6d0b3 into 0.12.0-dev Sep 26, 2026
7 of 9 checks passed
@marlon-costa-dc
marlon-costa-dc deleted the fix/cli-fail-loud-remaining branch September 26, 2026 13:17
marlon-costa-dc added a commit that referenced this pull request Sep 26, 2026
…act (#204)

## Summary

- flext-core PR #504 (merged `8de52fa6b`) removed the `on_error`
parameter from `u.process` (`FlextUtilitiesCollection.process`). The
first processor failure now returns `r.fail` carrying the processor's
original `exception`, with no skip/collect mode.
- `tests/unit/test_utilities_cov.py:37` dropped the removed
`on_error="fail"` kwarg (no behavior change — `"fail"` was already the
only real mode).
- `tests/unit/test_utilities_cov.py:43` (the `on_error="skip"` test) is
deleted — that behaviour no longer exists.
- Added
`test_process_fails_loud_with_original_exception_and_stops_at_first_failure`,
a behavioural test through the public `u` facade proving the run stops
at the first failing item (later items are never visited) and
`result.exception` is the processor's original raised exception.
- `src/flext_cli/_utilities/validation.py` and
`tests/unit/test_cmd_runtime_validation_branch_cov.py` were checked for
leftover `process_mapping` modes from flext-cli PR #203 — none found;
that cleanup already landed on the tip.
- `make upg` re-resolved the lock, pinning `flext-core` to `8de52fa6b`
(the exact PR #504 merge commit); `make gen` regenerated the managed
projections that follow from that lock bump (docs, Makefile,
pyproject.toml, mkdocs.yml, a couple of generated catalog files).

## Evidence (worktree
`/home/marlonsc/flext-work/v8-process-consumers/flext-cli`)

- `make upg`: exit 0 — lock pins `flext-core @
git+...@0.12.0-dev#8de52fa6b117ef54a840ac655c4518aff26ba027`.
- `make setup`: exit 0.
- `make gen` x2: exit 0 both times; identical working-tree diff after
the second run (fixed point).
- `make fix`: exit 0, 0 errors.
- `make fmt`: exit 0, 1 file reformatted (unrelated cosmetic reflow in
`tests/unit/test_runtime_process_descendants.py`).
- `make check`: exit 0, `Total: 1 Success: 1 Failed: 0 Skipped: 0`, 15
gates at 0 errors (duplication/codemod/boundary/namespace/runtime-census
gates suspended by operator authority flext-itpd1.3).
- `make test`: exit 0, `1193 passed` — includes
`test_process_fails_when_processor_raises`,
`test_process_fails_loud_with_original_exception_and_stops_at_first_failure`,
`test_process_predicate_excludes_items_before_processing`, all passing.
- Runtime proof, direct `u.process` call through the installed flext-cli
package (not just pytest):
  ```
  success: [10, 5, 2]
failure ok? True exception= ZeroDivisionError('integer division or
modulo by zero') error= Processing failed for item 0: integer division
or modulo by zero
  ```

## Test plan

- [x] `make check` green (15 gates, 0 errors)
- [x] `make test` green (1193 passed)
- [x] Runtime proof of `u.process` fail-loud contract through the
flext-cli public `u` facade

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Rewires `u.process` test consumers to the fail-loud contract from
flext-core PR #504, which removed the `on_error` parameter. The first
processor failure now returns `r.fail` carrying the processor's original
exception, with no skip/collect mode.

- Dropped the removed `on_error="fail"` kwarg — no behavior change,
since `"fail"` was the only real mode.
- Deleted the `on_error="skip"` test, as that mode no longer exists.
- Added a behavioral test asserting the run stops at the first failure,
never visiting later items, and that `result.exception` is the
processor's original exception.
- Pinned `flext-core` in `uv.lock` to the PR #504 merge commit and
regenerated the managed projections (docs, Makefile, pyproject.toml,
mkdocs.yml).

<sup>Written for commit 62b3156.
Summary will update on new commits.</sup>

<a
href="https://cubic.dev/pr/flext-sh/flext-cli/pull/204?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>

<!-- End of auto-generated description by cubic. -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant