Skip to content

ctx-payload-guard: enforce the redirect-then-search anti-pattern on sandbox payloads (currently advisory) #90

Description

@halindrome

Problem

rules/ctx-rules.md documents two anti-patterns and then disarms them:

It is advisory — no PreToolUse hook enforces it on sandbox payloads; compliance depends on the agent following the guideline.

Compliance does not hold. In a long session in a consumer repo, an agent following these
rules perfectly wherever a hook blocked violated both advisory rules repeatedly.

The observed split is sharp:

Rule Enforcement Outcome over one ~4h session
Use CMM, not grep, for code search cmm-grep-nudge (blocks) never violated
Route output through ctx_execute ctx-execute-enforcer (blocks) never violated
Don't truncate/redirect-then-search captured output prose, explicitly advisory violated repeatedly

Hook-backed rules held under load. Prose-only rules degraded exactly when the session got
long and the agent got busy — which is when they matter most.

What it costs

The failure mode is not context bloat, it is lost evidence and wrong conclusions.

A mutation-testing harness was written as:

( cd "$ISO" && bash test/suite.sh >"$ISO/out.txt" 2>&1 )   # redirect: nothing indexed
grep -aE "$guessed_pattern" "$ISO/out.txt"                  # search for a predicted string
rm -rf "$ISO"                                               # evidence destroyed

Three separate harms, all of which landed:

  1. Nothing was indexed. stdout was empty, so intent= went inert and ctx_search
    could not answer anything about the run afterwards.
  2. The pattern was guessed. One mutation reported 699 passed, 2 failed; the grep
    matched only one of the two. The second failure was never seen — and because the
    isolated tree was deleted per iteration, it was unrecoverable.
  3. A full re-run was required (~4 min per suite run, several times) to recover
    information that would have been free had stdout reached the index.

This is the exact shape ctx-rules already warns about — "a grep that misses is not a check
that found nothing, it is a check that never ran" — reproduced by an agent that had that
rule in context the whole time.

Proposal

Extend hooks/project/ctx-payload-guard.sh (or a sibling) to inspect the code payload of
ctx_execute / ctx_execute_file / ctx_batch_execute and block, with the existing
enforcer's remediation style, when it finds:

  1. Redirect-then-search: a > / >> to a path, and a later grep/awk/sed/rg
    reading that same path in the same payload. Remediation: drop the redirect and let
    stdout hit the index; pass intent=; or tee if the file must persist.
  2. Pipe-to-truncate: | head -N, | tail -N, | grep -m N, | sed -n '1,Np',
    | awk 'NR<=N'. Bare head file / tail file stays exempt, as ctx-rules already
    carves out.

Suggested severity: block #1 (silent evidence loss), warn on #2 (already partly covered by
existing guidance and has legitimate uses).

Escape hatch consistent with the rest of the stack: an inline # ctx-raw-ok marker for the
genuine case where a file must persist for a later command.

Acceptance criteria

  • A payload that redirects a command's stdout to a file and greps that file is blocked, and
    the message names intent= and tee as the two correct alternatives.
  • cmd | tee f followed by a grep on f is not blocked (stdout still reaches the index).
  • head somefile / tail somefile are not blocked.
  • # ctx-raw-ok suppresses the block.
  • rules/ctx-rules.md is updated: the "It is advisory — no PreToolUse hook enforces it"
    sentence becomes a statement of what is now enforced and what remains advisory. That
    sentence currently reads as permission.

Note on scope

The related failure in the same session — an agent writing "an earlier version did X"
narration into instruction files — is not from this repo's rules; no such rule exists
here or in the consumer repo. That one has been fixed locally by writing the rule down. It
is mentioned only to make clear this issue is scoped to the two ctx anti-patterns above.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions