Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 3 additions & 10 deletions apps/cli/src/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -623,20 +623,13 @@ try {
let shouldLaunchMain = true;
const parsedInteractiveArgs = parseArgs(compatibility?.args ?? stepCodeArgs);
if (
parsedInteractiveArgs.completionReview &&
!parsedInteractiveArgs.completionCheck &&
!parsedInteractiveArgs.help &&
!parsedInteractiveArgs.version
) {
throw new Error("--completion-review requires --completion-check git-committed");
}
if (
parsedInteractiveArgs.completionCheck &&
(parsedInteractiveArgs.completionCheck || parsedInteractiveArgs.completionReview) &&
!parsedInteractiveArgs.help &&
!parsedInteractiveArgs.version &&
resolveAppMode(parsedInteractiveArgs, process.stdin.isTTY, process.stdout.isTTY) === "interactive"
) {
throw new Error("--completion-check requires print or JSON mode; use --print or --mode json.");
const flag = parsedInteractiveArgs.completionCheck ? "--completion-check" : "--completion-review";
throw new Error(`${flag} requires print or JSON mode; use --print or --mode json.`);
}
const interactiveStartup = isStepInteractiveLoginStartup({
stdinIsTTY: process.stdin.isTTY,
Expand Down
12 changes: 12 additions & 0 deletions apps/cli/test/plan-review-tool-row.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,18 @@ describe("exit_plan_mode tool row", () => {
}
});

it("renders an external approval handoff without implying approval or clipping the plan", () => {
const component = createRow({ ...dismissed, outcome: "external_approval_required", planModeExited: true });
const out = rows(component);
expect(out.some((row) => row.includes("External approval required"))).toBe(true);
expect(out.some((row) => row.includes("Plan approved"))).toBe(false);
expect(out.some((row) => row.includes("# Plan: tank game"))).toBe(true);
expect(out.some((row) => row.includes("Delete the directory."))).toBe(true);
expect(out.some((row) => row.includes("to expand"))).toBe(false);
component.setExpanded(true);
expect(rows(component)).toEqual(out);
});

it("reports the approved and feedback outcomes too", () => {
const approved = rows(createRow({ ...dismissed, outcome: "approved" }));
expect(approved.some((row) => row.includes("Plan approved"))).toBe(true);
Expand Down
105 changes: 76 additions & 29 deletions docs/completion-check.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,48 @@
# Print-mode completion check
# Print-mode completion checks and task self-review

The Step CLI can perform a bounded completion check in the existing session:
The Step CLI can request a task self-review or a bounded Git completion check in
the existing session. Both features default off and support print and JSON modes:

```sh
step --print --completion-review "Complete the task."
step --mode json --completion-review "Complete the task."
step --print --completion-check git-committed --completion-check-attempts 2 "Complete the task and commit the changes."
step --mode json --completion-check git-committed "Complete the task and commit the changes."
step --mode json --completion-check git-committed --completion-review "Complete the task and commit the changes."
```

The feature is off unless `--completion-check git-committed` is supplied. The
attempts option counts **additional prompts**, defaults to 2, and accepts integers
1 through 3. Both flags accept `--flag=value` syntax. Attempts without the check,
unsupported values, interactive mode, RPC, and SDK stdio are rejected. Piped
print mode is supported. Direct `runPrintMode` callers can supply the same
`completionCheck` and `completionCheckAttempts` options. The boolean
`--completion-review` flag takes no value, requires `--completion-check
git-committed`, and defaults off. Direct callers can set `completionReview: true`.
`--completion-review` alone requests exactly one additional semantic review
prompt after the initial prompt, its images, and all additional user messages.
It does not require a Git repository, a new commit, or a clean worktree. The review
asks the same session and model to compare the work with the original visible
task, check task coverage, public interfaces and types, boundary cases and the
final diff, and confirm relevant tests and checks ran after the last edit. It asks
the assistant to run missing verification, fix issues, and provide a brief final
answer while preserving unrelated user changes and respecting permission denials.
It adds no requirement to commit. Original task instructions still apply.

The boolean review flag takes no value. Its standalone budget is fixed at **one
additional prompt**; `--completion-check-attempts` requires the Git check and cannot
extend standalone review. Direct `runPrintMode` callers can set
`completionReview: true` without a `completionCheck` option.

`--completion-check git-committed` enables the Git conditions described below.
`--completion-check-attempts` counts **additional prompts**, defaults to 2, and
accepts integers 1 through 3. Both value-taking flags accept `--flag=value` syntax.
Direct callers can supply `completionCheck` and `completionCheckAttempts` options.
Combining `--completion-review` with the Git check uses one slot of that existing
budget for review; it does not add a separate review call or increase the budget.

Attempts without the Git check, unsupported values, interactive mode, RPC, and
SDK stdio are rejected. Piped print mode is supported. Neither option creates a
new session or resets the invocation's time budget. A review is not a correctness
score, a grader, or independent verification; the canonical task verifier still
owns correctness.

Before binding extensions or sending the first prompt, the check requires a Git
worktree with an existing HEAD commit and saves that HEAD. It then sends the
initial prompt, its images, and all additional user messages in their original
order. Once those prompts finish, completion requires all of these conditions:
When the Git check is enabled, before binding extensions or sending the first
prompt, the check requires a Git worktree with an existing HEAD commit and saves
that HEAD. It then sends the initial prompt, its images, and all additional user
messages in their original order. Once those prompts finish, completion requires all of these conditions:

- `starting-HEAD..HEAD` contains at least one commit. A preexisting commit or
moving HEAD backwards is insufficient.
Expand All @@ -43,10 +65,10 @@ not separately to each user message. These prompts consume the original trial's
time budget; no trial timeout is extended or reset, and no new attempt is started.
The checker neither changes source files nor commits changes or runs hidden tests.

With `--completion-review`, the first eligible completion follow-up requests one
generic self-review even if the Git conditions and final text already pass. It
asks the same session and model to compare the work with the original visible
task, check public interfaces and types, boundary cases and the final diff, and
With both `--completion-check git-committed` and `--completion-review`, the first
eligible completion follow-up requests one generic self-review even if the Git
conditions and final text already pass. It asks the same session and model to
compare the work with the original visible task, check public interfaces and types, boundary cases and the final diff, and
confirm relevant tests and checks ran after the last edit. It asks the assistant
to fix issues, commit remaining task changes, and provide a final answer while
preserving unrelated user changes and permission denials.
Expand All @@ -60,12 +82,24 @@ optional experiment, not a grader; its benefit to task scores is unproven. It
introduces no hidden tests, external grading feedback, automatic commits, or
independent correctness judgment.

A successful `exit_plan_mode` tool result with structured details containing
`outcome: "external_approval_required"` is a handoff to the caller, not permission
to execute the plan. The headless result also carries `planHash` and
`planModeExited: true`. This handoff suppresses automatically generated review or
completion feedback for the rest of the invocation. Explicit caller messages,
including approval, cancellation, refinements, or new requests, are still delivered
in their original order, subject to the terminal failure and cancellation guards
below. The automatic guard remains sticky even after those messages: completion
control does not infer approval from prose or an arbitrary non-error plan-exit
call. Ordinary recoverable tool errors retain their existing behavior and do not
trigger this guard.

An explicit terminating tool denial, or an assistant error/abort observed during
this invocation, prevents further prompts from the checker, including when a
native retry subsequently succeeds. Pending user messages also stop at such a
terminal outcome when the check is enabled. Native provider retry policies are
unchanged. Explicit runtime session replacement continues to rebind listeners and
extensions, but the checker does not carry automatic feedback into another
terminal outcome when either completion feature is enabled. Native provider retry
policies are unchanged. Explicit runtime session replacement continues to rebind
listeners and extensions, but the checker does not carry automatic feedback into another
session or working directory.
These rules also suppress review, including after an error recovered by native
retry. If cancellation or replacement occurs while waiting for stdout, a planned
Expand All @@ -76,20 +110,33 @@ code **0** even if Git conditions remain unsatisfied. The canonical task verifie
owns the score; an ordinary failed task must not become an infrastructure error
that resamples the attempt. Missing/thinking-only final output returns **2** with
an explicit incomplete diagnostic. Existing terminal denials and final assistant
errors keep exit code **1**. Invalid configuration or failed Git preflight returns
**1** before a model call. If Git becomes unreadable after the model runs, the
checker stops adding prompts and reports that state; final text still returns 0,
and missing final text returns 2.
errors keep exit code **1**. A successful external-approval handoff produced by the
latest prompt keeps exit code **0** even when the last message is its tool result.
An earlier handoff does not excuse missing output from a later explicit prompt:
that prompt must provide final text or a fresh handoff, otherwise the exit code is
**2**. Automatic follow-ups stay blocked for the rest of the invocation.
Invalid configuration or failed Git preflight returns **1** before a model call.
If Git becomes unreadable after the model runs, the checker stops adding prompts
and reports that state; final text still returns 0, and missing final text returns 2.

Text stdout contains only the last assistant answer. Diagnostics use stderr.
JSON mode keeps the ordinary session event stream, including the added user
prompts, and adds `completion_check` events. Each successful inspection includes
`check`, `attempt` (follow-ups already used, starting at 0), `maxAttempts`,
prompts, and adds `completion_check` events. For the Git check, each successful
inspection includes `check`, `attempt` (follow-ups already used, starting at 0), `maxAttempts`,
`hasNewCommit`, `hasCommittedChanges`, `trackedDirty`, `untrackedFiles`, `hasFinalText`, `status`
(`passed`, `follow_up`, or `exhausted`), and `willFollowUp`. A failed inspection
emits `status: "unavailable"` and `willFollowUp: false`. No filenames, file
contents, diffs, commit messages, or Git stderr appear in check feedback/events.

Standalone review emits `completion_check` events with `check: "task-review"`,
`maxAttempts: 1`, `attempt`, `hasFinalText`, `status`, and `willFollowUp`. It includes
no Git-state fields. The status is `follow_up` when scheduling review, `completed`
when final assistant text is present after the single review opportunity, or
`exhausted` when final text is still missing. `completed` records the end of this
bounded flow, not successful verification or a correct task result. It does not
prove that an extension allowed delivery of the review; use `review.sent` for that.
These events contain no correctness score.

Only when review is enabled, completion events also contain
`review: { requested: boolean, sent: boolean }`. `requested` becomes true when the
first eligible inspection schedules review. `sent` becomes true only when the
Expand All @@ -107,8 +154,8 @@ messages rather than the number of events. Subsequent inspections retain the
review state. With review disabled, these fields and the receipt are absent and
the existing event shape is unchanged.

Git is invoked directly with fixed argument arrays, no shell, and only a
validated starting object ID as a variable argument. Reads use `rev-parse`,
Only `--completion-check git-committed` invokes Git. It uses fixed argument arrays,
no shell, and only a validated starting object ID as a variable argument. Reads use `rev-parse`,
`rev-list --max-count=1`, `diff --quiet <starting-HEAD> HEAD --`, and NUL-delimited
porcelain `status --no-renames` with normal untracked-directory reporting. The
tree diff disables external diffs, text conversion, and rename detection. Only
Expand Down
100 changes: 100 additions & 0 deletions docs/plan-approval.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# Plan approval handoff

`exit_plan_mode` submits a written proposal for review. Leaving plan mode and
approving the proposal are separate events. In headless and RPC sessions, a valid
proposal exits plan mode and restores the previous tools, while the caller remains
responsible for obtaining approval before execution. This also applies to RPC
contexts with `hasUI: true`; their UI bridge does not provide interactive approval.

Successful headless and RPC exits return the existing model-facing text plus
structured `details`, for example:

```json
{
"outcome": "external_approval_required",
"approvalSource": "external",
"planFilePath": "/workspace/.stepcode/plans/session-example.md",
"planContents": "# Plan\n\n1. Update the parser.\n2. Run focused tests.\n",
"planHash": "64a6a6c86ef0197c43f9bfc081792cca8d33807fbe33390d2439308a790f7520",
"planModeExited": true
}
```

`outcome: "external_approval_required"` identifies a handoff awaiting the caller's
decision. `planModeExited: true` describes the resulting mode state; it does not
mean approval was granted. The result still has `terminate: false`, so the host
must gate execution through its own approval flow. These details do not add an
approval command or enforce an execution barrier.

`approvalSource` identifies who owns the decision: `"user"` for native interactive
review, including feedback and dismissal, and `"external"` for the headless/RPC
caller's approval flow. It does not indicate that approval has occurred. Source
and outcome are independent: a future caller-owned `"approved"` outcome can keep
`approvalSource: "external"` without claiming the user clicked the TUI. Native
headless/RPC exits currently emit `"external_approval_required"` only.

`planHash` is the lowercase hexadecimal SHA-256 digest of `planContents` encoded
as UTF-8. It covers the complete snapshot read from the file, including any BOM,
line endings, leading or trailing whitespace, and the end of a long proposal.
Nothing is trimmed, normalized, or truncated before hashing. The path identifies
the source file; the hash identifies the returned contents, even if that file
later changes.

A caller can verify the returned snapshot before presenting it for approval:

```js
import { createHash } from "node:crypto";

const details = toolResult.details;
if (details?.outcome === "external_approval_required") {
const expectedHash = createHash("sha256")
.update(details.planContents, "utf8")
.digest("hex");
if (details.planModeExited !== true || details.planHash !== expectedHash) {
throw new Error("Incomplete or inconsistent plan handoff");
}
// Present details.planContents and bind the caller's decision to details.planHash.
// Allow execution only after that caller-owned approval flow succeeds.
}
```

A caller-owned extension can use the existing `tool_result` hook to apply its
approval policy before the result reaches the runner. After that policy actually
approves the snapshot, it can retain the hash and contents, set
`outcome: "approved"` with `approvalSource: "external"`, and deliver an explicit
caller-approval message to the model. Merely changing metadata is not a substitute
for obtaining approval. Execution-time permission checks continue to apply.

This also lets an opted-in completion check run after a caller-approved plan.
Once an unapproved handoff has already reached the runner, automatic feedback
stays suppressed for that invocation; explicit caller messages are still delivered.
The runner does not infer an approval decision from their wording.

If the proposal changes, obtain a new review and approval for the new hash. A
matching hash identifies content; it is not an approval token.

Native outcomes keep their existing behavior:

| Outcome | `approvalSource` | `planModeExited` | Behavior |
| --- | --- | --- | --- |
| `approved` | `user` | `true` | The user selected execution; restore tools and continue. |
| `feedback` | `user` | `false` | Keep planning and steer the user's notes into the next response. |
| `dismissed` | `user` | `false` | Keep planning and terminate the turn to wait for the user. |
| `external_approval_required` | `external` | `true` | Headless/RPC handoff; the caller must gate execution. |

The TUI still selects execution by default, and Enter confirms that selection.
Interactive result details also include the hash of the reviewed snapshot.
Transcripts display external handoffs as **External approval required**, with the
complete plan, rather than displaying an approved result.

Missing, empty, whitespace-only, unreadable, and non-regular plan files retain
their existing explanatory text and leave plan mode active. Their `details` are
`undefined`; they never receive a successful handoff outcome or hash. Calling
`exit_plan_mode` outside plan mode likewise returns no handoff details. A
non-error tool result alone is therefore insufficient to recognize a handoff.

`approvalSource`, `planHash`, and `planModeExited` are optional in `PlanReviewDetails`
for compatibility with existing producers and older transcripts. Native results
with a valid plan populate all three fields. Consumers should handle their absence
explicitly and must not infer approval or its source from missing fields, or infer
approval from plan-mode exit alone.
Loading
Loading