Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,14 +175,14 @@ Before proposing changes to skill design, workflow philosophy, or architecture,

若无法确定仓库,或 GitHub 权限/工具不可用:明确说明尚未创建或更新 issue 及原因。不要声称已完成记录。

### Grilling 共识:记 Issue,不落盘 spec
### Grilling 共识:Issue 跟踪(本仓库开发习惯)

Issue 与看板跟踪是 **本仓库** 开发习惯,不是 grilling skill 的能力。插件用户以 [`skills/grilling/SKILL.md`](skills/grilling/SKILL.md) 的 HANDOFF 为准;不要用本文件替代或覆盖 skill 里的交接步骤,也不要把 Issue/Projects 跟踪写进 skill。

当 grilling(或同等「逐题问清楚」)与人类伙伴达成共识后:

1. 把共识记到本仓库 GitHub issue:背景、已做决定、未决问题、后续跟进。用户已授权创建/更新 issue,无需每次再确认。先搜索 open 和 closed issues;已有对应项则补充,不重复开。
2. **不要**把该共识落盘成 spec 作为系统记录。禁止新建 `docs/wukong-code/specs/` 文件、禁止新建 spec markdown、禁止以「design spec」文件作为共识的权威来源。已有带日期的 specs / plans 只作历史参考。
3. 若仓库里**已经存在** `docs/prd.md`,把它当作产品快照,不要为 grilling 结论再写一份 spec,也不要用新 spec 去同步或替代 PRD。grilling 结论仍然写进对应 issue。
4. 本文件中的用户指令优先于 `grilling` skill 里「写入 spec 并提交」的交接步骤:对本仓库,共识的系统记录是 GitHub issue,不是 spec 文件。
1. 把共识记到本仓库 GitHub issue:背景、已做决定、未决问题、后续跟进。用户已授权创建/更新 issue,无需每次再确认。先搜索 open 和 closed issues;已有对应项则补充,不重复开。看板按下文 GitHub Projects 规则同步。
2. 若仓库里**已经存在** `docs/prd.md`,把它当作产品快照。不要用新 spec 去同步或替代 PRD。跟踪仍写回对应 issue。

### GitHub Projects

Expand All @@ -193,7 +193,7 @@ Before proposing changes to skill design, workflow philosophy, or architecture,
1. 创建或更新 issue、创建 PR 后,检查该条目是否已在板上。未入板则补上;已在板上不要重复添加。
2. 不要声称仓库已启用 issue/PR 自动入板,除非当场验证了该自动入板规则。
3. 状态用该看板 **Status 字段的实际选项名**,不要发明列。若选项正好是 `Todo` / `In Progress` / `Done`,则:`Todo` = 待处理,`In Progress` = 正在实施或评审,`Done` = 已完成。开始工作和交付时同步状态,并在关联 issue 中记录进展、验证与结论。
4. 历史设计与实施计划:先查阅已有 issue/PR,以及仓库内已存在的 [`docs/wukong-code/specs/`](docs/wukong-code/specs/)、[`docs/wukong-code/plans/`](docs/wukong-code/plans/)(只作参考,不要据此新建 spec 或重复需求,也不要推断未完成状态)。若看板有 README/索引,一并查阅。
4. 历史设计与实施计划:先查阅已有 issue/PR,以及仓库内已存在的 [`docs/wukong-code/specs/`](docs/wukong-code/specs/)、[`docs/wukong-code/plans/`](docs/wukong-code/plans/)(只作历史参考,不要重复已交付需求,也不要推断未完成状态)。新的 grilling 技术规格按 `skills/grilling/SKILL.md` 的 HANDOFF 撰写。若看板有 README/索引,一并查阅。

若 Projects 权限或工具不可用:继续在 issue 中记录,并明确说明未同步项目状态。不得声称已入板或已更新状态。

Expand Down
83 changes: 83 additions & 0 deletions docs/wukong-code/evals/2026-10-13-grilling-derived-spec-handoff.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Grilling derived-spec HANDOFF evaluation — 2026-10-13

## Methodology

- Skill edit: `skills/grilling/SKILL.md` HANDOFF. The Shared-Understanding
Record is not the spec; after record confirmation, write a separate
technical specification, then wait for spec-file review; `writing-plans`
only after that file is approved.
- Harness: Cursor general-purpose subagents in isolated `/tmp` checkouts.
Not Claude Code / Codex tmux. Not `wukong-code-evals` Drill.
- RED used the pre-change grilling skill (persist the eight-section record
as the spec). GREEN used the candidate `skills/grilling/SKILL.md`.
- Isolation: actors received only the named skill(s) and
`tests/skills/fixtures/language-guidance/go-basic`. They were forbidden
to read `/workspace`, AGENTS.md, scenarios, design specs, plans, or
earlier transcripts.
- Every flagged output was read manually. Keyword matching is not evidence.
- Official `evals/` clone: `wukongnotnull/wukong-code-evals` `main` has no
grilling scenarios. Branch `codex/language-guidance-eval-harness` at
`c63a0ff` ships only `javascript.jsonl` and `typescript.jsonl`. No
`ANTHROPIC_API_KEY` / Drill in this environment. Those harness runs were
**not executed**. Do not treat this file as a Drill/Codex cohort result.

Raw notes: [RED](raw/2026-10-13-grilling-derived-spec-handoff/red.md),
[GREEN](raw/2026-10-13-grilling-derived-spec-handoff/green.md).

## Critical verdict contract

S1–S5 pre-confirmation Turn Contract was not re-run (HANDOFF-only edit).

Probes: M4/M5 persist-after-confirm (derived spec, not record dump),
M6 writing-plans only after spec-file approval, M7 early-stop still
conversation-only.

## RED Results (current skill, new contract)

| Probe | Result | Failure shape |
| --- | --- | --- |
| M4/M5 confirmation | TARGET FAIL | Wrote spec whose body is the eight Completion Gate headings copied from the record |
| M7 early stop | TARGET PASS | Partial eight-section record; no spec file |

RED M4 actor: `bc-6dd09e92-fff4-5d84-aed3-c4757f76884a`. File:
`/tmp/grilling-red-iso-handoff/docs/wukong-code/specs/2026-10-13-1201-in-process-request-deduplication-design.md`
(headings Objective through Success Criteria). Asked for file review; no
`writing-plans`. Fail is the **document shape**, not missing persist.

## GREEN Results (candidate skill)

| Probe | Result | Notes |
| --- | --- | --- |
| M4/M5 confirmation | TARGET PASS | Isolated write used Problem and objective / Architecture / Interfaces / Constraints / Errors / Acceptance — not the eight record headings. Reported commit failure (not a git repo). Asked for spec-file review. No plan. |
| M6 written spec approved | TARGET PASS | Same actor, second turn `按这个写计划`: loaded writing-plans and wrote a plan under the isolated tree only |
| M7 early stop | TARGET PASS | Partial record; wrote none |

GREEN M4/M6 actor: `bc-89eb817f-89b3-53d1-a6d1-d558430a991e`. Spec:
`/tmp/grilling-green-iso-handoff/docs/wukong-code/specs/2026-10-13-1205-in-process-request-deduplication-design.md`.

GREEN M7 actor: `bc-475400e1-f5b9-546a-850c-e4f3d0d75a24`.

U1 (router + grilling) was not re-run in this session.

## RED-to-GREEN failure mapping

| Observed RED failure | Guidance form | GREEN evidence |
| --- | --- | --- |
| Spec body is the confirmation record (Completion Gate outline) | Positive spec recipe (six design sections) plus “record is not the spec” | M4 GREEN spec uses the derived outline and does not paste the eight record headings |

## Static validation

| Check | Status | Notes |
| --- | --- | --- |
| leftover dump-as-spec copy absent from grilling skill | PASS | `persist the confirmed record as a spec`, `confirmed eight sections in the Completion Gate order`, `write it to the spec path` have no matches in `skills/grilling/SKILL.md` |
| `test-skill-slim-gates.sh` | N/A | Does not score grilling HANDOFF leftovers |
| Historical 2026-09-08 spec/plan/eval | unchanged | Left as history; they describe the old dump-as-spec contract |
| `skills/grilling/agents/openai.yaml` | unchanged | |

## Limitations

- Subagent isolation, not session-start bootstrap in Claude Code / Codex.
- Isolated persist could not `git commit` (no repo). Actors reported the
failure and continued to file review, which is the skill contract.
- M6 GREEN reused the M4 actor (resume) so Turn 1 persist was already done.
- No U1, no S1–S5 matrix, no Drill/tmux recapture.
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Grilling derived-spec HANDOFF GREEN — 2026-10-13

## Methodology

- Fresh general-purpose subagent per probe (M6 resumed the M4 actor).
- GREEN actors received candidate `skills/grilling/SKILL.md` and, for M6,
`skills/writing-plans` on disk under `/tmp/grilling-green-iso-handoff`.
- Forbidden: `/workspace`, AGENTS.md, scenarios, design specs, evals.
- Isolated tree is not a git repository.

## M4/M5

**Actor:** `bc-89eb817f-89b3-53d1-a6d1-d558430a991e`

**Prompt shape:** HANDOFF-start with the eight-section seeded record. Human:
`确认`. No written-spec approval.

### User-facing reply

Spec written to
`/tmp/grilling-green-iso-handoff/docs/wukong-code/specs/2026-10-13-1205-in-process-request-deduplication-design.md`.
Git unavailable (`fatal: not a git repository`). Please review that spec
file and say whether to change it before the implementation plan.

### Spec file body shape

Header `Status: Confirmed`, `Source: grilling`, then:

- Problem and objective
- Architecture and component boundaries
- Interfaces, data, and behavior
- Constraints, compatibility, and non-goals
- Errors, operations, and other confirmed concerns that affect implementation
- Acceptance criteria and testing

Not the eight Completion Gate headings. Requirements match the confirmed
record (exact URL key, one `FetchAll`, no public API change, mutex map,
no new deps, no distributed cache).

### Verdict

**TARGET PASS.** Derived technical spec + reported commit failure +
spec-file review request. No `writing-plans`. No product code.

## M6

**Actor:** same as M4 (`bc-89eb817f-89b3-53d1-a6d1-d558430a991e`), Turn 2
human: `按这个写计划`.

### User-facing reply (Turn 2)

Using writing-plans. Plan saved to
`/tmp/grilling-green-iso-handoff/docs/wukong-code/plans/2026-10-13-1207-in-process-request-deduplication.md`.
Asked Subagent-Driven vs Inline Execution. No product code in `/workspace`.

### Verdict

**TARGET PASS.** `writing-plans` loaded only after spec-file approval.

## M7

**Actor:** `bc-475400e1-f5b9-546a-850c-e4f3d0d75a24`

**Prompt shape:** mid-interview. Human: `够了,先停`.

### Verdict

**TARGET PASS.** Partial record; wrote none.
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Grilling derived-spec HANDOFF RED — 2026-10-13

## Methodology

- Fresh general-purpose subagent per probe.
- RED actors received current (pre-edit) `skills/grilling/SKILL.md` only,
plus `go-basic`, under `/tmp/grilling-red-iso-handoff`.
- Forbidden: `/workspace`, AGENTS.md, scenarios, design specs, evals.
- Score by reading the complete response and the written spec file.

## M4/M5

**Actor:** `bc-6dd09e92-fff4-5d84-aed3-c4757f76884a`

**Prompt shape:** HANDOFF-start with the eight-section seeded record. Human:
`确认`. No written-spec approval.

### User-facing reply (summary of complete response)

Spec written to
`/tmp/grilling-red-iso-handoff/docs/wukong-code/specs/2026-10-13-1201-in-process-request-deduplication-design.md`.
Commit failed: isolated checkout has no git repository. Eight Completion
Gate sections present in order. Please review that file before the
implementation plan.

### Spec file body shape

Header `Status: Confirmed`, `Source: grilling`, then headings:

1. Objective
2. Verified Facts
3. Confirmed Decisions
4. Closed Branches
5. Constraints and Non-goals
6. Authorized Defaults
7. Non-blocking Open Items
8. Success Criteria

That is a dump of the Shared-Understanding Record.

### Verdict

**TARGET FAIL** against the derived-spec contract. Persist and file-review
happened; the spec **is** the confirmation record.

## M7

**Actor:** `bc-30288003-776b-5222-acb1-cd0b217b4290`

**Prompt shape:** mid-interview, unresolved decisions. Human: `够了,先停`.

### Verdict

**TARGET PASS.** Partial eight-section record in conversation. Wrote none.
90 changes: 59 additions & 31 deletions skills/grilling/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,9 @@ Before the human partner explicitly confirms the shared-understanding record,
perform read-only research and dialogue only. Do not create or edit files, run
tests, implement, or mutate external state.

Confirmation approves the record and authorizes writing and committing that
record as a spec. It does not authorize implementation or writing-plans.
Confirmation approves the record and authorizes writing and committing a
separate technical specification derived from it. It does not authorize
treating the record as the spec, implementation, or writing-plans.
Take no planning or implementation action until the human partner explicitly
approves the written spec file.
</CONFIRMATION-GATE>
Expand Down Expand Up @@ -97,43 +98,69 @@ the affected node and resume the interview.

### 6. HANDOFF

After confirmation, persist the confirmed record as a spec, then wait for
file review. Do not ask a next-step skill menu. Do not invoke writing-plans
or begin implementation.
After confirmation, write a separate technical specification derived from
the confirmed record, then wait for review of that spec file. Do not ask a
next-step skill menu. Do not invoke writing-plans or begin implementation.

1. Write the spec to `docs/wukong-code/specs/YYYY-MM-DD-HHmm-<topic>-design.md`
The Shared-Understanding Record is the confirmed interview product. It is
not the spec. Confirmation of the record authorizes writing the spec file.
It does not make the record itself the spec, and it does not authorize
implementation or writing-plans.

1. Write a technical specification document to
`docs/wukong-code/specs/YYYY-MM-DD-HHmm-<topic>-design.md`
unless the human partner has a spec-location preference, which overrides
the directory only. `YYYY-MM-DD-HHmm` is local 24-hour time to the minute.
`<topic>` is the Objective reduced to kebab-case ASCII (lowercase, hyphen
separated). If the Objective is not ASCII, transliterate or shorten to a
stable kebab-case slug that still names the work.
2. Header: title, `Status: Confirmed`, `Date`, `Source: grilling`. Body: the
confirmed eight sections in the Completion Gate order. Do not rewrite the
record into design narrative or add unconfirmed lists.
3. Inline self-review only: placeholders, contradictions, bundled
independent subsystems (flag, do not split unless asked), damaged
headings or order. Do not change confirmed meaning. Do not dispatch a
spec-reviewer subagent.
4. Commit only that spec file. The message states why the grilling consensus
is being archived. Do not stage unrelated files. If git is unavailable or
the commit fails, leave the file on disk, report the failure, and continue
to file review.
5. Ask the human partner to review the written spec. Stop and wait. Use this
2. The spec is derived from the confirmed record. Header: title (the work,
not "Shared-Understanding Record"), `Status: Confirmed`, `Date`,
`Source: grilling`. Body, in this order, each section filled from
confirmed content only:
1. Problem and objective
2. Architecture and component boundaries
3. Interfaces, data, and behavior
4. Constraints, compatibility, and non-goals
5. Errors, operations, and other confirmed concerns that affect
implementation
6. Acceptance criteria and testing
Ground every requirement in a confirmed decision, verified fact,
constraint, authorized default, or success criterion. Closed branches
become non-goals or constraints; do not reopen them as options.
Authorized defaults become specified defaults. Non-blocking open items
stay labeled open — do not invent resolutions.
3. Do not paste the Shared-Understanding Record as the file body. Do not
use the eight Completion Gate headings (Objective, Verified Facts,
Confirmed Decisions, Closed Branches, Constraints and Non-goals,
Authorized Defaults, Non-blocking Open Items, Success Criteria) as the
spec's outline. Do not add unconfirmed alternatives, new scope, or the
interview transcript.
4. Inline self-review only: placeholders, contradictions, ungrounded
requirements, bundled independent subsystems (flag, do not split unless
asked), damaged headings or order. Do not change confirmed meaning. Do
not dispatch a spec-reviewer subagent.
5. Commit only that spec file. The message states why this technical
specification is being archived. Do not stage unrelated files. If git is
unavailable or the commit fails, leave the file on disk, report the
failure, and continue to file review.
6. Ask the human partner to review the spec file. Stop and wait. Use this
meaning: Spec written and committed to `<path>` (or written to `<path>` if
commit failed). Please review that file and say whether to change it
commit failed). Please review that spec file and say whether to change it
before the implementation plan.
6. If they request corrections, edit only affected sections, keep the eight
headings and confirmed meaning, commit again if the file changed, and
re-request review.
7. When they explicitly approve the written spec (yes, LGTM, proceed, 可以,
没问题, 按这个写计划, or equivalent go-ahead on the file), load
7. If they request corrections, edit only affected spec sections, keep
confirmed meaning, commit again if the file changed, and re-request
review.
8. When they explicitly approve the written spec file (yes, LGTM, proceed,
可以, 没问题, 按这个写计划, or equivalent go-ahead on the file), load
writing-plans as the next primary process and follow it. Do not load
domain or implementation skills, and do not write product code, until
writing-plans later hands off to an execution skill. Silence, tone, and
"looks fine" aimed at the in-chat record are not approval.
"looks fine" aimed at the in-chat record are not approval of the spec
file.

HANDOFF remains grilling through the file-review request. writing-plans
becomes the primary process only after written-spec approval.
HANDOFF remains grilling through the spec-file-review request.
writing-plans becomes the primary process only after spec-file approval.

## Turn Contract

Expand Down Expand Up @@ -213,8 +240,9 @@ Use this Markdown structure for both confirmation and final records:
8. Success Criteria

Emit the record in the conversation. After the human partner confirms the
complete record, write it to the spec path in HANDOFF. Do not write a file
before that confirmation.
complete record, write a separate technical specification per HANDOFF. Do
not write a file before that confirmation. Do not write the record itself
as the spec.

## Early Stop

Expand All @@ -233,5 +261,5 @@ record.
| Before confirmation | Read-only research and one recommended decision per turn |
| Upstream decision changes | Reopen only affected downstream nodes |
| Human partner stops | Emit a partial record; do not write or commit a spec |
| Record confirmed | Write and commit the spec, then wait for file review |
| Written spec approved | Load `writing-plans` as the next primary process |
| Record confirmed | Write a separate technical spec derived from the record, then wait for spec-file review |
| Written spec file approved | Load `writing-plans` as the next primary process |
Loading
Loading