Skip to content

Apply the prompt-audit fixes and stop Active Technologies duplication - #342

Merged
IanMayo merged 2 commits into
mainfrom
claude/youthful-allen-dongcr
Sep 27, 2026
Merged

IanMayo merged 2 commits into
mainfrom
claude/youthful-allen-dongcr

Conversation

@IanMayo

@IanMayo IanMayo commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

Summary

A prompt audit of the repository's model-facing text (CLAUDE.md, the constitution, the vendored spec-kit commands and templates) found factual rot rather than dated prompting in CLAUDE.md, three constitution rules the codebase had outgrown, and model-dated text in spec-kit 0.4.2. This PR applies the High and Medium findings, and fixes the script that was generating the worst of the rot.

Audit fixes (commit 1)

CLAUDE.md

  • The file list promised every path exists and was missing pointerScope.js and SymbolSizeTrial.js; both added
  • "Recent Changes" still called the 167/166 work "Planned" while the file above described it as done; trimmed to the three entries the template asks for
  • Fifteen near-duplicate "Active Technologies" lines become the two facts they carried
  • Four lines describing a previous version of the code ("is gone", "at first", "since spec 167") keep their reason and drop the history

Constitution 1.0.0 → 1.0.1 (corrections of fact; Sync Impact Report updated)

  • Principle III named three of the five modes (Pan and Sidebands missing). /speckit.analyze grades any conflict with the constitution as CRITICAL, so this made every Pan/Sidebands plan look non-compliant
  • Principle IV forbade the <audio> first row that spec 168 introduced
  • Quality Gates listed three of the eleven checks CI runs

Speckit commands and templates

  • The plan template states the repository's constants and real tree instead of generic placeholders and a three-option structure menu
  • Bold-caps openers, a CRITICAL table-formatting block, a 40-line ignore-file lookup table, a rule stated five times in the checklist command, an implementation choreography that contradicted the tasks template, and a research fan-out per technology are rewritten as plain statements of the goal
  • The extension-hook blocks (~280 lines, gated on a .specify/extensions.yml this repo does not have) are removed from the four commands that carried them. Decline this if you plan to adopt spec-kit extensions.

Stop the duplication (commit 2)

update-agent-context.sh deduped Active Technologies entries by exact substring, so every rewording of the same stack in a plan appended a line. Three layers now prevent that:

  • The script compares entries by normalised text (case, spacing, punctuation, (branch) suffix ignored), treats a Storage line that says there is none as no entry, and mints a Recent Changes line only when a feature actually adds a technology, so hand-written entries are not evicted by no-ops
  • yarn hygiene fails an Active Technologies entry that restates another or states nothing (baseline 0)
  • A Vitest test sources the script and pins the rules on a scratch file

Two upstream bugs in the same function surfaced and are fixed: the tech-section branch consumed the ## Recent Changes heading so that section was never trimmed when the two sat together (how the file reached ten entries), and (( count++ )) from zero returns 1, which set -e read as failure the first time the trimming code ran.

Verification

  • yarn test:unit 362 passed (including the new script test)
  • yarn typecheck, yarn lint, yarn hygiene all clean at baseline
  • Dry run: two /speckit.plan-style runs restating the template's constants leave CLAUDE.md byte-identical; a run adding a real technology appends one entry and one change line
  • Playwright suite not run locally (no src/ changes); CI will run it

Not changed

Low-confidence flags from the audit are left as they were: the personal Codespaces/Edge note, the "I'm Groot" shell-quoting line in the commands, the generic SaaS examples in the spec template, issue-number citations, and docs/archive/prompts/.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HVGXdCBws4z1ut7BSfD4eE


Generated by Claude Code

The 2026-09 prompt audit found factual rot rather than dated prompting in
CLAUDE.md, three constitution rules that the codebase had outgrown, and
model-dated text in the vendored spec-kit 0.4.2 commands. This applies
the High and Medium findings; the flagged items are left as they were.

CLAUDE.md
- The file list said every path exists and was missing pointerScope.js
  and SymbolSizeTrial.js; both are added from their own headers
- "Recent Changes" still called the 167/166 work "Planned" while the
  file above it described that work as done; the section now holds the
  three most recent entries the template asks for
- Fifteen near-duplicate "Active Technologies" lines become the two facts
  they carried, spelled exactly as the plan template now states them
- Four lines that described a previous version of the code ("is gone",
  "at first", "since spec 167") keep their reason and drop the history

Constitution (1.0.0 → 1.0.1, corrections of fact)
- Principle III named three of the five modes; Pan and Sidebands were
  missing, and /speckit.analyze grades any conflict as CRITICAL
- Principle IV forbade the <audio> first row that spec 168 introduced
- Quality Gates listed three of the eleven checks CI runs

Speckit commands and templates
- The plan template states the repository's constants in Technical
  Context and its real tree in Source Code, instead of generic
  placeholders and a three-option menu every plan had to prune
- The bold-caps opener, a CRITICAL table-formatting block, a forty-line
  ignore-file lookup table, a rule stated five times in the checklist
  command, an implementation choreography that contradicted the tasks
  template, and a research fan-out per technology are rewritten as plain
  statements of the goal
- The extension-hook blocks (~280 lines, gated on a .specify/extensions.yml
  this repository does not have) are removed from the four commands that
  carried them

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVGXdCBws4z1ut7BSfD4eE
update-agent-context.sh deduped Active Technologies entries by exact
substring, so every rewording of the same stack in a plan's Technical
Context appended a line, and CLAUDE.md collected fifteen copies of one
fact. Three layers now prevent that:

- The script compares entries by their normalised text (case, spacing,
  punctuation and the "(branch)" suffix ignored), treats a Storage line
  that says there is none ("N/A", "None", "Unchanged — …") as no entry,
  and mints a Recent Changes line only when a feature actually adds a
  technology, so the hand-written entries are not evicted by no-ops
- yarn hygiene fails an Active Technologies entry that restates another
  or states nothing, at a baseline of zero
- A Vitest test sources the script and pins the rules on a scratch file

Two upstream bugs in the same function surfaced on the way and are fixed:
the tech-section branch consumed the "## Recent Changes" heading, so that
section was never trimmed when the two sat together (which is how the
file reached ten entries); and `(( count++ ))` from zero returns 1, which
`set -e` read as failure the first time the trimming code ran. Trimming
now keeps three whole entries, continuation lines included.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVGXdCBws4z1ut7BSfD4eE
@github-actions

Copy link
Copy Markdown
Contributor

PR Preview

Preview deployment is ready! View Preview

Last updated: 2026-09-25T11:15:53.258Z

@IanMayo
IanMayo merged commit 60f5c50 into main Sep 27, 2026
6 checks passed
@IanMayo
IanMayo deleted the claude/youthful-allen-dongcr branch September 27, 2026 14:56
github-actions Bot added a commit that referenced this pull request Sep 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants