Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,8 @@ over repo-wide scans.
- `src/engine/` — the findings engine: `finding.ts` (Finding/Lens/ranking) · `ledger.ts`
(committed improvement memory + `resolveEntry` for dismiss/accept) · `run.ts` (lens registry +
audit composition) · `fleet.ts` (sweep + manifest check + fleet-scope wall findings) ·
`premise.ts` (the task-as-instruction check: bare-token promotion, entities, the agent brief)
`premise.ts` (the task-as-instruction check: bare-token promotion, foreign-path scoping,
entities, the agent brief)
- `src/lenses/` — the lenses: `instruction-truth/` (claims extraction, the shared truth checks
in `checks.ts`, and the headline truth lens — instruction files AND state documents, incl.
decision-reference resolution) ·
Expand Down Expand Up @@ -115,8 +116,8 @@ over repo-wide scans.
- `src/lenses/instruction-truth/checks.ts` — the ONE implementation of the command, path,
doc-reference, and decision-reference truth checks. `instruction-truth` (files, state docs) and
`premise` (the task) both call it; a check re-implemented beside it would let the two surfaces
drift apart, which is the failure this tool exists to catch. Tier and wording are parameters,
the rules are not. What a check examined is returned beside what it flagged, so a caller never
drift apart, which is the failure this tool exists to catch. Tier, wording, and the task-only
outside-repo path scope are parameters, the rules are not. What a check examined is returned beside what it flagged, so a caller never
re-derives coverage by running the extractors a second time.
- `src/lenses/instruction-truth/claims.ts` — claim extraction. Invariant: precision over recall;
every skip class (builtins, flagged invocations, globs/URLs/placeholders, unrecognized
Expand Down
10 changes: 8 additions & 2 deletions docs/decisions/010-premise-the-task-is-an-instruction.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,15 @@ brief.
recognized extension; a directory claim needs its trailing slash AND a first segment that
exists here; a script needs the `run` form (or `npm test` / `npm start`) — a bare `pnpm X` in
a sentence is a phrase as often as an invocation; a first segment shaped like a host name is a
URL. Everything backticked is read exactly as an instruction file would be. Every class left as
URL. Everything backticked is read exactly as an instruction file would be — with one
task-surface exception: a mention behind a namespace prefix (`pc:`, `lk:`, a repo shorthand)
points into another repo's tree, not this one. Every class left as
prose is counted and disclosed, never silently dropped — the same skip-class discipline as the
file lens.
file lens. And a missing path is only accused when it is plausibly repo-relative at all: its
first segment must start where a directory of this repo does. A task routinely quotes paths
from OTHER repositories (a clone in a scratchpad, a legend-prefixed token); absence of such a
path here proves nothing, and calling it "missing in the repo" was a live false positive that
outranked the task's real content.
- **Handed over, never guessed:** the three semantic premises, in `.etymd/premise-brief.md`
(only where `.etymd/` exists — a repo that never opted in takes zero writes and gets the brief on
stdout). The brief lists what was checked, found or not, then the questions only an agent can
Expand Down
6 changes: 6 additions & 0 deletions docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,12 @@ would be. A bare `pnpm X` in a sentence, a scheme-less URL (`github.com/…`), a
(`input/output/`) and a bare file name (`config.ts`) are prose — each class is counted and
disclosed, never flagged; backtick one to have it checked.

A task routinely quotes other people's trees, so for the task surface a path is only accused of
missing when it is plausibly repo-relative: the mention carries no namespace prefix (`pc:`,
`lk:`, a repo shorthand — another repo's tree, skipped and disclosed), and the path's first
segment starts where a directory of this repo does. A path that starts elsewhere is outside this
repo — reported as unverifiable, never as a missing file here.

Tiers: a missing path the task is _about_ is **risk** (the task would solve the wrong problem
precisely), a dead script is **risk**, a dead doc or decision reference is **gap**.

Expand Down
115 changes: 88 additions & 27 deletions src/engine/premise.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,24 @@ import {
type ExaminedClaim,
type TruthEnv,
} from "../lenses/instruction-truth/checks.js"
import { KNOWN_EXTENSIONS, PATH_TOKEN_RE } from "../lenses/instruction-truth/claims.js"
import {
KNOWN_EXTENSIONS,
NAMESPACE_IDENT,
NAMESPACE_STOP,
PATH_TOKEN_RE,
} from "../lenses/instruction-truth/claims.js"
import { rankFindings, type Finding } from "./finding.js"

// `etymd premise` — the task an agent is about to be handed is an instruction too (decision 010).
// Before anything acts on it, the things it NAMES are checked against the repo with the same
// precision rules instruction files get; the premises only an agent can verify — that the named
// things are the ones meant, that the mechanism the task assumes actually runs, that the state it
// assumes holds — are handed over in a brief, never guessed at here. Reading files is the whole
// of what this command does (decision 005: anything beyond that is out of scope by construction).
// precision rules instruction files get — except that a task routinely QUOTES other people's
// trees (a scratchpad clone, a legend-prefixed token), so a path is only accused of missing when
// it is plausibly repo-relative: no namespace prefix on the mention, and a first segment that
// starts where a directory of this repo does. The premises only an agent can verify — that the
// named things are the ones meant, that the mechanism the task assumes actually runs, that the
// state it assumes holds — are handed over in a brief, never guessed at here. Reading files is
// the whole of what this command does (decision 005: anything beyond that is out of scope by
// construction).

export const PREMISE_LENS = "premise"
export const PREMISE_BRIEF_FILE = path.join(ETYMD_DIR, "premise-brief.md")
Expand Down Expand Up @@ -67,6 +76,8 @@ export interface PromotionSkips {
hostnameLike: number
/** `input/output/` — a slash-joined phrase whose first segment is no directory here. */
unrootedDirs: number
/** `pc:src/x.ts`, `lk: src/x.ts` — a namespace-prefixed mention of ANOTHER repo's tree. */
namespaced: number
}

export interface PromotionContext {
Expand Down Expand Up @@ -113,6 +124,7 @@ export function promoteBareTokens(text: string, ctx: PromotionContext): Promoted
proseScripts: 0,
hostnameLike: 0,
unrootedDirs: 0,
namespaced: 0,
}
const promoted = text
.split(CODE_SPAN_RE)
Expand All @@ -136,31 +148,60 @@ function promoteProse(segment: string, ctx: PromotionContext, skips: PromotionSk
}
return `\`${core}\`${trail}`
})
return withCommands.replace(/[^\s`]+/g, (token) => {
const m = WRAP_RE.exec(token)
if (!m) return token
const [, lead = "", core = "", trail = ""] = m
if (!core || core.includes("://") || !PATH_TOKEN_RE.test(core)) return token
const first = core.slice(0, core.indexOf("/"))
const hostSuffix = HOSTNAME_RE.exec(first)?.[1]?.toLowerCase()
if (hostSuffix && !KNOWN_EXTENSIONS.has(hostSuffix)) {
skips.hostnameLike += 1
// Token-wise with one token of lookback: a legend prefix (`pc:`) reaches the path AFTER it.
const parts = withCommands.split(/(\s+)/)
for (let i = 0; i < parts.length; i += 2) {
parts[i] = promoteToken(parts[i] as string, i >= 2 ? (parts[i - 2] as string) : "", ctx, skips)
}
return parts.join("")
}

// The namespace label on a path mention — attached to it (`pc:src/x.ts`) or in the previous
// token (`pc: src/x.ts`). Prose introducers ("note:", "see:") are not namespaces (claims.ts).
function namespaceOf(core: string, prev: string): string | null {
const ns =
new RegExp(`^(${NAMESPACE_IDENT}):`).exec(core)?.[1] ??
new RegExp(`^(${NAMESPACE_IDENT}):$`).exec(prev)?.[1]
return ns && !NAMESPACE_STOP.has(ns.toLowerCase()) ? ns : null
}

function promoteToken(
token: string,
prev: string,
ctx: PromotionContext,
skips: PromotionSkips,
): string {
const m = WRAP_RE.exec(token)
if (!m) return token
const [, lead = "", core = "", trail = ""] = m
if (!core || core.includes("://")) return token
const namespace = namespaceOf(core, prev)
const body =
namespace && core.startsWith(`${namespace}:`) ? core.slice(namespace.length + 1) : core
if (!PATH_TOKEN_RE.test(body)) return token
const first = body.slice(0, body.indexOf("/"))
const hostSuffix = HOSTNAME_RE.exec(first)?.[1]?.toLowerCase()
if (hostSuffix && !KNOWN_EXTENSIONS.has(hostSuffix)) {
skips.hostnameLike += 1
return token
}
if (body.endsWith("/")) {
// A directory claim in prose is only a claim when it starts where a real directory does;
// `input/output/` in a sentence is a slash-joined phrase, and prose is full of them.
if (!ctx.rootedDirs.has(first)) {
skips.unrootedDirs += 1
return token
}
if (core.endsWith("/")) {
// A directory claim in prose is only a claim when it starts where a real directory does;
// `input/output/` in a sentence is a slash-joined phrase, and prose is full of them.
if (!ctx.rootedDirs.has(first)) {
skips.unrootedDirs += 1
return token
}
return `${lead}\`${core}\`${trail}`
}
} else {
// The extractor reads a file claim only with a recognized extension; `and/or` stays prose.
const ext = core.toLowerCase().match(/\.([a-z0-9]{1,8})$/)?.[1]
const ext = body.toLowerCase().match(/\.([a-z0-9]{1,8})$/)?.[1]
if (!ext || !KNOWN_EXTENSIONS.has(ext)) return token
return `${lead}\`${core}\`${trail}`
})
}
if (namespace) {
skips.namespaced += 1
return token
}
return `${lead}\`${body}\`${trail}`
}

/** Directory names a prose dir claim may start with — mirrors where `pathResolves` looks. */
Expand All @@ -183,7 +224,10 @@ export async function runPremise(opts: PremiseOptions): Promise<PremiseResult> {
const facts = await scanProject(root)
const env = await buildTruthEnv(root, facts)
const counters = emptyCounters()
const promoted = promoteBareTokens(task, { rootedDirs: await listRootedDirs(env) })
// One rooting notion for both halves: where prose dir claims may start, and where a missing
// path's first segment must start to be plausibly repo-relative at all.
const rootedDirs = await listRootedDirs(env)
const promoted = promoteBareTokens(task, { rootedDirs })
const text = { path: TASK_LABEL, text: promoted.text }

const findings: Finding[] = []
Expand All @@ -198,6 +242,8 @@ export async function runPremise(opts: PremiseOptions): Promise<PremiseResult> {
subject: "The task",
missingPathTier: "risk",
maxPathFindings: MAX_PATH_FINDINGS,
rootedFirstSegments: rootedDirs,
treatNamespacedPrefixes: true,
whyCommand:
"The task is built on a command that does not exist — an agent will run it and fail, or quietly substitute something else and report success.",
actionCommand: "Fix the task before handing it over: name the real script, or restore it.",
Expand Down Expand Up @@ -262,6 +308,21 @@ export async function runPremise(opts: PremiseOptions): Promise<PremiseResult> {
`${skips.unrootedDirs} slash-joined phrase(s) ending in \`/\` start with no directory that exists here (e.g. \`input/output/\`) — read as prose; skipped, not flagged. Backtick one to have it checked.`,
)
}
if (skips.namespaced) {
disclosures.push(
`${skips.namespaced} path mention(s) in prose carry a namespace prefix (\`pc:src/x.ts\`, \`lk: src/x.ts\`) — another repo's tree, never this one; skipped, not flagged.`,
)
}
if (counters.namespacedSkipped) {
disclosures.push(
`${counters.namespacedSkipped} backticked path mention(s) sit directly after a namespace prefix (\`pc: \`src/x.ts\`\`) — another repo's tree, not this one; skipped, not flagged.`,
)
}
if (counters.outsideRepoSkipped) {
disclosures.push(
`${counters.outsideRepoSkipped} path(s) the task names start at no directory of this repo — typical of a path quoted from another repository or a scratchpad clone; outside this repo, not missing here. Skipped, not flagged.`,
)
}
if (counters.unverifiableCommands) {
disclosures.push(
`node_modules is not installed — ${counters.unverifiableCommands} command(s) matching no package script could not be checked against installed binaries; skipped, not flagged.`,
Expand Down
31 changes: 30 additions & 1 deletion src/lenses/instruction-truth/checks.ts
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,10 @@ export interface ClaimCounters {
qualifiedRefsSkipped: number
/** Decision references with no `## D-NNN` ledger to resolve against. */
unresolvableRefs: number
/** Path claims whose every mention sits behind a namespace prefix (`pc:`) — another repo's. */
namespacedSkipped: number
/** Missing paths starting at no directory of this repo — quoted from elsewhere (task only). */
outsideRepoSkipped: number
}

export function emptyCounters(): ClaimCounters {
Expand All @@ -105,6 +109,8 @@ export function emptyCounters(): ClaimCounters {
placeholderSkipped: 0,
qualifiedRefsSkipped: 0,
unresolvableRefs: 0,
namespacedSkipped: 0,
outsideRepoSkipped: 0,
}
}

Expand Down Expand Up @@ -141,6 +147,19 @@ export interface TextClaimsOptions {
actionCommand?: string
whyPath?: string
actionPath?: string
/**
* Directories a missing path may start from and still be plausibly repo-relative — the root,
* workspace packages, and their src/ scripts/. Supplied by the task surface (`etymd premise`)
* only: a task quoting a path from ANOTHER repository (a clone in a scratchpad) starts where
* no directory of this repo does, and its absence here proves nothing. Instruction files keep
* the stricter reading — their references are written against this repo.
*/
rootedFirstSegments?: ReadonlySet<string>
/**
* Read namespace-prefixed path mentions (`pc: `src/x.ts``) as another repo's tree — the task
* surface only; instruction files keep every backticked span as a claim of this repo.
*/
treatNamespacedPrefixes?: boolean
}

export interface TextClaimsResult {
Expand Down Expand Up @@ -202,9 +221,12 @@ export async function checkTextClaims(
}

// Path claims: a path the text points agents at must exist.
const { paths, prospective, placeholder } = extractPathClaims(file.text)
const { paths, prospective, placeholder, namespaced } = extractPathClaims(file.text, {
namespaces: opts.treatNamespacedPrefixes,
})
counters.prospectiveSkipped += prospective.length
counters.placeholderSkipped += placeholder.length
counters.namespacedSkipped += namespaced.length
const missing: string[] = []
for (const claim of paths) {
if (await env.pathResolves(claim)) examined.push({ kind: "path", value: claim, exists: true })
Expand All @@ -217,6 +239,13 @@ export async function checkTextClaims(
const gitignored = new Set((ignoredOut ?? "").split("\n").filter(Boolean))
let pathFindings = 0
for (const claim of missing) {
// Outside this repo (task surface only): a path that starts at no directory the repo has
// is quoted from elsewhere, and "missing here" would be a false accusation.
if (opts.rootedFirstSegments && !opts.rootedFirstSegments.has(claim.split("/")[0] ?? claim)) {
counters.outsideRepoSkipped += 1
examined.push({ kind: "path", value: claim, exists: null })
continue
}
if (gitignored.has(claim)) {
counters.gitignoredSkipped += 1
examined.push({ kind: "path", value: claim, exists: null })
Expand Down
51 changes: 47 additions & 4 deletions src/lenses/instruction-truth/claims.ts
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,26 @@ function isPlaceholderClaim(token: string): boolean {
.some((seg) => PLACEHOLDER_PREFIX_RE.test(seg) || PLACEHOLDER_SEGMENTS.has(seg.toLowerCase()))
}

// A namespace prefix (`pc:`, `lk:`, a repo shorthand) directly before a path mention labels it
// as ANOTHER repo's tree — a legend token in a multi-repo prompt, not a reference into this
// repo. Such a mention is skipped and counted, never resolved against the cwd — on the task
// surface only, where quoting foreign trees is routine. Prose introducers that merely happen to
// precede a path ("note:", "facts:") are not namespaces; the stop-list is grown from corpus
// finds, and an unlisted one costs a disclosed skip, never a false accusation.
export const NAMESPACE_IDENT = "[A-Za-z][A-Za-z0-9-]{0,15}"
export const NAMESPACE_STOP = new Set(
(
"note notes see warning caution caveat caveats example examples ex eg ie nb ps re per " +
"file files path paths hint tip tips todo fixme step steps rule rules ref refs " +
"fact facts output outputs input inputs result results summary status overview " +
"next prev then thus hence plus goal goals spec specs context"
).split(" "),
)

function isNamespace(ident: string | undefined): boolean {
return Boolean(ident) && !NAMESPACE_STOP.has((ident as string).toLowerCase())
}

/**
* The prose around one occurrence: its own line, plus the lead-in line when the claim sits in a
* list item or table row ("Files this creates:" followed by bulleted paths is the common shape).
Expand Down Expand Up @@ -275,19 +295,33 @@ export interface PathClaims {
paths: string[]
/** Claims whose every mention sits in create-this prose — skipped, counted, disclosed. */
prospective: string[]
/** Claims whose every mention sits behind a namespace prefix (`pc:`) — another repo's tree. */
namespaced: string[]
/** Naming stand-ins (`my-custom-skill`) — never real claims. */
placeholder: string[]
}

export interface PathClaimOptions {
/**
* Read namespace-prefixed mentions (`pc: `src/x.ts``) as another repo's tree — the task
* surface, where prompts quote foreign repos behind a legend. Instruction files keep every
* backticked span as a claim of this repo.
*/
namespaces?: boolean
}

/**
* Repo-relative path claims from single-token inline spans, conservatively filtered. The
* load-bearing precision rule (learned from real corpus prose): an extensionless bare token
* (`research/trust`, `milestone/mNN`) is prose — a dir claim must end with `/`, a file claim
* must carry an extension.
* must carry an extension. With `namespaces`, a span whose every mention sits behind a
* namespace prefix names another repo's tree, not this one.
*/
export function extractPathClaims(text: string): PathClaims {
// Per claim: does EVERY mention sit in create-this prose? One plain reference makes it a claim.
export function extractPathClaims(text: string, opts: PathClaimOptions = {}): PathClaims {
// Per claim: does EVERY mention sit in create-this prose / behind a namespace prefix? One
// plain reference makes it a claim.
const prospectiveOnly = new Map<string, boolean>()
const namespacedOnly = new Map<string, boolean>()
const placeholder = new Set<string>()

for (const m of text.matchAll(/`([^`\n]+)`/g)) {
Expand Down Expand Up @@ -317,15 +351,24 @@ export function extractPathClaims(text: string): PathClaims {
}
const prospective = CREATION_CONTEXT_RE.test(claimContext(text, m.index ?? 0))
prospectiveOnly.set(claim, (prospectiveOnly.get(claim) ?? true) && prospective)
if (opts.namespaces) {
// A namespace prefix ends the text right before this mention (`pc: `, `lk:`) — the span
// points into another repo's tree.
const nsLead = new RegExp(`(${NAMESPACE_IDENT}):[ \\t]*$`).exec(text.slice(0, m.index ?? 0))
const prefixed = isNamespace(nsLead?.[1])
namespacedOnly.set(claim, (namespacedOnly.get(claim) ?? true) && prefixed)
}
}

const paths: string[] = []
const prospective: string[] = []
const namespaced: string[] = []
for (const [claim, only] of prospectiveOnly) {
if (only) prospective.push(claim)
else if (namespacedOnly.get(claim)) namespaced.push(claim)
else paths.push(claim)
}
return { paths, prospective, placeholder: [...placeholder] }
return { paths, prospective, namespaced, placeholder: [...placeholder] }
}

export interface DecisionRefs {
Expand Down
Loading
Loading