Skip to content

fix(sel): anchor the chain lock on the log directory - #13308

Merged
iamwhatever merged 2 commits into
mainfrom
fix/sel-chain-lock-path-7077
Sep 28, 2026
Merged

iamwhatever merged 2 commits into
mainfrom
fix/sel-chain-lock-path-7077

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Keep the SEL audit log's hash chain unforkable across processes, and stop a degraded install from silently destroying the key that signs every record it holds.

The SEL audit log's cross-process chain lock was taken on a path derived from the
resolved HMAC key file, not from the log. Those two are not the same thing,
and the key's location differs between processes on one log directory:

  • a process whose key migration completed resolves trust/sel_hmac.key, so it
    locked <sel dir>/trust/security_events.lock;
  • a process whose migration failed (read-only config dir, an unremovable planted
    trust link) keeps signing from the legacy <sel dir>/sel_hmac.key, and
    _chain_lock_path() then returned that key file itself as the lock.

Two such processes hold locks on different inodes while appending to the same
log. The serialization that keeps two writers from chaining off one prev_hash
is gone, and the hash chain forks.

The legacy-fallback branch carried a second, worse consequence. Because the lock
was opened with O_CREAT, a process pinned to the legacy path creates a 0-byte
sel_hmac.key
beside the log. On the next boot the migration block promoted
that 0-byte file over the real key at the destination, destroying the only copy
of the key that signed every existing record. The minimum-length check then
failed init on every later boot, so the loss was unrecoverable rather than
merely wrong.

Measured on 1fb8c9eeec before the fix, read-only:

step reading
lock path, process pinned to legacy key sel_hmac.key
lock path, sibling finished migration security_events.lock
same path False
after the pinned process opens its lock sel_hmac.key exists, 0 bytes
next boot real key 32 -> 0 bytes, init raises

Anchoring the lock on the log directory then depends on that directory being
trustworthy, and one case was not.
An unremovable planted trust symlink is
handled on the key path by design -- _load_or_create_hmac_key deliberately
stays on the legacy key so the linked destination is never used -- but the lock
path still resolved the sidecar under trust/.
Path.mkdir(mode=0o700, parents=True, exist_ok=True) cannot detect that: CPython's
exist_ok check is self.is_dir(), which calls stat() and follows the link, so
the target reports as a usable directory and the FileExistsError is swallowed.
The existing is_link_or_junction check and O_NOFOLLOW both cover only the
final component, so the sidecar was created inside storage the audited agent
controls
, where it can be unlinked and recreated between two writers' acquires:
two flocks on two inodes, and the same permanent chain fork. mode=0o700 is
ignored for a directory that already exists, so the protection the code claimed in
its own docstring was void.

Reproduced on the pre-fix branch: after init the link target is empty, and the
first append creates security_events.lock inside it --
_chain_lock_target() returning (trust/security_events.lock, True).

Why it matters

The SEL log is the tamper-evidence record of what an agent was allowed to do. Its
value rests entirely on the hash chain being unforked: a forked chain cannot
distinguish a concurrent append from a tampered one, so verify_integrity()
stops being a meaningful signal about the record it guards.

The key loss is the sharper harm. It is silent when it happens and terminal
afterwards: the gateway raises on SEL init on every subsequent start, and the
records already written can never be verified again because the key that signed
them is gone. There is no recovery short of restoring a backup of the key.

The linked-directory case is narrower but strictly worse in kind, because it puts
the lock inside storage the audited party controls. An agent that can unlink
and recreate that sidecar between two acquires can fork the chain on demand
rather than only under an unlucky race.

All of these need only an ordinary degraded install to trigger: a read-only
config directory, a mixed-binary window during an upgrade, or a link planted
before the upgrade that the migration cannot remove.

Not a goal

  • Making the HMAC key's own location uniform across processes. The key still
    lives under trust/, and its migration keeps the failure modes that made the
    old lock path vary. This PR removes the coupling that let that non-uniformity
    fork the chain; it does not make key resolution itself uniform. That is a
    separate change with a separate risk profile.
  • Broader hardening of the trust directory. Only the chain lock's use of it
    is in scope here. Other consumers of trust/ are untouched.
  • Fixing test_a_contended_write_waits_instead_of_failing. That Windows
    wall-clock race (assert 0.29599999999982174 >= 0.3) surfaced on an earlier
    head of this PR. Its failure direction cannot be produced by this diff, it
    lives in a file this change surface never imports, and it matches closed
    precedent CI flake: test_an_unchanged_process_still_gets_its_rate divides by a rewind plus real elapsed time (Windows) #13761. Folding a clock fix for it into a SEL PR would put an
    unrelated test in this diff.

What changed (motivation -> approach -> change)

Symptom: two writers append to one log unserialized; a 0-byte key file
appears beside the log and later eats the real key; and on a linked-trust
install the lock sidecar is created inside agent-controlled storage.

Immediate cause: _chain_lock_path() returned self._hmac_key_file whenever
the key had not been relocated, so the lock path was a function of migration
state. And _chain_lock_target() created the sidecar's directory with
mkdir(exist_ok=True), whose EEXIST test follows a link.

Root cause: the lock's identity was computed from a value resolved per
process. Two processes serializing on one file must agree on that file by
construction, and the key's location is precisely the quantity that differs
between processes on one log directory. Anchoring on the log directory fixes
that, and makes the directory's own trustworthiness load-bearing -- which the
link check did not yet cover, because it tested only the final component.

The fix, part one. The sidecar's path is now a constant under the trust
subdirectory, read from the log directory alone. _chain_lock_path() returns
<sel dir>/trust/security_events.lock and reads nothing about the key, so the
value is identical in every process whatever the key's state: no candidate key on
the append path, and no in-process transition for the _CHAIN_HOLDS registry to
straddle. The sidecar stays where it already was for every healthy install, so
the file the lock is taken on does not move on upgrade.

The fix, part two. The lock directory is refused when it is a link, and
the refusal is raised into the same fallback an uncreatable directory already
took. That restores the guarantee _chain_lock_path's docstring makes: the
audited agent can neither unlink the sidecar mid-hold nor hold it out from under
the writers.

The screen stops at the trust leaf, which is both the whole reachable threat
and the parity _load_or_create_hmac_key screens this same directory by
(is_link_or_junction(trust_dir)). trust is the one component
below the log directory the audited agent can write. Every ancestor is the
operator's own layout -- a home directory that is itself a symlink, a crew dir
moved to another volume -- which the agent cannot plant, so refusing on one buys
no protection and costs the whole log: the refusal routes into the legacy-key
fallback, and an install with no legacy key gets every best-effort audit dropped
and every critical operation denied while nothing is wrong. The link-screen gate
states that rule directly in
test_an_ordinary_link_layout_is_not_broken_by_the_hold: what a screen owes is
that it and the use reach the same object, not that no reparse point exists on
the host.

That guard declares itself to the link-screen gate. Refusing a linked lock
directory means screening a path and then handing the same name to mkdir, which
is the screen-then-use shape test/test_link_screen_hold_pin.py enumerates across
the tree, so the guard creates a site that gate did not know. It is declared in the
generated baseline test/link_screen_sites.py, written by
test/gen_link_screen_sites.py rather than by hand, and deliberately not added
to HELD_SITES (test/test_link_screen_hold_pin.py:186, which holds a single
non-SEL entry): the guard raises rather than vetting a target and continuing, so it
never traverses a link and holding the directory open across the check would buy it
nothing. Its siblings on this same directory, _load_or_create_hmac_key and
_chain_lock, are already baselined unheld for that reason.

What happens when trust/ cannot be used at all. The old fallback existed
for a real reason: an install that cannot create that directory must keep
auditing rather than fail every acquire and thereby drop every best-effort event
and deny every critical action. That reason survives, so the fallback survives,
but it is re-keyed, and the rule it is keyed by is the one every branch of the
resolution now obeys: only a condition no actor can flip between two writers'
acquires may select a lock target.
A target that differs across two concurrent
writers is two flocks on two inodes, which is the fork the lock exists to
prevent.

Two conditions qualify, both properties of the filesystem that every writer on
the install reads alike: whether trust/ is creatable, and whether it is a
real directory rather than a link. Either refusal falls back to the legacy
key file, opened without O_CREAT -- which is what removes the 0-byte
artifact at the root, since nothing on the lock path can now bring that file into
being.

What admits that fallback is _hmac_key_file -- where this process actually
resolved its HMAC key, settled once at init -- not the legacy file merely
existing. The existence test cannot be the gate, because sel_hmac.key is a
sandbox read-write leaf just as trust is: the audited agent can drop a file
there on a perfectly healthy install, and the lock path never reads its bytes, so
any content would do. Gating on existence therefore let that agent arm the
fallback at will
. Keying on where the process signs means a planted file buys
nothing, and it is the install-wide fact the fallback was always argued from: a
process signing at the legacy location is one whose trust/ could not be created
or migrated out of.

A pin refusal fails the acquire and never selects another target. This is
the one refusal the fallback must not reach: _pin_lock_dir raises precisely
when it catches trust being replaced mid-acquire, so it is per-acquire and
agent-triggered and can be true for one writer and false for the next. It is
taken outside the fallback so it cannot select a target at all.

Where nothing qualifies, the acquire fails, because inventing a lock elsewhere is
worse than failing: an unserialized append forks the chain.

The hold registry stays keyed on the constant sidecar path rather than on the
file finally locked. A key that tracked the fallback would hand two instances in
one process different keys, and two real flocks on two inodes is the fork this
lock exists to prevent; a stable key makes them join one hold instead.

Third fix, for the upgrade window: a legacy file too short to be a key no
longer replaces a destination that holds a usable one. The unconditional
"legacy key wins" rule rests on the two files carrying the same bytes, and a file
that cannot load as a key carries none. The guard is deliberately narrow: only
that one pairing is skipped, so a genuine legacy key still wins over a
pre-existing destination file, and a short legacy file with no usable
destination still reaches the length check exactly as before. This covers the
mixed-binary window where an old process can still leave the artifact behind.

The owning spec is updated in the same change, because both the superseded lock
contract (target derived from the key's migration state at init) and the
unqualified legacy-wins rule were still documented as current.

The fallback is the part most worth a reviewer's attention, because it is the one
place where two files can still be locked on one install. Its remaining exposure
is stated rather than argued away: the legacy file is itself agent-writable,
so on an install signing there the agent can replace it between two acquires and
reach the same fork. That is left undefended deliberately -- an agent that can
rewrite the key signing the chain can forge any MAC on it directly, which is
strictly more than forking it, so a lock-side screen would buy nothing the key's
own protection has not already lost. If a reviewer disagrees with that
reading, that is the sentence to push on.

Backwards compatibility

Compatible. The sidecar's path is unchanged for every install that already has a
real trust/ directory, so the file the lock is taken on does not move on
upgrade, and no caller signature or on-disk format changes.

The one behaviour that narrows is deliberate and is the fix: an install whose
trust/ is a link previously acquired a lock inside that link and now
refuses it. That install does not lose its lock -- the refusal
routes into the pre-existing fallback, and it is exactly the install where
_load_or_create_hmac_key also met the link and fell back, so the legacy key it
is already signing from is present and becomes the lock. The acquire fails only
when there is no legacy key either, which is the behaviour an uncreatable trust/
already had before this PR.

Nothing narrows for an install whose trust/ is a real directory reached through
a symlinked ancestor -- a home directory that is itself a symlink is a normal
layout on shared build hosts, and it keeps the real sidecar.
test_symlinked_data_home_still_takes_the_real_chain_lock pins that.

Writer sweep on origin/main @ 19b157da12c2: _chain_lock is module-private
with exactly two callers, both in src/kiro_crew/sel.py -- the append path
(_flush_batch) and the prune path (prune). Both already handle a refused
acquire, which _flush_batch turns into a rollback plus warning, or propagates
for a critical audit under the audit-or-deny contract. No caller outside the
module reaches it, and no test outside test/test_sel.py asserts the old path.

Tests

Six tests in the diff were measured to fail before the fix and pass after.
The first four were measured by copying this branch's test file into a worktree
checked out at the parent commit and running it against that commit's own source;
the last two by neutralising the guard in place, with the source restored from a
copy held outside the repository and verified by sha256.

  • test_lock_path_does_not_vary_with_the_keys_migration_state. On base one log
    directory yields trust/security_events.lock for a migrated process and
    sel_hmac.key for a pinned one; it fails with "the chain lock path diverges
    between two processes on one log dir".
  • test_short_legacy_file_does_not_destroy_a_usable_migrated_key. On base the
    0-byte legacy file is promoted over the real key and init then raises
    SEL HMAC key ... is too short (0 bytes; require >= 32). It also asserts the
    pre-existing chain still verifies and that a later boot still initializes.
  • test_the_fallback_never_creates_a_file_at_the_legacy_key_path. Pins the
    no-O_CREAT guarantee directly: with the sidecar's directory unusable and no
    legacy key present, the acquire must fail and leave nothing at the key path.
  • test_appends_survive_when_the_trust_dir_is_uncreatable gains an assertion
    that the resolved target is the existing legacy key with creation refused, so
    the audit-continuity behaviour and the new flag are pinned together.
  • test_unremovable_planted_link_never_hosts_the_chain_lock pins the linked
    directory. The pre-existing sibling pin covered init only, and asserted the
    link target stays empty at that point; the lock is taken on the first
    append
    , which is why a clean target there proved nothing. The new pin appends
    and then asserts _chain_lock_target(). With the guard neutralised it fails
    at test/test_sel.py:2543 --
    assert (PosixPath('.../trust/security_events.lock'), True) == (PosixPath('.../sel_hmac.key'), False)
    -- and with the guard in place it passes.
  • test_symlinked_data_home_still_takes_the_real_chain_lock pins the other
    direction, and is what narrowed part two's screen to the trust leaf. It builds
    the operator layout -- a real trust/ under a data home reached through a
    symlink, no planted link anywhere -- and asserts the real sidecar is still used
    and the append lands. Against the first version of the guard, which also walked
    ancestors, it failed with
    OSError: SEL chain-lock directory .../link/crew/sel/trust is or resolves through a link,
    i.e. the whole log refused on a healthy install; against the leaf-only guard it
    passes. Disabling the guard entirely passes this pin and fails the planted-link
    one, so the two pins bound the screen from both sides.

Two further tests pin the target-selection rule, added after review found the
pin refusal and the existence gate both able to switch targets. Each was proved
discriminating by a mutation harness (.kc-13308-mutations.sh) that restores the
pre-fix form, requires an explicit pytest N passed / N failed count, and
restores the source from a copy held outside the repository verified by sha256
-- never git checkout, which would revert to HEAD and delete the fix mid-run.
Unmutated control: 2 passed. Both mutations CAUGHT (1 failed each).

  • test_a_planted_legacy_key_does_not_arm_the_fallback_on_a_healthy_install.
    This process signs from trust/; a file is planted at the legacy key path and a
    refusal is then triggered. The acquire must fail closed, and the planted bytes
    must be neither locked nor rewritten. Mutation M1 restores the
    legacy_key.exists() gate and the test fails.
  • test_a_mid_acquire_directory_swap_fails_closed_instead_of_switching_targets.
    Set up in the most permissive state the fallback has -- a legacy key present
    and this process signing from it -- so neither condition can be what admits
    the refusal. The static screens pass, so only _pin_lock_dir can refuse, and
    it must raise rather than return the legacy key. Mutation M2 routes the pin
    refusal into the fallback and the test fails.

Also re-run on the amended head: -k "TestCrossProcessSafety or TestHmacKeyTrustDirMigration" -> 51 passed; mypy src/kiro_crew/sel.py ->
Success: no issues found in 1 source file; ruff check clean; docs-lint.sh
passes with its own --test self-check first, so the gate is proved to still
fire before its green is believed.

One further failure was the link-screen gate rather than a test in this diff.
test_every_site_is_declared scans the tree, and part two's guard made it red on
both Backend Tests (3.12, 2) and Backend Tests (Windows) (2) until the new site
was declared. Removing only that one baseline entry reproduces the CI failure
exactly, which is how the entry was confirmed to be the fix rather than a
suppression.

Runs, each read from its own exit status, -n 0 against a scratch
KIROCREW_HOME, on this PR's base 19b157da12c2:

run selection result
both owning files test/test_sel.py + test/test_link_screen_hold_pin.py 305 passed, 42.8s
owning file alone test/test_sel.py 288 passed
link-screen gate, new entry present test/test_link_screen_hold_pin.py 17 passed
link-screen gate, new entry removed same 1 failed, 16 passed
scoped -k "TestCrossProcessSafety or TestHmacKeyTrustDirMigration" 44 passed, 244 deselected
both link pins, guard disabled the planted-link pin + the symlinked-home pin 1 failed, 1 passed (the planted-link pin fails)
both link pins, ancestor-walking guard same two 1 failed, 1 passed (the symlinked-home pin fails)
both link pins, leaf-only guard same two 2 passed

Because test/test_link_screen_hold_pin.py imports test/source_corpus.py, and
main changed that module's file enumeration, the baseline was re-derived rather
than trusted: running test/gen_link_screen_sites.py on the merged tree leaves
the committed file byte-identical (150 declared sites), so this PR's entry is
exactly what the gate's own condition yields.

Gates, each read from its own exit status, run as CI invokes them on the rebased
tree: isort --check-only src/kiro_crew test conftest.py xdist_budget.py 0,
flake8 on the same paths 0, mypy src/kiro_crew/ 0 (1735 files), the baselined
scripts/check_black_formatting.py 0 (4 changed files in scope),
scripts/check_subprocess_encoding.py self-test and gate 0,
scripts/check_brand_name.py 0, and scripts/docs_lint.py 0.

Manual verification

The linked-directory leak was reproduced by hand on the pre-fix tree before any
code was written, because the two review findings that named it had to be
confirmed rather than taken on trust. With a trust link planted so its removal
fails, init leaves the key on the legacy path and the link target empty; the
first append then creates security_events.lock inside the link target, and
_chain_lock_target() returns (trust/security_events.lock, True). That is the
exact tuple the fix's new pin now asserts against, so the manual reproduction and
the automated pin agree on the same observation.

Related Issues

Closes #7077

Pattern harvest

Rule candidate: grep / semgrep
Pattern: a cross-process lock path derived from a separately resolved sibling
rather than from the resource it serializes.

The defect was not the fallback itself, it was that the lock's identity was
computed from self._hmac_key_file, a value resolved per process that varies
with migration state. A lock whose path depends on any value that can differ
between two writers of the same resource is forkable by definition, and the fork
is silent: both writers succeed, and only later verification sees it. When a
degraded-mode fallback is genuinely needed, key it off a property of the shared
filesystem, which every writer reads alike, not off per-process state.

A second, narrower candidate from the same commit: opening a key or secret file
with O_CREAT for a purpose other than reading or writing that secret. Here the
lock's O_CREAT | O_RDWR open was pointed at the HMAC key path, which
manufactured a 0-byte key file that the next boot's migration then promoted over
the real key. A file that holds a secret should only ever be created by the code
that owns the secret.

A third: Path.mkdir(exist_ok=True) as a link defence. Its EEXIST test is
is_dir(), which follows symlinks, so a planted link to a directory satisfies it
silently and the mode argument is ignored for a path that already exists. A
mkdir whose purpose is to establish a trusted directory needs an explicit
link check.

And a fourth, which this PR got wrong first: that explicit check must span
exactly the components the untrusted party can plant, and stop there. Screening
the whole path instead reads as strictly safer against the refusal direction
alone, while refusing the operator's own layout -- a symlinked home, a data dir
moved to another volume -- which no attacker controls. The cost is not a narrower
feature but the subsystem: this screen's refusal path drops every best-effort
audit and denies every critical operation, so an ancestor walk turns a supported
layout into a silently unauditable install. The bound to reach for is the sibling
screen on the same directory, not the widest probe available.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (fix(sel): ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (the owning spec, docs/system-specs/modules/sel.md)
  • No secrets, credentials, or internal references in the diff

@chenmingwei23
chenmingwei23 requested review from a team and hoang-phan98 September 24, 2026 08:46
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 24, 2026
@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of e6def7dd2146b59b0307dd7db056729bd32c9636 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Fix _pin_lock_dir's stale docstring: it says a pin refusal "routes into the legacy-key fallback," contradicting the caller, the spec, and mutation pin M2.

[DESIGN-REVIEWED] e6def7d

@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — ✅ PASS

Premise-level review of e6def7dd2146b59b0307dd7db056729bd32c9636 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

On Windows the pin degrades to the by-name junction probe, so the mid-acquire trust swap stays open there — confirm that residual is the intended read-side parity.

What this change ships

Inventory (8 items) — 8 justified

Intent: FIX — stop the SEL hash chain forking across processes and stop a degraded install destroying its own signing key.

  1. Chain lock is always trust/security_events.lock, identical in every process regardless of key migration state — justified
  2. Legacy-key lock fallback now requires this process to actually sign there, not a file merely existing — justified
  3. The fallback never creates the legacy key file, so no 0-byte key appears beside the log — justified
  4. A linked trust directory is refused as the lock's home instead of hosting the sidecar — justified
  5. Lock directory pinned by descriptor; a trust swapped mid-acquire fails the acquire rather than moving the lock (POSIX) — justified
  6. Init refuses, non-destructively and with a remedy, a too-short legacy key beside a loadable migrated key instead of promoting the stub over the real key — justified
  7. In-process hold registry keys on the constant sidecar path, so two instances join one hold across fallback states — justified
  8. sel.md rewritten to record the new lock-resolution rules, same commit — justified

[FIRST-PRINCIPLES-REVIEWED] e6def7d

@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ human override accepted

Human judgment by @chenmingwei23 overrides the GPT 5.6 finding for e6def7dd2146b59b0307dd7db056729bd32c9636; the recorded reason is authoritative for this commit.

[GPT-OVERRIDE] e6def7d

This comment is updated in place on each push.

The model was not re-run because an authorized human decision supersedes it.

False positive or not applicable? A repository writer can comment:
/ai-review override gpt e6def7dd2146b59b0307dd7db056729bd32c9636: <one-sentence reason>

@github-actions

Copy link
Copy Markdown
Contributor

Security Scope Review -- 🟡 CONCERNS

Which legitimate operations does 71e3c1e3912b86ea75bf8fbc20d604f23175f41d newly refuse? Updated in place on each push. A script-confirmed regression blocks; the model proposes candidates and never decides a refusal.

Verdict basis: the reviewer's own judgement; no confirmed regression.

Security scope review -- adjudicated candidates

  • macos-latest (platform posix): 11 classified, 0 newly refused, 11 still allowed.
  • ubuntu-latest (platform posix): 11 classified, 0 newly refused, 11 still allowed.
  • windows-latest (platform windows): 6 classified, 0 newly refused, 6 still allowed.

No candidate flipped from allowed to refused.

Scope-Verdict: CONCERNS

TIGHTENING: The sensitive-path read+write floor (_CREW_SECRET_LEAVES in src/kiro_crew/security/paths.py) gains the leaf "security_events.lock", expanded under both crew data-home prefixes -- the ONE threat it refuses is the audited agent tampering with the SEL chain-lock sidecar now that it sits beside the log: per the hunk, the agent "must not be able to unlink the sidecar (the next two writers then create separate inodes and fork the chain) nor hold it (every audited action either stalls behind it or is denied)". The entry is home-anchored by exact leaf (never basename-matched), and at the two home paths the pre-existing keystone artifact-suffix rule (.lock beside a keystone leaf, _is_keystone_publish_artifact) already refused the same names at base, so the adjudicable delta is expected to be nil.

ADJUDICATED: 16 candidates, 0 confirmed newly-refused
No verdicts were handed back for any candidate at review time; no regression is claimed. The candidates probe: direct siblings of the new leaf that must stay readable (config.json, sessions.db under both ~/.kiro/crew and ~/.kirocrew), listings and traversals of the leaf's parent (ls, dir, du), same-named lock files outside the fenced homes (pytest tmp SEL dirs, %TEMP% repro dirs -- a basename-keyed spelling of the rule would refuse them), the leaf name as a bare command token (grep/findstr/git grep over the source), the maintainers' own SEL test invocations on both hosts, and a workspace .lock-suffixed file (Cargo.lock) against a suffix-keyed widening.

PLATFORM COVERAGE: posix rows cover ~-anchored home spellings, /-separated test and grep invocations, and the out-of-home /tmp probe. windows rows cover the same operations in the other tokenizer's spelling: %USERPROFILE%- and %TEMP%-anchored paths, backslash separators, dir/type/findstr, and the .venv\Scripts\python.exe test lane -- this diff assumes Windows explicitly (the sidecar exists because msvcrt.locking() needs a movable-offset fd, and the leaf specs are POSIX-authored then re-joined per host), so both spellings of every home-path and test operation were written. One any row (git grep -n security_events.lock) is genuinely separator-free. No platform assumption the diff makes was left without a candidate; a UNC-rooted crew home has no candidate because the fence is home-anchored and a UNC home is not a spelling the diff introduces or changes.

UNADJUDICATED: Two parts of this tightening are OS-sandbox mask decisions the shell-command classifier settles nothing for. (1) sandbox.py adds security_events.lock to _CREW_CHILD_WITHHELD_LEAVES: the guard is the child-sandbox bind mask, reached when the gateway spawns a foreign/child harness and projects the withheld-leaf list into its mount view -- the child's own open()/unlink() of the sidecar then fails at the OS, never through the tool gate. The candidate inputs I would have submitted: a child harness's in-sandbox MCP server attempting a SEL append (an append takes the chain lock first, so a child that legitimately appended before this change would now stall or fail at lock acquisition); the diff's own comment asserts no child may append, but no verdict is available to confirm nothing legitimate chained through that path. (2) The companion _CREW_SANDBOX_VISIBLE_LEAVES addition keeps the lock read-write inside the crew's own sandbox -- a carve, not a tightening, but it is the only thing keeping the crew's in-sandbox SEL appenders alive against entry (1), and it likewise gets no classifier verdict. The sel.py changes themselves (lock path now a pure function of the log directory, the link/hardlink refusal relocated beside the log, the short-legacy-key promotion guard) are runtime behavior, not refusal matchers, and no classifier verdict exists for them either; the diff's own new tests (test_lock_path_does_not_vary_with_the_keys_migration_state, test_short_legacy_file_does_not_destroy_a_usable_migrated_key) are the evidence there. The honest verdict over these unadjudicated halves is CONCERNS, not PASS.

[SCOPE-REVIEWED] 71e3c1e

@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review — ✅ no blocking findings

Reviewed e6def7dd2146b59b0307dd7db056729bd32c9636 — this comment is updated in place on each push.

Review details

Two advisory findings: the new ambiguity guard cannot tell a VANISHED legacy key from a short one, and the newly written prose misstates when the legacy-key fallback applies.

FINDING — src/kiro_crew/sel.py:1503 — if not _trust_root_key_loads(legacy_path) and _trust_root_key_loads(key_path) returns False for an ABSENT legacy file exactly as for a short one, so when a sibling process completes os.replace(legacy_path, key_path) in the gap between the legacy_path.exists() gate at line 1467 and this os.stat, the second process raises RuntimeError("legacy SEL HMAC key … is too short to be a key") out of SecurityEventLog.__init__ — the benign concurrent-migration case the code below still handles gracefully at line 1561 ("SEL HMAC key migration raced; using already-migrated %s"), now a hard init failure whose message tells the operator to remove both key files → Fix: treat absence as not-ambiguous, e.g. gate the refusal on legacy_path.exists() and not _trust_root_key_loads(legacy_path) and _trust_root_key_loads(key_path) so a vanished legacy source falls through to the existing raced-migration branch.

FINDING — src/kiro_crew/sel.py:3667 — _pin_lock_dir's docstring claims it "Raises OSError on a refusal, so the caller routes it into the legacy-key fallback exactly as it routes an uncreatable directory", but the call at line 1005 is deliberately OUTSIDE the except OSError: fallback (line 979) and the caller's own comment plus test_a_mid_acquire_directory_swap_fails_closed_instead_of_switching_targets pin the opposite, fail-closed behaviour; the same paragraph-level mismatch sits at line 664, where _chain_lock's docstring says a link-or-uncreatable trust "falls back to the legacy key file" with no mention that the code only does so when self._hmac_key_file == self._dir / _HMAC_KEY_FILE and otherwise re-raises — so a change made on either docstring's authority reinstates the two-inodes-on-a-healthy-install fork this commit exists to prevent → Fix: rewrite _pin_lock_dir's last paragraph to say the refusal fails the acquire and is deliberately not routed into the fallback (citing sel.py:990–999), and add the "only when this process already resolved its key there" clause to the sentence at line 664.

[OPUS-REVIEWED] e6def7d

Verdict parsed from the review's SHA-scoped output markers for commit e6def7dd2146b59b0307dd7db056729bd32c9636.

False positive or not applicable? A repository writer can comment:
/ai-review override fable e6def7dd2146b59b0307dd7db056729bd32c9636: <one-sentence reason>

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 24, 2026
@chenmingwei23
chenmingwei23 force-pushed the fix/sel-chain-lock-path-7077 branch from 71e3c1e to b3ee04b Compare September 24, 2026 14:41
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 24, 2026
@chenmingwei23
chenmingwei23 force-pushed the fix/sel-chain-lock-path-7077 branch 2 times, most recently from 5bfcaaf to b66d7dd Compare September 27, 2026 08:12
@chenmingwei23
chenmingwei23 requested a review from a team as a code owner September 27, 2026 08:12
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 27, 2026
The cross-process chain lock was taken on a path derived from the resolved
HMAC key: the trust-dir sidecar when the key had migrated, the legacy key
file itself when it had not. That location is per-process state, so during
a migration two writers on one log directory locked DIFFERENT inodes and
appended unserialized, forking the hash chain the audit log's
tamper-evidence rests on. Worse, the legacy-key leg opened that path with
O_CREAT, so a writer still pinned to it left a 0-byte file at the key path
after a sibling relocated the real key; the migration block then promoted
that file over the usable key and every later boot failed init with "too
short (0 bytes; require >= 32)", losing the only copy of the signing key.

The sidecar's name is now a constant under the trust subdirectory, read
from the log directory alone and never from the key. When that directory
cannot be created at all, the lock falls back to the legacy key file only
when it ALREADY exists, and opens it without O_CREAT, so no acquire can
leave the 0-byte artifact behind. The fallback keys off the directory's
permissions, a filesystem fact every writer on the install reads alike,
where the old one keyed off each process's own migration timing. The hold
registry stays keyed on the constant sidecar path, so two instances join
one hold rather than taking two real flocks. A migration guard also
refuses to promote a legacy file that cannot load as a key over a
destination holding a usable one.

Anchoring on the directory exposed a second hole in that directory's own
trustworthiness. A `trust` symlink planted before the upgrade whose
removal fails leaves `_load_or_create_hmac_key` on the legacy key
precisely so the linked destination is never used, but `_chain_lock_target`
still resolved the lock under `trust/`. `Path.mkdir(exist_ok=True)` cannot
see that: its EEXIST check is `is_dir()`, which follows the link and
reports the target as a usable directory, so the FileExistsError was
swallowed and `(trust/security_events.lock, True)` returned.
`is_link_or_junction` and `O_NOFOLLOW` cover only the final component, so
the sidecar was then created inside storage the audited agent controls,
where it can be unlinked and recreated between two writers' acquires --
two flocks on different inodes and a permanent HMAC chain fork -- and
`mode=0o700` is ignored for an existing directory, voiding the protection
the code claimed. The lock directory is now refused when it is itself a
link, and the refusal routes into the same fallback: the
deny-list-protected legacy key when one exists, otherwise the failure.

That screen covers the `trust` leaf and nothing above it, which is both
the whole reachable threat and the parity `_load_or_create_hmac_key`
screens this same directory by. `trust` is the one component below the log
directory the audited agent can write. Every ancestor is the operator's
own layout -- a home directory that is itself a symlink, a crew dir moved
to another volume -- which the agent cannot plant, so refusing on one buys
no protection and costs the whole log: the refusal routes into the
legacy-key fallback, and an install with no legacy key gets every
best-effort audit dropped and every critical operation denied while
nothing is wrong. The link-screen gate states that rule directly, in
`test_an_ordinary_link_layout_is_not_broken_by_the_hold`: what a screen
owes is that it and the use reach the same object, not that no reparse
point exists on the host.

Also update the owning spec, which described the superseded contract (lock
target derived from the key's migration state at init) and stated the
legacy-wins rule without this branch's short-legacy-file exception.

The new guard makes ``_chain_lock_target`` a screen-then-use site under the
link-screen gate: it asks whether the lock directory is a link and then hands
the same name to ``mkdir``. It is recorded in the generated baseline
(``test/link_screen_sites.py``) via ``test/gen_link_screen_sites.py`` and
deliberately NOT added to ``HELD_SITES``, because it refuses every link rather
than vetting a target and continuing, so it never traverses one and a held
chain would buy it nothing. That is the same treatment its siblings on this
directory already have, ``_load_or_create_hmac_key`` and ``_chain_lock``.
@chenmingwei23
chenmingwei23 force-pushed the fix/sel-chain-lock-path-7077 branch from b66d7dd to a329d95 Compare September 27, 2026 12:20
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Sep 27, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Sep 27, 2026
@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Fable 5) — ⏭️ skipped

Revision e6def7dd2146b59b0307dd7db056729bd32c9636 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Advisory — does not block merge.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 28, 2026
@chenmingwei23
chenmingwei23 force-pushed the fix/sel-chain-lock-path-7077 branch from 882a524 to fff8573 Compare September 28, 2026 04:49
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 28, 2026
Three security-class findings on the chain-lock change, all reachable by the
audited agent because `trust` and `sel_hmac.key` are sandbox read-write leaves
(sandbox.py's _CREW_SANDBOX_VISIBLE_LEAVES).

The lock directory was screened by name and then created/opened through that
same name, so the agent could replace it in the window: both writers' screens
pass truthfully while their opens land in different directories, giving them
flocks on different inodes -- the forked chain the lock exists to prevent.
`_pin_lock_dir` now pins the directory by descriptor (O_DIRECTORY | O_NOFOLLOW
| O_NONBLOCK, a descriptor fstat directory check, and a name-to-descriptor
identity check), and the sidecar is opened relative to that descriptor, so the
directory screened is the directory the sidecar is opened inside.

That pin introduced the third finding, and it is the same class one level in:
which file gets locked must be decided by a condition no actor can flip BETWEEN
two writers' acquires, and a pin refusal is the opposite of that. It fires
exactly when the agent replaces `trust` mid-acquire, so it can be true for one
writer and false for the next -- and routing it into the legacy-key fallback
therefore handed those two writers different inodes on an install whose trust
dir is perfectly healthy. The pin now fails the acquire instead, and is taken
outside the fallback so it cannot select a target at all. The two screens that
still may are filesystem properties every writer reads alike.

The fallback's own gate was the same mistake in its admitting condition: it
keyed on a file EXISTING at the legacy key path, which the agent can create on
a healthy install, and the lock path never reads those bytes so any content
sufficed. It now keys on `_hmac_key_file` -- where this process actually
resolved its key, settled once at init -- which is the install-wide fact the
fallback was always argued from, and which a planted file cannot move. The
legacy FILE stays agent-writable, so on an install signing there the agent can
still replace it between two acquires; that is left undefended deliberately,
because an agent that can rewrite the key signing the chain can forge any MAC
on it directly, which is strictly more than forking it.

The short-legacy exception retained the destination key on a size test, and
length is not trust: truncating the legacy key below the minimum and planting a
long-enough destination turned a fail-closed length check into adoption of an
attacker-known key. That shape now resolves neither way -- nothing is adopted,
nothing is overwritten, both files survive, and init fails closed naming both
paths and the remedy. Authenticating against the log was considered and
rejected as the worst available anchor: it is a read-write leaf for the same
agent, so a forged newest row would turn the check into adoption of a key the
attacker knows.

Tests pin both new behaviours, and a mutation harness restoring each pre-fix
form reddens the matching test (M1 the existence gate, M2 the pin routed into
the fallback), so neither pin is a tautology.
@chenmingwei23
chenmingwei23 force-pushed the fix/sel-chain-lock-path-7077 branch from fff8573 to e6def7d Compare September 28, 2026 12:44
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 28, 2026
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

/ai-review override gpt e6def7d: Applied the reachability test; overriding on limb 1, actor already holds a strictly stronger capability. AGENTS.md:117 records sel_hmac.key as VISIBLE with no OS fence, and a spawned shell reaches it via open() outside the tool gate, so this agent reads the audit key and forges any MAC outright, which subsumes forking the chain. The same VISIBLE grant supplies the trust swap. Fail-closed would deny every critical audit on a writable-log, non-writable-dir install.

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@chenmingwei23 marked the gpt AI finding as false positive, not applicable, or explicitly accepted for e6def7dd2146b59b0307dd7db056729bd32c9636.

Applied the reachability test; overriding on limb 1, actor already holds a strictly stronger capability. AGENTS.md:117 records sel_hmac.key as VISIBLE with no OS fence, and a spawned shell reaches it via open() outside the tool gate, so this agent reads the audit key and forges any MAC outright, which subsumes forking the chain. The same VISIBLE grant supplies the trust swap. Fail-closed would deny every critical audit on a writable-log, non-writable-dir install.

This decision applies only to this commit. A new push requires a new judgment.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention labels Sep 28, 2026
@iamwhatever
iamwhatever merged commit 998b045 into main Sep 28, 2026
143 of 152 checks passed
@iamwhatever
iamwhatever deleted the fix/sel-chain-lock-path-7077 branch September 28, 2026 23:31
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 28, 2026
bolichen97 added a commit that referenced this pull request Oct 1, 2026
…15634)

Backport of #13308, #14302, #14860, #14961, #15070 to release/0.8.0, cherry-picked in main's merge order.

* fix(sel): anchor the chain lock on the log directory (#13308)
  (cherry picked from commit 998b045)
* fix(watchdog): decide the loop-stall exit on the monotonic clock (#14302)
  (cherry picked from commit 1d37ab1)
* fix(ops-mission-control): strict read in ledger resolve_conflict (#14860)
  (cherry picked from commit b8ea16b)
* fix(env): skip a mise candidate whose probe is denied instead of aborting boot (#14961)
  (cherry picked from commit e054c95)
* fix(lessons): retire only a finding on a contradiction verdict (#15070)
  (cherry picked from commit dd5e840)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SEL chain lock can fork when a sibling completes the trust-key migration

3 participants