Skip to content

fix(autonudge): set a damaged self-arm record aside before an arm rewrites it - #36206

Merged
javenciu merged 2 commits into
mainfrom
fix/lenient-self-arm-strict-rewrite
Nov 13, 2026
Merged

javenciu merged 2 commits into
mainfrom
fix/lenient-self-arm-strict-rewrite

Conversation

@javenciu

@javenciu javenciu commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: arming a self-armed loop over a damaged record keeps the damaged record's bytes, and an arm never fails where it succeeds today.

autonudge-self-armed.json holds one entry per loop a session armed for itself, and the fire-time guard refuses a self-armed loop whose entry is missing. record_self_arm read the record with _read_record, which answers an empty table for a file that is cut off or has the wrong shape, then wrote back only the new entry. The old bytes were replaced, so every other loop's entry was gone with no copy left to recover it from.

Why it matters

A refused fire looks like a loop that stopped working for no reason. With the damaged record set aside, the entries it held can still be read back by hand instead of being overwritten.

Not a goal

  • Repairing a damaged record or merging its entries back. Its bytes are kept in a .corrupt-<utc> file beside it.
  • Refusing an arm. A record that cannot be opened (a transient EACCES or EIO) reads as empty and the arm goes ahead, as it does today; there are no bytes to keep from a file that could not be read.
  • forget_self_arm, which writes only after finding the entry it removes.

What changed (motivation → approach → change)

Symptom → cause: the read-modify-write started from a read that turns a damaged file into an empty one and then writes over it.

  • src/kiro_crew/autonudge_selfarm.py: record_self_arm reads through _read_record(set_aside_damaged=True), the same reader the checks use, with one new keyword. With it, a record that opens but is cut off or has the wrong shape moves to a .corrupt-<utc> name reserved by the existing sqlite_quarantine.reserve_quarantine_name (through replace_with_retry; a move that fails removes the reserved placeholder) before the arm writes the new record. A record that cannot be opened reads as empty, and a set-aside that fails is logged; in both cases the arm goes ahead, so this read never refuses an arm. Without the keyword the reader answers exactly as before.
  • test/test_lenient_read_rewrite_ratchet.py: record_self_arm stays listed in _KNOWN, now with the reason above, and forget_self_arm is marked _FOUND_ONLY.
  • test/test_autonudge_self_arm_unreadable_rewrite.py: new.

Backwards compatibility

Compatible: every arm that succeeds today still succeeds, and a readable record and a missing one behave as before. The one change is that a cut-off or wrong-shaped record is moved aside to a .corrupt-<utc> file before the arm rewrites it, instead of being overwritten. Removes nothing.

Tests

  • Before: on main b9ddf797c7, 1 of the 4 new tests fails: arming over a cut-off record sets it aside. The other 3 pass on main and pin behavior that must not change: an arm over a record that cannot be opened still arms, an arm whose set-aside fails still arms, and an arm with no record yet creates it.
  • After: 4 of 4 pass.
  • Every test file that names the self-arm record (12 files, among them the conductor patrol and wake reset, the member default patrol, the work-ledger wake delivery and the ratchet): 829 passed. All test/test_autonudge*.py files: 2074 passed.
  • flake8, isort and mypy pass on the changed files, and the repo's black gate (scripts/check_black_formatting.py) passes.
  • Reproduce: python -m pytest test/test_autonudge_self_arm_unreadable_rewrite.py -n0
  • Proof: PROVEN test/test_autonudge_self_arm_unreadable_rewrite.py::test_arming_over_a_cut_off_record_sets_it_aside_and_starts_again (head 5219448)

Manual verification

N/A: the tests drive record_self_arm on a real record in a temporary data home and read it back through is_recorded_self_arm.

Sibling sweep

Swept: the _KNOWN sites in test/test_lenient_read_rewrite_ratchet.py marked "matches the shape; not audited".
Sibling: src/kiro_crew/autonudge_selfarm.py -- changed here
Sibling: src/kiro_crew/apps/backend_runtime/pidfile.py -- not affected -- fixed in #36012
Sibling: src/kiro_crew/apps/dev_mode.py -- not affected -- fixed in #35910

Pattern harvest

Rule candidate: review-prompt
Pattern: a writer of an authorization record replaces a file it could not parse, so the bytes that held every other entry are gone.

Related Issues

None filed; found while auditing the unaudited sites of the lenient-read ratchet.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

@javenciu
javenciu requested review from a team and KiroCrewCaptain as code owners November 12, 2026 23:43
@javenciu
javenciu requested a review from buluoray November 12, 2026 23:43
@javenciu
javenciu enabled auto-merge (squash) November 12, 2026 23:43
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 12, 2026
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 52194486f0e94316d1c0ec40b662346448c0f357 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 52194486f0e94316d1c0ec40b662346448c0f357 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

No design-level concerns. The fix stays inside the lock-held read-modify-write, adds one opt-in keyword, and reuses the existing quarantine-name helper.

Notes

  • A record that exists but hits a transient EACCES/EIO is still overwritten with only the new entry. Its bytes are on disk, so "no bytes to keep" isn't accurate. When that happens, every other loop's entry is lost, the same as before this PR. Fixing it would mean refusing the arm, which the description rules out as not a goal; it deserves its own follow-up rather than a change here.
  • The set-aside keeps the old entries only for reading back by hand. Every other self-armed loop is still refused when it next fires, and nothing in the product tells the user that a .corrupt-<utc> file exists to recover from.

[DESIGN-REVIEWED] 5219448

Description read

[DESCRIPTION-READ] c21355153a6d5ddc44f645bc54da5bb4eb07e7d5bb32af4c1344d079b5903677

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 52194486f0e94316d1c0ec40b662346448c0f357 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. The fix works at the root of the bug: the arm used to read a damaged file as empty and then write over it. Its one test fails on base, so the defect is proven.

What this change ships

Inventory (4 items) — 4 justified

Intent: when a loop is armed over a cut-off or wrong-shaped self-arm record, the other loops' entries can still be recovered instead of being destroyed. FIX.

  1. Arming over a cut-off or wrong-shaped record first moves it to trust/autonudge-self-armed.json.corrupt-<utc>, then writes the new record. Justified.
  2. If that move fails, the failure is logged, any reserved placeholder is removed and the arm still goes ahead. Justified.
  3. A record that cannot be opened (EACCES/EIO) still reads as empty and the arm still goes ahead. This is unchanged and is now pinned by a test. Justified.
  4. In the lenient-read ratchet, record_self_arm now gives its reason and forget_self_arm is reclassified as _FOUND_ONLY. Justified.

[FIRST-PRINCIPLES-REVIEWED] 5219448

Description read

[DESCRIPTION-READ] 447b32ae8364da2d8f13cbd9764cff1c921ab8e750c430769f15f12945410575

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 52194486f0e94316d1c0ec40b662346448c0f357 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 5219448

Verdict parsed from the review's SHA-scoped output markers for commit 52194486f0e94316d1c0ec40b662346448c0f357.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 52194486f0e94316d1c0ec40b662346448c0f357 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 52194486f0e94316d1c0ec40b662346448c0f357 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 5219448

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 52194486f0e94316d1c0ec40b662346448c0f357 false-positive: <reason, 20 to 500 characters>

@KiroCrewCaptain

KiroCrewCaptain commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: Approve

Why: the decider estimates a 12% chance that this change introduces a defect, within the 12% limit for approving on its own, and every check passes.

record_self_arm now reads through _read_record_for_update. If the record opens but is cut off or has the wrong shape, it is moved to a .corrupt- file using the existing reserve_quarantine_name and replace_with_retry helpers, and a failed move deletes the reserved placeholder. Only then is the record rewritten. Paths that are not damaged-but-openable behave exactly as on main: a missing record creates it, a readable record keeps its entries, a record that can't be opened (EACCES/EIO) is still rewritten, and a failed set-aside still rewrites it. forget_self_arm only writes after finding its entry, so it can't destroy a damaged record. The fire-time readers still treat a damaged record as "not recorded", which refuses the fire. The set-aside file stays inside the trust/ subtree, which is protected as a whole directory, and nothing enumerates that directory. The new cut-off-record test fails on main, and the ratchet _KNOWN entries are consistent with the code. No regression or trust-boundary change found.

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 9c32e0a8aa · 129 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Nov 13, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the whole diff at 4f35d74, tests included. Most of it is good, and one part needs to change.

Good: a record that opens but is cut off or has the wrong shape is moved to a .corrupt-<utc> file and the arm carries on, so one damaged file no longer drops every other loop's entry and the arm still succeeds. That keeps the arm working where main also works, and I have no concern with it. The ratchet entry change is fine too.

Needs to change: _read_record_for_update now raises OSError when the record cannot be opened (a transient EACCES or EIO), and record_self_arm lets it out. On main the same call goes ahead and arms the loop. So this adds a new condition under which an arm fails where main lets it pass. It is a refusal even though its purpose is to protect the other loops' entries, and test_arming_over_a_record_that_cannot_be_opened_keeps_it_and_refuses pins it. The change should not make a self-arm fail where main arms.

Suggested direction, either of which keeps main's behavior by default:

  • On an unopenable record, log a warning and carry on as main does, keeping the corrupt-set-aside part for the cut-off case. Or
  • If the failing behavior is wanted, put it behind a setting that is off by default, so the default path arms as main does.

The test for the unopenable case would change to match whichever you pick. Everything else can stay as it is. I am not approving until that part is changed. The author's auto-merge is left alone.

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: fix (3 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: arming a self-arm loop no longer rewrites the record from an empty read when the file cannot be opened; a cut-off record is set aside first. Tests added.

@bolichen97

Copy link
Copy Markdown
Contributor

A follow-up to my review at 4f35d74, with a third option for the unopenable-record case that protects the data without a refusal.

When the record cannot be opened (a transient EACCES or EIO), record_self_arm could skip writing the record, log a warning, and let the arm carry on as it does on main. The other loops' entries stay untouched on disk because nothing is written over them, and the new arm succeeds. The only thing lost is the new loop's own entry in the record for that one call, which is the same entry a retry or the next arm writes once the file opens again.

That keeps the data protection this change is after and adds no condition under which an arm fails where main arms. The cut-off and wrong-shape case can stay as written (set aside as .corrupt-<utc> and start again). The test for the unopenable case would then assert that the file is unchanged, the arm does not raise, and a warning is logged.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision labels Nov 13, 2026
@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

This review found no blocking defect at head 4f35d74. It cannot approve while the maintainer change request on this head is open. A maintainer decides how an arm should behave when the self-arm record cannot be opened.

What the review checked on this head:

  • The review ran the new test file against the merge-base and the head: 2 of 3 fail before the fix, and 3 of 3 pass after it. The lenient-read ratchet test passes on the head.
  • The fire-time readers is_recorded_self_arm and is_recorded_default_patrol still use the lenient read and still refuse when they cannot read the record. The record is written before the loop is added.
  • forget_self_arm writes only after it finds the entry, so its new _FOUND_ONLY label is accurate.

Done well: the tests drive a real record in a temporary data home and read it back through is_recorded_self_arm. Nothing in the module under test is mocked.

For the change request decision:

  • The only production caller of record_self_arm is the authorizer (autonudge_authz.py around lines 1525 to 1542). The authorizer refuses a self-armed slot only when record_self_arm raises. Suppose an unopenable record were skipped inside record_self_arm with only a warning. The authorizer would then report the loop as armed, and is_recorded_self_arm would refuse every fire of it, with no message to the session. Going back to main's behavior means rewriting the record from an empty read, and that is the bug this PR fixes. If the refusal should go, it is safer to decide it in the authorizer than inside record_self_arm.
  • If the refusal stays, the Backwards compatibility section should list it instead of "Removes nothing." It applies to self-arms on crew and member slots and to the default ledger patrol. The set-aside rename at autonudge_selfarm.py:133 can also raise, and nothing catches that today. fix(apps): do not rewrite an app-backend pidfile that cannot be read #36012 catches the same rename in pidfile.py.

Optional:

  • No test covers the wrong-shape half of the set-aside. A mutation that overwrites a record with no loops key, without setting it aside, passes all 3 tests. A case that writes {"version": 1} or [] would pin it.
  • The record_self_arm docstring says it raises only on a failed write, and the owning spec in docs/system-specs/modules/learn-cron-dashboard.md does not mention the .corrupt-<utc> set-aside. One line in each would match the code.

@javenciu javenciu changed the title fix(autonudge): do not rewrite a self-arm record that cannot be read fix(autonudge): set a damaged self-arm record aside before an arm rewrites it Nov 13, 2026
@javenciu

Copy link
Copy Markdown
Contributor Author

Thanks, agreed: this read should never refuse an arm. ac392d9d21 changes it so an arm goes ahead in every case where it does on main:

  • A record that cannot be opened (EACCES or EIO) reads as empty and the arm proceeds, with a warning logged.
  • A cut-off or wrong-shaped record is still moved to a .corrupt-<utc> file first. If that move itself fails, it is logged and the arm proceeds too.

The unopenable-case test now pins that the arm succeeds, and a new test pins the same for a set-aside that fails. Both pass on main, which is the point. The set-aside test is the only one that fails on main. record_self_arm is back in the lenient-read ratchet's _KNOWN with that reason, and the title and description now say what the change does. The 687 tests in the self-arm, authorizer, conductor and ratchet files pass.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 13, 2026
@javenciu
javenciu force-pushed the fix/lenient-self-arm-strict-rewrite branch from ac392d9 to 47bb927 Compare November 13, 2026 04:06
@javenciu

Copy link
Copy Markdown
Contributor Author

Rebased: ac392d9d21 -> 47bb9271f7 onto main 27e9359419, with no change to the diff. The Frontend Tests failures came from the i18n [changed-values] style checks on fr, ru and bn strings that main changed after the old base, and the GPT line finding points at src/kiro_crew/dashboard/handlers/secrets.py, which this PR does not touch. It changes only src/kiro_crew/autonudge_selfarm.py and two test files.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 13, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 13, 2026
@javenciu
javenciu force-pushed the fix/lenient-self-arm-strict-rewrite branch from 47bb927 to 9c32e0a Compare November 13, 2026 08:34
@javenciu

Copy link
Copy Markdown
Contributor Author

The First Principles note is right, and 9c32e0a8aa takes it. The set-aside now gets its name from sqlite_quarantine.reserve_quarantine_name, which already picks a collision-free .corrupt-<utc> name, instead of building a stamp inline. A move that fails removes the reserved placeholder, and the test that covers that case checks no .corrupt- file is left behind. The fix is still two commits, and the 687 tests in the self-arm, authorizer, conductor and ratchet files pass.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running labels Nov 13, 2026
record_self_arm read autonudge-self-armed.json with a reader that turns an
unopenable or cut-off file into an empty table, then wrote back only the new
entry. Every other loop's entry was gone, and each of those loops was refused at
every fire although it had been armed by its own session.

The rewrite reads through a strict reader. Only a missing file is empty. A file
that cannot be opened raises OSError and stays as it is, which the authorizer
already answers with "self-arm record unavailable". A file that opens but is cut
off or has the wrong shape is moved to a timestamped .corrupt- file and the
record starts again, so one damaged file does not stop every later self-arm.

The lenient-read ratchet drops record_self_arm and marks forget_self_arm
audited: it writes only after finding the entry.
@javenciu
javenciu force-pushed the fix/lenient-self-arm-strict-rewrite branch from 9c32e0a to 1f99b05 Compare November 13, 2026 10:27
@javenciu

Copy link
Copy Markdown
Contributor Author

Rebased: onto main b9ddf797c7, which has the RFC index fix from #36504. Source Gates on the old head failed only because main's own docs-lint failed at the time (rfc-app-agent-tools-third-party-grant.md had dropped out of the index). The two commits change the same lines as before. On the new base the one new test still fails on main and passes here, and the 12 test files that name the self-arm record pass (829 tests).

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 13, 2026
…e read

An unopenable record reads as empty and the arm proceeds, and a damaged record
that cannot be set aside is logged and the arm proceeds too, so this read never
refuses an arm that would succeed without it. A cut-off or wrong-shaped record is
still moved to a timestamped .corrupt- file first, so its bytes are kept.

The lenient-read ratchet lists record_self_arm again, with that reason.
@javenciu
javenciu force-pushed the fix/lenient-self-arm-strict-rewrite branch from 1f99b05 to 5219448 Compare November 13, 2026 12:15
@javenciu

Copy link
Copy Markdown
Contributor Author

The First Principles note is right, and 52194486f0 takes it. record_self_arm now reads through _read_record(set_aside_damaged=True), so the record is parsed and checked in one place: the separate _read_record_for_update and the _valid_entries split are gone, and the only new code is the move-aside branch (_set_aside_damaged). Without the keyword the reader answers exactly as it does on main. The lenient-read ratchet still lists record_self_arm, because a record that cannot be opened still reads as empty. The four new tests pass, the cut-off-record test still fails on main, and the 12 test files that name the self-arm record pass (829 tests).

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 13, 2026
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 13, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the whole diff at 5219448, tests included. This supersedes my earlier request for changes.

A damaged self-arm record (cut off or the wrong shape) is now moved to a .corrupt-<utc> file before the arm rewrites it, so its bytes are kept, and a failed set-aside is logged and does not stop the arm. A record that cannot be opened reads as empty, as on main, so an arm never fails on it. The change adds no new refusal: every case still arms, and the tests cover the cut-off record, the unopenable record, a set-aside that cannot happen and the no-record case. The unopenable-record path loses no more than main does.

One non-blocking idea for a follow-up: when the record cannot be opened, skipping the write with a warning would also keep the other loops' entries, which main loses today.

All checks that are live are green. The four cancelled runs on this head each have a newer passing run. The author holds auto-merge, so I approve without it.

@javenciu
javenciu merged commit ca48438 into main Nov 13, 2026
100 of 105 checks passed
@javenciu
javenciu deleted the fix/lenient-self-arm-strict-rewrite branch November 13, 2026 15:09
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants