Skip to content

fix(aws-control): back up the kiro-cli terminal conversations, token-safe (#12237) - #12485

Merged
iamwhatever merged 2 commits into
kirodotdev:mainfrom
chenmingwei23:fix/backup-cli-conversations-12237
Sep 24, 2026
Merged

iamwhatever merged 2 commits into
kirodotdev:mainfrom
chenmingwei23:fix/backup-cli-conversations-12237

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

run_sessions_backup packs two trees -- the crew sessions dir and <kiro home>/sessions/cli (the JSONL transcript pairs). The terminal's OWN conversation store is neither: kiro-cli keeps its chat in conversations_v2 inside ~/.local/share/kiro-cli/data.sqlite3, and that store is never named. An operator who enables the off-host backup (and nightly_sessions, #12190) still has no copy of their terminal conversations, while the manifest lists a cli tree and the run reports success.

Why it matters

The two stores are disjoint, not redundant: the JSONL pairs are the gateway's resume path (ACP session/load); the SQLite store is the terminal's. Losing the host loses every terminal conversation the backup was enabled to protect, and nothing about the failure is visible from the artifact. This is the opposite of cosmetic.

What changed

The symptom is a missing conversations/ root in the archive. The root cause is that the only place the terminal's chat lives is a file the backup is forbidden to carry: hooks.py classifies data.sqlite3 as a token path and #8085 recorded that it holds live bearer tokens, so tarring it would upload live credentials off-host -- strictly worse than the gap it closes. So the fix cannot be "add the file"; it has to be "carry the conversation rows without carrying the file".

The archive carries a table-scoped export. _export_cli_conversations (backup.py:3975) opens the store READ-ONLY and:

  1. Resolves the store through identity_stores.state_db_candidates against FIXED, home-anchored locations, deliberately not consulting the environment -- not XDG_DATA_HOME on POSIX, not LOCALAPPDATA or APPDATA on Windows. The fence that makes this store unreadable and unwritable by agent file tools is home-anchored (security.paths splices identity_stores.fenced_home_dirs, whose comment records that "a profile redirected outside the home directory is not covered"), so a redirected root sits OUTSIDE the fence, where an agent can author rows. This function feeds an archive uploaded off-host unattended, so honouring the variable would let an agent plant rows in a conversations_v2 table at a writable location and have a scheduled backup ship them; an uploaded object cannot be un-sent. kiro_prerequisite already records this decision for the same two variables and states the rule: a fixed anchor cannot be pointed at something the agent may write. The cost is real and is the direction chosen: an install whose store is relocated by one of those variables backs up no terminal conversations, reported in the run record as conversations_skipped. Being wrong that way is recoverable; the other way is not.

  2. Refuses a candidate reached through a redirection, and the order of that check is the guard: platform_compat.first_linked_ancestor runs BEFORE any is_file() on the candidate, because an ancestor junction whose target is \\host\share turns a stat on a local-looking path into an outbound SMB handshake that authenticates as this process, inside the stat, before any check of ours could reject it. A leaf-only test cannot see an ancestor. The leaf is then tested with platform_compat.is_link_or_junction, the predicate the rest of this module already uses, because islink answers False for a Windows junction. A fixed anchor bounds where a candidate may live but does not stop a link planted AT that anchor, so both guards are needed.

  3. Reads under a SINGLE read-only snapshot. The connection is mode=ro, so it writes no database page and runs no checkpoint against the operator's live store. That is not a promise of zero filesystem writes: on a WAL store SQLite may still update the -shm sidecar to take a read mark, which is how a reader participates in WAL at all. A mode=ro connection cannot checkpoint, and does not need to: SQLite applies committed -wal frames transparently on read, so one deferred read transaction (BEGIN) pins a consistent view including the WAL for the whole copy. A non-empty WAL is the normal steady state of a live terminal, so an approach that checkpointed-or-refused would silently omit conversations on the common path (a read-only checkpoint raises disk I/O error). Opening the store goes through the sanctioned credential-read audit (hooks.emit_internal_read_audit under the registered id aws_control.conversation_export) and FAILS CLOSED: an export whose access cannot be recorded is dropped rather than shipped unaudited.

  4. Copies ONLY the tables in _CONVERSATION_TABLES, an allowlist currently ("conversations_v2",). No identity or token TABLE can reach the archive even if kiro-cli adds a credential table later; a denylist would fail open the day that happened, an allowlist fails closed.

    The export is table-scoped, not column-pinned. _copy_table moves rows through a named column list built from PRAGMA table_info rather than a literal SELECT *, and that buys a stable, quoted column order shared by the SELECT and the INSERT -- it is not an allowlist and it holds no column back. A column added to conversations_v2 upstream, including a credential-bearing one, is enumerated by the same PRAGMA and copied. The fail-closed boundary is the table set, one level up. Whether an unknown column should be dropped or should refuse the export is a product decision and is tracked on aws-control: a dropped conversation export reads as "no store" in the run record, and the v1 conversations table is outside the allowlist #13142, not decided here.

The skip is reported, not silent, and it survives an unchanged run. Restricting to fixed anchors means a host that relocates its store backs up no terminal conversations, and a silent skip there is how an operator ends up believing they hold a backup they do not hold. So the run record carries conversations_skipped="store_relocated_outside_fence" when the environment names a store location this export declines to read. The reason travels on the export result beside rows and members; the rest of the archive still builds and still uploads, because the refusal is worth reporting and not worth failing a backup over. Detection is a comparison of two candidate path sets and touches no file at all -- deliberately, since probing a path outside the fence is what the ancestor guard above exists to prevent. A run that skipped nothing carries no such key, so the key's presence is itself the signal.

Every skip is named, and retention cannot make a gap permanent. An unreadable store, a missing allowlisted table and an unrecordable credential audit each carry their own reason, on the same standard the relocation path set -- a skip nobody can see is the failure, whichever branch produced it. And a run whose export came up short for a reason an earlier archive may not have shared SKIPS the retention sweep: the sweep protects only the key this run uploaded, so at keep=1 retiring the previous archive would erase the one copy that still held the conversations, and delete_object_versions erases versions outright. The gap recovers on the next successful run; the retired object does not. The suppression is a predicate on the field, not membership in a list of qualifying reasons: the sweep is skipped whenever conversations_skipped is non-empty. An earlier revision kept a frozenset of reasons and excluded the relocation one, on the argument that a relocation is a standing property of the install so no archive ever held those conversations. That argument is wrong and nothing enforced it -- the flag is read from THIS PROCESS's environment, so a daemon-launched run and a shell-launched run can disagree about XDG_DATA_HOME with the operator relocating nothing, and one run can write a complete archive while the next omits the conversations and prunes the only copy that held them. Once that reason is included the set holds every reason the module emits, so it was only a slower way of writing "any reason at all" -- and a sixth reason added later cannot be forgotten from a predicate, while it can absolutely be forgotten from a frozenset.

It does not over-suppress, and that direction is tested too -- but the line is not where it first looks. What may stay reasonless is a run that READ everything the host holds, or one where the operator withheld the permission. An ABSENT store does not qualify, and that was the last place this diff still reasoned about the store's state NOW while retention decides the fate of an archive written EARLIER: a store wiped to clear corruption, dropped by a reinstall, or on a volume not mounted at nightly-run time was present when last week's archive was written, so that archive holds rows this run cannot carry. Nothing pins a store's presence from one run to the next, which is the same reason the relocation flag could not be treated as standing. Absence now reports store_absent and suppresses the sweep.

A refused store is not an absent one. _kiro_cli_conversation_db returns (store, reason) rather than store | None. It declines a candidate whose leaf or ANY ancestor is a link, and that ancestor walk runs from / down and includes the home directory -- so an ordinary symlinked ~/.local/share, or a symlinked home, took that exit on every run. Returning a bare None made "I refused to read a store that is there" identical to "there is no store", and only the second is safe to prune on. Rejection reports store_rejected_link, an unreadable candidate reports store_unreadable, and nothing at any candidate reports store_absent. The second return value is never empty when no store was used.

The relocation detector compares only variables that MOVE a candidate, and APPDATA is not one. An earlier revision also compared the named roots against the home directory, to cover the Windows Roaming row, whose candidate state_db_candidates anchors to home whatever mapping it is given. That comparison is removed, because the premise under it does not hold: the same function's docstring records that "APPDATA is deliberately NOT followed here ... the current generation writes the LOCALAPPDATA location, and the roaming default is retained only as a legacy fallback". A store kiro-cli does not write to cannot be relocated away from this export, so an APPDATA mismatch is not evidence of a relocation -- it is evidence of a redirected Roaming folder, which is an ordinary enterprise configuration. Comparing it reported a relocation on every such host and so suppressed retention there permanently, while a readable Local store sat beside it inside the fence: the same unbounded-accumulation failure this PR spent its other rounds removing.

Nothing is given up by removing it, which is what makes it subtraction rather than a regression. LOCALAPPDATA and XDG_DATA_HOME re-root their own candidate and are caught by the set difference. A legacy install that really does keep its store at a redirected Roaming root has nothing at either fixed candidate, so the lookup reports store_absent and the sweep is suppressed on that path instead -- verified by test, not by reading the code, since a deletion whose safety rests on an untested claim is not a deletion worth making.

A stale leftover no longer masks the live store. The candidate table lists Local (the current layout) before Roaming (legacy), so taking the first cleared candidate would export an abandoned root's conversations and record success. identity_stores.selected_store arbitrates exactly this state by write time, and the reviewer was right that it should not be re-derived. It is NOT wrapped, though, and the measurement is the reason: that function stats its own candidates before anything has checked them for redirection, so calling it would put the outbound stat back ahead of the guard -- the hole round 1 closed. What is reused is its READING, _store_write_time, called only on candidates these guards have already cleared. That primitive reads the -wal sidecar as well as the main file, because a commit lands in the sidecar and the main file's mtime does not advance until a checkpoint, so a live store under write looks older by its main file alone. Ties keep table order, preferring the current layout, as selected_store does.

Suppressing the sweep suppresses the DELETION, never the audit. _audit_retention is only reachable from inside _prune_remote_archives, so skipping the sweep left the one path in the app that erases object versions permanently with no SEL event at all -- and that function's contract is that every terminal outcome files one, "including the ones that deleted nothing". A decline is a terminal outcome, and it is now filed as failed with the reason as error, matching the sweep's own refusal-to-act on a listing that does not show the archive just uploaded. This is also what keeps the accumulation visible: while the condition persists the archives pile past the keep count, and one event per run naming the reason is how an auditor sees that rather than inferring it from a sweep that silently never ran.

Every exit was walked against one sentence, rather than against the lines each review round named: every exit meaning "we tried to read this host's store and did not" carries a reason; only a run that read everything, or one the operator declined, may be reasonless. The sentence needed that qualifier. Its first form exempted "genuine absence", which reads as obviously safe and is not: absence describes the store now, and the question is what an earlier archive holds. Ten of the eleven _ConversationExport exits pass. The eleventh is the Layer B gate, and it is deliberately left reasonless: sessions_layer_b_enabled defaults false, so a reason there would fire on every default install and suppress retention everywhere, permanently. Unlike a relocation, the standing-state premise is genuinely enforced for a permission -- it is persisted account state read through _account_view, not per-process environment, so two runs cannot disagree about it by accident.

The residual that leaves is permission REVOCATION: granted one night and withheld the next, the older archive holds conversations the newer one does not, and at keep=1 the newer run prunes it. That is a consented withdrawal rather than a failure to read, and the general fix -- retention keyed on what each archive holds rather than on the key the current run uploaded -- covers it together with every other instance. Filed as item 5 on #13142. So the invariant's antecedent is properly "we TRIED to read it and could not": policy-declined is a third category alongside failure and absence.

The accepted cost, stated rather than engineered around: an install that stays in any of those states never prunes, so its archives accumulate until the operator resolves it. Lowering keep does NOT release it, and that correction replaces an earlier claim in this body that it would: the decline is not a function of the count. On this path the sweep is declined before the count is consulted, and the in-lock re-check refuses after the count has been re-read, so a declined sweep retires nothing at any value, including zero. What accumulation costs is storage; what a wrongly permitted sweep costs is the only copy. That is honest and recoverable -- the record says why on every run, and a retention SEL event fires every run naming it. The cost is bounded by the permission being owner-gated and OFF by default: an ordinary install prunes exactly as it did before this feature existed, and only a host that opted INTO conversation backup and then cannot be read stops pruning. For store_absent and store_rejected_link that combination is a standing misconfiguration the record now names -- Layer B enabled with no readable store -- rather than a working state. Making retention protect the newest COMPLETE archive instead of the just-uploaded one is the correct general fix and is tracked on #13142, not attempted here: it means changing shared code that deletes objects, and it has to answer what "complete" means for archives written before these fields existed.

Every exit of the export was enumerated, not just the one a reviewer named. Each return that carries no data was decided explicitly:

exit reason reported why
relocation detected store_relocated_outside_fence per-process env, so retention is suppressed too
discovery raised (Path.home(), candidate walk) store_discovery_failed guarded, or it would fail the whole backup
no store at any fenced location store_absent an earlier archive may hold rows from before it went missing
Layer B withheld none, deliberately consented withdrawal, and the default -- see below
store unreadable (OSError / sqlite3.Error) store_unreadable transient; an earlier archive may hold more
no allowlisted table present no_conversation_table same
credential audit could not be recorded credential_audit_unavailable same
store read, a value could not be sanitised conversations_unredactable dropping the row ships an archive a restore reads as complete; the raw value ships the credential
a field wider than the per-cell ceiling conversations_oversized the row count alone bounds nothing, so the width is refused before the first fetch
the copy exhausted memory conversations_memory_exhausted caught rather than propagated, so an allocation failure costs this member and not the transcript archive
the masked scratch root was unusable scratch_root_unusable the export has nowhere the agent cannot reach to write, so it carries nothing
the host cannot pin a directory traversal scratch_pinning_unavailable explicitly not applicable rather than a weaker read invented for it
the scratch file failed its pinned checks scratch_export_unsafe raised before the first tar write, so a substituted file costs the member and not a damaged archive
export succeeded none rows and members carry the answer
failure WRITING members into the tar none -- propagates addfile raising part-way leaves the archive undefined; uploading a damaged tarball under a success record is worse than losing the run

The scratch-directory creation is not guarded either, and that is reachable-by-construction rather than an omission: run_sessions_backup already created its own temporary directory and opened the tar inside it before calling this, so a host that cannot make one failed upstream and never arrives here.

_record_skip forwards both conversations_skipped and layer_b. That function REPLACES the run slot, and the sessions path takes it whenever the tree has not moved -- the ordinary nightly case -- so a field it did not forward would be erased, and the first unchanged run would quietly restore an assertion of complete coverage over a run that had reported a gap. The caller passes what it measured on the current run rather than copying the baseline's answer.

The relocation is detected BEFORE the fixed anchor is consulted, which is the ordering that matters rather than an implementation detail. A file may still sit at the old anchor after the environment says the store moved; checking the anchor first would export those stale conversations and report complete coverage, because the relocation would only become visible when the fixed lookup found nothing. identity_stores.selected_store arbitrates the same leftover-in-the-abandoned-root state by mtime, so this is a state the codebase already expects. A store the environment has moved away from is stale by definition, and exporting stale conversations under a record asserting complete coverage is worse than exporting nothing and saying so.

The export carries emitted archive members and copied rows as separate quantities, because they disagree on a real path: a present-but-empty allowlisted table is carried so a restore sees the real schema, which is a conversations/ root with two members and zero rows. Measuring content by rows alone let the "no session files to archive" guard discard members the tar already held, so the guard now consults both and the run record's layer_b flag keys off members.

The export rides under a new conversations/ archive root (beside crew and cli) with a CONVERSATIONS_MANIFEST.json recording the table set and per-table row count.

The export rides the Layer B permission

This is the one behavioural decision a reviewer should look at, and it changed while rebasing onto #12178.

#12178 landed an operator permission, sessions_layer_b_enabled, that gates the cli half of the archive. Its stated reason is sensitivity class, not path: the crew transcript is what was DISPLAYED, with display-time redaction applied, while Layer B is what the model actually held, unredacted, and cannot be redacted on the way out. Default OFF, because being wrong in the ON direction puts unredacted context somewhere it cannot be recalled from.

conversations_v2 is in that same class -- it is the terminal's own conversation content, what the model actually held rather than what a UI displayed. Its exported rows ARE scrubbed of credentials and exfiltration URLs by _redacted_row, which removes secrets and not the conversation, so the sensitivity class the permission prices is unchanged. So the export rides the SAME permission (backup.py:4157) rather than the crew half's terms. An ungated export would have carried unredacted terminal context out of an install whose operator had withheld exactly that, through a second path the permission does not watch. Three consequences, all deliberate:

  • A withheld run carries no conversations/ root, whatever the host's store holds.
  • The withdrawal recheck already in run_sessions_backup (refuse-on-withdrawal, keyed off layer_b) now covers the export too, at no extra code.
  • The run record's layer_b flag is (layer_b_files + conversation_rows) > 0 (backup.py:4316), so an archive carrying conversation rows is recorded as holding Layer-B-class content even when the cli tree was empty. The record describes the ARCHIVE, which is fix(aws-control): gate Layer B in the sessions archive on the operator's permission #12178's own rule for that field.

An earlier revision of this PR said the export "rides the existing nightly_sessions consent -- no third grant". That is no longer accurate and the module header docstring now states the permission it actually rides.

Design boundary (needs-human)

What counts as "conversation state" beyond conversations_v2 is the open question. This ships the defensible minimal version -- conversations_v2 only -- and declares the boundary in the module header docstring, which is the single place a reviewer looks and the single place widening _CONVERSATION_TABLES is expressed.

The module spec is corrected, because this change falsified two of its statements. docs/system-specs/modules/aws-control.md said _prune_remote_archives runs as the last step of every successful push, which is now conditional, and that the run record's layer_b comes from the kiro-cli file count, which is now either that count or conversation rows. It also described neither the conversations/ archive root nor the conversations_skipped field. Both corrections are made and both surfaces are documented, including that any reason in that field suppresses the sweep and that a declined sweep still files its retention event. Nothing else in the spec is touched: no restructuring, nothing documented that this PR did not change, and none of the deferred #13142 items. A statement this change made false is part of this change, on the same standard applied to the comments above -- a claim the code does not implement is worse than no claim, and a spec is read by more people than a comment.

The export's byte determinism is not pinned by a test, and the cost of that is stated rather than guessed. The exported database is an archive member, so its bytes feed _tree_fingerprint, which digests every member's path, kind, mode, size and content hash. Two causes must not be confused. A SQLite version change shifts the fingerprint once, costing one extra upload, after which it is stable again at the new version -- that one is self-correcting. Byte non-determinism WITHIN a version is different and much worse: the fingerprint would differ every run, so the unchanged-run skip would never fire again on any Layer B host, and each night would re-upload the whole terminal history rather than nothing. Measured across three runs on SQLite 3.50.4 the fingerprint is stable, so the second case is not observed here; it is also not guarded, and a test that pinned it would go red on a version bump for something that is not a defect. The trade is deliberate: no test, and the real exposure written down. Recorded on #13142 beside the other deferred items.

The Design Review's second watch item is also already tracked there and is not addressed here: a store that was never readable has no historical archive to protect, so suppressing retention on that host protects nothing and only accumulates. Its suggested mechanism -- condition the suppression on a persisted record that some earlier archive actually carried conversations -- is on #13142 as part of item 5, because it reads prior run records from inside the code that deletes object versions.

Backwards compatibility

Additive. Existing archives are unaffected. A new archive gains a conversations/ root only on an install whose operator has granted Layer B AND whose store sits at a fixed, home-anchored location. An install that relocates its store with XDG_DATA_HOME / LOCALAPPDATA gets no conversations/ root, deliberately: that root is outside the agent-file-tool fence and this archive is uploaded off-host unattended. On a platform without a store, or a store that cannot be read consistently, the export carries nothing and the transcript halves back up as before. One behaviour does change for those installs: the run reports why under conversations_skipped, and that suppresses the retention sweep for that run, so archives accumulate past the keep count while the state persists. Only an install that granted Layer B reaches it, since the export does not run otherwise.

The scratch export is descriptor-pinned

The conversation export writes a scratch copy into a TemporaryDirectory and then adds it to the tar. That add used to stat and open the file BY PATH, justified in a docstring by the directory being "private". The justification was wrong: mkdtemp mode 0700 excludes other USERS, not the same-UID agent this codebase's own sandbox notes describe planting links in the world-writable root, and the payout of the swap is a host file uploaded off-host with no recall.

It is now opened relative to a pinned directory descriptor with O_NOFOLLOW, and the descriptor is checked for a regular file and st_nlink == 1 before any byte reaches the archive, with the member size taken from that same fstat so the header cannot describe one file while the body streams another. Each of the three checks stops a different substitution, and each has its own mutation test: O_NOFOLLOW for a symlink, st_nlink for a hard link (which defeats the other two by construction), S_ISREG for a FIFO. The directory open is O_NOFOLLOW too, so replacing the scratch directory whole does not redirect the read. The scratch file is written under the agent-masked app data root, not the system temp directory. That ordering is the point: descriptor pinning cannot rescue a shared temp root, because a same-UID agent that replaces the temp DIRECTORY before the open hands over a directory of its own, in which all three checks pass on a file it chose. The masked root removes the reachability and the pinning is depth behind it. The root is guarded against a link planted at it, and the resolve is re-checked after mkdir because exist_ok=True accepts a pre-existing link. On a platform with no descriptor-pinned open the export reports scratch_pinning_unavailable rather than inventing a weaker read -- os.open cannot open a directory on Windows, O_NOFOLLOW/O_DIRECTORY do not exist there, and st_nlink from fstat is not a dependable link count -- which costs nothing in practice because run_sessions_backup already refuses the whole sessions kind on such a host.

The conversations-retained field is set on the run record BEFORE _locked_state_update, beside layer_b_scope, for the reason sequence is assigned there: the update can abort before its mutate ever runs (the read raising, a scanner holding the file, non-UTF-8 bytes, the lock timing out), and the recovery path then carries whatever the record already says. The account-level key stays inside the mutate, since it is part of the document being written.

A retention sweep declined inside the lock now audits as a failure with its reason. The decision rides on the exception rather than being re-derived from the reason string, because an unrecognised reason falls to the handler's successful branch with an empty error, which would make a refused permanent delete byte-identical in the SEL record to a healthy sweep that retired nothing. The suppression stops the delete, never the audit.

The OUTER archive has the same shape and is deliberately NOT changed here. It is built in the same shared temp root and re-opened by path four times after the tar closes -- the entry-set fingerprint, os.path.getsize, the AWS CLI --body, and the body fingerprint -- with no descriptor held across them. That is pre-existing on main and reaches storage.put_file, the shared upload helper every backup kind uses, so fixing it means changing how that helper receives its payload. Filed as #13259 rather than folded in. This change hardens the one file it introduces.

A refusal reports scratch_export_unsafe, which carries a reason and therefore suppresses the retention sweep, and it happens before the first tar write so a refusal never leaves a damaged archive.

Tests

test/test_aws_control_backup_conversations.py (new) pins the export itself, each box mutation-verified in both directions:

  • No auth byte reaches the archive (test_no_byte_of_the_auth_half_reaches_the_archive): the fixture DB carries a token table and a token column; the test asserts neither name nor the token value appears in the exported bytes, and the exported schema lists only conversations_v2. MUTATION: adding the token table to _CONVERSATION_TABLES leaks the token value and the test reddens.
  • Archived count matches source (test_archived_conversation_count_matches_source, test_return_value_is_the_source_row_count): manifest count, exported rows and return value all equal the source's 7. MUTATION: stopping _copy_table after the first partial batch drops the count to 3 and both redden.
  • Live-WAL rows exported (test_committed_rows_in_a_live_wal_are_exported): a store with a non-empty, non-checkpointed WAL still exports every committed row. MUTATION: reintroducing wal_checkpoint(TRUNCATE) on the read-only connection makes it raise and the export returns 0.
  • Credential-read audit fails closed (test_a_failed_credential_audit_drops_the_export, test_a_successful_export_emits_the_credential_read_audit, test_the_conversation_read_id_is_registered_in_hooks).
  • An env-relocated store is NOT consulted (test_an_env_relocated_store_is_not_consulted): a store placed via XDG_DATA_HOME is skipped, because that root is outside the agent-file-tool fence. MUTATION: pass os.environ instead of {} and it reddens. Plus a missing store as a silent no-op, and a present-but-empty table carried with its schema.
  • A linked ancestor is refused before the store is stat-ed (test_a_linked_ancestor_is_refused_before_the_store_is_stat_ed): is_file is replaced with a raiser for paths under the linked ancestor, so reaching the stat fails the test. That pins the ORDER, not just the refusal -- a leaf-first implementation also returns None for this layout and would pass an outcome-only assertion. MUTATION: restore if db.is_file() and not is_link_or_junction(db) and it reddens with the sentinel.
  • An emitted-but-empty export is not discarded (test_an_emitted_but_empty_export_is_not_discarded_as_nothing_to_archive): both transcript halves empty plus a zero-row store still uploads, so rows alone cannot carry the run past the guard and the members have to. MUTATION: guard on count == 0 alone, or fold members into the row count, and it reddens with RuntimeError("no session files to archive").
  • A relocated store is reported, not silently skipped (test_a_relocated_store_is_reported_not_silently_skipped): the record carries conversations_skipped, and the rest of the archive still uploads. MUTATION: restore the silent skip and it reddens with KeyError. Its complement (test_a_run_that_skipped_nothing_carries_no_skip_key) pins that the key is absent when nothing was skipped.
  • The skip key survives an unchanged run (test_the_skip_key_survives_an_unchanged_run): runs the same backup twice with an upload the baseline probe can prove, and asserts the second record still carries both coverage fields. MUTATION: drop conversations_skipped from the _record_skip call and it reddens with KeyError.
  • A discovery failure reports and does not fail the backup (test_a_discovery_failure_reports_and_does_not_fail_the_backup): Path.home() raising must not throw away a correct transcript archive. MUTATION: remove the try/except around discovery and it reddens with the RuntimeError.
  • An unreadable store is named in the record (test_an_unreadable_store_is_named_in_the_record): MUTATION: return a bare _ConversationExport(0, 0) from the except branch and it reddens.
  • An incomplete export does not let retention retire the older archive (test_an_incomplete_export_does_not_let_retention_retire_the_older_archive) and its two siblings (test_a_relocation_also_stops_retention, test_a_run_that_reported_nothing_still_prunes), so the predicate is pinned in both directions. MUTATIONS: restore the relocation exclusion and the first reddens; suppress unconditionally and the second reddens.
  • A stale store at the fixed anchor does not mask a relocation (test_a_stale_store_at_the_fixed_anchor_does_not_mask_a_relocation): a readable store at the anchor AND an active relocation; the stale store must not be exported. MUTATION: move the relocation check back under if db is None and it reddens.

Comment sweep

Three reviewer rounds each found a comment in this change claiming a security property the code does not enforce, so every comment the diff adds was read once against the code rather than fixed one per round. Five claims were corrected, the fifth found by walking the whole path asking only whether anything happens before the check that would have prevented it:

claim why it was wrong
"EXPLICIT column list ... never SELECT *" the list is built from PRAGMA table_info, so it is not an allowlist and holds no column back
"no identity row, no token column" can reach the archive _copy_table copies every column an allowlisted table declares
mode=ro means this "can never write" the store a WAL reader may still update the -shm sidecar; the claim holds for database pages and checkpoints
"One name so the reader and the registry cannot drift" hooks.py holds its own literal, so they are two strings; a test is what keeps them equal
"Opening the store goes through the ... audit" and fails closed the audit RECORDS the read after it happens; it is an access log, not an authorization. What fails closed is the shipping

The module header now also records the fixed-anchor restriction and the per-table bound, so a reader who stops at the header is not left with the stronger claim.

test/test_aws_control_backup.py::TestSessionsArchiveLayerBGate gained three tests pinning the permission decision above, both mutation-verified:

  • test_withholding_also_withholds_the_conversation_export -- a withheld run carries only crew/t.jsonl even with a populated store. MUTATION: dropping the if layer_b guard at the call site reddens it on the two conversations/ members.
  • test_the_permission_lets_the_conversation_export_ride -- with the permission recorded, both conversations/ members ride beside both transcript halves.
  • test_conversation_rows_alone_are_recorded_as_layer_b -- an empty kiro-cli dir plus exported rows still records layer_b=True. MUTATION: recording layer_b_files > 0 alone reddens it.

That class's autouse fixture now stubs _kiro_cli_conversation_db to None, and test_aws_control_backup_unchanged.py keeps its one-line isolation, because those tests assert on the aggregate archive and would otherwise pick up a live store on the test host.

test_an_absent_store_also_stops_retention pins that nothing at any candidate suppresses the sweep, and test_a_host_with_no_store_reports_absence pins the resolver reporting store_absent. They are split deliberately: the caller class stubs the resolver out so its member-list assertions are not perturbed by whatever store the CI host happens to have, so a single end-to-end test there would take its reason from the stub and no mutation of the resolver could redden it. test_a_run_that_reported_nothing_still_prunes holds the other direction of the predicate and now uses a SUCCESSFUL export, which is the case the no-over-suppression claim is actually true of.

335 tests pass across test_aws_control_backup.py, test_aws_control_backup_conversations.py and test_aws_control_backup_unchanged.py with -n0. Local gates clean: black, isort, flake8, mypy on backup.py and hooks.py, and the comment-history gate against this PR's base.

Not run against the live host store: it is credential-bearing and behind the path fence, so the export is exercised end to end against synthetic stores built in-test carrying both a conversation table and a token table.

Any other suggestions on the work

The export rides sessions_layer_b_enabled rather than a toggle of its own. That is decided here, not left to the reviewer. #12178's rule for this gate is that it is priced by the payload's sensitivity CLASS, not by whether a path is named in the permission's docstring, and conversations_v2 is that class: model conversation content. It differs from the cli half in one way that does not change the class -- its rows CAN be scrubbed on the way out and are, by _redacted_row, while the cli half cannot be because its thinking blocks carry a provider signature over their own content. Scrubbing removes credentials, not the conversation, so what the permission prices still ships.

Two alternatives were rejected.

A second, export-specific grant buys exactly one distinction -- an operator who wants terminal conversations but not Layer B. Nobody has stated that need, and the cost is concrete: a second operator grant, a second withdrawal recheck inside the upload gate, and a new product surface added inside a PR scoped to an export.

Leaving the export ungated, as the pre-rebase revision had it, is a second path into the class the permission governs. That is how a gate stops working without anyone removing it.

What the grant COVERS is named in the grant's own description, and the grant now records which scope it was made under. The two payloads differ in REACH even though their sensitivity class is identical: the cli half is this product's own session files, while conversations_v2 records every interactive kiro-cli use on the host, including work unrelated to this product's sessions. An operator reading only "unredacted context in the sessions archive" would price the first and receive both.

Naming it was not enough on its own. This change altered what an ALREADY RECORDED authorization authorizes, which is not an adjacent concern -- it is this change's own blast radius. A pre-existing sessionsIncludeLayerB=true would have shipped host-wide terminal context on the next scheduled run, off-host and unrecallable, on a consent that never mentioned it. So the export either does not go out, or the consent scope goes out with it.

The grant carries a scope marker, and it is not a second toggle. The permission stays one boolean and the operator gains no new control. sessionsLayerBScope records which payloads the recorded decision covers. A grant carrying cli+conversations covers both; a grant with no marker, or any value this code does not recognise, covers the cli half only -- the same fail-closed posture sessions_layer_b_enabled already takes on an unparseable grant. set_sessions_layer_b stamps the marker only when the CALLER NAMES that scope, through an optional scope field on the same POST /backup/{account}/layer-b request. The route keeps its shape and its meaning: enabled is still the only required field, a bare {"enabled": true} records the narrower grant, and there is no new endpoint and no new control. A bare {"enabled": true} is no statement about scope, so it KEEPS a stored marker only while the grant was already in force -- nothing about it changed, so neither widening nor narrowing was requested, and an idempotent retry must not revoke a consent the operator did give. An enable that RE-ESTABLISHES the grant clears the marker instead, because a marker records the scope in force when it was written: the document can hold enabled=false together with a marker, and preserving it across a re-enable would turn a bare {"enabled": true} into a host-wide grant nobody named. A disable clears the marker with the grant, an enable over a corrupted stored value or an account with no entry clears it, and an enable naming a scope this code does not recognise clears it too -- absent and unrecognised are different requests and must not collapse.

Naming it is required because the act of enabling carries no evidence of what the operator was shown. An idempotent retry, an automation, and a client still rendering older copy all send the same bare body as a deliberate re-consent. A transition test -- stamp only when the grant goes from off to on -- closes the retry but NOT a first enable from a stale client, where the operator reads the narrower description and the grant covers the whole host. The request is the only place the decision can travel.

An enable whose scope field is ABSENT neither widens nor narrows: the stored marker is left exactly as it is, because absence is no statement about scope, and treating it as a withdrawal would revoke a real consent on every retry from an older client. An enable that NAMES a scope this code does not recognise is a different request and CLEARS the marker -- the caller said what it wanted and it was not the conversation export, so an already-wide grant must not stay wide for it. Collapsing those two into one branch was a real defect: it kept a wide grant wide for a request that asked for something else entirely, which is the widening-without-a-request the field exists to stop. A disable removes the marker with the grant, and the response echoes the resulting scope whenever one was named, so a caller cannot believe it consented to the wider payload.

Both directions of a grant write file a SEL event naming what was decided. _audit_layer_b_grant carries the direction and the resulting scope. The route's own event records the operation and the path, not which way the decision went, so learning what the grant became would mean reading the state file -- the on-disk dependency the decision audit exists to remove. A narrowing is filed on the same footing, because a review reconstructing what an archive was allowed to carry needs the revocation as much as the grant. A write that failed files nothing, since nothing was decided.

Conversation TEXT is redacted on the way into the export. The allowlist bounds which TABLES ride and says nothing about what a conversation's own text holds: an operator who pasted a key into a terminal session put it in value, and this archive goes off-host unrecallably. Every text value now passes the two egress redactors this module already uses for object names, credential shapes and exfiltration URLs both, through one shared _redact_egress so a third redactor added later cannot reach one call site and miss the other -- which is the drift sanitize_label's own note warns about.

Two consequences were handled rather than assumed. The exported database is an archive member, so its bytes feed _tree_fingerprint: a redactor with a per-run placeholder would change them every night and the unchanged-run skip would never fire again on any Layer B host. Measured across separate processes on a fixed set of inputs, the pair is byte-identical, and a test pins that two exports of one unchanged store produce identical rows. It is measured rather than assumed because security.redaction does hold a per-process random key, for a different function. And a redactor that RAISES neither drops the row nor falls back to the raw value: the export carries nothing, reports conversations_unredactable, and so suppresses the retention sweep, because an earlier archive may hold what this run could not carry. Only str values are rewritten, which covers the measured schema -- key/conversation_id/value TEXT plus two INTEGER timestamps, every text column reporting typeof() == 'text' on a live store.

Peak memory is bounded by two named numbers, and a wider field is refused rather than copied. The copy fetches a fixed number of rows at a time, and the row count alone bounds nothing: one conversation field can be arbitrarily wide, so a batch of them exhausts memory -- and the allocation failure would escape this function's guards and lose the whole sessions backup over a sub-member. Both numbers come from a live store rather than a guess: its widest conversations_v2 value measures 2.5 MiB and the table totals 1.1 GiB across 3226 rows, so a 500-row batch of real data is roughly 180 MiB. The per-cell ceiling sits well above the widest real value so an ordinary host is never refused, and it is measured in SQL inside the copy's own read transaction before the first fetch, so length(cast(c as blob)) establishes the bound without handing Python the value and a writer committing a wider one mid-copy is outside the snapshot. A table holding a wider cell reports conversations_oversized and carries nothing, for the reason the redaction exit gives: a partial copy produces an archive a restore reads as complete. The redacted rows are produced as a generator rather than a list, because materialising them beside the raw batch doubles the peak for the length of the statement.

The scope read moved off the event loop. layer_b_grant_covers_conversations reaches the state file through a synchronous read with no thread seam, so the scope field added above put a file read on the loop for every request that carried one. It is now await asyncio.to_thread(...), matching the nightly_enabled read in the same module and the write two lines above it.

The grant audit reads the same pair the reader does. _audit_layer_b_grant derived its scope value from the stored marker alone, while layer_b_grant_covers_conversations requires the grant to be on first -- so the event was truthful only because the disable branch happens to remove the marker, making its correctness depend on a decision taken elsewhere in the same function. It now requires the grant too. Nothing observable changes, since the two already agree; what goes away is a caller that is right only because a sibling branch cleans up after it.

The scope is rechecked immediately before the upload, beside the grant. The grant staying on does not mean it still covers this payload: a disable followed by an enable naming no scope leaves the permission on with the marker gone, so the grant recheck passes while the conversations already written into that archive sit outside what the grant now covers. Only the withdrawn direction refuses, matching the grant's own recheck, since a scope granted mid-build leaves an archive without the conversations and that is the withholding default.

The consent decision reaches the SEL event, not only the run record. _audit_layer_b_decision now records layer_b and conversations as separate allowed-or-withheld values. That event exists so a consent question has an answer that does not depend on the run record still being on disk, and an event naming only the permission describes a run that shipped the terminal conversations identically to one that withheld them. The run record does carry layer_b_scope, but reading the scope back from there would put the audit on the very dependency it is there to remove. One supporting claim in that finding did not hold and is worth noting: the audit call already sits after the scope is resolved, so the ordering was fine; what was missing was the value in the event.

Three alternatives were rejected. Invalidating legacy grants outright also kills the cli backup on every existing Layer B install -- collateral damage to the half this change never touched. A fresh confirmation flow is a genuinely new product surface and is unnecessary, since the grant's write path already exists. Leaving it and deferring is not available: a fenced blocking finding removed "do nothing" from the decision space.

An out-of-scope grant records layer_b_scope, and deliberately NOT conversations_skipped. That split matters more than it looks. The skip field suppresses the retention sweep, so writing one here would freeze retention on every install that granted Layer B before this export existed -- all at once, which is the unbounded accumulation the suppression exists to prevent rather than an instance of it. An out-of-scope grant belongs with the other policy-declined exit, the Layer B gate itself: recorded as STATE, visible in the run record, and still pruning.

Pruning on that path is NOT safe on its own, and the sweep answers that separately. The reasonless exit was first justified by a measurement -- no released version wrote an archive carrying conversations, verified against this PR's base and current main, whose sessions archive has exactly the crew and cli roots. That bound covers only pre-release archives and expires on the second successful run after this ships: once a host has uploaded a conversation-bearing archive, a narrowed scope produces a run whose archive omits them, and at a keep count of one the sweep would retire the only copy with no recovery.

So the account records one persisted fact, sessionsConversationsRetained, set whenever a run uploads an archive carrying a conversations/ root, and the sweep is declined when this run carries none while that fact holds. Two independent conditions feed one decline: this run's own export coming up short, which is the conversations_skipped reason, and an older retained archive holding what this one does not. A narrowed scope sets no skip reason -- it is a policy decline rather than a failed read -- so the first condition structurally cannot see the second's case. Each is pinned by its own test and each mutation reddens only its own: removing the new condition leaves the absent-store test green, and removing the skipped condition leaves the new test green.

The fact is one boolean read through a predicate, not a list of archives, for the same reason the skip suppression is a predicate: a second list to keep in sync is a place to forget one, and the cost of forgetting is a permanent delete. It only ever goes true, which is correct rather than lazy -- while it holds the sweep is declined, so the archive it refers to is never retired, so the fact stays true. A run that DOES carry conversations prunes normally, because the newest archive holds them and retiring older ones loses nothing, and that is what lets retention resume rather than freezing once set. An install that never carried them prunes exactly as before, which has its own test in the over-suppression direction. The fact is carried on the run RECORD as well as on the account, and that is the correction to an earlier claim in this body that it had no failure mode of its own. It did: the account-level key is written in the state update, but when that write raises the recovery path holds only the run record and merges back records, uploads and versions -- no account-level key. So on a full or read-only filesystem the run survived and the fact did not, while the archive it protects stayed in the drive, and only another conversation-bearing run could set it again, which a narrowed scope makes impossible. Carried on the record, the fact travels the recovery path and the merge restores it. It is also set for a run whose record was superseded, because which record wins the slot says nothing about what the drive holds, and the merge only ever sets it -- a later record carrying no conversations cannot lower it. The fact is read through the SAME unpersisted overlay as the sweep's own candidate and version sets, and re-read inside the lock hold that already re-reads the keep count: two same-account sessions runs can overlap, because the owner-triggered path does not pass the upload gate, and without one snapshot the other run's key was a live candidate while the fact protecting it was invisible. ONE RESIDUE REMAINS and is stated rather than implied: a run held unpersisted by ANOTHER process keeps its fact in that process's memory alone, where no lock or overlay can observe it. Closing that would mean ordering the upload, the marker write and the remote delete in one cross-process protocol, which rewrites shared retention and locking machinery beyond this change's scope; it is filed on #13142.

The default-OFF consequence -- a stock install backs up no terminal conversations until the operator grants Layer B -- belongs to sessions_layer_b_enabled, and #12178 chose it for exactly this data class. If that default is wrong it is wrong for both halves, so changing it here would quietly re-decide #12178.

The Design Review asks for its two accepted costs to be confirmed, so they are confirmed here explicitly.

The retention freeze on a host that enabled Layer B and has no readable store is accepted for this PR, in the direction stated above: archives accumulating is recoverable and audited every run, while retiring the last archive that held the conversations is not recoverable at all. Stated plainly, because the marker deserves it: there is NO in-product way to clear sessionsConversationsRetained. Nothing in the app or the API resets it, and lowering keep does not reach the delete, so on a host that exported conversations and then narrowed the grant permanently the sessions sweep stays declined for good. The only thing that resumes pruning is a run that again carries conversations, which means granting the wider scope again -- an operator action, not a product one. A clearing path is new behaviour and is filed on #13142 beside the self-describing marker; this PR accepts the frozen state rather than shipping a reset it has not designed. The review's own clearing condition -- permit the sweep when the archive being retired carries the SAME standing skip reason in its own record, which the record already holds -- is the right shape and is recorded on #13142 beside the newest-complete-archive fix, because both read prior records inside the code that deletes objects. That is the scope this PR does not take.

The per-column residual is confirmed as tracked, not fixed: _copy_table enumerates whatever columns the source declares, so the fail-closed boundary is the table set one level up. The measured schema and the drop-versus-refuse options are on #13142.

One reviewer ask was answered by measurement rather than deferred. First Principles asked to confirm on a live install that the table is exactly conversations_v2, since a name mismatch would report no_conversation_table on every host and now suppress retention on all of them. Verified on a real store: conversations_v2 holds 3158 rows, and the v1 conversations table is present with 0 rows on this migrated host, which is the coverage item already on #13142.

Pattern harvest

Rule candidate: a file that is BOTH conversation state and auth state cannot be backed up whole; export it table-scoped with a table ALLOWLIST, so it fails closed against a credential table added later.

Rule candidate: to back up a live WAL-mode SQLite store read-only, open mode=ro and read under one BEGIN snapshot. Do NOT wal_checkpoint on a read-only connection (it raises disk I/O error) and do not refuse on a surviving WAL, or a live store -- whose WAL is non-empty in steady state -- silently exports nothing.

Rule candidate: reading a file the codebase classifies as a token path must go through the sanctioned audit gate and fail closed, even when the read copies only non-secret rows -- the audit is owed for opening the store, not for what you take from it.

Rule candidate: when a new payload joins an aggregate that a permission already gates, decide the permission question by the payload's sensitivity CLASS, not by whether its path is named in the permission's docstring. A second path into the same class is how a gate gets bypassed without anyone removing it.

Rule candidate: a decision about a stored artifact's fate must not be made from an observation of the CURRENT run only. "There is nothing here now" does not answer "does the thing I am about to delete hold something" -- a store absent today was present when last week's copy was written. Before exempting a case from a safety rule because it looks like it has nothing to lose, name what makes that true ACROSS runs and check that something enforces it.

Rule candidate: pin a value that is an ON-DISK contract as a literal in its test, not by comparing against the constant the production code uses. Routing both sides through one constant only proves the writer and the reader agree with each other -- they still agree after the literal changes, while every record already written under the old value silently reads as something else.

Rule candidate: a bound on the CONTAINER is not a bound on the FIELD. Batching a copy by row count bounds nothing when one cell can be arbitrarily wide, and the failure mode is an allocation error that escapes the guards written for the surrounding operation. Bound the widest field too, establish it before the fetch rather than after, and state the product of the two as the peak.

Rule candidate: prose that claims to enumerate every case is a list that has to stay in sync with the code, and it drifts silently because no gate reads it. Both this change's spec paragraph and its own PR table had fallen four exits behind before a reviewer touched either. State the RULE that decides which cases qualify, and name examples as examples.

Fixes #12237

@chenmingwei23
chenmingwei23 requested review from a team and smeyffret September 21, 2026 08:20
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 21, 2026
@chenmingwei23 chenmingwei23 changed the title aws-control: back up the kiro-cli terminal conversations, token-safe (#12237) fix(aws-control): back up the kiro-cli terminal conversations, token-safe (#12237) Sep 21, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 21, 2026
@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review (fork) — ✅ no blocking findings

Reviewed e1f8a6742faa5f0efb848a1ff386f3f076654535 via the fork AI-review pipeline; updated in place on each push.

Review details

Nothing blocks the merge; three advisory findings.

FINDING — src/kiro_crew/apps/builtins/aws_control/backend/backup.py:4896 — with tempfile.TemporaryDirectory(prefix="kc-conv-", dir=str(scratch_parent)) as tmp: sits outside the except (OSError, ValueError) guarding _conversation_scratch_parent() one line above, so an ENOSPC/EDQUOT on the data-home volume (the tar itself is built in the system temp, so it is still fine) raises out of _export_cli_conversations and run_sessions_backup into hooks._push_nightly's except Exception, losing the whole night's sessions archive instead of returning the "scratch_root_unusable" reason this function's own contract promises for a scratch failure → Fix: acquire it with tempfile.mkdtemp(prefix=..., dir=...) inside the existing try and release it in a finally: shutil.rmtree(tmp, ignore_errors=True).

FINDING — src/kiro_crew/apps/builtins/aws_control/backend/backup.py:279 — entry[SESSIONS_CONVERSATIONS_RETAINED_KEY] = True has no clearing writer anywhere, and the only code that removes an archive is _prune_remote_archives (sole caller of delete_object_versions) — the sweep this fact declines — so an operator who turns Layer B off, or re-enables it without naming cli+conversations (which set_sessions_layer_b only records when the caller states it), freezes sessions retention permanently: archives accumulate past keep indefinitely with a result="failed" retention SEL event every night, contradicting the "only while the condition persists" bound the comment claims → Fix: persist the conversation-bearing archive's key rather than a boolean and clear the fact once that key is no longer live — a new state field plus a clearing writer, so it cannot land inside these changed lines.

FINDING — test/test_aws_control_backup_conversations.py:275 — TestConversationExport stubs only _kiro_cli_conversation_db, but _export_cli_conversations first calls _store_relocated_outside_the_fence(), which diffs state_db_candidates(sys.platform, Path.home(), {}) against state_db_candidates(..., os.environ); no conftest pins XDG_DATA_HOME, so on any host exporting it every test in this class receives skipped == "store_relocated_outside_fence" and its member-list, skipped and scratch_pinning_unavailable assertions fail host-dependently → Fix: monkeypatch.setattr(backup, "_store_relocated_outside_the_fence", lambda: False) in the class's autouse fixture, as the sibling tests in test_aws_control_backup.py already do.

[OPUS-REVIEWED] e1f8a67

@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5, fork) — 🟡 CONCERNS

Premise-level review of e1f8a6742faa5f0efb848a1ff386f3f076654535 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

The consent scope ships a string vocabulary with exactly one legal value ever constructed — a boolean grant marker does the same job with less grammar.

Not justified as shipped

  • Item 2 — one consumer, generalized: SESSIONS_LAYER_B_SCOPE_WITH_CONVERSATIONS = "cli+conversations" is the only scope value the code ever writes (grepped sessionsLayerBScope|cli\+conversations on base: zero occurrences; in the diff, one constructed value), and layer_b_scope only ever holds "cli". The free-string form is what forces the unrecognised-value branch in set_sessions_layer_b, the echo semantics, and the "CLI+CONVERSATIONS"/list/dict rejection matrix — grammar a boolean marker (absent = no statement, true = named consent, anything else = 400) makes unnecessary while keeping the exact fail-closed, must-be-named properties the description argues for. A second scope value is "so we can later".
    Subtraction: replace the scope request string, SESSIONS_LAYER_B_SCOPE_KEY string marker and SESSIONS_LAYER_B_SCOPE_WITH_CONVERSATIONS with one boolean consent field; delete the unrecognised-scope clearing branch and its tests.
    Clears when: a second scope value gains a real consumer, or the boolean form replaces the string.

What this change ships

Inventory (10 items) — 9 justified

Intent: carry the kiro-cli terminal conversations in the off-host sessions backup without carrying the token-bearing store file (#12237). Titled fix, but functionally an ADDITION — a new payload behind a consent no operator has yet granted; nothing changes for anyone until they re-consent.

  1. Sessions archive gains a conversations/ root, table-scoped so the token file never ships — justified
  2. Reaching it requires naming a scope on the existing layer-b endpoint; legacy grants stay cli-only — one consumer, generalized (string vocabulary with one value)
  3. Conversation text is egress-redacted (credentials, exfil URLs) before shipping — justified
  4. A short or skipped export is named in the run record and survives unchanged runs — justified
  5. Any export gap declines that run's retention sweep; the decline is still SEL-audited — justified
  6. A persisted retained-conversations fact keeps the sweep off the last conversation-bearing archive after a scope narrowing — justified
  7. Grant writes and run decisions now file SEL events naming the conversation scope — justified
  8. Store lookup reads fixed home anchors only; relocations reported, linked ancestors refused before any stat (matches kiro_prerequisite.py:960) — justified
  9. Scratch export lives under the agent-masked app-data root, read via pinned descriptor; no-pinning platforms refuse — justified
  10. 16 MiB per-cell ceiling and 4-row batches bound memory; unredactable/oversized/OOM exports carry nothing and say why — justified

Capped at 10; smaller riders (the _redact_egress consolidation, the MemoryError guard) fold into items 3 and 10.

[FIRST-PRINCIPLES-REVIEWED] e1f8a67

@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5, fork) — 🟡 CONCERNS

Design-level review of e1f8a6742faa5f0efb848a1ff386f3f076654535 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

Retention can freeze permanently — sessionsConversationsRetained has no in-product reset, and standing skip states (symlinked home, absent store) stop pruning forever on scoped installs.

Watch

The diff ships two states a human should co-sign, not just the bot lanes: any standing conversations_skipped reason (an ordinary symlinked ~/.local/share takes store_rejected_link on every run) declines the sweep on every nightly, and once a conversation-bearing archive is uploaded and the scope narrowed, "there is NO in-product way to clear sessionsConversationsRetained" (author's own words) — archives accumulate until an operator acts outside the product. The direction (keep data, spend storage) is right and audited per run, but the clearing path is deferred to #13142, so the freeze ships before its exit exists.
Clears when: a maintainer explicitly accepts the frozen-retention states as-shipped, or the #13142 clearing condition (permit the sweep when the retired archive's own record carries the same standing reason) lands.

Suggestions

Record on #13142: an early-out on _store_write_time versus the run baseline, so an unchanged store (measured 1.1 GiB) is not re-read, re-redacted and re-gzipped every night solely to reproduce an identical fingerprint.

[DESIGN-REVIEWED] e1f8a67

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 21, 2026
@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review (fork) — ✅ no blocking findings

Reviewed e1f8a6742faa5f0efb848a1ff386f3f076654535 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] e1f8a67

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 21, 2026
@chenmingwei23
chenmingwei23 force-pushed the fix/backup-cli-conversations-12237 branch from 661e470 to 1ee81e3 Compare September 21, 2026 09:19
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 21, 2026
@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Security Scope Review (fork) — 🟡 CONCERNS

Which legitimate operations does e1f8a6742faa5f0efb848a1ff386f3f076654535 newly refuse? Model proposes, scripts/deny_diff.py decides — updated in place on each push. A script-confirmed regression blocks PR readiness.

the reviewer's own judgement; no confirmed regression

All candidates are written and the mechanism is mapped. Emitting the review now.

Scope-Verdict: CONCERNS

TIGHTENING: the Layer B consent gate for the aws-control sessions archive now carries a scope marker that the conversation export and the pre-upload recheck refuse on -- the one threat it refuses is shipping host-wide terminal conversations (conversations_v2, "records every interactive kiro-cli use on the host") off-host on a grant recorded before that payload was disclosed: "A grant with no marker, or with any value this code does not recognise, covers the cli half only -- reading it as covering the conversation store would ship host-wide terminal context off-host on a consent that named this product's session files". Behind it sit three sibling tightenings in the same feature: the store resolver refuses env-relocated and link-reached stores ("a location named by the environment falls outside the agent-file-tool fence"), the scratch read refuses anything that is not the descriptor-verified file this run wrote (scratch_export_unsafe), and the retention sweep is declined on any skip reason or retained-conversations fact.

ADJUDICATED: 14 candidates, 0 confirmed newly-refused
No verdicts were handed back for any row, and the diff gives the composite nothing to change: the only shared file it touches is src/kiro_crew/hooks.py, where it adds one entry to _AUDIT_ONLY_READ_IDS -- an internal SEL-audit registry consulted only by hooks.emit_internal_read_audit for the app's own Python readers (base hooks.py:4055, 4156), not by the four-check tool gate. The rows therefore probe that the composite is genuinely unchanged around the surfaces the diff hard-codes (kiro-cli store paths, sqlite, aws s3 listing/head, the relocation env vars), not the guards themselves.

PLATFORM COVERAGE: posix rows (6) cover the POSIX assumptions the diff makes -- the ~/.local/share/kiro-cli fixed anchor, XDG_DATA_HOME as the relocation signal, the symlink-ancestor walk, and the sanctioned .venv/bin/python -m pytest invocation for the new test file. windows rows (4) cover the assumptions the diff states for the other host -- %LOCALAPPDATA%/APPDATA, the AppData\Local vs Roaming layouts it arbitrates by write time, %VAR% expansion, PowerShell and cmd tokenization, and the .venv\Scripts\python.exe interpreter spelling. any rows (4) cover the platform-neutral aws/pytest operations in the touched area. macOS gets no separate spelling: the diff treats darwin as fixed-anchor-only with no relocation variable, and its shell spelling is the posix one, so the posix rows are its coverage as far as a shell row can carry it. One assumed platform state no candidate can probe: a host without descriptor-pinned open (Windows), where the diff refuses with scratch_pinning_unavailable rather than degrading -- that refusal is an in-process branch with no command-line expression, named below.

UNADJUDICATED: the entire tightening lives outside the shell deny composite; no classifier verdict exists for any of it, which is what caps this review at CONCERNS.

  1. Consent scope gate. Guard: layer_b_grant_covers_conversations / the scope branches of set_sessions_layer_b (backup.py). Call path: POST /api/apps/aws-control/backup/{account}/layer-b → routes._handle_backup_layer_b (which also newly 400s a non-string scope as invalid_scope) → set_sessions_layer_b(account, enabled, scope=...); read back by run_sessions_backup at start and again immediately before the PUT. Candidate legitimate inputs I would have submitted: a bare {"enabled": true} retry from an older client over a standing wide grant (must keep the marker -- the code preserves it only while the grant was already on); the same bare body as a first enable after a disable (marker cleared -- an operator who WAS shown the wider copy on a current client silently gets the narrow grant and their conversations stop being backed up with only layer_b_scope: "cli" in the record to say so); {"enabled": true, "scope": "cli+conversations"}; an enable naming an unrecognised scope over a wide marker (cleared by design).

  2. Store resolver. Guard: _kiro_cli_conversation_db + _store_relocated_outside_the_fence. Call path: run_sessions_backup → _export_cli_conversations → resolver, before any stat. Candidate legitimate inputs: a store under a symlinked ~/.local/share or a symlinked home directory -- the diff's own docstring concedes "an ordinary symlinked ~/.local/share or a symlinked home takes this exit permanently" -- and a host whose distro sets XDG_DATA_HOME as standard configuration. Both are ordinary installs whose operator opted into conversation backup and now permanently receives none of it. This is a NARROWING I can argue but not confirm: the refusal is deliberate and documented, but the population it costs (symlinked-home installs are common) is wider than the threat shape (an agent-writable relocated root), and a fence that admitted a link whose resolved target stays inside the fixed home anchor would keep those installs alive while still refusing agent-reachable redirections.

  3. Retention suppression compounding on 2. Guard: the decline predicate in run_sessions_backup plus the in-lock recheck in _delete_under_the_retention_gate. Call path: every sessions backup, last step. On the installs named in 2, store_rejected_link/store_relocated_outside_fence recurs on every nightly run, so the retention sweep is declined forever and archives accumulate without bound -- audited once per run, but never recovering, because the condition is a standing property of the host rather than the transient the recovery story assumes ("the gap recovers on the next successful run" never happens). The legitimate operation lost is the retention prune itself, permanently, on an opted-in symlinked-home install.

  4. Pinned scratch read. Guard: _conversation_scratch_parent + _open_pinned_scratch (O_NOFOLLOW, S_ISREG, st_nlink == 1), and the _CAN_PIN_TRAVERSAL refusal (scratch_pinning_unavailable). Call path: _export_cli_conversations between the sqlite copy and the first tar write. Candidate legitimate input: the export's own scratch file on ordinary filesystems (covered by the new tests); the platform-without-pinning branch is unreachable in production because the whole sessions kind is already refused there, so nothing legitimate is newly lost -- named only because no shell row can verify it.

  5. Mid-build scope-withdrawal refusal. Guard: the second layer_b_grant_covers_conversations check before storage.put_file. Call path: run_sessions_backup, after _authorize_upload. Candidate legitimate input: a backup in flight while the operator disables and re-enables the grant without re-naming the scope -- the run fails loudly and the operator restarts it, which is the withholding default and costs one run, not an operation.

[SCOPE-REVIEWED] e1f8a67

Security scope review -- adjudicated candidates

  • macos-latest (platform posix): 10 classified, 0 newly refused, 10 still allowed.
  • ubuntu-latest (platform posix): 10 classified, 0 newly refused, 10 still allowed.
  • windows-latest (platform windows): 9 classified, 0 newly refused, 9 still allowed.

No candidate flipped from allowed to refused.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 23, 2026
@chenmingwei23
chenmingwei23 force-pushed the fix/backup-cli-conversations-12237 branch from f750c54 to a4fcc4a Compare September 23, 2026 20:20
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 23, 2026
@chenmingwei23
chenmingwei23 force-pushed the fix/backup-cli-conversations-12237 branch 2 times, most recently from a2d4264 to 0570406 Compare September 23, 2026 21:27
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 23, 2026
@chenmingwei23
chenmingwei23 force-pushed the fix/backup-cli-conversations-12237 branch 3 times, most recently from 3b412f5 to 24a2249 Compare September 23, 2026 23:06
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 23, 2026
@chenmingwei23
chenmingwei23 force-pushed the fix/backup-cli-conversations-12237 branch from 24a2249 to 32d2375 Compare September 24, 2026 00:20
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 24, 2026
Route the export's gate-side log line through redact_log_via_context.

Resolve the store among FIXED, home-anchored candidates and do not consult the
environment: not XDG_DATA_HOME on POSIX, not LOCALAPPDATA or APPDATA on Windows.
The fence that makes this store unreadable and unwritable by agent file tools is
home-anchored (security.paths splices identity_stores.fenced_home_dirs, whose own
comment records that a redirected profile is not covered), so a relocated root
sits outside it, where an agent can author rows. This function feeds an archive
uploaded off-host unattended, so honouring the variable would let planted rows in
a conversations_v2 table ship on a scheduled backup, and an uploaded object cannot
be un-sent. kiro_prerequisite records the same decision for the same variables.

Detect the relocation BEFORE looking at the fixed anchor. A file may still sit at
the old anchor after the environment says the store moved, and checking the anchor
first would export those stale conversations and report complete coverage, because
the relocation would only be noticed when the fixed lookup found nothing.
identity_stores.selected_store arbitrates the same leftover-in-the-abandoned-root
state, so it is a state this codebase already expects.

Name EVERY skip, not just one. An unreadable store, a missing allowlisted table
and an unrecordable credential audit each carry their own reason now, on the same
standard the relocation path set: a skip nobody can see is how an operator ends up
believing they hold a backup they do not hold.

Do not let retention make such a gap permanent. The sweep protects only the key
this run uploaded, so at keep=1 retiring the previous archive erases the one copy
that still held the conversations, and delete_object_versions erases versions
outright. A run whose export came up short for a reason an earlier archive may not
have shared skips the sweep; the gap recovers on the next successful run, the
retired object does not. A relocation is deliberately excluded, since no archive
this install wrote carried those conversations either.

Report that refusal instead of skipping silently, and carry it through the skip
path. _record_skip now forwards conversations_skipped and layer_b: it REPLACES the
run slot and the sessions path takes it on every unchanged tree, so a field it does
not forward is erased and the first ordinary nightly run would restore an assertion
of complete coverage over a run that had reported a gap.

Consult first_linked_ancestor BEFORE any stat on a candidate, then test the leaf
with is_link_or_junction. An ancestor junction whose target is a UNC share turns
is_file() on a local-looking path into an outbound SMB handshake that
authenticates as this process, inside the stat, so the ancestor walk has to
precede it; islink alone answers False for a junction. A fixed anchor bounds where
a candidate may live but does not stop a link planted AT that anchor.

Track emitted archive members separately from copied rows. A present-but-empty
allowlisted table is carried so a restore sees the real schema, so rows and
members genuinely disagree, and the "no session files to archive" guard was
discarding a conversations root it had already written.

Sweep the comments for claims the code does not enforce. The allowlist bound is
per TABLE, not per column: every column an allowlisted table declares is copied,
and the named column list fixes a stable column order rather than acting as a
column allowlist. The read-only claim covers database pages and checkpoints, not
the WAL -shm sidecar a reader may touch. The audit id constant does not make the
reader and the hooks registry one string -- they are two literals and a test is
what keeps them equal. And the credential-read audit RECORDS the read rather than
gating it: what fails closed is the shipping, not the open.

Clear the scope marker on an enable that RE-ESTABLISHES the grant. A marker records
the scope in force when it was written, so a document holding enabled=false together
with a marker must not let a bare {"enabled": true} widen the grant to the whole
host: that request names no scope, so it is no statement about one, and the archive
that follows cannot be recalled. An enable over an ALREADY-ON grant still keeps the
marker, because nothing about the grant changed and an idempotent retry must not
revoke a consent the operator did give. An enable over an off grant, over a
corrupted stored value, or over an account with no entry yet clears it; a disable
clears it with the grant; and an enable naming an unrecognised scope clears it too,
since the caller said what they wanted and it was not this.
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

First Principles Review, item 2 -- rebutted. Head e1f8a6742faa5f0efb848a1ff386f3f076654535.

The item proposes replacing the sessionsLayerBScope string and its single cli+conversations value with a boolean consent field, and deleting the branch that clears the marker when a request carries an unrecognized scope value. Declining, for three reasons.

A string stores the name of the range; a boolean stores whether the range is present. Consent here is carried by two keys: sessionsIncludeLayerB says the grant is on, and sessionsLayerBScope says what it covers. A name still reports which payload set was consented to when a later build reads it. Presence has to be interpreted by whichever build reads it, so a boolean's meaning depends by definition on the reading build, while cli+conversations depends on no build at all.

That is the shape round 8 of this review caught, and it is why the scope key exists at all: consent was carried by sessionsIncludeLayerB alone, with no companion scope or version anywhere in the tree, so a build that widened the payload set read an older true as consent to the wider set. Collapsing back to the boolean is that state again one layer down -- the same defect, not a style preference.

The branch this would delete is not syntax. A boolean has no unrecognized value, so there is nothing left to fail closed on. Removing that branch removes a property -- an unrecognized scope name narrows to the narrow grant -- rather than removing complexity.

The count is of encodings, not of ranges. One legal string value is accurate. The range space in this change has two members: marker absent is cli, marker present is cli+conversations. "Only one legal value" therefore describes the encoding, not the design.

The other side, stated plainly: if the payload set never grows, a boolean is equivalent and simpler, and the refusal matrix really is syntax. This change already distinguishes two payload sets, and the durable direction recorded on #13142 is a self-describing, versioned marker -- which extends the string, so a boolean moves against it.

Recorded here as the author's rebuttal; the lane is advisory on this head and readiness is passing. A repository writer's disposition comment is what formally answers the item.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

aws-control: the sessions archive carries neither half of the kiro-cli conversation store, and the file it lives in also holds bearer tokens

2 participants