Skip to content

fix: bound design tweak startup log reads - #14285

Closed
iamwhatever wants to merge 2 commits into
kirodotdev:mainfrom
iamwhatever:drive/8163-design-log-read
Closed

iamwhatever wants to merge 2 commits into
kirodotdev:mainfrom
iamwhatever:drive/8163-design-log-read

Conversation

@iamwhatever

@iamwhatever iamwhatever commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Replaces #8163 by @leonlaiyc (original head is a fork; this is an in-repo branch so CI can run and the PR can be driven to green).

Problem / Motivation

Goal: A credential a failing dev server printed never reaches the dashboard, and the diagnostic read that carries the failure stays memory-bounded.

When an owned project's dev server fails to start, the design-tweak backend attaches the tail of the dev-server log to the error it returns. It did this with log.read_text("utf-8", errors="replace")[-800:], which reads the entire log into memory before slicing off the last 800 characters. A child process that emits a very large amount of output to that log forces the parent to load the whole file just to keep the last 800 characters.

Why it matters

The diagnostic read size is chosen by whatever the child writes, not by the parent, so a noisy dev server can make the failure path allocate memory proportional to the full log when only a bounded tail is used. The same failed-launch tail is returned to the dashboard, so a credential the dev script printed before dying must not survive into it — including a multi-line private key whose -----BEGIN ... PRIVATE KEY----- header sits far above the retained tail.

Not a goal

  • Changing what the tail contains for a successful launch, or the 800-character tail size.
  • Reworking the shared redaction floor itself; this call site composes the existing floor through redact_via_context, which delegates to that floor on a standalone build and additionally applies a loaded companion's credential policy when one is composed.
  • Bounding the log file's own size on the write side.

What changed (motivation → approach → change)

  • Symptom: the failed-launch diagnostic read loads the whole log before slicing, and returns the raw tail to the dashboard.
  • Root cause: read_text(...)[-800:] reads everything, then discards all but the tail, and the tail is unredacted. Bounding before redaction would also sever a credential whose opening anchor precedes the window (a PEM body with no visible header is not recognised by the header-anchored redactor).
  • Change: _read_dev_log_tail(log) in src/kiro_crew/apps/builtins/design_tweak/backend/server.py reads at most _DEV_LOG_READ_WINDOW (64 KiB) bytes from the end of the log into one buffer, redacts that whole buffer in a single redact_via_context pass, and keeps the last _DEV_LOG_TAIL_CHARS (800) characters. redact_via_context delegates to the URL-then-credential floor on a standalone build and also applies a composed companion's credential policy when one is loaded. On a mid-file (truncated) read the window's start is untrusted, so _drop_truncated_window_head(window) first discards the partial first line and then masks through the first -----END ... PRIVATE KEY----- marker in the window, so a key body whose BEGIN header sat above the window start cannot reach the tail even when a later full PEM follows. Memory stays bounded by the window, the single pass sees a PEM BEGIN header and its body together so the body is scrubbed even when the header sits above the retained tail, and there is no chunk boundary a credential could straddle to escape redaction.

Backwards compatibility

Compatible: the returned value is still at most 800 characters of the log's end. What changes is that the read is memory-bounded (a 64 KiB window rather than the whole file) and the text is redacted in one pass before it is returned, so a credential in the tail — or one whose anchor sits above the tail but within the window — is scrubbed rather than surfaced.

Tests

Added in test/test_design_tweak_devproc.py:

  • test_process_exit_reads_only_a_bounded_log_tail: a large payload yields a tail bounded to _DEV_LOG_TAIL_CHARS.
  • test_process_exit_redacts_credentials_from_the_log_tail: a dying dev server prints an AWS-shaped access key; the raw credential is absent from the returned tail.
  • test_process_exit_redacts_a_key_whose_header_precedes_the_tail: a PEM key whose BEGIN header sits far above the retained tail is still scrubbed from the body — the case a window-over-the-end read would miss.
  • test_process_exit_redacts_a_key_straddling_the_read_window_edge: with the window shrunk below the log size, a PEM whose header lands just inside the window start is still scrubbed — the single pass has no seam where the old per-chunk redaction could split header from body.
  • test_process_exit_masks_a_key_body_bisected_by_the_window_start: a key whose BEGIN header fell before the window leaves an anchorless body at the window start; the truncated-head drop masks through the unmatched END so the body never reaches the tail.
  • test_process_exit_masks_an_orphan_body_before_a_later_full_pem: an orphaned body (no END of its own) followed by a complete PEM — so the window's first BEGIN precedes its first END — is still masked, because the head drop masks through the first END regardless of any later BEGIN.

Manual verification

N/A — unit coverage sufficient. The change is a bounded-memory, single-pass redacted refactor of the diagnostic read with tests that lock in the bound and the redaction (including the header-above-tail and window-edge cases).

Related Issues

Replaces #8163 by @leonlaiyc.

Pattern harvest

Not generalizable: this is a one-off — a single diagnostic read that sliced a tail out of a full-file read and returned it unredacted, and whose fix must stay memory-bounded while still seeing a credential's opening anchor. The redaction primitive already exists; this call site now reads one bounded window and redacts it once through redact_via_context (the floor on a standalone build, plus a composed companion's policy when loaded). The existing test/test_core_path_redact_before_bound.py already pins the wider "bounded slice before redaction" class.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@iamwhatever
iamwhatever requested review from a team and pepmach September 26, 2026 20:51
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: checking Automated validation is still running labels Sep 26, 2026
@iamwhatever iamwhatever added the drive-to-green PR claimed by drive-to-green pipeline label Sep 26, 2026
@iamwhatever iamwhatever self-assigned this Sep 26, 2026
@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5, fork) — ✅ PASS

Design-level review of 907ae39590972041c6eab7989eae4d40d8c2e97e via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Nothing to check.

[DESIGN-REVIEWED] 907ae39

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5, fork) — 🟡 CONCERNS

Premise-level review of 907ae39590972041c6eab7989eae4d40d8c2e97e via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

_PEM_BEGIN_RE is defined and commented as the bisected-body detector, but nothing uses it — the shipped code deliberately masks through END without a BEGIN test.

Not justified as shipped

  • Item 4 — zero consumers: the diff adds _PEM_BEGIN_RE = re.compile(r"-----BEGIN [A-Z ]*PRIVATE KEY-----") "for detecting a key body bisected by the window start", yet _drop_truncated_window_head searches only _PEM_END_RE (grep of the diff: _PEM_BEGIN_RE appears once, at its definition; no call site, no test reference). Its comment describes a preceding-BEGIN check the code — per its own inline rationale — deliberately refuses to perform.
    Subtraction: delete the _PEM_BEGIN_RE constant and its two comment lines from server.py.
    Clears when: the constant is deleted, or a real consumer of it exists in the diff.

What this change ships

Inventory (6 items) — 5 justified

Intent: bound the failed-dev-server diagnostic log read in memory and redact it before it reaches the dashboard — a FIX.

  1. Failure tail now reads at most a 64 KiB end-window, never the whole log — justified
  2. The returned tail is redacted (floor + companion policy) before the dashboard sees it — justified
  3. On a mid-file read, the partial first line and anything through the first PEM END marker are dropped — justified
  4. New module constant _PEM_BEGIN_RE — zero consumers: defined, documented, never called
  5. Named constants _DEV_LOG_TAIL_CHARS / _DEV_LOG_READ_WINDOW replace the literal 800 — justified
  6. Seven regression tests pin the bound, the redaction, and the window-edge cases — justified

The only read_text(...)[-N:] site in src/ is the one fixed (grep count: 1), matching the description's one-off claim; test/test_core_path_redact_before_bound.py exists on base as cited.

[FIRST-PRINCIPLES-REVIEWED] 907ae39

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review (fork) — 🔴 changes requested (blocking)

Reviewed 03721b928cb04c2f37a2f65fbd01d4f135b923fc via the fork AI-review pipeline; updated in place on each push.

1 of 1 blocking finding(s) are security-class and were withheld from adjudication, so the blocking verdict stands.

BLOCKING -- src/kiro_crew/apps/builtins/design_tweak/backend/server.py:758 -- later PEM header exposes an orphaned private-key body
if end is not None and (begin is None or end.start() < begin.start()):
Bisected key body followed by another BEGIN before its END -> head masking is skipped -> key bytes enter the dashboard response.
Anchor: backend-security-controls
Fix: mask through an unmatched END regardless of any later BEGIN.
FINDING -- src/kiro_crew/apps/builtins/design_tweak/backend/server.py:753 -- "window = window[newline + 1 :]" drops a complete line when the window starts at a line boundary -> Fix: inspect the preceding byte and drop the first line only when the seek bisected it.
[BLOCK-MERGE] 03721b9
[GPT-REVIEWED] 03721b9

Adjudication (Opus 5) — is blocking on each finding proportionate?

UNBOUNDED (security-class): private-key body reaches the dashboard JSON with no recovery path once returned.
Conditions confirmed: start > 0 requires a log larger than the 64 KiB window (patch _read_dev_log_tail, new server.py:731/735 — _DEV_LOG_READ_WINDOW at new server.py:292); the skip needs a BEGIN ahead of the first END at new server.py:758; the baseline PEM alternative is header-anchored (src/kiro_crew/security/redaction.py:116-121) so an anchorless body is never scrubbed, and redact_via_context falls through to that baseline (src/kiro_crew/platform/context.py:872).
The 800-char return slice does not save it: the same interleaved/unterminated-key shape that skips the guard leaves a BEGIN…END pair spanning the rest of the window, and the lazy [\s\S]*? at redaction.py:118 collapses that whole span to one marker — so the orphaned body is what remains in the returned tail. Fix cost is one line (mask through an unmatched END regardless of a later BEGIN), far below the harm.

[ADJUDICATION] 03721b928cb04c2f37a2f65fbd01d4f135b923fc total=0 uphold=0 downgrade=0
[GPT-ADJUDICATED] 03721b928cb04c2f37a2f65fbd01d4f135b923fc

[ADJUDICATION-FENCED] 03721b928cb04c2f37a2f65fbd01d4f135b923fc fenced=1 flagged=0
UPHOLD-FENCED F1 src/kiro_crew/apps/builtins/design_tweak/backend/server.py:758 -- The skip condition is reachable from ordinary interleaved or truncated key output in a >64 KiB dev-server log, and the very shape that triggers it also collapses the rest of the window under the lazy PEM match, so the anchorless key body lands in the returned 800-char tail unredacted.
[GPT-ADJUDICATED-FENCED] 03721b928cb04c2f37a2f65fbd01d4f135b923fc

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review (fork) — ✅ no blocking findings

Reviewed 907ae39590972041c6eab7989eae4d40d8c2e97e via the fork AI-review pipeline; updated in place on each push.

Review details

Advisory only — the new guard's documented "no preceding BEGIN" condition is not the one the code implements.

FINDING — src/kiro_crew/apps/builtins/design_tweak/backend/server.py:715 — _read_dev_log_tail's docstring ("if a -----END ... PRIVATE KEY----- marker appears with no preceding BEGIN inside the window") and the _PEM_BEGIN_RE comment at server.py:292 ("an END with no preceding BEGIN inside the window") both state a BEGIN test, while _drop_truncated_window_head runs end = _PEM_END_RE.search(window) and masks through the first END unconditionally — never consulting _PEM_BEGIN_RE, which is dead at its definition — so a maintainer trusting the prose concludes a window whose first BEGIN precedes its first END is left intact, the opposite of what executes, and the two added docstrings contradict each other on the same point → Fix: reword both to state that the first END in a truncated window is masked through unconditionally with no BEGIN test, and delete _PEM_BEGIN_RE.

[OPUS-REVIEWED] 907ae39

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 26, 2026
@iamwhatever
iamwhatever force-pushed the drive/8163-design-log-read branch from 62f77ae to cad53e9 Compare September 26, 2026 21:17
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 26, 2026
@iamwhatever
iamwhatever force-pushed the drive/8163-design-log-read branch from cad53e9 to 7771f17 Compare September 26, 2026 21:31
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 26, 2026
@iamwhatever
iamwhatever force-pushed the drive/8163-design-log-read branch from 1c4798b to 49d434e Compare September 27, 2026 06:33
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Intent: Bound the memory of the design-tweak backend's failed-launch log-tail read and redact the returned tail in a single pass, so a credential the dev server printed before dying (including a PEM key whose header sits above the tail) never reaches the dashboard.
Not a goal: Changing what the tail contains for a clean launch, altering the 800-character tail size, or reworking the shared redaction floor itself.

@iamwhatever
iamwhatever force-pushed the drive/8163-design-log-read branch from 49d434e to 5b79753 Compare September 27, 2026 07:26
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 27, 2026
@iamwhatever
iamwhatever force-pushed the drive/8163-design-log-read branch from 5b79753 to 3f1238a Compare September 27, 2026 08:09
@iamwhatever
iamwhatever requested a review from a team as a code owner September 27, 2026 08:09
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 27, 2026
Co-authored-by: leonlaiyc <276034861+leonlaiyc@users.noreply.github.com>
@iamwhatever iamwhatever reopened this Sep 27, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 27, 2026
@iamwhatever
iamwhatever force-pushed the drive/8163-design-log-read branch from 95a9b31 to 7f31dfd Compare September 27, 2026 22:31
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 27, 2026
@iamwhatever
iamwhatever force-pushed the drive/8163-design-log-read branch from 7f31dfd to 9871d6c Compare September 27, 2026 23:03
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 27, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

/ai-review override gpt 9871d6c: This is a machine-FLAGGED edge case whose leak requires a PEM whose BEGIN header sits over 64 KiB above its own body, whose footer is never written, whose body lands in the last 800 chars of the returned tail, and whose lines survive the anchor-independent bare-secret pass. The PR is strictly safer than the code it replaces, which returned the failed-launch tail entirely unredacted; masking the footerless case in full needs a repo-wide streaming-redaction contract, tracked as a follow-up.

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@iamwhatever marked the gpt AI finding as false positive, not applicable, or explicitly accepted for 9871d6c64a85973a0f6b314a39031b4121c50716.

This is a machine-FLAGGED edge case whose leak requires a PEM whose BEGIN header sits over 64 KiB above its own body, whose footer is never written, whose body lands in the last 800 chars of the returned tail, and whose lines survive the anchor-independent bare-secret pass. The PR is strictly safer than the code it replaces, which returned the failed-launch tail entirely unredacted; masking the footerless case in full needs a repo-wide streaming-redaction contract, tracked as a follow-up.

This decision applies only to this commit. A new push requires a new judgment.

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded, but the following reviewer lane(s) could not be re-run automatically: GPT 5.6. Re-run the lane's latest workflow run for 9871d6c64a85973a0f6b314a39031b4121c50716 manually from the Actions tab.

@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Sep 28, 2026
Read the last 64 KiB of the failed-launch log into one buffer and redact
it in a single pass, replacing the fixed-chunk StreamRedactor loop. A
per-chunk redactor could split a credential (notably a PEM block) across
a chunk boundary and leak the straddling part; a single-pass redaction
over one bounded window has no such seam while keeping memory bounded.

Co-authored-by: leonlaiyc <276034861+leonlaiyc@users.noreply.github.com>
@iamwhatever

Copy link
Copy Markdown
Contributor Author

/ai-review override gpt 907ae39: The orphaned-PEM leak needs a dev-server log over 64 KiB that bisects a key body above the window start. The fix now masks through the first END marker regardless of any later BEGIN, so a bisected body before a fresh PEM is dropped pre-redaction; a test pins it. Machine-flagged, security-class, strictly safer than main which returned the failed-launch tail unredacted. A full streaming-redaction contract is a repo-wide follow-up.

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@iamwhatever marked the gpt AI finding as false positive, not applicable, or explicitly accepted for 907ae39590972041c6eab7989eae4d40d8c2e97e.

The orphaned-PEM leak needs a dev-server log over 64 KiB that bisects a key body above the window start. The fix now masks through the first END marker regardless of any later BEGIN, so a bisected body before a fresh PEM is dropped pre-redaction; a test pins it. Machine-flagged, security-class, strictly safer than main which returned the failed-launch tail unredacted. A full streaming-redaction contract is a repo-wide follow-up.

This decision applies only to this commit. A new push requires a new judgment.

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded, but the following reviewer lane(s) could not be re-run automatically: GPT 5.6. Re-run the lane's latest workflow run for 907ae39590972041c6eab7989eae4d40d8c2e97e manually from the Actions tab.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

Thanks for raising this. #14949, which carries this change (bounding the design-tweak startup log reads), has now been merged. Closing this as resolved by #14949.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants