Skip to content

test: make prompt refusal coverage portable across platforms - #4

Draft
knowlet wants to merge 1 commit into
mainfrom
test/portable-prompt-refusal
Draft

knowlet wants to merge 1 commit into
mainfrom
test/portable-prompt-refusal

Conversation

@knowlet

@knowlet knowlet commented Nov 17, 2026 •

Copy link
Copy Markdown
Owner

Problem / Motivation

Goal: Keep the KAS disallowed-prompt regression meaningful on Windows as well as POSIX.

Main CI on a6ae3d4 failed test_a_disallowed_prompt_path_still_refuses in Windows shard 5: https://github.com/knowlet/KiroCrew/actions/runs/44019710868/job/133830332554 . Checkout is on D:, while pytest's temp directory is on C:. The POSIX /proc/self/environ fixture resolves to C:\proc\self\environ, but the pseudo-filesystem anchor resolves against D:. The absent Windows file correctly takes the documented unreadable-file fallback. This is not evidence of process secrets being read.

Why it matters

The portable regression must prove that a genuinely denied path cannot be downgraded to the default prompt. A POSIX pseudo-filesystem name cannot reliably express that condition on Windows.

Not a goal

  • No production path-policy, credential, sandbox, or fallback behavior changes.
  • No runner provisioning, secret configuration, or unrelated CI repairs.
  • No claim of native Windows validation before hosted tests execute.

What changed (motivation → approach → change)

Use a synthetic Path.home() and its protected .config/gh/hosts.yml path for the all-platform rejection test. The file is never created and no real credentials are read. Keep both exception assertions. Preserve the original procfs assertion as a separate POSIX-only test, rather than skipping the portable refusal regression on Windows.

Backwards compatibility

Compatible: only one test file changes; runtime behavior is unchanged.
Removes nothing: generic refusal coverage remains mandatory on Windows; procfs coverage remains on POSIX.

Tests

Proof: not provable -- test-only change

  • Linux Python 3.12.14: both prompt test modules pass, 27 tests.
  • Synthetic mutation disables the prompt-path guard only inside a disposable test process: the portable refusal test fails as intended (DID NOT RAISE). Production source remains unchanged.
  • Determinism checks pass: 20 serial repetitions of both changed tests; 10 parallel repetitions of both modules (27 tests each), plus a seeded shuffled run (27 passed).
  • Changed file passes black, isort, flake8, AST parsing, and git diff checks.
  • Independent review confirmed the platform mismatch and unchanged runtime security rules.

Manual verification

Original Windows failure and C:/D: paths verified in hosted logs. Native Windows acceptance remains pending CI. Repository local-gate backend passes; its frontend leg fails before tests because the default npm cache is not writable; frontend dependencies are not installed in this cloud checkout. Full-repository validation is not claimed green. Existing base CI also has unrelated missing kirocrew-runners-windows and internal-scan configuration blockers.

Screenshots / video

N/A: no UI change.

Related Issues

The exact main CI job is linked above; no separate issue.

Sibling sweep

Swept: git grep -n 'file:///proc/self/environ' -- test
Sibling: test/test_kas_agents.py -- not affected -- accepts both denied-location and unreadable-file errors explicitly on Windows.
Sibling: test/test_kas_prompt_paths.py -- not affected -- existing POSIX-layout suite is explicitly platform-scoped.

Pattern harvest

Not generalizable: the portable assertion needs a platform-independent protected-path fixture, not a broader production denylist.

Checklist

  • One Conventional Commit; test-only scope
  • Focused tests and mutation check completed
  • Self-review completed; changed-file style checks pass
  • No runtime documentation contract changed
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contribution under the repository's existing Apache-2.0 license; no separate agreement signed.

Summary by Sourcery

Ensure prompt refusal regression tests remain meaningful across Windows and POSIX without changing runtime behavior.

Enhancements:

  • Make disallowed prompt-path regression coverage platform-independent by testing a synthetic protected home path on all platforms while retaining procfs coverage on POSIX.

Tests:

  • Separate POSIX pseudo-filesystem prompt-path coverage from the portable refusal test and preserve both error assertions.

Summary by cubic

Makes the KAS disallowed-prompt regression test pass meaningfully on Windows.

Replaces the POSIX-only /proc/self/environ fixture, which resolves to an irrelevant drive letter on Windows, with a protected path under a synthetic home directory that also refuses on every platform. The procfs assertion survives in a separate POSIX-only test, and no runtime behavior changes.

Written for commit 024ec3f. Summary will update on new commits.

View guided diff Turn on auto-fix

Verification update

At head 024ec3fffa50a89665e0fb98169903fdc03131e7, Build 44037232807 completed successfully (wheel and Linux desktop jobs; Windows installer was skipped). CI 44037233020 has passed backend lint/type checks and Linux test shards, but Windows shard 5 and other Windows shards are still pending; no native-Windows acceptance is claimed yet. The Windows Pod Boot Canary remains an independent failed job. PR merge ref c28a89079deab6229b2f5921327a9485878649da has the stated base/head parents and the same tree a9db9f91e474ff1ca40fb1de89e0ace035eda172 as the published test-only change.

The local broad gate pipeline has now completed: mypy reports no issues in 4,812 source files; comment-history, brand and file-line-cap checks pass. Local frontend limitations above remain, and the overall PR is not green.

Native Windows verification completed

CI 44037233020 has now finished for unchanged head 024ec3fffa50a89665e0fb98169903fdc03131e7: all eight Windows backend jobs succeeded, including the previously failing shard 5. That job reports 35,358 passed, 2,047 skipped and 19 xfailed on Python 3.12.10; this is not a zero-skip claim. Its checkout is the verified merge ref c28a89079deab6229b2f5921327a9485878649da.

The overall CI run remains failure, solely within that workflow due to Pod Boot Canary (Windows): GitHub annotation says Required runner group 'kirocrew-runners-windows' not found. No runner or test steps ran for that job. Other PR review/security gates may also remain blocked by repository configuration; no global-green or merge-ready claim is made. No rerun, workflow changes, production changes, or security-policy changes were used to obtain the Windows test result.

@coderabbitai

coderabbitai Bot commented Nov 17, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@sourcery-ai

sourcery-ai Bot commented Nov 17, 2026

Copy link
Copy Markdown

Reviewer's Guide

Tests

Make denied-prompt coverage use a protected path under a synthetic home on every platform, while retaining the procfs-specific refusal check only on POSIX.

  • test/test_kas_prompt_file.py L12, L71-84

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@github-actions github-actions Bot added the readiness: action required A blocking check or review needs attention label Nov 17, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: action required A blocking check or review needs attention

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant