Repository navigation
fix(apps): stop an app's backend on uninstall and observe the port instead of trusting the stop - #13304
Conversation
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Verify one live end-to-end [DESIGN-REVIEWED] e31164d |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS The uninstall handler re-copies the stop-and-observe sequence that Not justified as shipped
What this change shipsInventory (10 items) — 9 justifiedIntent: make removing an app actually stop its backend and its scheduled jobs, and stop reporting success when it did not — a FIX.
[FIRST-PRINCIPLES-REVIEWED] e31164d |
Opus 5 Review — ✅ no blocking findingsReviewed Review detailsNothing blocks — but the messages this change adds reach no surface that displays them. FINDING — src/kiro_crew/apps/routes.py:1524 — the new still-listening-port message is invisible to both dashboard uninstalls, contradicting the comment that FINDING — src/kiro_crew/cli_commands.py:1228 — the new delegation routes the CLI through the handler that runs FINDING — src/kiro_crew/apps/routes.py:1526 — FINDING — src/kiro_crew/cli_commands.py:856 — [OPUS-REVIEWED] e31164d Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
980b32c to
0c0ebab
Compare
0c0ebab to
e867365
Compare
e867365 to
a58693d
Compare
a58693d to
51f47e4
Compare
7384f10 to
f14ebc0
Compare
f14ebc0 to
498d71c
Compare
498d71c to
7bcd1ae
Compare
…ares Both uninstall entry points take part in the positive-evidence stop contract that disable and trust-revoke already hold. The CLI hands uninstall to a running gateway the way enable and disable do, carrying --purge-data. When no gateway answers it reports that the backend was not stopped instead of printing success, and names restarting the gateway as the action, since on Windows and in sandboxed shells this path is taken even while a gateway runs. The gateway handler stops the backend for every app rather than only when the app-written resources field reads gateway, and observes the recorded port afterwards, reporting unstopped_backend_port when something still accepts connections. An app-written field cannot switch off the app's own teardown, and a discarded stop result cannot stand in for a stop that happened. Scheduler cleanup loses the same gate, for the same reason and one more: an app:<name> job is persisted in the gateway's own cron store and fired by the gateway's own service, which applies no app-admission check at fire time. Gated, such a job outlived the uninstall and kept running its payload against a deleted app directory until the next boot reconciled the store, which also left uninstall cleaning up less than the strictly less destructive disable. Deregistration still honors the field: the process and the gateway's own scheduler entries are not the app's to keep, but the agents, skills and routes it registered for itself are. A cleanup that RAISED is now told apart from one that found nothing. deregister_app_crons_from_service answers 0 for both, and the rows cannot be re-counted to settle it because a failing save leaves them filtered out of the in-memory job list until a reload, so the exception is the only evidence there is. deregister_app_crons_reporting_failures is that evidence: the same removal, reporting every failure. The uninstall handler consumes it and refuses with a retryable 409 before the non-idempotent onUninstall script runs, so a write that never landed no longer deletes the app while its still-enabled rows keep firing from disk. The wrapper's own answer is unchanged, deliberately. It is what the CLI's cron cleanup calls on disable as well as uninstall, and reporting a failed write is a decision only a caller about to do something irreversible needs to make. The gateway disable path does not use it at all, driving CronSDK directly. The uninstall completes rather than aborting on a still-listening port. An app the operator asked to remove has already run its own onUninstall script, which is not idempotent, so an unremovable app is worse than a reported port. The scheduler steps abort instead, which is sound because they run first, before anything destructive. A reported port means the listener survived, and adoption reads health and owner PIDs rather than the code the listener executes, so a later install of the same name can adopt that survivor. Closing that requires provenance on the adoption path itself, which is a separate change on a file this one does not touch, and is left to its own issue. Adds test/test_apps_uninstall_backend_stop.py covering both entry points, the delegation, the ungated scheduler cleanup, the write-failure abort, the disable path's unchanged disposition, and the port report.
7bcd1ae to
e31164d
Compare
…ares (kirodotdev#13304) Both uninstall entry points take part in the positive-evidence stop contract that disable and trust-revoke already hold. The CLI hands uninstall to a running gateway the way enable and disable do, carrying --purge-data. When no gateway answers it reports that the backend was not stopped instead of printing success, and names restarting the gateway as the action, since on Windows and in sandboxed shells this path is taken even while a gateway runs. The gateway handler stops the backend for every app rather than only when the app-written resources field reads gateway, and observes the recorded port afterwards, reporting unstopped_backend_port when something still accepts connections. An app-written field cannot switch off the app's own teardown, and a discarded stop result cannot stand in for a stop that happened. Scheduler cleanup loses the same gate, for the same reason and one more: an app:<name> job is persisted in the gateway's own cron store and fired by the gateway's own service, which applies no app-admission check at fire time. Gated, such a job outlived the uninstall and kept running its payload against a deleted app directory until the next boot reconciled the store, which also left uninstall cleaning up less than the strictly less destructive disable. Deregistration still honors the field: the process and the gateway's own scheduler entries are not the app's to keep, but the agents, skills and routes it registered for itself are. A cleanup that RAISED is now told apart from one that found nothing. deregister_app_crons_from_service answers 0 for both, and the rows cannot be re-counted to settle it because a failing save leaves them filtered out of the in-memory job list until a reload, so the exception is the only evidence there is. deregister_app_crons_reporting_failures is that evidence: the same removal, reporting every failure. The uninstall handler consumes it and refuses with a retryable 409 before the non-idempotent onUninstall script runs, so a write that never landed no longer deletes the app while its still-enabled rows keep firing from disk. The wrapper's own answer is unchanged, deliberately. It is what the CLI's cron cleanup calls on disable as well as uninstall, and reporting a failed write is a decision only a caller about to do something irreversible needs to make. The gateway disable path does not use it at all, driving CronSDK directly. The uninstall completes rather than aborting on a still-listening port. An app the operator asked to remove has already run its own onUninstall script, which is not idempotent, so an unremovable app is worse than a reported port. The scheduler steps abort instead, which is sound because they run first, before anything destructive. A reported port means the listener survived, and adoption reads health and owner PIDs rather than the code the listener executes, so a later install of the same name can adopt that survivor. Closing that requires provenance on the adoption path itself, which is a separate change on a file this one does not touch, and is left to its own issue. Adds test/test_apps_uninstall_backend_stop.py covering both entry points, the delegation, the ungated scheduler cleanup, the write-failure abort, the disable path's unchanged disposition, and the port report.
What is the problem?
Removing an app does not stop the backend process that app runs, and the command still reports success.
Two entry points remove an app, and neither takes part in the teardown contract the rest of the lifecycle already holds:
kirocrew app uninstallruns the whole uninstall inside the CLI process. The CLI is a separate process from the gateway, so it holds no handle on the child the gateway spawned. It deletes the app's files and prints a success line while the backend keeps its port, its app secret, and its proxied routes.resourcesfield readsgateway, and it discards the stop's return value.resourcesis written by the app itself, so an app that declaresresources: "app"switches off its own teardown. Even on the branch that does stop, aFalseanswer is thrown away.The same
resourcesgate sits on the handler's scheduler cleanup, one step earlier. Anapp:<name>cron job is persisted in the gateway's own cron store and fired by the gateway's ownCronService, which applies no app-admission check at fire time. So for an app whoseresourcesis notgateway, uninstall deleted the app directory and left its enabled jobs to keep running their command, script or agent payload until the next gateway boot reconciled the store.on_app_disablekeys that cleanup on the app's cron permission and not onresources, so uninstall cleaned up less than the strictly less destructive disable.One step further in, a cron cleanup that FAILED could not be told from one that found nothing.
deregister_app_crons_from_serviceanswers0for both: it is the honest answer for an app that owned no jobs, and it is also what an unexpected write failure looks like from outside, because the bridge logs and audits that case and then returns0. The handler recordedremoved=0as completed and carried on throughonUninstalland app deletion while the still-enabled rows were still on disk.disableand trust-revoke already stop and then observe the port correctly.One half of the issue's premise no longer holds. The issue names forgeable evidence in
stop_recorded_app_backend, but that function is not onmain: it was extracted from PR #7901 and never merged.app_backends.pids.jsonsurvives only as a cache the startup reap reads, and every row carriespidplusstart_timeand is matched positively before any signal is sent. No stop decision onmainreads pidfile absence.Why this issue matters to the user
After an uninstall the user has no app and no way to reach it, yet the process keeps running. It holds its TCP port, so reinstalling the app or installing a different one on that port fails. It still holds a valid app secret. The gateway's proxy routes for it are gone, so nothing inside the product can reach it to shut it down. The only remedy left is finding the pid by hand or restarting the gateway.
The scheduler half is worse than a held port, because it keeps executing. A removed app's jobs go on firing their payload against a directory that no longer exists, and nothing left in the store says they belonged to a removed app.
The success line the command prints is what makes any of this hard to notice at all.
How our fix solves it
Symptom: uninstall reports success while the app's backend is still executing and its scheduled jobs still fire.
Root cause: the uninstall path is the only lifecycle path that gates teardown on app-written metadata, and the one signal that could have caught the remaining case is a count that means two different things.
src/kiro_crew/cli_commands.py:app uninstalldelegates to a running gateway through_run_app_action_through_gateway, the same wayenableanddisablealready do, carrying--purge-dataover the wire. The gateway owns the child, so the gateway is the process that can stop it. When no gateway answers, the file-only path no longer implies a clean stop: it states that the backend was not stopped, names the app, and names restarting the gateway as the action, carrying the caveat the neighbouring_print_file_only_app_resultalready states — on Windows and in sandboxed shells this path is taken even while a gateway is running, so "start the gateway" would not be actionable.src/kiro_crew/app_lifecycle_client.py:toggle_appcarries thepurge_dataflag, so the delegated uninstall preserves the flag's meaning. A bodyless request would silently turn--purge-datainto a data-preserving uninstall.src/kiro_crew/apps/routes.py:handle_uninstall_appstops the backend for every app rather than only whenresourcesreadsgateway, and captures the recorded port before the stop. After the stop it probes that port, and reports a port still accepting connections asunstopped_backend_portwith the port number.src/kiro_crew/apps/routes.py: the scheduler cleanup loses the same gate, and runs for every app. Deregistration still honoursresources: the process and the gateway's own scheduler entries are not the app's to keep, but the agents, skills and routes an app registered for itself are.src/kiro_crew/apps/bridges.py:deregister_app_crons_reporting_failuresperforms the same removal and reports every failure instead of answering0. The uninstall handler consumes it and refuses with a retryable 409 before the non-idempotentonUninstallscript runs, so a write that never landed no longer deletes the app while its rows keep firing.docs/system-specs/modules/app-kit-platform.md: the owning spec is corrected to match. Its teardown chain recorded cron cleanup as gateway-managed and backend stop plus deregistration as gateway-managed only, and its CLI-routing paragraph recorded that uninstall is deliberately not routed through a running gateway. This change reverses all three, and that paragraph now says so explicitly and supersedes the old statement. AGENTS.md requires reading this spec before touching this area, so a recorded decision left contradicting shipped behaviour is how the behaviour gets reverted by accident.The exception is the only evidence available, which is why the fix propagates rather than re-checks. A post-hoc count cannot settle it:
CronService's rollback keeps the removed rows filtered out of the in-memory job list after a failing_save()and only a reload restores them, solist_jobsreports a removal that did not persist.cron.pyis untouched here — that rollback is load-bearing for the children whose ownership it restores.deregister_app_crons_from_servicekeeps its own answer, deliberately. It is what the CLI's cron cleanup calls on disable as well as uninstall, and reporting a failed write is a decision only a caller about to do something irreversible needs to make. The gateway's disable path does not use it at all:on_app_disabledrivesCronSDKdirectly.The port report is recorded in both
warningsanduninstall_log, because their consumers are disjoint and neither shows it twice.print_resultrenderswarningsand neveruninstall_log; the dashboard's uninstall readsuninstall_log(website/src/pages/AppDetailPage.tsx:1168) and neverwarnings.A probe that observed nothing is also reported, rather than read as a clean stop.
unstopped_backend_portdocumentsNoneas "nothing observed", not "definitely stopped", and the distinction is load-bearing here: with no gateway-recorded port the probe falls back to the port declared inapp.json, which sits inside the app directory, andonUninstallhas by then already run app-controlled code there. So an untracked fixed-port backend could relabel its port — or drop itsentryPoint— and the fallback would probe the wrong port and answerNone, leaving a live process and a valid app secret behind a success line, with the files gone and the pidfile record the next start would reap from already dropped. When no recorded port existed, the result now says the stop could not be verified. The warning is gated on a backend being declared in the installed record captured BEFORE the hook ran, so an app cannot suppress it by rewriting its manifest, and an app that never had a backend collects no warning about one.The backend stop reports rather than aborts, while the scheduler steps abort. The difference is position: the scheduler steps run first, before anything destructive, so an abort there leaves nothing changed and the retry is safe. By the time the stop runs, the non-idempotent
onUninstallhas already executed, and an app that cannot be removed is worse than a port named in the result.What tests we did
Every path that removes or stops an app backend was enumerated with a verdict, not just the ones that were edited.
resources: gatewayresources: appresources: app0read as "owned nothing", app deletedonUninstall00New file
test/test_apps_uninstall_backend_stop.py, 20 tests across both entry points. Both directions of the widened scope are pinned: an uninstall whose cron write fails is refused with a retryable 409 and never reaches the teardown script, and the same failure throughderegister_app_crons_from_serviceis still answered0so the disable path is unaffected. A third test holds0as the honest answer on both entry points when the app genuinely owned nothing.Each decision was verified by mutation, one at a time, each reddening exactly one intended test with the rest green:
resources, port report droppedresourcestest_the_scheduler_is_cleaned_even_for_a_self_managed_appwarningsonlytest_the_port_reaches_both_consumers0test_an_unexpected_write_failure_is_still_answered_as_zeroLocal runs, all with
-n0and naming files rather than directories: the new file withtest_app_routes.pyandtest_app_bridges.py, 251 passed;test_app_slot_ownership.pywithtest_apps_routes_coverage.pyandtest_trusted_apps_api.py, 341 passed;test_cli_commands_coverage.pywithtest_app_slot_ownership.py, 294 passed;test_cron_arbiter_items.py,test_cron_store_unreadable_boundaries.pyandtest_spawn_audit.py, 71 passed. flake8, isort, black and mypy are clean on the changed sources, and the comment-history gate passes.Four existing test files change as a direct consequence, none of their assertions touched.
test_app_routes.py's four cron-abort tests patch the bridge entry point the handler calls, by name, so they are retargeted. The other three drivehandle_uninstall_appwith a blanketMagicMockstate, whose auto-createdcronsattribute makes the handler drive a fake removal that raises — harmless while that was swallowed, and now correctly an abort. Those tests supply no cron service so they stay scoped to the step each is about:test_app_slot_ownership.py's pointer-flush and teardown cases,test_apps_uninstall_keep_specific.py's keep-list parsing, andtest_apps_uninstall_hook_registries.py's hook teardown. Every test that drives the handler was enumerated rather than found one CI round at a time; the fifth,test_trusted_apps_api.py, already supplies a cron service and needs no change.Any other suggestions on the work
The invariant, swept
Every fix above is one instance of a single rule: app-controlled evidence, and the absence of evidence, must not stand in for proof. Four instances were found one per review round, so the remaining sites on the uninstall, disable and stop paths were enumerated deliberately rather than waiting for a fifth. Each place that reads
app.json, reads a manifest, or could treat "cannot see it" as "it is not there":app.json, via_app_declares_backendapp.json, sinceget_appreadsapp_dir(name)/app.jsononUninstalllookup_app_declares_backendon an unreadable manifestTrueunstopped_backend_portmanifest fallbackapp.jsonbackend.py, outside this change; the unverifiable report above is what makes it visibleon_app_disablescheduler cleanuppermissions.cron_absence_is_genuineThe one residual inside this change is the handler's unverifiable-stop gate, and it is stated rather than closed because the union that fixes the CLI cannot fix it: that branch is only reached when no recorded port exists, so the gateway-owned half of the union is empty there by construction. Warning anyway would fire on every uninstall of an app that never had a backend, and the only signal that distinguishes the two is the app's own manifest. Closing it properly needs a gateway-owned record that the app declared a backend, which does not exist today —
installed.jsoncarries the app's metadata but not its backend declaration. That belongs with the adoption-provenance work on #13403, which already needs gateway-owned provenance for the same reason.The two pre-existing instances are recorded, not fixed.
unstopped_backend_port's fallback is in a file this change does not touch, andon_app_disable's permission gate is on the disable path, whose behaviour this change holds byte-identical on purpose.The swallowed-write hole this closes is pre-existing on
mainand is not created by this change:bridges.py's class-agnostic handler andcron.py's rollback are both untouched upstream. For an app whoseresourcesis notgateway,mainnever cleaned crons on uninstall at all, so this change is a strict improvement for exactly the apps that were worst off, and the window it leaves is narrower than the one it removes — a failed write is now refused, where previously every such app's jobs were orphaned unconditionally.A backend that outlives its uninstall meets a second gap that this PR deliberately does not close. The gateway's start path adopts an existing listener on an app's declared port when a health probe answers; it records which PIDs hold the socket, but nothing reads the code that listener executes, and an adopted instance is excluded from the startup stale-reap. So a listener that survived one removal can be adopted by the next install of the same name. That is pre-existing and wider on
main, and reporting the surviving listener is what makes it visible. It is left to #13403, because the fix lands insrc/kiro_crew/apps/backend.pyand needs the record it consults to be a leaf covered by the sandbox security floor — the app's own backend receivesKIROCREW_HOMEin its spawn env and can otherwise delete the record that fences it, and a guard forgeable by the process it exists to fence is the same class as the defect being closed.The stop-and-observe sequence is spelled twice:
_stop_backend_and_observehere and the same triple inline inteardown_app_runtimeatsrc/kiro_crew/apps/teardown.py:330-341. Sharing it needs a helper in that module, which also serves disable and trust withdrawal, so it is filed as #13462 rather than folded in.The port probe establishes that something still listens on the recorded port. It cannot establish that the listener is the app's backend rather than an unrelated process that claimed the port after the stop, and it is a single instant, so a backend that rebinds after the probe reads as stopped. Both halves are carried by #13403. What the probe can no longer do is stay silent about not knowing: an absent recorded port is reported as unverifiable rather than as a clean stop.
app_backends.pids.jsonstays a cache. Nothing here makes a stop decision from it, and the startup reap's positivepidplusstart_timematch is untouched.The CLI's no-gateway path still cannot stop a backend, and its cron cleanup still answers
0on a failed write. It has no gateway-owned port to observe and no 409 to return; the row it leaves in the pidfile is what the next gateway start reaps. This change makes that limit visible instead of hiding it behind a success line.That second half is a fair hit against this change's own stated criterion, so it is recorded rather than argued away: if reporting a failed write is a decision a caller about to do something irreversible needs to make, then
_cleanup_app_crons_from_scheduleris such a caller too, since the CLI's local uninstall reachesderegister_appanduninstall_appafter it. It is not fixed here because that helper is shared with CLI disable, whose disposition this change deliberately leaves byte-identical, so telling the two apart means splitting the helper. That belongs with the shared-helper work already filed as #13462.Pattern harvest
Rule candidate: semgrep
Pattern: a stop or teardown helper invoked as a bare statement, its boolean return discarded
Three further patterns here are not mechanical enough for semgrep, so they belong in a review prompt instead. First, a teardown step gated on a field that the managed component itself authors, which lets that component switch off its own cleanup — worth checking against every sibling step in the same handler, not only the one a report names, since this change found two under one gate. Second, a reuse decision made from a liveness probe alone, where the probe answers "something is here" and the code reads it as "the thing I expect is here". Third, a count used as a success signal where zero is also the honest answer for an empty set: the caller cannot tell those apart, and if what it does next is irreversible, the failure has to arrive as something other than a number.
Closes #9396