Skip to content

Add miren app disable and enable - #1431

Open
phinze wants to merge 22 commits into
mainfrom
phinze/app-disable-enable
Open

phinze wants to merge 22 commits into
mainfrom
phinze/app-disable-enable

Conversation

@phinze

@phinze phinze commented Nov 5, 2026 •

Copy link
Copy Markdown
Member

This carries @jcasimir's #1322 home. The feature and most of the work are his, with Evan's scheduler follow-ups, and those commits are here unchanged. Main kept moving under that branch, so we're finishing it from our side.

miren app disable turns an app completely off without deleting it, and miren app enable brings it back. While disabled, every service stays at zero, scheduled tasks pause, deploys are refused, and visitors get a holding page with a 503. miren app run still works, and addons keep running. #1322 has the full walkthrough, and the design and review history live there too.

On top of that branch, this merges main and adds five changes:

  • Authorization. App operations moved to resolved authorization after Add miren app disable and enable #1322 was written, which left these two methods admin-only on Cloud clusters. They now use the deploy action, matching app restart.
  • Deployer roles. Anyone who can deploy can already ship code that refuses connections, so app-deployer and cluster-deployer get disable and enable too. That settles the open question in Should deployer roles be able to disable and enable apps? #1354.
  • One component for disabled state. disabled_at and disabled_reason become a disabled component on the app, following route maintenance: presence is the state, and enabling clears it. last_enabled_at was really a run scheduler watermark, so it's now schedules_resumed_at. The status RPC and app status --json don't change.
  • Disable blocks deploys. Add miren app disable and enable #1322 recorded new versions without booting them, but trigger = "deploy" tasks still ran, and nothing could verify a version without booting the app. Now miren deploy refuses, and maintenance mode stays the option when deploys should keep flowing. Rollback and env changes still work.
  • A small gap. sandbox pool set-desired could raise a disabled app's pool, and now it refuses.

disabled reaches Cloud as a new health value. Cloud needs to learn it, or those apps will show as Unknown.

Supersedes #1322.

Closes #1354
Closes MIR-2061

jcasimir and others added 19 commits October 13, 2026 08:18
An App gains disabled_at and disabled_reason, an explicit record that the
operator turned it off, as opposed to an app at zero because something
broke. Disable marks the app and scales its pools to zero; enable clears the
mark and returns fixed-mode pools of the active version to their configured
count, leaving autoscaled services to start on the next request. Restart
refuses on a disabled app, since restoring fixed counts would undo it until
the next resync. App health reports a new disabled state, and
getConfiguration returns when and why.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The launcher keeps every pool of a disabled app at zero through resyncs and
deploys, skipping the one-instance boot it does for deploy verification. The
activator answers requests with ErrAppDisabled, served as 503, instead of
waiting out the pool timeout for capacity that won't come. The pool manager
clears crash state on a disabled app's pools, so cooldown can't hand them a
replacement and revive the app.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
app list shows disabled apps, app status shows when and why, and a deploy to
a disabled app finishes with a verdict instead of waiting for an instance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The run scheduler skips a disabled app and forgets its tasks' frontiers, so
enabling the app resumes the schedule from that moment instead of firing
every tick missed while it was off. On a fallback copy left disabled on an
old cluster, a nightly job would otherwise run on both clusters against
shared state. Manual runs are unaffected.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The disabled check moves up into resolveIngressTarget, which already reads
the App on every request. It now runs before auth, as maintenance does, and
before the lease cache, so a warm app stops serving the moment it's disabled
instead of when its sandboxes exit. DoRequest gets the same check ahead of
its lease cache.

The response goes through the maintenance holding page: no-store, the JSON
shape with error "disabled", and the page worded as unavailable. Templates
get a Disabled field alongside Maintenance, which stays true so a template
written for maintenance still renders a holding page. The disable reason is
left off the page; it's an operator note for app status.

The activator check stays as a backstop for a request already past ingress
when the app is disabled.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A disabled app with a route in maintenance printed two separate yellow
lines. They now read as one "Not serving" section: when and why the app
was disabled, then the routes showing their maintenance page instead. An
app that's only in maintenance keeps its existing line.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Maintenance mode's docs said background work keeps running and to stop it
the way you normally would; app disable is that way, and pairing the two
keeps the holding page up while workers and schedules stop. The app disable
docs now say scheduled tasks pause and point to route down for taking only
traffic down.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The disabled override lived inside the branch that lists the active
version's pools, so a never-deployed app read as unknown and a failed pool
lookup dropped it, while list and the cloud health feed said disabled. It now
applies after those branches.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Neither method was in any workload role, so even a cluster-admin workload
was refused before reaching the handler. They go in a new appLifecycle
block granted to app-admin, for its own app through the existing
rpc.AllowApp guard, and to cluster-admin for any app. Deployers don't get
them: like create and destroy, turning an app off is lifecycle, not
deployment.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The scheduler only pauses an app when a sweep sees it disabled. A disable
and enable that both land between two sweeps left no trace, so a tick that
came due while the app was off still fired. Enable now records the window it
closes, last_disabled_at and last_enabled_at, and the scheduler skips any
tick inside it. Ticks that came due before the disable fire as usual.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Replace disabled-window history with a single enable boundary. Discard pending ticks up to enable, preserve normal scheduling afterward, and cover repeated enables and boundary behavior.
Check disabled state before scheduled-run admission, preserving manual runs and recording skipped work as terminal so enable cannot replay it.
# Conflicts:
#	api/app/app_v1alpha/rpc.gen.go
#	api/app/rpc.yml
#	api/core/core_v1alpha/schema.gen.go
#	controllers/deployment/launcher_test.go
The conflicts were small: the holding page needed Disabled alongside
Loading from the loading page work, and "Not serving" in app status
moved onto the named value list that replaced printed lines. A named
value is one aligned row, so its parts are joined with a semicolon
rather than spread over several lines.
main moved app operations to appclient.Authorize and the AppAction
list after these handlers were written. They kept the older AllowApp
check, so on Cloud clusters they fell through to the method gate,
which only admits cluster admins: an owner who could restart an app
could not disable it.

They use the deploy action, matching restart. Anyone who can deploy
can already ship code that refuses connections, so disable grants no
new power.
The sandbox pool RPC could raise a disabled app's desired count without
checking the flag, booting sandboxes the app is meant to be without.
Lowering stays allowed.
disabled_at, disabled_reason and last_enabled_at read like three pieces
of state for one yes/no, and allowed combinations that mean nothing,
such as a reason on an app that is not disabled. The disabled component
follows route maintenance (RFD-100): presence is the state, and
enabling clears all of it.

The remaining field is not app state. It is the run scheduler's
watermark, needed because its position lives in memory, so a disable
and enable between sweeps leaves no other trace. schedules_resumed_at
says who it is for. The status RPC and JSON keep their names.
Anyone who can deploy can already ship code that refuses connections,
so disable is a convenience rather than a new power. Moving both calls
into the deploy block keeps workload roles in line with the deploy
action that Cloud callers need. Restart and config stay where they
are.
miren-code-agent[bot]

This comment was marked as outdated.

Enable cleared the flag and then restored fixed pools, so a failed
restore returned an error for an app that was already enabled, and a
retry could only say it wasn't disabled. The launcher's resync restores
those pools anyway, so a restore failure is now a warning.

A new pool for a disabled app also fell into the branch that seeds it
from the current pool size, which can be above zero while Disable is
scaling down.
Services stayed at zero through a deploy, but trigger = "deploy" tasks
still booted sandboxes and ran, which is the shared-state risk the
schedule pause exists to prevent. Skipping them would activate code
without its migrations, and a deploy that boots nothing can't be
verified by a health check unless the app boots, which defeats the
disable.

So disable blocks builds, and maintenance mode remains the option when
deploys should keep flowing. The check runs before any upload or build
work and again ahead of deploy tasks, for a disable that lands
mid-build. Rollback and env changes run no deploy tasks and still work.
@phinze

phinze commented Nov 5, 2026

Copy link
Copy Markdown
Member Author

@jcasimir one behavior change from your PR worth a look. We noticed tasks with trigger = "deploy" were still running when everything else was disabled. Deciding how to handle that made us realize deploy-while-disabled also sidesteps any notion of health checking unless we actually boot the app, which circumvents the intent of disable.

The lack of a coherent set of behaviors led us to the simpler call: disable blocks deploys, and maintenance mode stays the option for when you want deploys to flow. Rollback and env changes still work on a disabled app.

Does your fallback flow need to deploy to a disabled copy, or is enable-if-needed enough?

miren-code-agent[bot]

This comment was marked as outdated.

The docs said running sandboxes shut down, which reads as covering task
runs too. Disable only scales pools, so a run already in progress
finishes, and addons sit outside the app's pools and keep running.

@miren-code-agent miren-code-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🍪 biscuit: ✅ ready to merge — auto-review, non-blocking

This is still ready for human review. I'm still not recommending approval, for the same two reasons as last time.

The new commit (c048e8a) closes the last code-side point. app-disable.md now says task runs already in progress finish and addons keep running. The generated page matches its source, appDisableDescription (cli/commands/commands.go:1515), so regenerating the docs won't undo the change. Letting in-flight runs finish is now a stated choice rather than a gap, so I'm resolving that thread.

I also traced the disable paths again: Disable → launcher hold-at-zero, the pool manager clearing crash state, the SetDesired guard, the schedules_resumed_at watermark in sweepTask, and the SCHEDULE check in start(). I didn't find a new defect. One minor note: running miren app disable a second time without --reason clears the earlier reason, because the CLI always sends one. The comment at servers/app/disable.go:37 says a repeat call "only updates the reason", so this looks intended.

Two decisions still need a human, unchanged since last time:

  1. Deployer roles can disable and enable apps (pkg/workloadroles/roles.go). The linked ticket is exactly the question of whether they should. This PR answers yes, and the roles table in in-cluster-api.md matches. cluster-deployer can now take every app on a cluster offline, so whoever owns the roles model should sign off.
  2. Cloud has to learn the disabled health value, or it will show these apps as Unknown. Someone needs to decide whether merging waits for that.

There's also an open question on the PR for @jcasimir: does the fallback flow need to deploy to a disabled copy? If it does, that pushes back on "disable blocks deploys", so it's worth answering before merge.


🍪 full review note · reviewed at c048e8a

@phinze
phinze marked this pull request as ready for review November 5, 2026 17:54
@phinze
phinze requested a review from a team as a code owner November 5, 2026 17:54

@evanphx evanphx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking these up @phinze! The issues from the other PR look handled.

I agree on using the deployer role here, it's changing what the cluster serves for the app.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Should deployer roles be able to disable and enable apps?

3 participants