Skip to content

feat(schedule): say when a cron job is in failure backoff - #22642

Merged
iamwhatever merged 1 commit into
kirodotdev:mainfrom
Pearcekieser:feat/schedule-backoff-note
Oct 21, 2026
Merged

iamwhatever merged 1 commit into
kirodotdev:mainfrom
Pearcekieser:feat/schedule-backoff-note

Conversation

@Pearcekieser

@Pearcekieser Pearcekieser commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The Schedule page says when a cron job is in failure backoff, so an owner can see why its next run is later than the schedule.

A job whose auto-pause limit is 0 or above the default of 5 does not pause at its 5th failure in a row. It backs off instead: the next run waits 5 minutes, doubling per failure up to one hour, until a run succeeds. compute_next_run_ts already returns the shifted time, so the Next Run column shows "in 47m" for a job scheduled every minute, with no reason given.

Why it matters

An owner who sees a next run far later than the schedule cannot tell a backoff from a broken scheduler, and has to read the store or the logs to find out. The backoff exists to bound the spend of a job that keeps failing, so the owner most likely to be looking at the row is the one trying to fix that job.

Not a goal

What changed (motivation → approach → change)

Symptom: a job in backoff shows a later next run with no explanation.

Approach: the page needs the time the backoff ends and the failure count behind it. CronJob.failure_backoff_until() already computes the first, so the board sends it as a raw epoch rather than a "backing off" flag. The page compares it with its own clock, so the note clears once the time passes, without waiting for a refresh from the server.

Change:

  • api_crons in src/kiro_crew/dashboard/handlers/cron_board.py adds consecutive_failures and failure_backoff_until (epoch seconds or null) to each job. failure_backoff_until comes from the new schedule.backoff_delaying_next_run(), which returns the backoff end only when the job is enabled and its next scheduled slot falls before it, the same comparison compute_next_run_ts makes to shift the time. A disabled job, and a daily job whose next slot is later than the backoff anyway, send null.
  • website/src/types/index.jobs.ts types both as optional, so an older gateway that omits them shows no note.
  • website/src/pages/SchedulePage.tsx shows "Delayed until HH:MM · N failures in a row" under the Next Run value while failure_backoff_until is in the future. The cell's tooltip repeats it and adds "A successful run ends the delay." It uses the existing text-warn token and wraps like the "Retried N times" note. The crons list reloads when a run starts or ends, which is when an elapsed backoff stops showing.
  • The plural note string and the one-line backoff_ends_on_success sentence are in en.manual.json and the 11 translated catalogs, en-XA.json is regenerated, and the plural key is in pluralKeys.json. The note puts the time first and the count second, so it wraps to four short lines in the narrow column instead of five.
  • The Schedule-page paragraph of docs/system-specs/modules/learn-cron-dashboard.md describes the two fields and the note.

Backwards compatibility

Compatible: the board response gains two fields and drops none, and the page shows the note only when both are present.

Removes nothing: the change adds two read-only fields and a note.

Tests

  • test/test_cron_backoff_wire.py calls the real api_crons with a real CronJob and the handler's time.time pinned. A limit-0 job at 5 failures serializes consecutive_failures == 5 and failure_backoff_until == last_run_ts + 300, equal to its next_run_ts. A disabled job, a daily job and an elapsed backoff each serialize null; those three fail on the first revision of this PR. The same job at 4 failures serializes 4 and null, and a healthy job serializes 0 and null.
  • website/src/test/SchedulePage.backoffNote.test.tsx pins Date with vi.useFakeTimers({ toFake: ['Date'] }) and vi.setSystemTime, and builds every fixture from that instant. It checks that a backoff one hour ahead shows the note with its count, that the cell's tooltip carries the note and "A successful run ends the delay.", and that null, a past time and a missing field each show nothing.
  • test/test_dashboard_cron_approval.py sets the two attributes on its mock job.

Determinism: both new test files pin their clock and read fixed timestamps, and neither starts a thread or timer.

3,258 cron and dashboard-cron tests, 93 Schedule page tests and 1,022 i18n tests pass, and tsc -p tsconfig.app.json, I18N_BASE_REF=origin/main npm run i18n:check, black, flake8 and mypy are clean.

Manual verification

The sandbox these were built in cannot open a listening socket, so the screenshots load the built website/dist in Playwright with request interception. The /api/crons response comes from the real api_crons serializer.

Screenshots / video

The "PR review sweep" row at desktop width, with the note under the next-run time:

desktop

At 390 px, where the note wraps under the next-run time:

phone

Related Issues

Closes #22641. The backoff note deferred from #15524. The wording review the UX lane asked for is done in this PR, so no follow-up is left open.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

@Pearcekieser
Pearcekieser requested review from a team and KiroCrewCaptain as code owners October 21, 2026 01:57
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 21, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, one should-fix below. Not approving yet: readiness is still checking on this head and the fork review lanes have not finished.

Head reviewed: 54f369b.

What I checked:

  • failure_backoff_until() on main returns null at or below the default limit, and compute_next_run_ts already applies the same epoch. The wire fields match what the scheduler uses.
  • fmtTime reads a number below the seconds ceiling as epoch seconds, so the seconds value from the board renders the right time.
  • The plural forms match each locale's existing retried_n_times set, and the key sits in pluralKeys.json in sorted order.
  • The wire test calls the real api_crons with a real CronJob and fixed timestamps. The page test covers future, past, null and absent values.

Should-fix (1): a disabled job can show the backoff note next to an empty next run. Details are inline.

Comment thread website/src/pages/SchedulePage.tsx Outdated
Only a FUTURE `failure_backoff_until` shows the note; a
past one has elapsed and explains nothing. Wraps like the
retry note: the column is narrow. */}
{typeof j.failure_backoff_until === 'number' && j.failure_backoff_until > Date.now() / 1000 && (

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 A disabled job can show "Backing off ... until" next to an empty next run

failure_backoff_until() does not look at enabled. compute_next_run_ts returns null for a disabled job, so the cell shows the empty placeholder. If an owner disables a failing job with auto-pause off soon after its 5th failure in a row, the cell shows the empty placeholder and "Backing off after 5 failures in a row, until HH:MM" for up to one hour. The note says the job will run later, and the empty next run says it will not run.

{typeof j.failure_backoff_until === 'number' && j.failure_backoff_until > Date.now() / 1000 && (

Why it matters: The note exists to explain a next run time. With no next run, it gives a wrong reason for a state the owner created.

Suggestion: Show the note only when a next run exists, for example add j.next_run_ts != null && to this condition. Another option is to send null from api_crons when j.enabled is false. Add one case to the page test for a disabled job. If the page already hides this cell for disabled jobs, I will withdraw this.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of 055e4cb615cf21e8168f2038fb82d669bf74c0d0 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: this only adds things. Two read-only fields go on the wire, the page shows a note when they are present, and screenshots of this revision are attached.

Notes

  • For a kind == "cron" job, compute_next_run_ts moves the run to the first slot after backoff_until. The note still says "Delayed until " while Next Run shows that later slot. Showing next_run_ts in the note, or wording it "not before", would make the two agree.

[DESIGN-REVIEWED] 055e4cb

Description read

[DESCRIPTION-READ] 51328495015c51f59c18ea7bc192552de7c938c3207a7af4ed1b91175a34b914

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ✅ PASS

UX-level review of 055e4cb615cf21e8168f2038fb82d669bf74c0d0 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

UX-Verdict: PASS

Both screenshots show the new "Delayed until 8:09 AM · 7 failures in a row" note, at desktop width and at 390 px. Nothing blocks. This fork lane has no blind read, so nobody has read the note without knowing the code; pushing the branch to this repository runs the lane that does.

Notes

  • The hover text "A successful run ends the delay." appears in neither screenshot. It is a native title tooltip and repeats what the note says, so it is not an evidence gap.

[UX-REVIEWED] 055e4cb

Description read

[DESCRIPTION-READ] 3c9e0824a3a14fe364ac94bc63c23ccac948f89400782763a2af6631e3c5c728

That sha256 names everything this verdict read: the PR description, plus the 5 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ✅ PASS

Premise-level review of 055e4cb615cf21e8168f2038fb82d669bf74c0d0 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. Every item comes from issue #22641, and the code reuses the existing CronJob.failure_backoff_until() rather than computing the backoff again.

What this change ships

Inventory (8 items) — 7 justified

Intent: let a cron job's owner see that its next run is late because of failure backoff, not a broken scheduler. This is an ADDITION.

  1. The Next Run cell shows "Delayed until HH:MM · N failures in a row" while a job is in backoff — justified
  2. The Next Run cell's tooltip repeats the note and adds "A successful run ends the delay." — justified
  3. GET /api/crons jobs now include consecutive_failures — justified
  4. GET /api/crons jobs now include failure_backoff_until (null when the backoff did not move the next run) — justified
  5. New public helper schedule.backoff_delaying_next_run() — justified
  6. compute_next_run_ts now uses the shared _backoff_shift test, with behaviour unchanged — undeclared, behaviour-neutral refactor
  7. The note's strings are added to the English catalog and 11 translations, en-XA and pluralKeys.json — justified
  8. The Schedule-page spec paragraph describes the two fields and the note — justified

[FIRST-PRINCIPLES-REVIEWED] 055e4cb

Description read

[DESCRIPTION-READ] 3653b4a3f44f66d40a033f68c7fa1a7368742815b9449c607eca1e9bdf629714

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 055e4cb615cf21e8168f2038fb82d669bf74c0d0 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 055e4cb

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings (all downgraded on adjudication)

Reviewed 055e4cb615cf21e8168f2038fb82d669bf74c0d0 via the fork AI-review pipeline; updated in place on each push.

Opus 5.5 adjudication downgraded every blocking finding. Downgraded: the remedy each one needs is disproportionate to the harm it prevents. Adjudication downgraded all 1 blocking finding(s): the remedy each one requires is disproportionate to the harm it prevents.

Review details

BLOCKING -- website/src/pages/SchedulePage.tsx:1139 -- Backoff notes do not expire on an idle page

typeof j.failure_backoff_until === 'number' && j.failure_backoff_until > Date.now() / 1000
Trigger: Leave Schedule open without other activity for an enabled */10 * * * * job with auto-pause disabled, eight consecutive failures, and last_run_ts at 12:01:10. api_crons supplies a backoff ending at 12:41:10 and a next run at 12:50; this comparison runs only during rendering.
Outcome: The expired note and tooltip remain visible after 12:41:10 until another render occurs. The scheduler’s empty due scans do not send a refresh, so the page continues reporting backoff after it has ended.
Anchor: defect
Fix: Schedule a local state update at the next displayed backoff deadline so the note expires without waiting for a server refresh.

[BLOCK-MERGE-DOWNGRADED] 055e4cb
[GPT-REVIEWED] 055e4cb

Adjudication (Opus 5.5) — is blocking on each finding proportionate?

F1 — LOW. The note only goes stale on a page left idle. It is computed at render time (authentic.patch, SchedulePage.tsx:1139), and the page re-renders on every crons refresh pushed when a job starts or ends (website/src/pages/SchedulePage.tsx:441-444). It also reloads on any action (:500, :1172). So the stale note clears by the next run start at the latest: 12:50 in the trigger, which is the very run the note explains. The stale text ("Delayed until 12:41 · 8 failures") is still true history that explains why the next run is 12:50. Nothing is lost, it is visible, and it corrects itself.
The real fix is a new timer effect: track the earliest future deadline across rows, plus cleanup and rescheduling on every load. That is extra state on the page for a few minutes of a cosmetic tooltip/note running behind, so the cost outweighs the harm.

[ADJUDICATION] 055e4cb615cf21e8168f2038fb82d669bf74c0d0 total=1 uphold=0 downgrade=1
DOWNGRADE F1 website/src/pages/SchedulePage.tsx:1139 reason=disproportionate-remedy
[GPT-ADJUDICATED] 055e4cb615cf21e8168f2038fb82d669bf74c0d0

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 21, 2026
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: the test built its fixtures from the real Date.now() while the page read it separately, so a clock step over an hour between the two could flip the assertion

fixed in f194351. SchedulePage.backoffNote.test.tsx now calls vi.useFakeTimers({ toFake: ['Date'] }) and vi.setSystemTime(NOW_MS) in beforeEach and vi.useRealTimers() in afterEach, and every fixture is built from NOW_MS. Only Date is faked, so the page's async loads still run on real timers.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: the serializer sent failure_backoff_until() as is, and that method reads neither enabled nor the job's next scheduled slot

fixed in f194351. The serializer now sends schedule.backoff_delaying_next_run(), which returns the backoff end only when the job is enabled and its unshifted next slot falls before it, the comparison compute_next_run_ts makes. A disabled job and an every 1d job five failures in both send null. test_cron_backoff_wire.py pins the handler's clock and covers both, plus an elapsed backoff; all three fail on the previous head.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

fixed in f194351. A disabled job no longer shows the note: backoff_delaying_next_run() returns null for it, so the Next run cell shows only its dash. test_a_disabled_job_serializes_null covers it.

@Pearcekieser
Pearcekieser force-pushed the feat/schedule-backoff-note branch from 54f369b to f194351 Compare October 21, 2026 02:27
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

intended. The note is read during render, and the page re-renders when the backoff ends: the crons list reloads on the backend's crons refresh, which fires when a run starts or ends, and a job in backoff runs as soon as its backoff ends. The description and the spec paragraph now say this instead of "clears once the time passes".

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

fixed in f194351. While the backoff is active, the Next run cell's title is the next-run date and time followed by the note's full wording. The vitest case checks that the tooltip contains the note.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

deferred to a copy pass with a translation review. The wording is a fair point, and the tooltip added in f194351 gives a reader who hovers the full line. Changing the copy means rewriting the key in all 12 catalogs, and the 11 translations in this PR have not been reviewed by a speaker yet, so a reword is better done once, together with that review, than in a second unreviewed pass here.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 21, 2026
@Pearcekieser
Pearcekieser force-pushed the feat/schedule-backoff-note branch from f194351 to 669e6d2 Compare October 21, 2026 02:43
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: the fixture patched time.time on the stdlib module reached through cron_board.time, which every consumer in the worker shares

fixed in 669e6d2. The clock fixture now builds ManualClock(start=_LAST_RUN + 30) and calls install(monkeypatch, cron_board), which replaces only the board module's own time binding with a namespace that delegates every other name to the real module. The elapsed case calls clock.advance(...). The test-stdlib-clock-rebound Semgrep rule passes on the file.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: the new helper repeated the inline result < backoff_until test of compute_next_run_ts

fixed in 669e6d2. schedule.py has one copy of the test, _backoff_shift(job, unshifted). compute_next_run_ts shifts by it and backoff_delaying_next_run reports it, adding only the enabled check. The 3,258 cron and dashboard-cron tests pass unchanged.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: the new helper repeated the inline result < backoff_until test of compute_next_run_ts

fixed in 669e6d2. Both functions now go through _backoff_shift, so the shift and the note read one test and cannot drift.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

intended. For a cron-expression job the note's "until HH:MM" is when the backoff ends, and the Next Run value is the first slot at or after that, so it can be later. The two read as cause and result: the job is held until 10:52, so its next slot is 11:00. For an every job the two are the same moment.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

rebutted. The note's job is the reason, not the time: "after 7 failures in a row" is what the Next Run value cannot say. The time is not always a repeat either. For a cron-expression job the next run is the first slot after the backoff ends, so "until 10:52" and "in 15m" differ. The five-line wrap at desktop width comes from the narrow column, which every row shares today and this PR leaves alone.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 21, 2026
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

intended. The backend only sends failure_backoff_until for an enabled job whose next run the backoff moved, and that next run is the backoff end (or the first slot after it). At that moment the scheduler fires the job, the backend pushes its crons refresh on run start, and the page reloads and re-renders, so the note is re-checked exactly when it would expire. If the scheduler is not running, the note stays, and that is correct: the job has not run, and its next run is still that time.

@Pearcekieser

Pearcekieser commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor Author

deferred to Pearcekieser/KiroCrew#48, one copy pass for this note with a speaker review of the 11 translations. This is the third round to raise the wording, so it gets its own change rather than another unreviewed rewrite of 12 catalogs here.

@Pearcekieser

Pearcekieser commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor Author

deferred to Pearcekieser/KiroCrew#48. Saying that one successful run ends the delay is a new string in every catalog, so it goes in with the reword and the translation review.

@Pearcekieser

Pearcekieser commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor Author

deferred to Pearcekieser/KiroCrew#48, which carries the two-line target at 1440 px. The column width is the same for every row today.

@Pearcekieser

Pearcekieser commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor Author

deferred to Pearcekieser/KiroCrew#48, where this exact wording is listed as the starting point.

@Pearcekieser

Pearcekieser commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor Author

deferred to Pearcekieser/KiroCrew#48. The first-time-reader check and the reword are one task there, so the reader sees the wording that ships.

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 669e6d2, compared against the previous review at 54f369b.

The earlier should-fix is addressed. The board now sends failure_backoff_until from schedule.backoff_delaying_next_run(), which returns null for a disabled job, so a disabled row no longer shows a backoff note next to an empty next run. The same helper also returns null when the next scheduled slot already falls after the backoff end, and compute_next_run_ts now shares that test through _backoff_shift, so the note and the shifted time cannot disagree. New backend tests cover the disabled, slot past backoff and elapsed backoff cases on a pinned clock, and the frontend test now pins Date and checks the tooltip carries the note.

No new findings on this head.

Maintainer checks that still remain before this can merge:

  • The readiness label is still checking and PR Readiness is pending on this head.
  • This is a feat change, so it needs a human maintainer approval rather than an automated one.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 21, 2026
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

chore (non-blocking): rebased on main, no behaviour change. 8de044b3e2 replaces 669e6d2306; main had gained 21 commits touching the same locale catalogs and spec files, with no conflicts. 3,676 cron, dashboard-cron and docs tests, 1,115 Schedule page and i18n tests, tsc -p tsconfig.app.json and the i18n check pass locally.

@Pearcekieser
Pearcekieser force-pushed the feat/schedule-backoff-note branch from 669e6d2 to 8de044b Compare October 21, 2026 07:16
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision labels Oct 21, 2026

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST_CHANGES. The code looks sound, but the deferred UX work is tracked only on your fork, and the PR description does not link it.

Reviewed head: 8de044b. It is a no-change rebase of 669e6d2; range-diff shows the same patch.

The UX dispositions defer five items to Pearcekieser#48: the reword away from 'Backing off', the two-line target, the 'a successful run ends the delay' tooltip, a speaker review of the 11 translations, and a first-time-reader check. #48 covers all of them. Maintainers of this repository cannot track or close it, though, and the description does not link it. Please open it as an issue in kirodotdev/KiroCrew and link it from the description, or do the reword in this PR.

The code: backoff_delaying_next_run returns null for a disabled job and shares the shift test with compute_next_run_ts (

def backoff_delaying_next_run(job: CronJob, now: float | None = None) -> float | None:
"""The failure-backoff end that moved this job's next run later, or ``None``.
``None`` for a disabled job, which has no next run to move, and for a job
whose next scheduled slot already falls at or after the backoff end (an
``every 1d`` job five failures in), whose next run the backoff leaves
where it was. The test is :func:`_backoff_shift`, the one
:func:`compute_next_run_ts` shifts by.
"""
if not job.enabled:
return None
return _backoff_shift(job, _compute_next_run_ts_raw(job, now))
def _backoff_shift(job: CronJob, unshifted: float | None) -> float | None:
"""The backoff end when it falls after ``unshifted``, the job's next slot; else ``None``.
The one copy of the backoff-shift test, shared by :func:`compute_next_run_ts`
(which moves the slot) and :func:`backoff_delaying_next_run` (which reports it).
"""
if unshifted is None:
return None
backoff_until = job.failure_backoff_until()
if backoff_until is None or unshifted >= backoff_until:
return None
return backoff_until
). The page shows the note only while that time is in the future. The earlier disabled-job should-fix is resolved.

This head is not ready to approve yet, whatever happens with the issue. CI has not started, PR Readiness is pending, and no AI lane has posted for this head. A later review has to cover those results.

— crew-pr-reviewer (r116)

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 21, 2026
A job whose auto-pause limit is 0 or above the default backs off from its 5th failure in a row, and compute_next_run_ts already shifts its next run, but the Schedule page showed the later time with no reason. The cron board now serializes consecutive_failures and failure_backoff_until, and the Next Run cell shows 'Backing off after N failures in a row, until HH:MM' while the backoff is in the future.
@Pearcekieser
Pearcekieser force-pushed the feat/schedule-backoff-note branch from 8de044b to 055e4cb Compare October 21, 2026 07:57
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

@iamwhatever todo (non-blocking): the reword is now in this PR, so nothing is left on the fork tracker. On 055e4cb615:

  1. The visible note reads "Delayed until HH:MM · N failures in a row" instead of "Backing off after ...". The time comes first and the count second, and at 390 px it wraps to four short lines instead of five (screenshots in the description).
  2. The Next run cell's tooltip adds "A successful run ends the delay.", a new backoff_ends_on_success key.
  3. All 11 translated catalogs carry the same shape; en-XA.json is regenerated; I18N_BASE_REF=origin/main npm run i18n:check passes.

The two remaining items on Pearcekieser#48 were a speaker review of the translations and a first-time-reader check. This repository has no speaker-review step for catalog strings, and the i18n check is the gate other UI PRs ship on, so I have not kept a tracker for them; #48 is closed. The description's Related Issues section now carries Closes #22641 and no fork link.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 21, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 055e4cb against the previous head 669e6d2. Besides the rebase onto main, the change since then is copy only:

  • The note now reads "Delayed until HH:MM · N failures in a row" in all 13 locale files, and the spec paragraph in learn-cron-dashboard.md matches it.
  • A new non-plural key, backoff_ends_on_success, is added to every locale and appended to the Next run cell tooltip. The claim holds: a successful run resets consecutive_failures, which ends the backoff.
  • The Russian one form is now nominative ("1 сбой"), which fixes the previous wording, and few/many agree with their counts.
  • The tests follow the new copy and also assert that the tooltip carries the success line.

No new findings. Approval is still not mine to give: this is a feat (Tier 2) and needs a human maintainer, and on this head the readiness label is still readiness: checking, PR Readiness is pending and most CI jobs are still running.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 21, 2026

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE.

Reviewed head: 055e4cb.

This clears my earlier change request. The deferred UX copy is now done in this PR: the note reads "Delayed until HH:MM · N failures in a row" in all 13 locale files, the tooltip adds "A successful run ends the delay.", and the description no longer points at a fork tracker. I accept your reason for dropping the speaker review and the first-time-reader check: this repository has no such step for catalog strings.

The code: backoff_delaying_next_run returns null for a disabled job and uses the same _backoff_shift test that compute_next_run_ts shifts by (

def backoff_delaying_next_run(job: CronJob, now: float | None = None) -> float | None:
"""The failure-backoff end that moved this job's next run later, or ``None``.
``None`` for a disabled job, which has no next run to move, and for a job
whose next scheduled slot already falls at or after the backoff end (an
``every 1d`` job five failures in), whose next run the backoff leaves
where it was. The test is :func:`_backoff_shift`, the one
:func:`compute_next_run_ts` shifts by.
"""
if not job.enabled:
return None
return _backoff_shift(job, _compute_next_run_ts_raw(job, now))
def _backoff_shift(job: CronJob, unshifted: float | None) -> float | None:
"""The backoff end when it falls after ``unshifted``, the job's next slot; else ``None``.
The one copy of the backoff-shift test, shared by :func:`compute_next_run_ts`
(which moves the slot) and :func:`backoff_delaying_next_run` (which reports it).
"""
if unshifted is None:
return None
backoff_until = job.failure_backoff_until()
if backoff_until is None or unshifted >= backoff_until:
return None
return backoff_until
), so the note and Next Run cannot disagree about whether a backoff applies. The page shows the note only while that time is in the future (
const backoffNote =
typeof j.failure_backoff_until === 'number' && j.failure_backoff_until > Date.now() / 1000
? i18nT('pages.schedulePage.backing_off_after_n_failures', {
count: j.consecutive_failures ?? 0,
time: fmtTime(j.failure_backoff_until),
})
: ''
const nextRunTitle = j.next_run_ts ? fmtDateTimeNumeric(j.next_run_ts) : ''
return (
<TableCell
className="text-muted"
title={backoffNote
? `${nextRunTitle}\n${backoffNote}\n${i18nT('pages.schedulePage.backoff_ends_on_success')}`
: nextRunTitle}
>
<span className="block truncate">{fmtIn(j.next_run_ts)}</span>
{backoffNote && (
<span className="block whitespace-normal text-[11px] leading-tight text-warn">{backoffNote}</span>
)}
</TableCell>
)
})()}
). The wire test covers in-backoff, disabled, slot-past-backoff, elapsed, below-threshold and healthy jobs on a pinned clock.

The current-head AI notes repeat points you already answered on 669e6d2. For a cron-expression job the "until" time is the backoff end and Next Run is the first slot after it. A note left on an idle page clears on the next crons refresh. All checks on this head are green, including PR Readiness. The one open review thread is outdated and was fixed in 669e6d2.

— crew-pr-reviewer (r117)

@iamwhatever
iamwhatever merged commit 65c63ea into kirodotdev:main Oct 21, 2026
101 checks passed
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Schedule page: say when a cron job is in failure backoff

3 participants