Add --max-tasks-jitter to spread out worker restarts

CHANGELOG.md

@@ -4,6 +4,18 @@
 
 ### Added
 
+- **`--max-tasks-jitter J` on `run_database_tasks`.** Each worker adds a
+  random number from 0 to J, drawn once when it starts, to `--max-tasks`,
+  the way gunicorn's `max_requests_jitter` does, so the workers of
+  `--workers N --continuous --max-tasks K` no longer all reach K and restart
+  at the same moment. The draw is made by the worker, so every worker and
+  every restart gets its own, and from the operating system's random source,
+  so a project that seeds `random` does not make them alike; the limit drawn is printed at startup and
+  carried as a new `max_tasks` field on `Worker started`. Without
+  `--max-tasks` the option is ignored. The default is 0, which leaves
+  `--max-tasks` exactly as it was, and the only change to existing output is
+  the added log field, so existing projects need no changes.
+  ([#64](https://github.com/tokibito/django-database-task/issues/64))
 - **A summary of the `--workers` supervisor's state in its log records.**
   When every worker is exiting with an error within ten seconds of starting
   at once, the supervisor logs one `Every worker is failing to start`

README.md

@@ -256,6 +256,7 @@ python manage.py run_database_tasks [options]
 | `--continuous` | Keep polling even when no tasks |
 | `--interval` | Polling interval in seconds (default: 5) |
 | `--max-tasks` | Maximum number of tasks to process (0=unlimited) |
+| `--max-tasks-jitter` | Add a random number from 0 to this to `--max-tasks`, drawn once per worker at start (default: 0). Ignored without `--max-tasks`. See [Running several workers](#running-several-workers) |
 | `--source` | Where to look for tasks: `auto` (default), `db`, `broker` or `both`. See [Task sources](#task-sources) |
 | `--wait-time` | Seconds to wait for a broker message before looking again (default: 20) |
 | `--max-messages` | Maximum number of broker messages to receive at a time (default: 1) |
@@ -766,7 +767,24 @@ Shutdown complete: every worker exited.
   time.
 - **`--max-tasks`** goes to each worker. With `--continuous` that is
   gunicorn's `max_requests`: a worker exits after that many tasks and is
-  replaced, which bounds a leak in task code.
+  replaced, which bounds a leak in task code. The N workers start together
+  and process at similar rates, so on their own they reach the limit at
+  about the same time and restart together, leaving the host without a
+  worker for that moment. Add `--max-tasks-jitter J`, gunicorn's
+  `max_requests_jitter`, to spread them out: each worker draws a number
+  from 0 to J when it starts, a restarted one included, and stops after
+  `--max-tasks` plus that many tasks. A jitter of a tenth of `--max-tasks`
+  or more is a reasonable start:
+
+  ```bash
+  python manage.py run_database_tasks --continuous --workers 4 \
+      --max-tasks 1000 --max-tasks-jitter 100
+  ```
+
+  The limit a worker drew is printed at startup
+  (`Max tasks: 1037 (1000 + jitter 37 of up to 100)`) and carried as
+  `max_tasks` on its `Worker started` log record. A worker run by a process
+  manager rather than `--workers` takes the option the same way.
 - **Signals.** `SIGTERM` and `SIGINT` (Ctrl-Break too on Windows) are handled
   by the supervisor, which forwards them to every worker and waits for the
   workers to finish their running task, within `--shutdown-timeout` if one is

django_database_task/locale/es/LC_MESSAGES/django.po

@@ -6,7 +6,7 @@ msgid ""
 msgstr ""
 "Project-Id-Version: django-database-task 0.1.0\n"
 "Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2026-10-02 05:07+0000\n"
+"POT-Creation-Date: 2026-10-12 16:27+0000\n"
 "PO-Revision-Date: 2026-09-30 00:00+0000\n"
 "Last-Translator: Shinya Okano <tokibito@gmail.com>\n"
 "Language-Team: Spanish\n"
@@ -135,31 +135,42 @@ msgstr "Número de tareas a procesar de una vez (predeterminado: 1000)"
 msgid "Show what would happen without changing anything"
 msgstr "Muestra lo que ocurriría sin cambiar nada"
 
-#: management/commands/run_database_tasks.py:63
+#: management/commands/run_database_tasks.py:70
 msgid "Execute tasks queued in the database"
 msgstr "Ejecuta las tareas encoladas en la base de datos"
 
-#: management/commands/run_database_tasks.py:70
+#: management/commands/run_database_tasks.py:77
 msgid "Queue name to process (all queues if not specified)"
 msgstr "Nombre de la cola a procesar (todas las colas si no se indica)"
 
-#: management/commands/run_database_tasks.py:76
+#: management/commands/run_database_tasks.py:83
 msgid "Backend name (default: default)"
 msgstr "Nombre del backend (predeterminado: default)"
 
-#: management/commands/run_database_tasks.py:81
+#: management/commands/run_database_tasks.py:88
 msgid "Continuous mode (keep polling even when no tasks)"
 msgstr "Modo continuo (sigue sondeando aunque no haya tareas)"
 
-#: management/commands/run_database_tasks.py:87
+#: management/commands/run_database_tasks.py:94
 msgid "Polling interval in seconds for continuous mode (default: 5)"
 msgstr "Intervalo de sondeo en segundos en modo continuo (predeterminado: 5)"
 
-#: management/commands/run_database_tasks.py:93
+#: management/commands/run_database_tasks.py:100
 msgid "Maximum number of tasks to process (0=unlimited, default: 0)"
 msgstr "Número máximo de tareas a procesar (0=ilimitado, predeterminado: 0)"
 
-#: management/commands/run_database_tasks.py:100
+#: management/commands/run_database_tasks.py:108
+msgid ""
+"Add a random number from 0 to this to --max-tasks, drawn once when the "
+"worker starts, so workers started together do not all reach the limit at "
+"once. Ignored without --max-tasks (default: 0)"
+msgstr ""
+"Suma a --max-tasks un número aleatorio de 0 hasta este valor, sorteado una "
+"vez cuando el worker se inicia, para que los workers iniciados a la vez no "
+"alcancen el límite al mismo tiempo. Se ignora sin --max-tasks "
+"(predeterminado: 0)"
+
+#: management/commands/run_database_tasks.py:119
 msgid ""
 "Where to look for tasks: 'db' polls the database, 'broker' receives from the "
 "backend's broker, 'both' does the broker first and falls back to the "
@@ -171,7 +182,7 @@ msgstr ""
 "datos, y 'auto' (predeterminado) significa 'both' cuando el backend tiene un "
 "broker del que recibir y 'db' en caso contrario"
 
-#: management/commands/run_database_tasks.py:112
+#: management/commands/run_database_tasks.py:131
 msgid ""
 "Seconds to wait for a broker message before looking again. Replaces --"
 "interval as the idle wait when receiving from a broker (default: 20)"
@@ -180,12 +191,12 @@ msgstr ""
 "Sustituye a --interval como espera inactiva al recibir de un broker "
 "(predeterminado: 20)"
 
-#: management/commands/run_database_tasks.py:122
+#: management/commands/run_database_tasks.py:141
 msgid "Maximum number of broker messages to receive at a time (default: 1)"
 msgstr ""
 "Número máximo de mensajes del broker a recibir de una vez (predeterminado: 1)"
 
-#: management/commands/run_database_tasks.py:131
+#: management/commands/run_database_tasks.py:150
 msgid ""
 "Run this many worker processes from one command, each a copy of this command "
 "without --workers. With --continuous a worker that exits is replaced; "
@@ -196,18 +207,18 @@ msgstr ""
 "se reemplaza; sin él, el comando termina cuando todos los workers lo hayan "
 "hecho (predeterminado: 1)"
 
-#: management/commands/run_database_tasks.py:143
+#: management/commands/run_database_tasks.py:162
 msgid ""
 "Run tasks in this many threads inside the worker. The task code must be "
 "thread-safe. The worker still polls once per interval; each thread has a "
 "database connection of its own (default: 1)"
 msgstr ""
-"Ejecuta las tareas en esta cantidad de hilos dentro del worker. El código "
-"de las tareas debe ser seguro para hilos. El worker sigue consultando una "
-"vez por intervalo; cada hilo tiene su propia conexión a la base de datos "
+"Ejecuta las tareas en esta cantidad de hilos dentro del worker. El código de "
+"las tareas debe ser seguro para hilos. El worker sigue consultando una vez "
+"por intervalo; cada hilo tiene su propia conexión a la base de datos "
 "(predeterminado: 1)"
 
-#: management/commands/run_database_tasks.py:154
+#: management/commands/run_database_tasks.py:173
 msgid ""
 "Maximum seconds to wait for the running task after receiving SIGTERM/SIGINT "
 "before forcing exit (0=wait indefinitely, default: 0)"
@@ -216,15 +227,15 @@ msgstr ""
 "SIGINT antes de forzar la salida (0=esperar indefinidamente, predeterminado: "
 "0)"
 
-#: management/commands/run_database_tasks.py:163
+#: management/commands/run_database_tasks.py:182
 msgid ""
 "Do not install SIGTERM/SIGINT handlers; the process is terminated "
 "immediately, even while a task is running"
 msgstr ""
 "No instala manejadores de SIGTERM/SIGINT; el proceso termina de inmediato, "
 "incluso con una tarea en ejecución"
 
-#: management/commands/run_database_tasks.py:173
+#: management/commands/run_database_tasks.py:192
 msgid ""
 "Exit with this code when no task was processed, so a job scheduler can tell "
 "an idle run from a real one (0=exit normally, default: 0)"
@@ -233,7 +244,7 @@ msgstr ""
 "planificador de trabajos distinga una ejecución vacía de una real "
 "(0=terminar normalmente, predeterminado: 0)"
 
-#: management/commands/run_database_tasks.py:184
+#: management/commands/run_database_tasks.py:203
 msgid ""
 "Exit with this code when at least one task failed or could not be run. Takes "
 "precedence over --empty-exit-code (0=exit normally, default: 0)"

django_database_task/locale/ja/LC_MESSAGES/django.po

@@ -7,7 +7,7 @@ msgid ""
 msgstr ""
 "Project-Id-Version: django-database-task 0.1.0\n"
 "Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2026-10-02 05:07+0000\n"
+"POT-Creation-Date: 2026-10-12 16:27+0000\n"
 "PO-Revision-Date: 2025-12-27 00:00+0000\n"
 "Last-Translator: Shinya Okano <tokibito@gmail.com>\n"
 "Language-Team: Japanese\n"
@@ -129,31 +129,41 @@ msgstr "一度に処理するタスク数 (デフォルト: 1000)"
 msgid "Show what would happen without changing anything"
 msgstr "何も変更せずに、何が行われるかを表示"
 
-#: management/commands/run_database_tasks.py:63
+#: management/commands/run_database_tasks.py:70
 msgid "Execute tasks queued in the database"
 msgstr "データベースのキューに登録されたタスクを実行"
 
-#: management/commands/run_database_tasks.py:70
+#: management/commands/run_database_tasks.py:77
 msgid "Queue name to process (all queues if not specified)"
 msgstr "処理するキュー名 (指定しない場合はすべてのキュー)"
 
-#: management/commands/run_database_tasks.py:76
+#: management/commands/run_database_tasks.py:83
 msgid "Backend name (default: default)"
 msgstr "バックエンド名 (デフォルト: default)"
 
-#: management/commands/run_database_tasks.py:81
+#: management/commands/run_database_tasks.py:88
 msgid "Continuous mode (keep polling even when no tasks)"
 msgstr "継続モード (タスクがなくてもポーリングを続ける)"
 
-#: management/commands/run_database_tasks.py:87
+#: management/commands/run_database_tasks.py:94
 msgid "Polling interval in seconds for continuous mode (default: 5)"
 msgstr "継続モードのポーリング間隔（秒）(デフォルト: 5)"
 
-#: management/commands/run_database_tasks.py:93
+#: management/commands/run_database_tasks.py:100
 msgid "Maximum number of tasks to process (0=unlimited, default: 0)"
 msgstr "処理する最大タスク数 (0=無制限、デフォルト: 0)"
 
-#: management/commands/run_database_tasks.py:100
+#: management/commands/run_database_tasks.py:108
+msgid ""
+"Add a random number from 0 to this to --max-tasks, drawn once when the "
+"worker starts, so workers started together do not all reach the limit at "
+"once. Ignored without --max-tasks (default: 0)"
+msgstr ""
+"--max-tasks に 0 からこの値までの乱数を加える。乱数はワーカーの起動時に 1 回"
+"だけ引くので、同時に起動したワーカーが一斉に上限に達することがなくなる。--"
+"max-tasks がないときは無視される (デフォルト: 0)"
+
+#: management/commands/run_database_tasks.py:119
 msgid ""
 "Where to look for tasks: 'db' polls the database, 'broker' receives from the "
 "backend's broker, 'both' does the broker first and falls back to the "
@@ -165,19 +175,19 @@ msgstr ""
 "バックする。'auto' (デフォルト) は、受信できるブローカーがバックエンドにあれ"
 "ば 'both'、なければ 'db' になる"
 
-#: management/commands/run_database_tasks.py:112
+#: management/commands/run_database_tasks.py:131
 msgid ""
 "Seconds to wait for a broker message before looking again. Replaces --"
 "interval as the idle wait when receiving from a broker (default: 20)"
 msgstr ""
 "再確認するまでブローカーのメッセージを待つ秒数。ブローカーから受信する場合、"
 "アイドル時の待機として --interval の代わりに使われる (デフォルト: 20)"
 
-#: management/commands/run_database_tasks.py:122
+#: management/commands/run_database_tasks.py:141
 msgid "Maximum number of broker messages to receive at a time (default: 1)"
 msgstr "一度に受信するブローカーメッセージの最大数 (デフォルト: 1)"
 
-#: management/commands/run_database_tasks.py:131
+#: management/commands/run_database_tasks.py:150
 msgid ""
 "Run this many worker processes from one command, each a copy of this command "
 "without --workers. With --continuous a worker that exits is replaced; "
@@ -187,42 +197,41 @@ msgstr ""
 "このコマンドのコピー。--continuous 指定時は終了したワーカーを再起動し、未指定"
 "時は全ワーカーが終了するとコマンドも終了する (デフォルト: 1)"
 
-#: management/commands/run_database_tasks.py:143
+#: management/commands/run_database_tasks.py:162
 msgid ""
 "Run tasks in this many threads inside the worker. The task code must be "
 "thread-safe. The worker still polls once per interval; each thread has a "
 "database connection of its own (default: 1)"
 msgstr ""
-"ワーカー内でこの数のスレッドでタスクを実行する。タスクのコードはスレッドセーフである必要がある。ポーリングは引き続き "
-"interval ごとに 1 "
-"回で、スレッドごとにデータベース接続を 1 本持つ "
-"(デフォルト: 1)"
+"ワーカー内でこの数のスレッドでタスクを実行する。タスクのコードはスレッドセー"
+"フである必要がある。ポーリングは引き続き interval ごとに 1 回で、スレッドごと"
+"にデータベース接続を 1 本持つ (デフォルト: 1)"
 
-#: management/commands/run_database_tasks.py:154
+#: management/commands/run_database_tasks.py:173
 msgid ""
 "Maximum seconds to wait for the running task after receiving SIGTERM/SIGINT "
 "before forcing exit (0=wait indefinitely, default: 0)"
 msgstr ""
 "SIGTERM/SIGINTを受信してから強制終了するまで、実行中のタスクを待つ最大秒数 "
 "(0=無期限に待つ、デフォルト: 0)"
 
-#: management/commands/run_database_tasks.py:163
+#: management/commands/run_database_tasks.py:182
 msgid ""
 "Do not install SIGTERM/SIGINT handlers; the process is terminated "
 "immediately, even while a task is running"
 msgstr ""
 "SIGTERM/SIGINTハンドラーを設定しない。タスクの実行中でもプロセスは即座に終了"
 "する"
 
-#: management/commands/run_database_tasks.py:173
+#: management/commands/run_database_tasks.py:192
 msgid ""
 "Exit with this code when no task was processed, so a job scheduler can tell "
 "an idle run from a real one (0=exit normally, default: 0)"
 msgstr ""
 "タスクを1件も処理しなかった場合にこの終了コードで終了する。ジョブスケジュー"
 "ラーが空振りの実行を区別できるようにする (0=通常どおり終了、デフォルト: 0)"
 
-#: management/commands/run_database_tasks.py:184
+#: management/commands/run_database_tasks.py:203
 msgid ""
 "Exit with this code when at least one task failed or could not be run. Takes "
 "precedence over --empty-exit-code (0=exit normally, default: 0)"