Skip to content

Add pid and worker_index to the worker's log records - #71

Merged
tokibito merged 2 commits into
mainfrom
join-worker-logs
Oct 9, 2026
Merged

tokibito merged 2 commits into
mainfrom
join-worker-logs

Conversation

@tokibito

@tokibito tokibito commented Oct 9, 2026

Copy link
Copy Markdown
Owner

First half of #65. Under --workers the supervisor's records carry a worker's worker_index and pid, while the worker's own records carry the worker id it picked for itself, so one host's logs could not be joined. This adds pid and worker_index to the worker's records, under the names the supervisor already uses, so the two sides join on pid.

What it does

  • The supervisor sets DJANGO_DATABASE_TASK_WORKER_INDEX in each child's environment to the slot's index. A restarted worker keeps its slot's index and gets a new pid.
  • The worker reads the variable once at startup and adds worker_index and its own pid to Worker started, Worker finished, the failed broker receive, every task record (through task_log_fields()) and the executor-thread failure record. With -v 1 and up, the banner shows Worker index: N after Worker ID: when there is one.
  • The value must be a whole number of 1 or more in ASCII digits. int() alone would take -1, +3, 1_0 or non-ASCII digits, none of which the supervisor ever sets.
  • Because the variable is read once, task code that changes os.environ afterwards does not change what the records say. The variable is left in the environment, so a task, and anything it starts, can read which worker it runs in.
  • Without the variable, a worker records its pid and no worker_index.

Things to look at

  • The variable is documented for process managers too. A worker started without the supervisor uses the variable when it is set, so a systemd template unit can pass %i and "worker 2" names the same unit across restarts. The README has a Worker index subsection under Structured logging, with a template unit example.
  • An invalid value is reported and ignored, not refused. The worker logs an Ignoring the worker index WARNING, writes the same line on stdout, and runs without an index. The index only labels the records, and a worker that will not start over a label seemed worse than one that runs without it.
  • One existing test changed. test_fields_describe_the_task compares the whole task_log_fields() dict, so it now includes pid. That is the behaviour change this PR makes: task records gain pid everywhere, including tasks run from the admin actions.

Compatibility

Added fields, one added WARNING record for an invalid value, and one added banner line when the variable is set. No logger name, message or existing field changes, so filters written against the current records keep working. Existing projects need no changes. The CHANGELOG entry is under Unreleased.

Tests

  • tests/test_logging.py: the fields on the worker and task records, no index without the variable, an index the environment changes after startup is not picked up, a task changing the variable does not change later records, a task can read the variable, an invalid value is reported and ignored, and the banner line.
  • tests/test_supervisor.py: read_worker_index() accepting and refusing values, each child seeing its own index matched to the pid in Worker process started, a restarted worker keeping its slot's index, and the supervisor's own environment left alone.

Run on Python 3.13 / Django 6.0 (SQLite), Python 3.14 / Django 6.1 with -W error::DeprecationWarning, and the whole suite against PostgreSQL 16.

Refs #65

Under --workers the supervisor's records carry a worker's index and pid,
and the worker's own records carry the worker id it picked for itself, so
one host's logs could not be joined. The supervisor now passes each child
its slot's index in DJANGO_DATABASE_TASK_WORKER_INDEX, and the worker adds
the index and its own pid to Worker started, Worker finished, the failed
broker receive and every task record, under the names the supervisor
already uses. A worker started without the supervisor records its pid and
no index.

Added fields only; no logger name, message or existing field changes.

Refs #65
The index was read from the environment on every record and parsed with
int(), so task code could change what later records said, and values the
supervisor never sets (-1, +3, 1_0, non-ASCII digits) were recorded. The
worker now reads it once at startup, reports and ignores anything that is
not a whole number of 1 or more, and leaves the variable in the
environment for what the tasks start. The README documents the variable
for process managers that start the workers themselves.

Refs #65
@tokibito
tokibito merged commit 7135e3c into main Oct 9, 2026
10 checks passed
@tokibito
tokibito deleted the join-worker-logs branch October 9, 2026 05:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant