[sankalpsthakur]

Description

• flexmeasures jobs run-worker looked up the exception handler with the whole --queue option. A worker started on several queues, as documentation/host/installation.rst sets it up (--queue "scheduling|forecasting|ingestion|reporting"), therefore got the generic fallback handler for every job: a failed scheduling job lost its traceback in the worker log, and a failed forecasting job was not counted in its meta.
• The worker now gets one handler, handle_exception_by_queue, which passes each failed job on to the handler of the queue the job came from (job.origin), as run-job already does since Run a job once, on its own queue, in flexmeasures jobs run-job #2480. It returns the inner handler's result, because RQ stops calling further handlers when one returns False.
• Added changelog item in documentation/changelog.rst

Look & Feel

No visible change, apart from what the worker logs for a failed job.

How to test

pytest flexmeasures/cli/tests/test_jobs.py

• test_run_worker_handles_a_job_with_the_handler_of_its_own_queue runs run-worker --queue "scheduling|forecasting" with an in-process burst worker on a failing forecasting job, and checks that the forecasting handler counted the failure in the job's meta. On main the generic handler runs instead, which stores only the exception summary, so the assertion fails. The forecasting queue comes second in the option, so giving every job the first queue's handler would fail it too.
• test_handle_exception_by_queue_falls_back_and_passes_on_the_result checks that a job goes to its own queue's handler, that a queue without one falls back to the generic handler, and that a False from the handler reaches RQ.

I could not run the tests myself: the suite needs a Postgres database, which I don't run locally, and Actions are disabled on my fork, so the CI of this PR is their first run. black 26.3.1 and mypy 2.3.1 (with the flags of the type-check task) pass on the two changed files, and flake8 7.1.1 with the repo's .flake8 reports nothing on the changed lines.

Further Improvements

None planned.

Related Items

Fixes #2526. The run-job part of the issue was already fixed by #2480, so this PR only changes run-worker.

---

AI/LLM disclosure

• AI coding tools (Claude Code) helped write this change, its tests and this description.

---

Sign-off

• I agree to contribute to the project under Apache 2 License.
• To the best of my knowledge, the proposed patch is not based on code under GPL or other license that is incompatible with FlexMeasures

[read-the-docs-community]

Documentation build overview

📚 flexmeasures | 🛠️ Build #35758705 | 📁 Comparing 1e1e5a0 against latest (055d54d)

  🔍 Preview build  

4 files changed

± changelog.html
± genindex.html
± _autosummary/flexmeasures.cli.jobs.html
± api/v3_0.html

[BelhsanHmida]

Looks good, thanks for the fix! Just the changelog wording.

[BelhsanHmida]

RQ already logs the full traceback before any handler runs, so the traceback isn't what this fixes. The gain is on the forecasting side:

Suggested change

* Handle each failed job of a worker on several queues, such as one started with ``flexmeasures jobs run-worker --queue "scheduling|forecasting"``, with the exception handler of the job's own queue, so a failed scheduling job shows its traceback in the worker log [see `PR #2850 <https://www.github.com/FlexMeasures/flexmeasures/pull/2850>`_]

* Handle each failed job of a worker on several queues, such as one started with ``flexmeasures jobs run-worker --queue "scheduling|forecasting"``, with the exception handler of the job's own queue, so a failed forecasting job counts its failures in its meta and, if an automation created it, is recorded as failed in the automation run [see `PR #2850 <https://www.github.com/FlexMeasures/flexmeasures/pull/2850>`_]