Skip to content

Move recurring event fetching to rq.cron.CronScheduler #2

Closed

Description

@Flix6x

Follow-up from review of #1.

Problem

VenFetchJobScheduler in flexmeasures_openadr3/utils/ven_jobs.py keeps its daily ingestion running by re-enqueueing itself at the end of _execute() (in a finally block). This makes the schedule a chain: each run is only scheduled by the previous one.

The finally guards against most in-job failures, but if a worker is restarted (or killed) while handling the job, before it reaches the re-scheduling call, the chain breaks silently and events are never fetched again.

Proposal

Move to rq.cron.CronScheduler, which owns the schedule independently of any single job run. This requires a dedicated rq-cron-scheduler service in docker-compose.yml running something like:

from rq.cron import CronScheduler

cron = CronScheduler(connection=app.redis_connection)
cron.start()

and registering the recurring job with something like:

cron.register(
    self._execute,
    queue_name=app.queues[FETCH_EVENTS_QUEUE_NAME],
    cron_string="0 0 * * *",
    kwargs={"ven_id": ven_client.id, "config_name": sensor_config.name},
)

Note that utc_trigger_time is currently per sensor config, so the cron string would need to be derived per registration rather than hard-coded.

_Originally posted by @Flix6x in https://github.com/FlexMeasures/flexmeasures-openadr3/pull/1#discussion_r3309216061_

            

Activity

  1. marked Move to rq.cron.CronScheduler #3 as a duplicate of this issue on Aug 7, 2026
  2. changed the title [-]Move to `rq.cron.CronScheduler`[/-] [+]Move recurring event fetching to `rq.cron.CronScheduler`[/+] on Aug 7, 2026
  3. nicburgt commented on Sep 3, 2026

    @nicburgt
    Collaborator

    Implemented in #1 now as well

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions