Skip to content

@Cron jobs run on every replica — bounty expiry and idempotency cleanup execute N times concurrently in a multi-instance deploy #586

Closed

Description

@chonilius

Problem

BountyExpiryScheduler.sweepOverdueBounties and IdempotencyCleanupService.removeExpired are plain @Cron(EVERY_HOUR) handlers with no leader election or lock. With 3 replicas, each sweep runs 3 times at the same moment. That makes the read-then-save race in expireOverdue (#460) routine, and once expiry triggers refunds (#26), it would attempt duplicate on-chain refunds.

Suggested fix

Guard each job with a Postgres advisory lock (SELECT pg_try_advisory_lock(<job id>)), or make scheduling opt-in per instance (RUN_SCHEDULERS=true).

Related: #26, #460

Activity

  1. added
    bugSomething isn't working
    architectureArchitecture/design issue
    Stellar WaveIssues in the Stellar wave program
    and removed
    Stellar WaveIssues in the Stellar wave program
    on Oct 13, 2026
  2. yodakkak commented on Oct 19, 2026

    @yodakkak

    @yodakkak has applied to work on this issue as part of the Stellar Wave Program's 10th wave.

    Hello! I'd like to take this on and guard the @Cron sweeps against running duplicate jobs across replicas. Ready to open a PR with tests right away — kindly assign me!

    ℹ️ Repo Maintainers: To accept this application, review their application or assign @yodakkak to this issue.

  3. devpeter999 commented on Oct 19, 2026

    @devpeter999
    Contributor

    @devpeter999 has applied to work on this issue as part of the Stellar Wave Program's 10th wave.

    Hi maintainer! I’d like to work on this issue. I’ve reviewed the requirements and would be happy to take it up. Please assign it to me.

    ℹ️ Repo Maintainers: To accept this application, review their application or assign @devpeter999 to this issue.

  4. drips-wave commented on Oct 19, 2026

    @drips-wave

    Congratulations, @devpeter999! 🎉 Your application was accepted by the repo's maintainers, and the issue is due on October 22, 2026.

    🧑‍💻 @devpeter999: Please resolve the issue such that the repo's maintainers have enough time to review your contribution before the due date. You'll earn Points for completing the issue on-time, which will make you eligible for a share of the Stellar Wave Program's reward pool.

    Warning

    When opening a PR, please link it to this issue to ensure it gets tracked accurately. Points are awarded when this issue is marked as completed by the maintainer.

    🤠 Repo maintainers: Please keep an eye on the contributor's progress and review their work before the due date. You can manage this issue, including adjusting its complexity and points, here.

    🌊 Happy Wave 🌊

  5. drips-wave commented on Oct 20, 2026

    @drips-wave

    This issue has been completed by @devpeter999 as part of the Stellar Wave Program's 10th Wave 🥳

    😎 @devpeter999: You earned 200 Points for completing this issue! Once the Wave's rewards are finalized, you'll be eligible for a percentage of the Wave's reward pool based on the percentage of total points you've earned. Learn more here. You can also Leave a review to share your experience working on this issue.

    🧑‍💻 Repo maintainers: How'd the contributor do? Leave a review to share your experience working with them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Stellar WaveIssues in the Stellar wave programarchitectureArchitecture/design issuebugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions