fix(relayer): default wallet-job concurrency to the wallet count - #919
Open
harrymove-ctrl wants to merge 1 commit into
Open
harrymove-ctrl wants to merge 1 commit into
harrymove-ctrl wants to merge 1 commit into
Conversation
WALLET_JOB_CONCURRENCY was read in two places, each falling back to a hard-coded 8, while the TS sidecar sized its Walrus upload semaphore from SERVER_SUI_PRIVATE_KEYS.length. Two processes, two unrelated defaults for one number, and nothing logged the resolved value — production ended up on 7 workers against 8 upload slots without anyone choosing that pair. Resolve it once, from the env when set and from the wallet count otherwise, so an unset variable leaves the worker and the sidecar on the same number. The env stays the source of truth where it is set. Log the resolved value at boot, and warn when it exceeds the wallet count, since the sidecar allows one upload per wallet and the surplus only queues on its semaphore.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
WALLET_JOB_CONCURRENCYwas read in two separate places inmain.rs, eachfalling back to its own hard-coded
8:Meanwhile the TS sidecar sizes its Walrus upload semaphore from a completely
different source — the wallet count:
Two processes, two unrelated defaults for one number, and nothing logged the
resolved value. Production drifted to
WALLET_JOB_CONCURRENCY=7against 8upload slots (8 wallets,
WALRUS_UPLOAD_MAX_CONCURRENCYunset) — nobodypicked that pair, and answering "which limiter actually binds?" currently
means reading two languages plus the Railway env.
Change
Resolve the number once, in
types::wallet_job_concurrency_from_env:WALLET_JOB_CONCURRENCYwins when set — production sets it on Railway andthat stays the source of truth
uses, so the two processes cannot drift to different hard-coded defaults
100, mirroring the sidecar's clamp0wallets still yields one worker, so the queue drains into a clearper-job error instead of stalling
main.rsnow derives both the worker count and the advisory-lock pool fromthat single value, and logs it at boot — it was previously invisible. When
concurrency exceeds the wallet count it warns, because the sidecar allows one
upload per wallet (
WALRUS_UPLOAD_PER_WALLET_CONCURRENCY, default1), sothe surplus jobs only queue on its semaphore rather than adding throughput.
Behaviour
WALLET_JOB_CONCURRENCY7(production today)500No change for production as configured. The fallback only moves when the
wallet count is not 8, which is exactly the drift this removes.
Tests
5 new unit tests on the pure resolver in
types.rs, all passing:cargo check --bin memwal-serverclean.cargo fmt --checkreports no diff inthe touched files (the two pre-existing diffs in
storage/sui.rsare ondevand left alone).
Not addressed here
The redundant
SERVER_SUI_PRIVATE_KEY(singular) alongside the plural form.It is dead config for the sidecar —
config.ts:85only reads it when theplural list is empty — but
types.rs:677,688still uses it as a Rust-sidefallback, so removing it needs its own check.