feat: single source of truth for public config in config/<env>.env - #65
Merged
Merged
Conversation
Load per-environment config from config/<name>.env with machine-local override files, and wire npm scripts to --env.
Document the secret keys expected in override env files for local and CI use.
NETWORK meant three different things: the keeper required base-mainnet, the subgraph manifests and market-maker YAML used base, and the Hardhat config declared a network literally named base-mainnet. The GitHub environment that deploys mainnet already sets NETWORK=base, so the keeper and Hardhat were the outliers. Settle on base / base-sepolia / hardhat everywhere. The keeper still accepts base-mainnet as a deprecated alias, since task definitions registered before this change carry it, and keeps Alchemy's own base-mainnet spelling for the RPC subdomain.
Contract addresses, start blocks and runtime knobs were duplicated across GitHub environment variables, the deploy workflows and ad-hoc loaders, so the same value could disagree between local runs, the subgraphs and ECS. Make config/dev.env and config/prd.env the single source for public values, and have every consumer read them: the keeper and market-maker via node --env-file, the subgraph renderers via ENV_FILE, and the deploy workflows by sourcing the file. Secrets stay in the repo-root .env locally and in GitHub Secrets in CI; both still win, because the real environment takes precedence over --env-file. The two ECS workflows built their environment block by naming each variable four times: once in the step env, once as a jq --arg, once in the jq object, and once more as a :- default. They now derive the block from the keys declared in the config file and name only secrets and computed values, so a public setting is written exactly once. Empty values are dropped so apps apply their own defaults, which were checked to match the defaults the workflows used to hardcode. Also drop the market-maker's custom dotenv loader in favour of --env-file, disable the stg deploy path, and remove a dead compose script in points-indexer. config/prd.env is still missing the mainnet addresses and start blocks; they exist nowhere in GitHub either, and are marked with TODOs.
pnpm runs scripts with sh, which is dash on the CI runners. POSIX `.` searches PATH when its operand contains no slash, so `ENV_FILE=.env.example` failed with ".env.example: not found" while passing locally, where /bin/sh is bash and falls back to the current directory. Prefix a bare filename with ./ before sourcing, leaving paths that already contain a slash untouched.
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Public, per-environment configuration now lives in
config/dev.envandconfig/prd.env, and every consumer reads those same files.env-loaderplugin resolvesconfig/<name>.envvia--env <name>, then applies machine/secret override files fromenvLoader.overrideEnvFilesnode --env-file, replacing the market-maker's custom dotenv loaderENV_FILE=… pnpm prepare:env, replacingset -a && source ../.envSecrets stay in the repo-root
.envlocally and in GitHub Secrets in CI. Both still win over the config file, because the real environment takes precedence over--env-file.Workflows no longer name each variable four times
The two ECS workflows previously declared every value in the step
env:, again as ajq --arg, again in thejqobject, and once more as a:-default. They now derive the environment block from the keys in the config file and name only secrets and computed values, so a public setting is written exactly once. Empty values are dropped so apps apply their own defaults; those defaults were checked against the ones the workflows used to hardcode (DRY_RUNfalse,DISCOVERY_MODEevents,SWEEP_INTERVAL_MS60000,HEALTH_PORT3000,MAKER_HEALTH_PORT3001) and they match.NETWORKcanonicalized tobaseNETWORKmeant three different things: the keeper requiredbase-mainnet, the subgraph manifests and market-maker YAML usedbase, and Hardhat declared a network literally namedbase-mainnet. The GitHub environment that deploys mainnet already setsNETWORK=base, so the keeper and Hardhat were the outliers. The keeper still acceptsbase-mainnetas a deprecated alias, since task definitions registered before this change carry it.Also
stgremoved from the deploy workflows,config/stg.envandcontracts'run:stgdeleteddocker composescript removed frompoints-indexer(no compose file existed)config/*.envvalues are unquoted, sincedocker run --env-filedoes not strip quotes in every CLI versionTest plan
keeperandmarket-maker: typecheck, lint, and 506 tests passconfig/dev.env, dropping empty values and including secretsloadConfig()(keeper) andloadPortfolioConfig()(market-maker) both load end-to-end fromconfig/dev.env+ dummy secretsNETWORKresolves forbase,base-sepolia, thebase-mainnetalias, and errors clearly on an unknown valueconfig/dev.envand from.env.example(the CI path)../.env/.envvalues win over the named env file for overlapping keysFollow-ups
config/prd.envstill lacksPERPS_ADDRESS,FUTURES_ADDRESS,HASHPRICE_USD_ADDRESS,BTC_USD_FEED_ADDRESSand the start blocks. These exist nowhere in GitHub either — themainenvironment has no variables at all — so they are marked with TODOs. The keeper and maker fail loudly at startup without them.VAULT_START_BLOCKhas never been set, so the vault subgraph has been rendering an emptystartBlock. Pre-existing, not introduced here.NETWORK,PERPS_ADDRESS, …) are now unread and can be deleted. OnlyKEEPER_DESIRED_COUNT,MAKER_DESIRED_COUNTand the Goldsky/AWS ones still matter.