Skip to content

feat(secrets): seed secrets from TEMPER_SECRET_ environment variables when temper serve starts - #521

Merged
arun-pathiban-ddog merged 6 commits into
mainfrom
secrets-from-environment
Oct 4, 2026
Merged

arun-pathiban-ddog merged 6 commits into
mainfrom
secrets-from-environment

Conversation

@arun-pathiban-ddog

@arun-pathiban-ddog arun-pathiban-ddog commented Oct 4, 2026 •

Copy link
Copy Markdown
Collaborator

What and why

temper serve fills its secrets cache in two ways: from secrets stored through the secrets API once it is running, and at start from two fixed environment variables (ANTHROPIC_API_KEY becomes the secret anthropic_api_key, EXA_API_KEY becomes exa_api_key). An operator who wants a module to have any other secret from the moment the server is up cannot supply it with the server's configuration. They have to wait for the server to answer, authenticate, and call the secrets API, and do it again after every restart of a server that keeps its cache in memory only.

This adds the general form of what the two fixed variables already do. Every variable named TEMPER_SECRET_<NAME> with a non-empty value is seeded at start as the secret <name> in lower case: TEMPER_SECRET_BUILD_TOKEN=abc supplies build_token. With no such variable set, start-up does and logs nothing new.

It adds a source of secrets and changes nothing in how secrets are stored, encrypted, authorized or read. A module's get_secret call still needs a policy in its tenant that permits access_secret on Secret::"<name>". A {secret:<name>} template in an integration config is resolved without that check, as it is for every secret today; see the first point under "Behaviour worth a reviewer's attention".

The rule

Name <NAME> is one or more of A to Z, 0 to 9 and _, starting with a letter. Lower-casing it is then one-to-one, so no two variables can supply the same secret.
Badly named variable Skipped and logged once at startup as a warning that gives the variable's name (TEMPER_SECRET_ alone, a lower-case letter, a dash). The server still starts.
Empty value The same as an unset variable. Nothing is seeded or logged.
Value over 8192 bytes Skipped and logged by variable name. It is the limit of the secrets API.
Reach The platform layer of the vault, like the two fixed variables: every tenant reads it, including tenants created later.
Storage In memory only. Nothing is written to storage.
Precedence, highest first 1. A secret the tenant stored through the secrets API, for that tenant only, stored before or after start. 2. What the server sets itself at start (anthropic_api_key from ANTHROPIC_API_KEY, exa_api_key from EXA_API_KEY, and the addresses it gives its own modules). 3. The prefixed variable.
What is logged One line with the number of secrets seeded, and one warning per skipped variable with its name. Never a value. Start-up prints nothing about the two fixed variables today, so the names of seeded secrets are not listed either.

Behaviour worth a reviewer's attention

  • A seeded secret can be read by the same two routes as any secret, and only one of them is policy-checked. get_secret from a module goes through access_secret. A {secret:<name>} template in an integration config is resolved from the vault when the integration runs, with no such check (resolve_secret_templates); that is how {secret:anthropic_api_key} resolves today. This change does not touch that. What it does change is how much can be in the platform layer: whatever the operator supplies reaches every tenant, so whoever may deploy a spec in any tenant can use it in a template. The guide says so and tells the operator to supply this way only what every tenant's specs may use. Gating templates by access_secret would change existing deployments and is left for a change of its own.
  • The prefixed variables are seeded last and never overwrite. They run after the two fixed variables and the server's own platform secrets, and a name that is already present is skipped and logged. So TEMPER_SECRET_ANTHROPIC_API_KEY supplies anthropic_api_key only when ANTHROPIC_API_KEY is not set, and an existing deployment sees no change. Seeding last also means the prefixed variables cannot use up the platform budget (100 secrets) before the server's own secrets are in.
  • ANTHROPIC_API_KEY set to an empty string is cached as an empty secret today. That is unchanged, and it still wins over the prefixed form.
  • A seeded secret behaves like any platform secret in the other places too. GET /api/tenants/{tenant}/secrets lists its name, and the existing fallback that starts the Discord transport from a discord_bot_token secret will find one supplied this way when neither the flag nor DISCORD_BOT_TOKEN is given (read from the code, not exercised). temper_api_key stays unavailable to modules: the resolver refuses that name before it looks anything up.
  • A tenant cannot delete a seeded secret, as it cannot delete the two fixed ones. It can store its own secret of the same name, which wins for that tenant; deleting that one uncovers the seeded value again.
  • The report goes through tracing, after the subscriber is installed, like the rest of the server's logs. It follows the log filter, and the readability ratchet's println count does not grow.
  • The environment is read with std::env::vars_os(). std::env::vars() panics on a variable that is not Unicode anywhere in the environment. A prefixed variable whose name or value is not UTF-8 is skipped and logged by name.
  • Where the code is. The rule is temper_server::secrets::seed_platform_secrets_from_environment, which takes the variables as an argument and reads no environment itself, so it stays deterministic. temper serve passes it the process environment in one call.

Not done

  • Templates are not gated by access_secret. Existing behaviour for every secret, described above, and not changed here.
  • The two fixed variables still have no size check. Only the prefixed ones are held to the 8192 bytes of the secrets API.
  • No decision record. This adds one more source to the platform secrets layer of ADR-0044 and changes none of its decisions. Say if you would like one.
  • General access to environment variables from a module is out of scope. Only the prefixed variables are read, and only into the secrets cache.

How it is tested

  • crates/temper-server/src/secrets/environment/tests/, 19 unit tests in four files (authorization, naming, precedence, reporting). The two authorization tests seed a vault, attach it to a ServerState, load Cedar policies for the tenant, and read through authorized_wasm_secret_resolver with CedarWasmAuthzGate: the lookup a module's get_secret call goes through. The logging tests capture every log line and span of the seeding at TRACE level.
  • crates/temper-cli/tests/serve_environment_secrets.rs, 2 tests. They start the built temper serve as a process with a clean environment, an empty home and an empty working directory, wait until it listens, and assert on what it wrote.
  • The built server was also run by hand with and without prefixed variables. Without one, its start-up output is the same as on main (see check 7).

Acceptance checks

All commands were run locally on the content of the final commit of this branch.

# Criterion Status Evidence
1 With TEMPER_SECRET_BUILD_TOKEN=abc set, the secret build_token resolves to abc for a module that is permitted to read it; the test fails first because the secret does not exist passed permitted_module_reads_a_secret_seeded_from_a_prefixed_variable. Before the seeding existed (an empty function of the same signature): left: Err("secret not found: build_token"), right: Ok("abc"). After it: passes.
2 A module that is not permitted to read build_token is still refused passed module_without_permission_is_still_refused_a_seeded_secret (a module the policies do not name: authorization denied for secret 'build_token') and permitted_module_is_refused_a_seeded_secret_its_policy_does_not_name.
3 Several such variables are all seeded passed several_prefixed_variables_are_all_seeded, seeded_secret_reaches_every_tenant.
4 An empty value seeds nothing passed empty_value_seeds_nothing_and_is_not_reported.
5 A badly named variable is skipped and reported once at start by name; the server starts passed badly_named_variables_are_skipped_and_each_reported_once_by_name (seven names, among them TEMPER_SECRET_ alone, a lower-case letter and a dash; the well-named variable beside them is still seeded). Against the real process: prefixed_variables_are_reported_by_count_and_bad_names_once_and_the_server_starts.
6 With ANTHROPIC_API_KEY set, anthropic_api_key resolves as it does today, with or without TEMPER_SECRET_ANTHROPIC_API_KEY passed name_the_server_already_set_keeps_its_value. The process test shows start-up applies it in that order: with both set, TEMPER_SECRET_ANTHROPIC_API_KEY is the variable reported as skipped. prefixed_form_of_a_fixed_name_is_seeded_when_the_server_has_not_set_it covers the other case.
7 With no TEMPER_SECRET_ variable, start-up output is identical to before the change passed no_prefixed_variable_seeds_nothing_and_logs_nothing, variables_without_the_prefix_are_ignored, and the process test start_up_without_a_prefixed_variable_reports_nothing_about_them. By hand: temper serve --port 0 --no-observe built from main at 0d7ae427 and from this branch, same clean environment; 457 lines of stdout and 26 of stderr each, identical once timestamps, the port and the temporary directory are normalised and the lines sorted.
8 No test output, log line or span contains a seeded value passed no_report_log_line_or_span_contains_a_value runs every outcome (seeded, badly named, already set, too large, over the budget) and searches the report and the captured log, with span events on. The process test searches the server's whole stdout and stderr for the five values it set.
9 The behaviour when a stored secret has the same name is covered by a test and stated in the documentation passed stored_tenant_secret_wins_over_a_seeded_one_for_that_tenant_only: stored before or after the seeding, the tenant's own secret wins for that tenant, other tenants read the seeded one, and removing the stored one uncovers it. Stated under "Precedence" in the guide.
10 The workspace's tests, lints and format checks show nothing new; CI is green passed locally; CI green on the first three commits cargo fmt --check, cargo check --workspace, cargo clippy --workspace --all-targets -- -D warnings, the readability ratchet, the storage dispatch boundary check, the TODO and unwrap() scans, the dependency isolation check, check_instrumentation and cargo test --doc --workspace all pass. cargo nextest run --workspace -E 'not test(dst_)', the command CI uses: 3526 tests run, 3526 passed, 65 skipped. The three observe-gated cargo test commands, cargo test --locked -p temper-cli verify:: and the four simulation suites CI runs on a pull request pass. CI passed on the first three commits; it has not yet run on the three that follow.
11 The documentation of the server's environment variables lists the prefix, the naming rule and the precedence passed "Secrets from the environment" in Appendix C of docs/AGENT_GUIDE.md, and the module documentation of temper_server::secrets::environment.

Also covered: a value or a name that is not UTF-8 (value_that_is_not_utf8_is_skipped_and_reported_by_name, name_that_is_not_utf8_is_skipped_and_reported), a value over the size limit (value_over_the_size_limit_is_skipped_and_reported_by_name), the template route (integration_config_template_resolves_a_seeded_secret_like_any_other), the platform budget (variables_over_the_platform_budget_are_skipped_in_name_order, which lists the variables in reverse to show the outcome follows the names), and the log line itself (log_gives_the_count_of_seeded_secrets_and_no_names).

Lint baseline against the result

Check main at 0d7ae427 This branch
cargo fmt --check clean clean
cargo clippy --workspace --all-targets -- -D warnings 0 warnings 0 warnings
Readability ratchet, blocking metrics pass pass, every blocking metric unchanged (PROD_PRINTLN_COUNT 245, PROD_FILES_GT500 78, PROD_FILES_GT1000 24, PROD_UNWRAP_CI_OK_COUNT 126)
Readability ratchet, advisory PROD_FILES_GT300 253 253
TODO and unwrap() scans, dependency isolation, storage dispatch boundary, check_instrumentation pass pass

crates/temper-cli/src/serve/mod.rs goes from 992 to 997 lines; the rule and its tests are in new files.

Departures from the plan

  • The rule lives in temper-server, not in temper serve. The plan was to seed through cache_platform_secret_if_present, the private helper in temper-cli that the two fixed variables use. The new function calls SecretsVault::cache_platform_secret directly, which is all that helper does after unwrapping an Option. temper-cli has no logging dependency and none is added, and the authorization tests need the server's own resolver.
  • "The fixed variable wins" is done by seeding the prefixed variables last and skipping a name that is present, not by seeding them first and letting the fixed ones overwrite. The outcome for the two fixed variables is the same. It also covers the server's other platform secrets, keeps the budget for them, and lets the skipped variable be reported.
  • Three more reasons to skip a variable than the naming rule: a value that is not UTF-8, a value over the size limit, and the platform budget. All are logged by variable name.
  • Order of tests and code. The 17 unit tests of the first commit were written before the seeding and run against an empty function of the same signature: 12 failed, and the 5 that assert a refusal or that nothing is seeded passed, as they should. The process test was written after the call in temper serve, and was then seen to fail with the call removed (left: 0, right: 1 on the count line). The size-limit test was seen to fail before the check (left: ["at_limit", "over_limit"]).

After the first review

Three commits answer the automated review of the first three:

  • test(secrets): split the environment seeding tests by topic: the test file was 525 lines, over the 500-line rule. It moves the tests without changing them.
  • fix(secrets): skip a TEMPER_SECRET_ value over the secret size limit.
  • docs(secrets): say what each way of reading a seeded secret checks: the guide said "authorization is unchanged" next to a template example, which read as if the permit covered both routes. The guide and the module documentation now say what each route checks.

🤖 Generated with Claude Code

RetriggerConfidence Score: 4/5

The PR does not appear safe to merge while an integration template can pass a seeded secret to a module whose policy denies access to it.

Fix All in Claude CodeFindings

  1. P1 Security Seeded secrets bypass module permissions ▶
Fix with agent prompt
### Issue 1
crates/temper-cli/src/serve/mod.rs:undefined-230
If an authorized spec submitter puts `{secret:build_token}` in a WASM integration config, this new platform secret is resolved and passed to the module even when that module lacks `access_secret` permission for `build_token`. Template resolution reads the vault without the Cedar check used by the module’s `get_secret` call, so the module can receive a secret its policy denies.

**How this was verified:** A submitted integration config reaches the WASM invocation context through template resolution, which reads platform secrets without calling the module secret-access gate.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
Summary

The PR seeds non-empty TEMPER_SECRET_<NAME> variables into the in-memory platform secrets layer at startup.

  • It validates names and value sizes, preserves secrets already set by the server, and logs counts and skipped variable names without logging values.
  • It adds unit and process tests and documents the precedence and two secret-reading routes.

Reviews (2) · Last reviewed commit: "docs(secrets): say what each way of read..."

arun-pathiban-ddog and others added 3 commits October 4, 2026 16:19
Add seed_platform_secrets_from_environment to the server's secrets module.
Given the environment as name and value pairs, it caches every variable named
TEMPER_SECRET_<NAME> with a non-empty value as the platform secret <name> in
lower case, so TEMPER_SECRET_BUILD_TOKEN supplies build_token to every tenant.

<NAME> is A-Z, 0-9 and _, starting with a letter, so no two variables can
supply the same secret. A variable that does not fit, a value that is not
UTF-8, a name the server already holds a platform secret for, and a variable
beyond the platform budget are skipped and logged once by variable name.
Seeded secrets are logged as a count. Values are never logged or returned.
Nothing is logged when no prefixed variable is set.

The function reads no environment and writes nothing to storage. Reading a
seeded secret is authorized as before: the tests go through the resolver a
module's get_secret call uses, with Cedar policies that permit and refuse it.

Co-Authored-By: Claude Code <noreply@anthropic.com>
temper serve seeded two fixed variables into the secrets cache at start,
ANTHROPIC_API_KEY and EXA_API_KEY, and had no way to supply any other secret
with the server's configuration: it had to be stored through the secrets API
once the server answered, and again after every restart of a server that
keeps its cache in memory only.

Seed every TEMPER_SECRET_<NAME> variable after the secrets the server sets
itself, so those keep their values and an existing deployment sees no change.
With no such variable set, start-up does and logs nothing new.

The test starts the built server as a process, with and without prefixed
variables, and checks the count, one warning per skipped variable, that the
fixed variable wins over its prefixed form, that no value appears in the
output, and that the server listens.

Co-Authored-By: Claude Code <noreply@anthropic.com>
… precedence

Add the prefix to the environment variable appendix of the guide, with the
naming rule, the reach across tenants, the unchanged authorization, the
precedence against a tenant's stored secret and the secrets the server sets
itself, what start-up logs, and the budget.

Co-Authored-By: Claude Code <noreply@anthropic.com>
// TEMPER_SECRET_<NAME> — any other secret the operator supplies. Runs
// last, so every name seeded above keeps its value.
// determinism-ok: environment read once at startup
temper_server::secrets::seed_platform_secrets_from_environment(vault, std::env::vars_os());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 security Seeded secrets bypass module permissions
If an authorized spec submitter puts {secret:build_token} in a WASM integration config, this new platform secret is resolved and passed to the module even when that module lacks access_secret permission for build_token. Template resolution reads the vault without the Cedar check used by the module’s get_secret call, so the module can receive a secret its policy denies.

How this was verified: A submitted integration config reaches the WASM invocation context through template resolution, which reads platform secrets without calling the module secret-access gate.

Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/temper-cli/src/serve/mod.rs
Line: 230

Comment:
**Seeded secrets bypass module permissions**
If an authorized spec submitter puts `{secret:build_token}` in a WASM integration config, this new platform secret is resolved and passed to the module even when that module lacks `access_secret` permission for `build_token`. Template resolution reads the vault without the Cedar check used by the module’s `get_secret` call, so the module can receive a secret its policy denies.

**How this was verified:** A submitted integration config reaches the WASM invocation context through template resolution, which reads platform secrets without calling the module secret-access gate.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code Fix in Codex Fix in Cursor

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The route is real, and it is not new. A {secret:KEY} template in an integration config is resolved from the vault when the integration runs, for every secret the tenant can read, without the access_secret check that a module's get_secret call goes through (resolve_secret_templates). That is how {secret:anthropic_api_key} resolves today, for the two fixed variables and for secrets stored through the API. This pull request does not change how secrets are read, so it leaves that as it is. Gating templates by access_secret would change existing deployments and belongs in a change of its own.

What was wrong here is the wording. The guide said "authorization is unchanged" next to a template example, which read as if the permit covered both routes. Fixed in "docs(secrets): say what each way of reading a seeded secret checks": the guide and the module documentation now say what each route checks, and that, because a seeded secret reaches every tenant, an operator should supply this way only what every tenant's specs may use. The same commit adds a test that states the template behaviour for a seeded secret, and the pull request description says the same under "Behaviour worth a reviewer's attention".

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clarification. I agree this is existing behavior rather than a regression introduced by this change, so gating template resolution is out of scope for this PR. The updated guide/module documentation and regression test make the distinction between template resolution and the policy-checked get_secret route clear, and appropriately call out that platform-seeded secrets are available across tenants. This concern is resolved; no code change is needed here.

Tip: You can customize Greptile's behavior for this repo with .greptile/rules.md and .greptile/config.json.

Comment thread crates/temper-server/src/secrets/environment.rs
Comment thread crates/temper-server/src/secrets/environment_test.rs Outdated
arun-pathiban-ddog and others added 3 commits October 4, 2026 17:21
The test file was 525 lines, over the 500-line rule. Move it to a tests
directory under the module: shared helpers, and one file each for
authorization, naming, precedence and reporting. No test is changed.

Co-Authored-By: Claude Code <noreply@anthropic.com>
The secrets API refuses a value over 8192 bytes. A prefixed variable was
cached whatever its size, so the limit depended on how a secret arrived.
Skip a larger value and log the variable by name, like the other skipped
variables. A value of exactly the limit is seeded.

Co-Authored-By: Claude Code <noreply@anthropic.com>
The guide said authorization is unchanged, next to an example of a
{secret:<name>} template. That holds for a module's get_secret call, which
needs a policy that permits access_secret. A template in an integration
config is resolved when the integration runs without that check, as it is for
every secret. Since a seeded secret reaches every tenant, say so, and say to
supply this way only what every tenant's specs may use. Add the size limit to
the guide, and a test that states the template behaviour for a seeded secret.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@arun-pathiban-ddog
arun-pathiban-ddog merged commit 1be4b7b into main Oct 4, 2026
12 checks passed
rita-aga added a commit to arni-labs/temper that referenced this pull request Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant