From caf4ad5d614c153d3e577183ac297f8b69aeb319 Mon Sep 17 00:00:00 2001 From: Lily Shen <115414357+lilyshen0722@users.noreply.github.com> Date: Wed, 28 Oct 2026 23:16:48 -0700 Subject: [PATCH 1/3] docs(deployment): DEV_HELM_HIDDENMCP_ALLOWLIST is a YAML overlay, not a word list The deploy-dev workflow writes the secret's whole value to allowlist.yaml and passes it to helm as the last -f, so the value must be a complete Helm values overlay of hostedMcp.hiddenEntryUserAllowlist (entry -> allowed user ids), not a comma/word list. Documents the shape (placeholder ids only), how to edit it, and the distinction from the in-repo hiddenEntries list. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01MDs9DS64Df3HsKsCHiKBWN --- docs/deployment/README.md | 1 + .../hidden-mcp-allowlist-overlay.md | 60 +++++++++++++++++++ 2 files changed, 61 insertions(+) create mode 100644 docs/deployment/hidden-mcp-allowlist-overlay.md diff --git a/docs/deployment/README.md b/docs/deployment/README.md index e1e6d44b6d..83e43489bf 100644 --- a/docs/deployment/README.md +++ b/docs/deployment/README.md @@ -11,6 +11,7 @@ This directory contains deployment, DevOps, and infrastructure documentation. | [SELF_HOSTED.md](./SELF_HOSTED.md) | Current local single-machine installation, operations, and production boundaries | | [KUBERNETES.md](./KUBERNETES.md) | Kubernetes and Helm deployment notes | | [DEPLOYMENT.md](./DEPLOYMENT.md) | Historical Docker, CI/CD, scaling, and monitoring notes; do not use it as the current self-hosting quick start | +| [hidden-mcp-allowlist-overlay.md](./hidden-mcp-allowlist-overlay.md) | Why the `DEV_HELM_HIDDENMCP_ALLOWLIST` deploy secret is a YAML overlay, not a word list, and how to edit it | ## Quick Start diff --git a/docs/deployment/hidden-mcp-allowlist-overlay.md b/docs/deployment/hidden-mcp-allowlist-overlay.md new file mode 100644 index 0000000000..588ff34c15 --- /dev/null +++ b/docs/deployment/hidden-mcp-allowlist-overlay.md @@ -0,0 +1,60 @@ +# The `DEV_HELM_HIDDENMCP_ALLOWLIST` secret is a YAML overlay, not a word list + +**One-line trap:** `DEV_HELM_HIDDENMCP_ALLOWLIST` (a GitHub Actions secret used +by `deploy-dev.yml`) is **not** a comma- or space-separated list of entry names. +Its entire value is the text of a Helm **values overlay file**. Setting it to +`atlassian,sentry,airtable` silently produces an invalid overlay. + +## How it is consumed + +The `Deploy Dev` workflow materializes the secret to a file and passes it to +`helm` as the last `-f`, after `values-dev.yaml`: + +```yaml +# .github/workflows/deploy-dev.yml (step: Materialize hosted-MCP allowlist overlay) +printf '%s\n' "${DEV_HELM_HIDDENMCP_ALLOWLIST}" > "$RUNNER_TEMP/allowlist.yaml" +# ... later ... +helm upgrade ... -f k8s/helm/commonly/values-dev.yaml \ + -f "$RUNNER_TEMP/values-private.yaml" \ + -f "$RUNNER_TEMP/allowlist.yaml" ... +``` + +So the secret's value **is** `allowlist.yaml`. It must be a complete, valid +Helm values document whose keys merge onto the chart. If the secret is unset or +empty the step writes an empty overlay (a no-op merge), which is the intended +"no allowlist" default — not an error. + +## What the overlay must contain + +It overrides `hostedMcp.hiddenEntryUserAllowlist` — a **map** of hidden +hosted-MCP entry name → list of Mongo user `_id`s permitted to walk that entry. +This block used to live in `values-dev.yaml`; it was moved into the secret so +the per-user allowlist stays out of the public repo. + +```yaml +# shape only — real user ids are NOT committed; they live in the secret +hostedMcp: + hiddenEntryUserAllowlist: + atlassian: + - "" + sentry: + - "" + airtable: + - "" +``` + +## Editing it + +- To allowlist a newly-hidden entry, add a **key** under + `hiddenEntryUserAllowlist` with its user-id list — edit the YAML, not a word + list. +- `gh secret set DEV_HELM_HIDDENMCP_ALLOWLIST --repo --body-file `. + Never paste ids on the command line or into logs. +- The set of **hidden** entries themselves is a separate field + (`hostedMcp.hiddenEntries`, a plain list) that stays in `values-dev.yaml`. + Only the per-user allowlist is overlaid from the secret. +- Each entry's list is YAML; an entry present with an empty list is not the same + as the entry being absent. + +> Earned 2026-10-29: the name reads like a feature-flag word list, and treating +> it as one would drop every allowlisted user on the next deploy. From beef6f8ba40f2e63bbd99d15950d9e9f858fa672 Mon Sep 17 00:00:00 2001 From: Lily Shen <115414357+lilyshen0722@users.noreply.github.com> Date: Wed, 28 Oct 2026 23:27:43 -0700 Subject: [PATCH 2/3] docs(deployment): correct the allowlist overlay failure modes (lumen gate) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per review 5693674306: a bare word list fails helm loudly (scalar→map unmarshal), it does not drop silently. The silent drops are (a) the secret unset/empty at the first hide-entries deploy → chart default {} removes every exception, and (b) a semantically-invalid overlay → visibility.ts discards the whole allowlist. Add the un-hiding trap (an allowlist key for a no-longer-hidden entry voids the entire allowlist) and the merge-order step. Correct the empty-list-vs-absent line: both leave the entry hidden for everyone. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01MDs9DS64Df3HsKsCHiKBWN --- .../hidden-mcp-allowlist-overlay.md | 73 ++++++++++++------- 1 file changed, 47 insertions(+), 26 deletions(-) diff --git a/docs/deployment/hidden-mcp-allowlist-overlay.md b/docs/deployment/hidden-mcp-allowlist-overlay.md index 588ff34c15..8b4ba90491 100644 --- a/docs/deployment/hidden-mcp-allowlist-overlay.md +++ b/docs/deployment/hidden-mcp-allowlist-overlay.md @@ -1,9 +1,20 @@ # The `DEV_HELM_HIDDENMCP_ALLOWLIST` secret is a YAML overlay, not a word list **One-line trap:** `DEV_HELM_HIDDENMCP_ALLOWLIST` (a GitHub Actions secret used -by `deploy-dev.yml`) is **not** a comma- or space-separated list of entry names. -Its entire value is the text of a Helm **values overlay file**. Setting it to -`atlassian,sentry,airtable` silently produces an invalid overlay. +by `deploy-dev.yml`) is the text of a Helm **values overlay file** — a map, not a +comma- or space-separated list of entry names. A bare word list like +`atlassian,sentry,airtable` does **not** fail silently: `helm` refuses to +unmarshal a scalar string into the map-typed value (`cannot unmarshal string +into Go value of type map`) and the deploy fails loudly, so the running release +keeps its current allowlist. The failures that drop users *silently* are +different, and they are the ones to guard against: + +- **The secret is unset or empty when the hidden-entries feature deploys.** The + chart default is `{}`, so every hidden entry is then hidden from *all* its + users — every exception is removed. **Set the secret before that deploy.** +- **The overlay parses but is semantically invalid** — see "What the overlay + must contain". The backend then throws the whole allowlist away (`{}`), with + the same effect. ## How it is consumed @@ -19,42 +30,52 @@ helm upgrade ... -f k8s/helm/commonly/values-dev.yaml \ -f "$RUNNER_TEMP/allowlist.yaml" ... ``` -So the secret's value **is** `allowlist.yaml`. It must be a complete, valid -Helm values document whose keys merge onto the chart. If the secret is unset or -empty the step writes an empty overlay (a no-op merge), which is the intended -"no allowlist" default — not an error. +So the secret's value **is** `allowlist.yaml`. It must be a complete, valid Helm +values document whose keys merge onto the chart. An unset or empty secret writes +an empty overlay, which merges the chart default `{}` — i.e. no exceptions. ## What the overlay must contain It overrides `hostedMcp.hiddenEntryUserAllowlist` — a **map** of hidden -hosted-MCP entry name → list of Mongo user `_id`s permitted to walk that entry. -This block used to live in `values-dev.yaml`; it was moved into the secret so -the per-user allowlist stays out of the public repo. +hosted-MCP entry name → list of authenticated user `_id`s permitted to see that +entry. (It feeds `HOSTED_MCP_HIDDEN_ENTRY_USER_ALLOWLIST`, read by +`backend/integrations/hostedMcp/visibility.ts`.) The per-user allowlist was +moved out of `values-dev.yaml` into the secret so the ids stay out of the public +repo. ```yaml # shape only — real user ids are NOT committed; they live in the secret hostedMcp: hiddenEntryUserAllowlist: atlassian: - - "" - sentry: - - "" - airtable: - - "" + - "" # a 24-hex-char user _id ``` +**The backend validates the whole overlay atomically.** Each key must also be in +`hostedMcp.hiddenEntries`, must match `^[a-z0-9][a-z0-9-]*$`, and every id must +be 24 hex characters. If **any** key or id fails, the backend discards the +**entire** allowlist (not just the bad entry) and grants no exceptions. One +malformed id silently hides every allowlisted entry from everyone. + ## Editing it -- To allowlist a newly-hidden entry, add a **key** under - `hiddenEntryUserAllowlist` with its user-id list — edit the YAML, not a word - list. +- To grant a user access to a hidden entry, add or extend that entry's key with + their 24-hex-char `_id`. Edit the YAML, not a word list. - `gh secret set DEV_HELM_HIDDENMCP_ALLOWLIST --repo --body-file `. Never paste ids on the command line or into logs. -- The set of **hidden** entries themselves is a separate field - (`hostedMcp.hiddenEntries`, a plain list) that stays in `values-dev.yaml`. - Only the per-user allowlist is overlaid from the secret. -- Each entry's list is YAML; an entry present with an empty list is not the same - as the entry being absent. - -> Earned 2026-10-29: the name reads like a feature-flag word list, and treating -> it as one would drop every allowlisted user on the next deploy. +- **Merge order:** set the secret before the change that first hides entries + deploys, or that deploy's chart-default `{}` removes every exception. +- **Un-hiding trap:** `hiddenEntries` (the list of hidden entries) lives in + `values-dev.yaml`; the per-user allowlist lives in the secret. When you remove + an entry from `hiddenEntries` (un-hide it), remove its key from the secret in + the **same change** — an allowlist key for an entry that is no longer hidden + fails validation and voids the *entire* allowlist, silently dropping every + allowlisted user across all entries. +- An entry with an empty list and an entry omitted entirely both leave that entry + hidden for everyone; neither grants access. Access requires a non-empty list of + valid ids. + +> Earned 2026-10-29: the name reads like a feature-flag word list. A word list +> would at least fail the deploy loudly; the real hazard is the silent +> exception-drop when the secret is unset/empty at the first hide-entries deploy, +> or when an un-hidden entry's key is left behind in the overlay. From 55d4f10ad5b82950b32a5099389cebd463420741 Mon Sep 17 00:00:00 2001 From: Lily Shen <115414357+lilyshen0722@users.noreply.github.com> Date: Wed, 28 Oct 2026 23:40:08 -0700 Subject: [PATCH 3/3] docs(deployment): anchor the silent-drop to the map-removal deploy (lumen gate) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per review 5693777878: entries have been hidden since an earlier change, so the exception-drop risk is not at a "hide-entries" deploy — it is at the deploy that removes hiddenEntryUserAllowlist from values-dev.yaml, when the chart default {} takes over if the secret is unset/empty. Corrected the line-12 bullet, the merge-order bullet, and the Earned note to say "the deploy that removes the map from values-dev.yaml". Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01MDs9DS64Df3HsKsCHiKBWN --- .../deployment/hidden-mcp-allowlist-overlay.md | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/docs/deployment/hidden-mcp-allowlist-overlay.md b/docs/deployment/hidden-mcp-allowlist-overlay.md index 8b4ba90491..2f626c6bb1 100644 --- a/docs/deployment/hidden-mcp-allowlist-overlay.md +++ b/docs/deployment/hidden-mcp-allowlist-overlay.md @@ -9,9 +9,12 @@ into Go value of type map`) and the deploy fails loudly, so the running release keeps its current allowlist. The failures that drop users *silently* are different, and they are the ones to guard against: -- **The secret is unset or empty when the hidden-entries feature deploys.** The - chart default is `{}`, so every hidden entry is then hidden from *all* its - users — every exception is removed. **Set the secret before that deploy.** +- **The secret is unset or empty at the deploy that removes + `hiddenEntryUserAllowlist` from `values-dev.yaml`.** The chart default is `{}`, + so every hidden entry is then hidden from *all* its users — every exception is + removed. **Set the secret before that deploy.** (Entries themselves have been + hidden since an earlier change; the exposure is the map moving out of the + committed values into the secret.) - **The overlay parses but is semantically invalid** — see "What the overlay must contain". The backend then throws the whole allowlist away (`{}`), with the same effect. @@ -63,8 +66,8 @@ malformed id silently hides every allowlisted entry from everyone. their 24-hex-char `_id`. Edit the YAML, not a word list. - `gh secret set DEV_HELM_HIDDENMCP_ALLOWLIST --repo --body-file `. Never paste ids on the command line or into logs. -- **Merge order:** set the secret before the change that first hides entries - deploys, or that deploy's chart-default `{}` removes every exception. +- **Merge order:** set the secret before the deploy that removes the map from + `values-dev.yaml`, or that deploy's chart-default `{}` removes every exception. - **Un-hiding trap:** `hiddenEntries` (the list of hidden entries) lives in `values-dev.yaml`; the per-user allowlist lives in the secret. When you remove an entry from `hiddenEntries` (un-hide it), remove its key from the secret in @@ -77,5 +80,6 @@ malformed id silently hides every allowlisted entry from everyone. > Earned 2026-10-29: the name reads like a feature-flag word list. A word list > would at least fail the deploy loudly; the real hazard is the silent -> exception-drop when the secret is unset/empty at the first hide-entries deploy, -> or when an un-hidden entry's key is left behind in the overlay. +> exception-drop when the secret is unset/empty at the deploy that removes the map +> from `values-dev.yaml`, or when an un-hidden entry's key is left behind in the +> overlay.