Repository navigation
feat(discord): encrypt the webhook URL at rest, keyed by (integrationId, kind) (TASK-124 part 2) - #1898
Merged
Conversation
…Id, kind) (TASK-124 part 2) `discord_integrations.webhookUrl` was the last connector credential stored in plaintext outside the ESO path. The URL embeds the webhook's own token, so holding it is posting AS that channel. Writers now put it in the connector-secret envelope (AES-256-GCM, ring from CONNECTOR_SECRET_KEYS) and the row keeps only an opaque ref at `Integration.config.webhookUrlRef`; readers go through `utils/discordWebhookUrl`. Wren's ruling: option A, keyed with `kind`. The same change fixes the three blockers the design exposed, which is what makes the encryption safe to turn on rather than a new way to lose a live credential: 1. `sweepOrphanedConnectorSecrets` kept a secret only on a Slack `botTokenRef`/`pendingBind.botTokenRef` pair, so it revoked a Discord secret ten minutes after it was written (`ORPHAN_SECRET_GRACE_MS`). Its keep-condition is now drawn from the kinds registry. 2. `sweepUnavailableSlackSecretKeys` marked ANY row with a Slack-named reason, and since #1878/#1881 that string is user-facing — a Discord owner read "Slack connector secret key is unavailable". Renamed and made kind-aware; Slack's sentence is byte-identical. 3. `upsertEncrypted` filtered on `{ integrationId }` alone, on both the upsert and the 11000 retry, so a second kind did not collide: it MATCHED the first kind's row and $set the new ciphertext onto it, leaving the first ref valid and decrypting to the second secret. The filter now carries `kind`, the model declares ONE index on the pair, and `services/connectorSecretKinds` is the single definition all four sites read. Also: `webhookUrl` and `webhookUrlRef` join `SERVER_OWNED_CONFIG_KEYS` (the URL is derived by the server on both writers, and a planted ref would point a row at another row's secret), and `webhookUrlRef` is listed in `INTEGRATION_SECRET_CONFIG_KEYS` so the pointer never rides out either. The migration is a separate gated step: `scripts/encrypt-discord-webhook-url` (dry-run by default, `--apply` to write, digests not URLs, idempotent, reads both plaintext stores and unsets both, refuses to write at all when the key ring is unusable). Run it after this deploys — the old image reads the plaintext field directly. Evidence: full backend tier 446 suites / 4167 tests green; tsc 0; eslint .ts 0 errors at the 1210-warning baseline. Eight-arm mutation ledger, no survivors: each arm reverts one piece and every arm reddens a named test.
…nd drop the inert pre-flight vera's hold at `e6d4077c`: the ordering that prevents data loss in `encrypt-discord-webhook-url` was unwitnessed. Moving both `$unset`s above the `connectorSecrets.put` (the arrangement where a failing put destroys the only copy) left the suite 8/8, and deleting the `listWithUnavailableKey()` pre-flight reddened nothing. Both observations were right, and the second one was the more interesting: `listWithUnavailableKey` returns refs whose `keyId` is not in the ring. It cannot fail. A call that cannot fail is not a guard, so it is gone — the script no longer claims to fail fast, and its comment now states the invariant that actually protects the data: per row, the `put` completes before either plaintext is cleared, so ANY failure leaves that row whole and a re-run reports it as still waiting. Two witnesses replace the claim: - a failed `put` leaves both plaintexts in place and reports nothing migrated (asserts the platform store, the legacy `config.webhookUrl` store, the absence of a ref, and that a following dry run still sees `candidates=1, migrated=0`); - clearing happens only after the row's own ref is stored, one row further in: the second row's `put` rejects, so the first must have completed end to end (the test cannot pass vacuously) and the second must be untouched. The empty-ring test keeps its place as the ring-shaped instance, with its comment corrected: what protects it is the ordering, not the deleted pre-flight. Mutation ledger, script only: vera's mutation reddens 3 named tests (both new witnesses + the empty-ring one); removing the dry-run guard reddens 1 as a control that the ledger sees this file at all; re-adding the deleted pre-flight is a SURVIVOR by design, which is the evidence for removing it rather than witnessing it.
…d pre-flight
vera's correction at 74198, and she is right: `listWithUnavailableKey` CAN fail.
Its first statement is `const { keys } = parseKeyRing()` (services/connectorSecrets.ts:171),
and `parseKeyRing` throws `ConnectorSecretConfigurationError` on an unset,
malformed or duplicated ring and `ConnectorSecretKeyMissing` when the active key
is absent (:49-79). I wrote "it cannot fail" by quoting the find line and
dropping the line above it.
So the pre-flight was redundant, not inert: `put` computes
`encrypt(material, parseKeyRing())` before `upsertEncrypted` touches anything
(:138-146), so an unusable ring already fails the first row ahead of either
`$unset`. Deleting it remains right; the reason recorded for the deletion was
wrong, and on the one script in this lane whose damage is irreversible.
The comment now says what is true, in vera's wording; the same false claim is
corrected in the test comment that cited it. Comment-only, so the clearance at
`d015ad25` stands: the only changed lines in this commit begin with `//`, `*` or
`/**`.
samxu01
pushed a commit
that referenced
this pull request
Sep 25, 2026
…its locators wren's hold at 272ca93: rule 24 is the right home and the suite claim holds, but two locators do not, and the PR merges after #1898. Both halves are the same fact — the declaration (`webhookUrlRef: String` in the `config` subdocument) and the witnessing assertion (`integrations.discordTokenCopy.test.js`) exist only on the unmerged #1898: `git grep webhookUrlRef origin/main -- backend frontend docs` returns nothing. So the clause now names #1898 as the carrier of both, states the pre-merge grep as the rev this was earned at (true forever, and it is the state a reader arriving early would find), and says the merge order outright. The witness is described by its mechanism — the create route against a MongoMemoryServer-backed model, read back with `.lean()` — and the census is the measured one: of the five suites that name the ref, the other four mock `Integration` or seed the ref themselves, so none of them writes it through the route. Docs-only, one line.
samxu01
pushed a commit
that referenced
this pull request
Sep 25, 2026
…oun for `config` wren's two corrections (74203/74204, relayed by connector-ops at 74205) applied, both verified at the premise first: 1. The writer is not `updateOne`. On #1898 the connect route writes with `Integration.findByIdAndUpdate(..., { $set: { 'config.webhookUrlRef': ref } })` (`routes/integrations.ts:474`) and the backfill does the same (`services/discordService.ts:345`). The only `updateOne` is the migration script's, and it runs on `mongoose.connection.collection()` — a raw-collection write, which strict mode never touches. So the clause now teaches the split instead: **a write through the Model is stripped, a raw-collection write is not**, and it says why the obvious grep points the wrong way (grepping `updateOne` finds the one writer that cannot drop). 2. `config` is not a subdocument. Measured from the model at 715e187 rather than restated: `schema.path('config')` is `undefined`, `schema.nested.config` is true, `schema.paths` carries no `config` key, and `schema.get('strict')` is true. The clause now says "nested path of a strict schema" and points at rule 34, which already describes this same `config` that way. The backfill's drop is marked *derived*, not observed (same schema, same `$set`, and both writers landed in one PR, so the declaration fixed both before either merged). #1898 itself is untouched: its model comment repeats "subdocument", and moving a cleared head that is in the enqueue for a noun is not worth it. Docs-only, one line.
samxu01
pushed a commit
that referenced
this pull request
Sep 25, 2026
…tance wren's third correction: the instance was earned ON #1898, and there `webhookUrlRef` appears in 15 backend files, not zero. My sentence described main before #1898 lands — a state nobody reading the merged checklist will ever see, and not the rev the lesson came from. Re-measured to confirm her premise: `git grep -l webhookUrlRef origin/kai/124-webhookurl-encryption -- backend` = 15. The replacement says the thing that is actually instructive: the name was already everywhere (readers, both Model writers, five suites) and the schema declaration was the one place it was missing, so a grep for the name cannot find this defect — what finds it is rule 24's opening comparison, every path the code reads against the schema that must declare it. Docs-only, one line.
lilyshen0722
added a commit
to thantiklermcirony/commonly
that referenced
this pull request
Oct 9, 2026
…t drop (Team-Commonly#1900) * docs(review): rule 24 gains the update shape of the strict-subdocument drop vera asked (74186) where Team-Commonly#1898's strict-subdocument finding belongs. The class is already rule 24's — a `strict` schema silently drops an undeclared path on the write, so a fixture that builds its own document proves the reader can read a field and never that the field can be there — and rule 37 answers a different question (whether a read tells you the key exists, the value is non-empty, or the value can authenticate). Filing a write-side drop under the read-side rule would hide it from the next reviewer grepping for it, so the instance goes where the class already lives. The added instance is the update shape of the same defect, which rule 24 did not carry: `$set { 'config.webhookUrlRef': ref }` on a row the Discord connect route had just created was dropped in silence because `Integration.config` is a strict subdocument that did not declare the path. Same tell as the constructor case and the same instrument closes it — the only test that saw it asserts against real Mongo after the create call and reads the ref back. Docs-only, one file. * docs(review): point rule 24's second instance at the PR that carries its locators wren's hold at 272ca93: rule 24 is the right home and the suite claim holds, but two locators do not, and the PR merges after Team-Commonly#1898. Both halves are the same fact — the declaration (`webhookUrlRef: String` in the `config` subdocument) and the witnessing assertion (`integrations.discordTokenCopy.test.js`) exist only on the unmerged Team-Commonly#1898: `git grep webhookUrlRef origin/main -- backend frontend docs` returns nothing. So the clause now names Team-Commonly#1898 as the carrier of both, states the pre-merge grep as the rev this was earned at (true forever, and it is the state a reader arriving early would find), and says the merge order outright. The witness is described by its mechanism — the create route against a MongoMemoryServer-backed model, read back with `.lean()` — and the census is the measured one: of the five suites that name the ref, the other four mock `Integration` or seed the ref themselves, so none of them writes it through the route. Docs-only, one line. * docs(review): name the Model-vs-raw-collection split, and the right noun for `config` wren's two corrections (74203/74204, relayed by connector-ops at 74205) applied, both verified at the premise first: 1. The writer is not `updateOne`. On Team-Commonly#1898 the connect route writes with `Integration.findByIdAndUpdate(..., { $set: { 'config.webhookUrlRef': ref } })` (`routes/integrations.ts:474`) and the backfill does the same (`services/discordService.ts:345`). The only `updateOne` is the migration script's, and it runs on `mongoose.connection.collection()` — a raw-collection write, which strict mode never touches. So the clause now teaches the split instead: **a write through the Model is stripped, a raw-collection write is not**, and it says why the obvious grep points the wrong way (grepping `updateOne` finds the one writer that cannot drop). 2. `config` is not a subdocument. Measured from the model at 715e187 rather than restated: `schema.path('config')` is `undefined`, `schema.nested.config` is true, `schema.paths` carries no `config` key, and `schema.get('strict')` is true. The clause now says "nested path of a strict schema" and points at rule 34, which already describes this same `config` that way. The backfill's drop is marked *derived*, not observed (same schema, same `$set`, and both writers landed in one PR, so the declaration fixed both before either merged). Team-Commonly#1898 itself is untouched: its model comment repeats "subdocument", and moving a cleared head that is in the enqueue for a noun is not worth it. Docs-only, one line. * docs(review): drop the false pre-merge grep from rule 24's second instance wren's third correction: the instance was earned ON Team-Commonly#1898, and there `webhookUrlRef` appears in 15 backend files, not zero. My sentence described main before Team-Commonly#1898 lands — a state nobody reading the merged checklist will ever see, and not the rev the lesson came from. Re-measured to confirm her premise: `git grep -l webhookUrlRef origin/kai/124-webhookurl-encryption -- backend` = 15. The replacement says the thing that is actually instructive: the name was already everywhere (readers, both Model writers, five suites) and the schema declaration was the one place it was missing, so a grep for the name cannot find this defect — what finds it is rule 24's opening comparison, every path the code reads against the schema that must declare it. Docs-only, one line. * docs(review): rule 24's second instance — drop the count, fix the referent wren's two swaps, exactly as given: "at the rev this was earned the name already appeared in **15 backend files**" -> "the name already appeared across the backend" "the comparison this rule opens with" -> "the comparison this rule's check opens with" She is right on the second: the rule opens with its thesis ("A fixture that builds its own document proves the reader can read a field"), and the comparison is the first step of its check ("The check is two greps and one round-trip: for each field the code reads cross-layer, confirm the writing schema declares that exact path"). Nothing else changed: no other path, no other word on the line.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Encrypts
discord_integrations.webhookUrlat rest and fixes the three blockers that made the encryption safe to switch on. Wren's ruling: option A, keyed withkind(74127). Gate: @vera.The shape
connector_secretsis the envelope that already held Slack's bot token. A secret's identity becomes the pair(integrationId, kind); writers store only a ref (Integration.config.webhookUrlRef), readers resolve through one new module.services/connectorSecretKinds.ts— the single definition:kind,provider, the row paths that hold that kind's ref, and the sentence written for the person reading the Connectors page. Four sites encode the pair (the index, the upsert filter, the 11000 retry's filter, the orphan sweep's keep-condition) and all four read this instead of literals.utils/discordWebhookUrl.ts— the one precedence definition: encrypted ref wins, the two legacy plaintext stores (platformIntegration.webhookUrl,Integration.config.webhookUrl) are fallbacks. A ref that cannot be resolved throws rather than falling through to a plaintext copy: a wrong webhook URL does not fail authentication, it authenticates as a different connector (vera 74141 — a missing secret is loud, a substituted one is not).routes/integrations.ts) and the lazy backfill (discordService.connect) bothputthe URL and$setthe ref; onlywebhookId, which is not a secret, stays on the platform document.webhookUrlandwebhookUrlRefjoinSERVER_OWNED_CONFIG_KEYS(the URL is derived server-side on both writers; a planted ref would aim a row at another row's secret, and a planted URL would be picked up as the resolver's legacy fallback).webhookUrlRefalso joinsINTEGRATION_SECRET_CONFIG_KEYSso the pointer never rides out in a response.The three blockers
sweepOrphanedConnectorSecretskept a secret only on a SlackbotTokenRefpair → revoked a Discord secret 10 min afterputsweepUnavailableSlackSecretKeysmarked any row "Slack connector secret key is unavailable" — user-facing since #1878/#1881upsertEncryptedfiltered{ integrationId }on both the upsert and the retry → second kind silently overwrote the first, first ref decrypting the second secretkindin the filter; one index on the pairMigration — separate gated step
backend/scripts/encrypt-discord-webhook-url(same shape as #1891): dry-run by default,--applywrites, digests instead of URLs, reads both plaintext stores (discord_integrations.webhookUrland the legacyIntegration.config.webhookUrl), and clears a row only after its own ref is stored — per row,putfirst, then both$unsets, so a failure of any kind leaves that row whole in both stores and a re-run reports it as still waiting. Corrected after vera's hold: the first draft also calledlistWithUnavailableKey()as a pre-flight and claimed it failed fast. That call does parse the key ring before it queries, so it would throw on an unusable ring — but so does everyput, ahead of either$unset, so the pre-flight was redundant rather than protective and was removed rather than witnessed; the two suite witnesses below hold the ordering instead. (vera's correction at 74198: the first version of this note said the call “cannot fail”, which was false —listWithUnavailableKey's first statement isparseKeyRing().)Run it after this deploys: the current image reads the plaintext field directly.
Two things found while building, both disclosed
configonIntegrationis a strict subdocument, so an undeclared path in$setis dropped in silence. The ref write was a no-op untilwebhookUrlRef: Stringwas declared besidewebhookUrl. The witness that caught it is the created-row assertion inintegrations.discordTokenCopy.test.js(it readundefinedwhile every mocked-model suite stayed green).Verification
tsc -p tsconfig.build.json --noEmitexit 0;eslint . --ext .ts0 errors at the 1210-warning baseline;kind(2 red), M2 a second unique index (2 red), M3 Slack-only keep-condition (1 red), M4 Slack-only reason (1 red), M5 fall back instead of throw (2 red), M6 connect writes plaintext again (1 red), M7 the strip entries removed (1 red), M8 the schema declaration removed (1 red). On the migration script: vera's mutation — both$unsets above theput— reddens 3 named tests (both ordering witnesses + the empty-ring one), removing the dry-run guard reddens 1, and re-adding the deleted pre-flight is a survivor by design, and the reason it survives is that it is redundant — both it andputreach the sameparseKeyRing()before any write, so no test can distinguish them;oauthControllertotal-count flake (TASK-134) andadmin.analytics.human-filter, both green on re-run (24 passed).Not changed, deliberately:
installable/projectors/webhookProjector.tsstill revokes only Slack refs on uninstall — the orphan sweep now covers a leftover Discord ref within the grace window, so a second revoke path would be duplicate machinery rather than a fix.