Skip to content

feat: seal MCP event subscriptions into an envelope - #132

Open
ChiragAgg5k wants to merge 2 commits into
feat/events-protocolfrom
feat/events-envelope
Open

ChiragAgg5k wants to merge 2 commits into
feat/events-protocolfrom
feat/events-envelope

Conversation

@ChiragAgg5k

@ChiragAgg5k ChiragAgg5k commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

Stack

feat/events ← #131 ← #132 ← #133 ← #134 ← #135 (events/subscribe / events/unsubscribe)

Merges into feat/events; feat/events → main lands as one feature after appwrite/appwrite#14293. Each PR's diff shows only its own layer, and every layer passes the full checklist on its own.

Summary

Part 3 of 4 for MCP Events (#127). Adds mcp_server_appwrite.events.envelope, the sealed subscription envelope that lets the hosted server stay stateless: each subscription is one Appwrite project webhook, and everything the ingress needs to deliver an event is sealed into that webhook's authPassword, which Appwrite sends back as HTTP Basic auth on every delivery.

This is a self-contained library. Nothing is wired into server.py or http_app.py; the subscribe and ingress PRs consume it.

Blocked from end-to-end use by appwrite/appwrite#14293. Today authPassword fails above ~99 chars (encrypted column sized for plaintext), and its API validator is Text(256). Every envelope this produces is longer than both (see the size table). The state lives only in authPassword: nothing is split into authUsername or the URL.

What's in it:

  • Subscription frozen dataclass: id, project, name, arguments (sorted, read-only mapping), callback, secrets (current whsec_ first, previous during rotation), expires (epoch ms), principal (digest, never raw tokens).
  • Principal(issuer, subject, client).digest: base64url of the first 128 bits of SHA-256 over canonical JSON of OAuth iss, sub, client_id (22 chars).
  • subscription_id(principal, callback, name, arguments): sub_ + 32 hex chars of SHA-256 over canonical JSON. Exactly 36 chars, valid for Appwrite's CustomId/Key validator ([A-Za-z0-9._-], no leading special char). Argument order does not change it.
  • Keyring (from MCP_EVENTS_SEALING_KEYS), seal / open, typed EnvelopeError with an EnvelopeFailure reason, and Subscription.expired() kept separate from opening so the ingress can tell "expired, 2xx drop" from "invalid".
  • Derived Appwrite webhook secret (Keyring.signing_key) and appwrite_signature / verify_appwrite_signature for X-Appwrite-Webhook-Signature.
  • cryptography is now a direct dependency (it was already locked at 49.0.0 through pyjwt[crypto]).

Format

v1.<key id>.<base64url(nonce[12] | ciphertext | tag[16])>
  • AES-256-GCM, random 96-bit nonce. The cipher key is HKDF-SHA256(sealing key, mcp-events/seal/v1).
  • Plaintext is canonical compact JSON with one-letter keys: n name, a arguments, c callback, s secrets, e expiry (epoch ms), p principal digest.
  • The subscription id and project id are not in the plaintext. They are bound as associated data, canonical_json(["v1", key id, subscription id, project id]), so an envelope copied onto another webhook or project, or relabelled with another key id, fails authentication. On open, the contents must also hash back to the subscription id (binding_mismatch otherwise).
  • The envelope alphabet is [A-Za-z0-9._-], safe as a Basic auth password. Appwrite only sends Basic auth when authUsername is non-empty too, so the subscribe PR must set a fixed, non-secret username.

Example (throwaway key; tablesdb.row.created, 79-char callback URL, one 32-byte secret):

id:        sub_4cccc364e443bf72d88e96cc1cb4943a
plaintext: {"a":{"database_id":"main","project_id":"6630f1a2b3c4d5e6f7a8","table_id":"support_tickets"},"c":"https://chatgpt.com/backend-api/mcp/events/webhook/3f2c9a7e","e":1760003600000,"n":"tablesdb.row.created","p":"OPxPhaa1hwu4ngsYV4zRuw","s":["whsec_9mUpy68IzPkNGko759eURWGH0b4aNRvfor2kJSxrurY="]}
envelope:  v1.k1.n8zJQVHwUd8QCqqjZ5PtArjiTZF1u1K-8iNN4vfeyN9C0tA4QwUY2gDmVigaVwyhjkc28xRa85yORoIPn1RxQRGlKrMuOegaqYIV3mRH3HqG6s21XJUTwsB--yRn9kJgR9vo4LHYeCE9p6xRsDIT9rPCHjrpoobBDe_ItaD2r_JgKQeuXjZvyY2t9q95X_HWpbN-vpLJomdqZ_fCTumYsio-6NcE3omTJzfQE7Q9TFaHZcFVrVkO7esjhjKIeCUlRZjxihfD4nazl7rvhaz3kHJ4mc46Z0RzTsTvcfYpOGYEIrYnqOcms8-VMNyLrs1pk-7BaUwxpqPW63wIhabiIe0dEdyRbMhl9x0jKpYyTPSuUQAsPOXZcL_IKvdLTqPB4U3CVOMBAgfo7p4gs9_cRP-qXmYv36HB3zXNb2ehM_Y   (433 chars)
secret:    db51823bd85cbec94d98ce123dfbc0c05292ce60b4f3f3a3397bb43c70015efe

Size budget

Measured with 36-char Appwrite ids, a 22-char principal digest, a 13-digit expiry and key id k1. Secrets are whsec_ + base64 of the stated byte length.

Case Envelope length
Example above (tablesdb.row.created, 79-char URL, one 32-byte secret) 433
users.user.created, 100-char URL, one 32-byte secret 440
tablesdb.row.created, 150-char URL, one 32-byte secret 646
tablesdb.row.created, 150-char URL, two 32-byte secrets (rotation) 717
functions.deployment.completed + status, 300-char URL, two 64-byte secrets 1005
Worst case: tablesdb.row.created, 300-char URL, two 64-byte secrets 1034
ENVELOPE_BUDGET (seal raises EnvelopeTooLarge above it) 2048
Appwrite authPassword today: reliable limit (encrypted column) ~99 (fails randomly from ~90)
Appwrite authPassword today: API validator Text(256) 256

Even the smallest realistic envelope (~430) is over both current limits, so a compact binary format would not help: the callback URL and the secrets dominate and are incompressible. What appwrite/appwrite#14293 must allow: authPassword of at least ENVELOPE_BUDGET = 2048 characters, stored intact. That means the column change already proposed there (httpPass → VAR_TEXT 65535) and raising the authPassword param validator on create and update from Text(256) to at least Text(2048), which the issue doesn't propose yet. SizeBudgetTests fails if the worst case grows past the budget; subscribe should map EnvelopeTooLarge to -32602 (callback URL too long).

Key management

  • MCP_EVENTS_SEALING_KEYS=<id>:<base64 32 bytes>[,<id>:<base64 32 bytes>...], read from the environment like the other hosted config (Keyring.from_env()), documented in .env.example and docs/development.md. In production it comes from Parameter Store.
  • The first key seals, every key opens; open uses the key id named in the envelope (no trial decryption).
  • Keys must be exactly 32 bytes (standard or URL-safe base64, padding optional) with at least 16 distinct byte values, which rejects zero, repeated and hand-typed keys. Key ids are [A-Za-z0-9_-]+ and must be unique. A missing variable raises KeyringError with a generation hint (openssl rand -base64 32).
  • Rotation: put the new key first; drop the old one after the longest subscription TTL has passed. Subscriptions refresh before they expire, so each refresh re-seals with the new key.
  • The Appwrite webhook secret is hex(HMAC-SHA256(HKDF(sealing key, "mcp-events/appwrite-signature/v1"), subscription id)): 64 chars, within Appwrite's Text(256, 8). Separate HKDF subkeys mean the AES key never doubles as an HMAC key. The ingress derives the secret from the key id in the delivery's envelope, since every webhook write carries an envelope and secret made from the same key.
  • Appwrite signs deliveries as base64(HMAC-SHA1(url . body, secret)) (src/Appwrite/Platform/Workers/Webhooks.php), where url is the webhook's configured URL. The ingress must rebuild the canonical ingress URL, not trust the URL the request arrived on.

Tests

Envelope behavior is covered end to end by the ingress flows in #134 (tests/e2e/test_events_ingress.py), which run against the real server:

  • Seal/open round trip: every delivered event passes through Keyring.seal and the ingress's open.
  • Tampering: a flipped envelope byte gets 401 envelope. An envelope for another project, or one copied from another webhook, also gets 401 envelope.
  • Key rotation by restarting the server: k1 delivers; restarted with k2,k1, the old envelope still delivers and a k2 one does too; restarted with k2 only, the k1 envelope gets 200 retired_key and nothing is delivered.
  • Expiry: an expired subscription gets 200 dropped expired and nothing is delivered.
  • Keyring parsing: the real entry point exits when MCP_EVENTS_SEALING_KEYS is missing, and short, all-zero, repeated, non-base64, id-less and duplicate-id keys stop startup with KeyringError.

Unit tests kept (tests/unit/test_events_envelope.py, 11 tests), and why they are not e2e:

  • Appwrite signature vector produced with PHP 8.5, exactly as the worker computes it, plus rejection of a changed URL, body, key or signature. This is external truth: the e2e harness signs deliveries with its own Python implementation of the same formula, so only this vector ties both to PHP.
    php -r '$url="https://mcp.appwrite.io/appwrite/webhooks/sub_0123456789abcdef0123456789abcdef"; $payload="{\"\$id\":\"6630f1a2b3c4d5e6f7a8\",\"status\":\"failed\"}"; $key="appwrite-signing-key-for-tests"; echo base64_encode(hash_hmac("sha1", $url . $payload, $key, true));'
    # SgM0XNLzyHAwzlgQA2iZykDQI8M=
  • Subscription id determinism and charset: same inputs give the same id, argument order doesn't matter, every input changes it, 200 generated ids satisfy Appwrite's CustomId rules, and foreign ids are refused. Also covered: the principal digest is stable and 22 chars, and the derived webhook secret fits Appwrite's Text(256, 8). Nothing derives an id from a request until events/subscribe (PR 5). Replace these with its e2e flow.
  • Size budget: realistic worst cases (300-char URL, two 64-byte secrets) fit ENVELOPE_BUDGET, and oversize envelopes raise EnvelopeTooLarge. These are deliberately extreme inputs, and they are the contract Encrypted and length-limited attributes return 500 instead of 400 (webhooks authPassword/authUsername/url, variables value) appwrite#14293 has to meet.

Everything else (canonical JSON, round trips, tampering per region, binding, rotation, keyring parsing, expiry) was removed in favor of the e2e flows above.

Verification

Run locally on Python 3.12.8, lockfile written with uv 0.11.22 (the CI version):

  • uv run --group dev ruff check src tests: pass
  • uv run --group dev black --check src tests: pass
  • uv run --group dev pyright: 0 errors
  • uv run python -m unittest discover -s tests/unit: 279 tests OK
  • uv run --group e2e python -m unittest discover -s tests/e2e: 2 tests OK (about 4 s)
  • docker build -t appwrite-mcp:e2e .: builds

@hansi-codes

hansi-codes Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

🔵 Tier A · Mergeable after minor fixes

The open subscription-ID format finding and the missing direct tests for envelope opening are minor issues to address.

This change adds an MCP Events catalog and protocol support, plus helpers for deterministic subscription IDs, sealed subscription envelopes, key rotation, and Appwrite webhook signatures. It also adds event documentation, configuration guidance, and an end-to-end test harness for hosted protocol discovery and catalog listing.

Verdict New comments Fixed Still open
💬 Commented 1 0 1

Note

Not approving while a bug finding is open: Restrict subscription IDs to the generated hex suffix

Finding Where
🟡 Add direct tests for envelope opening and tampering src/mcp_server_appwrite/events/envelope.py:390
Fix with agent prompt
### Issue 1
src/mcp_server_appwrite/events/envelope.py:390-392
**Add direct tests for envelope opening and tampering**

The envelope tests only seal subscriptions to measure their size; none call `Keyring.open`, and the added E2E tests do not exercise ingress. A regression in decryption or the ID/project associated-data binding could therefore pass the suite, so please add direct round-trip and rejection tests for this security boundary.

### Issue 2
src/mcp_server_appwrite/events/envelope.py:155-159
**Restrict subscription IDs to the generated hex suffix**

This accepts any 32-character alphanumeric suffix, such as `sub_zzzz...`, even though `subscription_id()` always emits 32 lowercase hexadecimal characters and the docstring says this checks IDs the module could produce. A caller using this helper to reject foreign IDs will mistakenly accept IDs outside that format.

```suggestion
return re.fullmatch(r"sub_[0-9a-f]{32}", value) is not None
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
📂 Walkthrough · 7
File Change
.env.example, docs/development.md, README.md, docs/events.md, docs/flags.md, AGENTS.md Document the Events feature, its configuration, and the new E2E test workflow.
.github/workflows/ci.yml, pyproject.toml, uv.lock Add the E2E CI job and dependency, and declare cryptography as a runtime dependency.
src/mcp_server_appwrite/events/__init__.py, catalog.py, errors.py, protocol.py Add the event catalog, error types, and feature-gated protocol registration.
src/mcp_server_appwrite/events/envelope.py Add subscription ID, keyring, sealing/opening, and Appwrite signature helpers.
src/mcp_server_appwrite/flags.py, src/mcp_server_appwrite/server.py Add the Events flag and register its protocol handlers for HTTP servers.
tests/e2e/support.py, tests/e2e/test_events_protocol.py Add a real-HTTP hosted-server harness and event protocol coverage.
tests/unit/test_events_catalog.py, tests/unit/test_events_envelope.py, tests/unit/test_events_protocol.py Cover catalog validation, ID and signature behavior, envelope size limits, and stdio gating.
⏳ Still open from earlier reviews · 1

Reviewed a995f8f · Details · Comment @hansi-codes review to re-run, or mention @hansi-codes with a question.

@hansi-codes hansi-codes Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Tier A · See the inline comments. Summary

Comment on lines +155 to +159
return (
len(value) == SUBSCRIPTION_ID_MAX
and value.startswith(SUBSCRIPTION_PREFIX)
and SUBSCRIPTION_ID_PATTERN.fullmatch(value) is not None
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restrict subscription IDs to the generated hex suffix

This accepts any 32-character alphanumeric suffix, such as sub_zzzz..., even though subscription_id() always emits 32 lowercase hexadecimal characters and the docstring says this checks IDs the module could produce. A caller using this helper to reject foreign IDs will mistakenly accept IDs outside that format.

Suggested change
return (
len(value) == SUBSCRIPTION_ID_MAX
and value.startswith(SUBSCRIPTION_PREFIX)
and SUBSCRIPTION_ID_PATTERN.fullmatch(value) is not None
)
return re.fullmatch(r"sub_[0-9a-f]{32}", value) is not None
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/mcp_server_appwrite/events/envelope.py
Line: 155-159

Comment:
**Restrict subscription IDs to the generated hex suffix**

This accepts any 32-character alphanumeric suffix, such as `sub_zzzz...`, even though `subscription_id()` always emits 32 lowercase hexadecimal characters and the docstring says this checks IDs the module could produce. A caller using this helper to reject foreign IDs will mistakenly accept IDs outside that format.

```suggestion
return re.fullmatch(r"sub_[0-9a-f]{32}", value) is not None
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

🟡 Minor · bug · Reply if this doesn't apply.

Each MCP Events subscription is stored only in its Appwrite webhook. The
envelope seals the delivery state into authPassword with AES-256-GCM, bound
to the subscription id and project, and the webhook signing key is derived
so the ingress can verify deliveries without any storage.
Round trips, tampering, binding, rotation and expiry are covered through
the ingress e2e flows in the layer above. What stays is the PHP-computed
Appwrite signature vector, subscription id determinism and charset until
subscribe has an e2e flow, and the size budget for appwrite/appwrite#14293.
@ChiragAgg5k
ChiragAgg5k force-pushed the feat/events-envelope branch from aef5109 to a995f8f Compare October 9, 2026 13:53
@ChiragAgg5k
ChiragAgg5k changed the base branch from main to feat/events-protocol October 9, 2026 13:53

@hansi-codes hansi-codes Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Tier A · See the inline comments. Summary

Comment on lines +390 to +392
plaintext = AESGCM(key.encryption).decrypt(
nonce, sealed, _associated(key.id, id, project)
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add direct tests for envelope opening and tampering

The envelope tests only seal subscriptions to measure their size; none call Keyring.open, and the added E2E tests do not exercise ingress. A regression in decryption or the ID/project associated-data binding could therefore pass the suite, so please add direct round-trip and rejection tests for this security boundary.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/mcp_server_appwrite/events/envelope.py
Line: 390-392

Comment:
**Add direct tests for envelope opening and tampering**

The envelope tests only seal subscriptions to measure their size; none call `Keyring.open`, and the added E2E tests do not exercise ingress. A regression in decryption or the ID/project associated-data binding could therefore pass the suite, so please add direct round-trip and rejection tests for this security boundary.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

🟡 Minor · testing · Reply if this doesn't apply.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant