Repository navigation
Description
User need and decision
Applications need selected business payloads to remain unreadable to the Server, database and their operators. Accept an opt-in client/worker payload-protection extension across embedded PHP and the PHP/Python/Rust service SDKs. durable-workflow/durable-workflow.github.io#306's application-owned encryption recipe is an interim explanation, not completion of this product requirement.
The PHP SDK currently requires AvroPayloadCodec. Preserve the official Avro value protocol. A protection transform is not permission to add a JSON fallback, arbitrary replacement wire codecs or a new codec-server service.
Acceptance
- Define a supported opt-in encode/decode hook and versioned authenticated ciphertext representation, including nonsecret key/version metadata. Specify the protected boundaries for workflow/activity inputs and results, signals, updates and queries. Unsupported boundaries must be explicit.
- Keep plaintext outside the Server when protection is enabled. Keys belong to caller/worker configuration, not Server settings, logs, histories or telemetry. Use established cryptographic libraries; do not implement cryptography.
- Preserve replay: encryption nonces/randomness are generated only at a new admission/commit boundary, never regenerated as a workflow side effect during replay. Canonical histories and duplicate/retry intent must remain stable. Coordinate request deduplication with Make signal and update retries safe after a lost response #216.
- Specify key rotation and retained-history compatibility. Old keys remain usable for required history; missing/wrong keys, tampered bytes and unknown versions fail explicitly without falling back to plaintext.
- Qualify exact typed int64/null/Unicode/maps and inline/external values with independently installed PHP/Python/Rust SDKs, real published Server admissions, cold-process replay and worker replacement. Include typed failures and payload-size accounting without weakening original deadlines or authority fences.
- Describe the plaintext metadata boundary: IDs, type/command names, namespaces/queues, searchable fields, memo/labels and error text are not automatically protected. Define deliberate redaction for operator surfaces. Waterline/CLI must display protected/unavailable data honestly rather than a corrupt or empty value.
- Correct the current website claims immediately: there is no released pluggable payload encryption hook or codec-server architecture. Final guides include a runnable supported hook and key-rotation example in every current locale. No compliance or data-residency claim follows from encryption alone.
Priority P2, additive product feature. No implementation is shipped. General owns the public SDK/Workflow implementation; native capability/catalog adoption requires separate durable-workflow/server#325 coordination.
Activity
rmcdaniel commentedon Oct 20, 2026
Additional need from source review (Slack miner case 2d1d1124): teams using KMS envelope encryption hit KMS rate limits when every decode during replay unwraps a data key. Two points for the hook's acceptance, if they aren't already covered:
rmcdaniel commentedon Oct 20, 2026
Ownership
A dedicated Claude CLI maintainer session now owns this issue through implementation across embedded PHP and the PHP/Python/Rust service SDKs, cross-language interoperability tests, qualification against the published Server, compatible publication and documentation in every current locale. Its assigned delivery order is shared #216 first, then workflow #871, then this issue.
Work lands on
claude/217-*branches with early draft pull requests. The shared envelope contract (versioning, authenticated metadata, key rotation, replay of retained history, external payloads, duplicate/intent comparison with #216, KMS data-key caching and per-key usage bounds, failure handling) will be recorded here for review before the first implementation PR leaves draft. Encryption stays opt-in and existing unencrypted behavior is preserved. Native Server adoption remains coordinated through server#325.rmcdaniel commentedon Oct 20, 2026
The operator transferred this assignment to General after the prior Claude checkpoint. Delivery order remains shared #216, Workflow #871, then this opt-in encryption feature. The source investigation and KMS/cache/usage-limit requirements are recovered. I retain full implementation, contract review, all-locale documentation and published/installed acceptance; no encryption capability or plaintext-protection claim follows from this handoff.
rmcdaniel commentedon Oct 20, 2026
Recovered source investigation
The prior source investigation is recovered. Current encryption is still unavailable; these findings do not advertise a codec or authorize production-key access:
After #216 and Workflow #871, I will finalize the portable threat/compatibility contract before implementation and verify PHP/Python/Rust/Core, external transport, operator surfaces, key rotation/limits and all eight documentation locales on published artifacts. No new dependency, wire, schema, Cloud/provider or frozen-actor adoption follows from this recovered investigation.
rmcdaniel commentedon Nov 4, 2026
General open-item pass, November 4. Owner: General for threat/compatibility design; SDK/Core owners for hooks; docs for truthful claims.
Next action: Finish the portable envelope/key-rotation/replay and216 intent interaction design before opt-in hooks; correct any currently implied shipped encryption capability now.
Action taken in this pass: Assigned docs the source-only truth check and it found the eight-locale workflow-ID wording to correct. No crypto implementation, provider/key provisioning or frozen wire adoption is admitted.
General retains the cross-repository follow-through. Existing operator stops and exact source/release/production gates remain binding; the item stays open until its actual acceptance is evidenced.
rmcdaniel commentedon Nov 4, 2026
The requested source-only website truth check found one remaining implication in the Workflow ID guide: it said payload codecs included payload encryption without stating that no supported hook is released. Docs draft #471 corrects that paragraph in all eight locales at exact
34df822b462a12967eca38f9e288ed3f04c4bd58, base563f81448607ed1562499d876150fa1566a78702, treea9cad92e7c1d595603608f5a78fe1193e4ca43d8.The guide now says Avro encoding does not encrypt payloads or protect identifiers, released packages have no supported client-side encryption hook, and this issue tracks that feature. All existing plaintext metadata, opaque-reference, application data-handling and authorization warnings remain, as do all executable examples. The source audit also checked the capability index and deployment guide: their Avro-interoperability, plaintext-ID, TLS and storage-owned at-rest statements do not claim shipped encryption hooks.
Named owner durable-workflow-github-io retains this immediate correction, with General retaining merge after normal hosted checks and independent Security review. Local translation/contract checks pass; hosted eight-locale qualification is running. Protected Pages and all eight live privacy paragraphs remain post-merge acceptance. General retains the feature contract/implementation and its #216 / Workflow #871 order. No payload protection, codec-server architecture, encryption library, key access or release is introduced or claimed by this docs slice.
rmcdaniel commentedon Nov 4, 2026
The immediate current-site truth correction requested by this issue is complete. Docs #471 merged normally as
2706af78b1950539a9918ff05a1f62cb54026dabafter 11 successful normal checks and independent Security PASS. The sole automatic protected Pages run 41915687309, deployment7558363450, and all six exact-main workflows succeeded.All eight live Workflow ID privacy paragraphs match accepted localized source and link this existing feature issue. They explicitly state that Avro does not encrypt payloads or protect identifiers and released packages have no supported client-side payload encryption hook. Plaintext metadata, opaque-reference, application data-handling and authorization warnings remain. All seven reviewed-source catalogs match the canonical English hash, and the entire live stable manifest is unchanged. The merged PR branch was deleted and exact absence verified.
Complete owning live receipt. This closes only the immediate misleading-public-wording slice. The opt-in encryption feature remains unshipped and open under General's #216 / Workflow #871 / #217 sequence, with no codec-server, key access, package/runtime or header-rollout claim.
rmcdaniel commentedon Nov 11, 2026
Workflow: not started. The embedded encryption hook comes after workflow#871 and the threat/compatibility contract (General). Owner: workflow.
rmcdaniel commentedon Nov 11, 2026
Waterline (2026-11-11): no Waterline slice assigned. If the envelope contract requires operator views to show protected payloads as unavailable, Waterline will take that as a follow-up issue then.
rmcdaniel commentedon Nov 11, 2026
Server: nothing needed yet; payloads stay opaque to the Server. I'll open a native capability issue if the frozen envelope needs one. Owner: General (design).
rmcdaniel commentedon Nov 11, 2026
PHP: not started. Waits on the envelope and key-rotation contract. Owner: General, then sdk-php.
rmcdaniel commentedon Nov 13, 2026
Design decision (General, 2026-11-13): payload protection envelope v1. This is the contract to build to; implementation PRs refine wording, not semantics.
Where it sits. Protection is a transform that runs after the official Avro value encoding on write and before Avro decoding on read. Avro stays the only wire codec: there's no JSON fallback, no replacement codec and no codec server. A protected payload is an Avro value with a reserved envelope type.
Envelope (v1):
{ "dw_protected": 1, "alg": "xchacha20poly1305-ietf", "key_id": "<caller-chosen, nonsecret>", "nonce": <24 bytes>, "ciphertext": <bytes> }sodium_*in PHP,cryptographyor PyNaCl in Python, RustCryptochacha20poly1305. No hand-rolled crypto. The 192-bit random nonce makes random nonces safe.dw_protectedversion,alg,key_id, namespace and the boundary kind (e.g.workflow_input,activity_result,signal_args). Ciphertext can't be moved to another namespace or to another kind of payload.Protected boundaries (v1): workflow input and result; activity input and result; signal args; update args and result; query args and result; heartbeat details; the details payload of failures (not the message text); child and continue-as-new inputs.
Not protected, and documented as such: IDs, workflow/activity/signal/update names, namespaces, queues, search attributes, memos, labels and failure message text. Applications must keep secrets out of those.
Keys. Keys live only in the client or worker configuration as a keyring: one active
key_idfor encrypting, plus every retained key for decrypting. They never go in Server settings, histories, logs or telemetry. Rotation means adding a new active key while keeping the old ones for as long as history that uses them is retained.Failures are typed and never fall back to plaintext:
payload_key_unavailable: an unknownkey_id;payload_authentication_failed: tampered bytes, or the wrong key or AAD;payload_envelope_unsupported: an unknown version or alg.A worker that hits one of these fails the task and doesn't continue on garbage.
Server. No change. Payloads stay opaque, and size limits count the envelope bytes.
Operator surfaces. Waterline and the CLI show
protected (key <key_id>)instead of a value, never an empty or corrupt one. Decoding there is out of scope for v1.Owners and order:
protecteddisplay.The qualification bar in the acceptance above still applies: independently installed SDKs, a published Server, cold replay and worker replacement.
rmcdaniel commentedon Nov 14, 2026
Security contract review — current SDK/embedded boundaries, exact AAD, and vectors
Disposition: HOLD implementation until the items below are settled in the v1 contract. I reviewed the accepted decision against current main at sdk-python
7a36e433, sdk-php92053fc1, sdk-rust1129d797, and workflow8bb05a63. This is a contract/source review only; I used synthetic keys and no Server, provider, customer data, or live runtime.1. Envelope recognition must be a distinct Avro writer type
The current Value schema has only null/scalar/array/
MapValuebranches, and Python converts every ordinary dictionary toMapValue(schema, map conversion). PHP and Rust have the same fixed fingerprint. A map with keysdw_protected,alg,key_id,nonce, andciphertextis therefore ordinary user data today. Key recognition would collide with valid application maps.Use a separate, top-level Avro single-object writer schema, not a magic map and not a recursive branch in
Value:{"type":"record","name":"ProtectedValue","namespace":"durable_workflow.protocol","fields":[{"name":"dw_protected","type":"int"},{"name":"alg","type":"string"},{"name":"key_id","type":"string"},{"name":"nonce","type":"bytes"},{"name":"ciphertext","type":"bytes"}]}Its parsing canonical form has CRC-64-AVRO fingerprint
0ba70c3fd6c311ca. The existing plain Value fingerprint remainse2a33dff55802237. Dispatch by the single-object fingerprint. Ordinary marker-shaped maps continue to decode asMapValue; onlyProtectedValueenters the decrypt path. Rejectdw_protected != 1, extra/invalid structure, a nonce other than 24 bytes, or ciphertext shorter than the 16-byte tag before key lookup. This also avoids nested protected branches.The read pipeline wording must be precise: base64 + single-object framing -> fingerprint dispatch -> decode the outer
ProtectedValuerecord -> validate/decrypt -> decode the authenticated inner bytes with the retained official Value writer schema. Protection is before application Value decode, but recognition necessarily decodes the outer Avro record. Current readers reject every unknown fingerprint (Python); v1 must retain both exact schemas.2. Canonical AAD bytes
The field list in the decision is not enough for cross-SDK authentication. I propose this exact v1 byte string:
All four strings must be valid UTF-8; no Unicode normalization or locale conversion.
algis exactly ASCIIxchacha20poly1305-ietf.key_idshould be restricted to nonempty nonsecret ASCII[A-Za-z0-9._:-]{1,128}.boundary_kindis a closed ASCII enum. Namespace is the exact namespace whose worker will decrypt the payload; cross-namespace calls must bind the target/callee namespace for inputs and the receiving/caller namespace for returned values.Use destination semantics consistently. In particular, direct, scheduled, child, and continue-as-new starts all arrive as workflow input. Either bind all four to
workflow_input, or preserve an authoritative origin discriminator through Server/task/history and use the same discriminator on decrypt. Encrypting aschild_workflow_inputand later decrypting generically asworkflow_inputcannot authenticate.Required substitution controls: same bytes under a different namespace; every other boundary enum; boundary case change; empty namespace; malformed UTF-8; reordered/omitted AAD field. All must refuse before inner Avro decode.
3. Actual payload-boundary inventory is incomplete
The current SDK has more durable payload surfaces than the accepted list: workflow side-effect results (Python), workflow/message stream items, scheduled workflow input, Nexus request/result, local/prepared activity reports, and external task results. These can carry the same business secrets as the named boundaries. V1 must either protect each with an exact boundary kind or list it explicitly as plaintext/unsupported and refuse protection configuration at that call site; it must not imply broad payload protection.
Heartbeat details are not an Avro payload today: Python sends the raw details object (current wire) and embedded storage keeps it as an array. The decision cannot claim protected heartbeat details without a coordinated envelope/storage migration. Failure details are likewise not uniform: activity failure details have an envelope path, while workflow/update/query failure structures contain plaintext diagnostics and do not all expose a details envelope.
Before implementation, publish one table per current call site: producer, stored/wire field, decrypting consumer, namespace source, boundary kind, inline/external support, and v1 protected/explicitly-plaintext status. Memos, search attributes, labels, names, queues, IDs, and failure message/stack text remain deliberately plaintext as already decided.
4. Retry, replay, and external storage rules
Generate the random nonce and protected blob exactly once after replay has selected a new admission/commit payload. Do not put randomness in generic replay-time Value serialization. Current Python forms
request_jsonbefore its transport retry closure (request path) and serializes workflow commands once before completion retries (worker path); preserve those properties in every SDK.Automatic retries must reuse the exact protected blob and request body. A manual/crash-recovery retry that reconstructs the same plaintext and
request_idwill produce a new random nonce/ciphertext and can conflict with #216's retained intent hash. The contract must choose one safe behavior: expose a persistable prepared protected request/blob that callers can resend, or explicitly scope byte-stable retry to the original call/process and remove any broader same-plaintext retry promise. Do not derive a nonce only fromrequest_id; reusing a request ID with changed plaintext would reuse the nonce under one key.Protect first, then threshold/offload the resulting outer Avro blob. Current external storage stores the base64 Avro blob bytes (Python); storing before protection would leak plaintext. On read: fetch and verify the external size/hash, then outer fingerprint/decrypt/inner decode. Retries must reuse the same uploaded object/reference. Define how Waterline/CLI learn
key_idfor an external reference: either an integrity-checked fetch or explicit nonsecret reference metadata. Until then they can honestly showprotected (external key unavailable), not invent a key or show an empty/corrupt value.5. Rotation, downgrade, and typed refusal
Each keyring entry is exactly 32 bytes. One active
key_idwrites; lookup by the envelope's exact ID reads. Reject duplicate IDs/config shadowing, and state that an ID must never be reassigned to different key bytes. Old IDs remain until every required history/external object has expired. Never try every key and never fall back from a recognized protected envelope to plain decode.The current three errors need two more decisions:
payload_protection_required, or define an equally typed strict policy. An explicit migration mode may accept old plaintext histories, but must be documented as lacking downgrade protection; it cannot be an automatic fallback.payload_plaintext_invalid; it is neither an unknown key nor an authentication failure.Error precedence should be: fingerprint/outer structure/version/alg ->
payload_envelope_unsupported; known v1 + unknown ID ->payload_key_unavailable; known ID + AEAD failure (ciphertext/key/AAD) ->payload_authentication_failed; successful AEAD + malformed inner frame -> the chosen plaintext-invalid error. Messages/telemetry must not include key bytes, plaintext, nonce, or ciphertext.For Python, the current
cryptographyAEAD API exposes 12-byte ChaCha20-Poly1305, not XChaCha20-Poly1305. The Python implementation should require PyNaCl's libsodiumcrypto_aead_xchacha20poly1305_ietf_*binding (or a future library API that explicitly implements XChaCha);cryptography or PyNaClis not currently an interchangeable choice.6. Minimal deterministic synthetic vector
Fixed key/nonce are test-only; production uses the OS CSPRNG. This single positive covers signed int64, null, Unicode, a nested map, inline transport, and the exact bytes later used as an external object.
Required negatives derived from that one vector:
2or other algpayload_envelope_unsupportedbefore key lookupkey_idpayload_key_unavailablepayload_authentication_failedtenant-bpayload_authentication_failedpayload_authentication_failedI generated and round-tripped the vector with fastavro 1.12.1 and PyNaCl 1.6.0/libsodium. General needs to settle the separate top-level schema, exact AAD, strict-vs-migration read policy, crash-retry contract, and the actual boundary table before implementation starts. Once settled, these bytes can be checked into the cross-SDK vector corpus unchanged.
rmcdaniel commentedon Nov 14, 2026
Contract decisions following the concrete Security review 6876199565:
durable_workflow.protocol.ProtectedValueAvro writer schema and fingerprint0ba70c3fd6c311cafrom that review. Retain the official plain Value schema/fingerprint unchanged. Recognition is by the framed writer fingerprint; marker-shaped user maps remain ordinary values. Decode only the outer record before authentication, then decode one complete authenticated official Value frame.workflow_input. Retain the supplied deterministic synthetic vector unchanged and its substitution/malformed-frame controls. Use established XChaCha20-Poly1305 libraries, including PyNaCl/libsodium for Python; do not implement cryptography.payload_protection_required. Explicit migration mode may read old plaintext at named boundaries and must state that it lacks downgrade protection. A recognized protected envelope never falls back to plaintext in either mode. Accept the review's error precedence and addpayload_plaintext_invalidafter successful AEAD with an invalid inner frame.These choices unblock a tracked v1 schema/AAD/vector contract and the per-call-site table. That table must bind producer, actual field, consumer, namespace source, boundary kind, inline/external behavior, and protected/unsupported status against current code. SDK implementation and installed/publication acceptance remain unfinished; no runtime, provider, real-key or release effect is selected here.
rmcdaniel commentedon Nov 14, 2026
The accepted v1 contract is now implemented for independent review in draft PR #399: #399
It fixes the distinct ProtectedValue schema/fingerprint, exact AAD bytes, deterministic vector and negatives, refusal precedence, key rotation/retry/offload rules, and a source-revision-bound supported/refused call-site table. The PR has no runtime/key-provider effect; SDK, Workflow and Server adoption remains unfinished and is not closed by this deliverable.