Skip to content

Remove crates/xmtp_sdk/conformance - #4458

Merged
neekolas merged 7 commits into
self-hostedfrom
nm/sdk-cleanup-6
Oct 7, 2026
Merged

neekolas merged 7 commits into
self-hostedfrom
nm/sdk-cleanup-6

Conversation

@neekolas

@neekolas neekolas commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

crates/xmtp_sdk/conformance no longer exists. Every durable proof moved and runs from a new recipe. A proof was rewritten only where it used an injected seam.

  • Bridge runtime unit tests moved to apps/xmtp_sdk_bindgen/runtime-tests/ts (just sdk test-bridge).
  • Chromium, real-WASM, storage, worker, package, logging, decode-once, stream-opening, pure-codec and panic proofs moved to sdks/browser/test/platform (just sdk test-browser).
  • Swift seam proofs are now XCTest cases (ReaderTeardownTests, one new ListenerGateTests case). They use @testable import and the existing fakes. The two Python injectors are deleted, so Swift CI no longer runs patched FFI glue.
  • Swift missing-bundle check and negative consumers moved to sdks/ios/Tests/Consumer (just ios check-consumer).
  • Kotlin negative consumers moved to sdks/android/library/src/test/negative (just android check-consumers).
  • Four CI double runs are removed.
  • The last sdk_conformance_* exports without a caller are deleted, with the conformance feature arms that only they used.

Targets self-hosted.

Changes

  • Moves (git renames):
    • 14 bridge-*.ts files, bridge.test.ts, codec-policy.test.ts, public-projection.test.ts, public-objects.test.ts, reader-consumer.test.ts, vitest.config.mts and logging-wait.ts go to apps/xmtp_sdk_bindgen/runtime-tests/ts/.
    • All other browser/* files, ts/logging-contract.ts and ts/transfer-control.mts go to sdks/browser/test/platform/.
    • The object-store relay fixture goes to sdks/browser/test/platform/object-store/.
    • Swift negative consumers go to sdks/ios/Tests/Consumer/Negative/. Kotlin negative consumers go to sdks/android/library/src/test/negative/.
  • sdk.just: the conformance* recipes are removed. New recipes: test-bridge, test-browser, test-browser-storage, test-browser-package.
  • Deleted drivers: conformance-browser, conformance-kotlin, conformance-swift, conformance-generate, stage-conformance-swift(.py), run-conformance-swift-app, check-swift-entrypoint, inject_reader_gate.py, inject_event_start_hook.py.
  • New just ios check-consumer and just android check-consumers. Both build against the repo with no patched copy. The Gradle check adds one negative file at a time to the unit test sources through the xmtpNegativeConsumer property and never writes to the tree.
  • CI:
    • test-sdk.yml: keeps the run_swift input and macOS swift job, which now runs just ios test-seams and just ios check-consumer. just android check-consumers runs in the android-stage job. The Kotlin JVM conformance step is removed. The other steps are renamed to "Bridge runtime unit tests" and "Browser platform proofs".
    • test.yml: keeps the sdk_swift filter and run_swift wiring. The sdk and sdk_swift filters list the moved sdks/ paths. sdks/browser/test/platform/** is added to the sdk filter.
    • test-ios.yml runs just ios test skip-seams so the seam proofs run once, in test-sdk.yml. test-ios.yml and test-android.yml no longer run the consumer checks.
  • Double runs removed: browser bridge/storage/package proofs (ran in two steps); test-sdk-artifacts.py (ran in two jobs); test-pure-wasm.mjs (ran in lint and package-scripts); bridge.real.mts and bridge.values.real.mts (ran again outside the generated test).
  • Lint scope: sdks/browser/tsconfig.json excludes test/platform, and .oxlintrc.json ignores it. These proofs need fixture builds under target/ that the JS lint job does not build. oxfmt --check covers them now.
  • Rust: deleted src/client/event_conformance.rs, conformance_queue_counts and the xmtp_events conformance feature. Dropped feature = "conformance" from the xmtp_mls builder.rs and server_configuration.rs cfgs. Deleted the native arms of sdk_conformance_emit and sdk_conformance_sink_error_count. The wasm32 arms and the decode-once and standard-sample exports stay, because moved proofs call them.
  • Flake fix in package.chromium.ts: a new waitForIdle() replaces a racy terminated === count wait ("worker termination missing: expected 8, got 9").
  • Generated Swift: StreamMethods.swift has two comment lines changed from // to ///. This is the dev/bindings output and matches its generator source. The drift came from feat(sdk): unify conversation and message stream methods #4438.
  • Docs: crates/xmtp_sdk/AGENTS.md, sdks/{browser,ios,android}/AGENTS.md, .github/workflows/AGENTS.md, apps/xmtp_sdk_bindgen/README.md, docs/specs/waivers.toml (PROC-041 waiver), docs/self-hosted/sdk-migration/handoff.md.

Deleted tests and their coverage

Old check Covering test
Swift checkReaderAppError (PROC-052, PROC-031, PROC-041) ReaderTeardownTests.testAppErrorEndsTheReaderWithoutAcknowledgingTheItem
Swift checkCooperativeReaderOpeningCancellation (PROC-041) ReaderTeardownTests.testCancellationReachesACooperativeOpen
Swift checkLateReaderOpeningCleanup (PROC-041) ReaderTeardownTests.testCloseWaitsForALateOpenToEndItsReader
Swift checkReaderReleasedBeforeIterationEnds ReaderTeardownTests.testIterationEndsAfterTheReaderIsReleased
Swift checkConversationReaderOpensBeforeDelivery ReaderTeardownTests.testConversationStreamDeliversAGroupCreatedAfterItsReaderOpens
Swift checkConnectionStateMonitor (PROC-044) ReaderTeardownTests.testStateMonitorStopsAtClosedAndReportsTheOpeningState
Swift checkDelayedListenerStop (EVENT-053) ListenerGateTests.testCallbackHeldAtItsStartDoesNotRunAfterStop
Swift --missing-bundle and 3 negative consumers sdks/ios/Tests/Consumer via just ios check-consumer
Kotlin negative consumers (2) just android check-consumers
Kotlin "conformance build compiles" program (no checks) None needed. The Android JVM unit tests cover the JNA cleaner branch.
Browser and bridge proofs Moved unchanged.

Each Swift port was shown to fail when its behavior was broken (for example, SDKReaderIterator.deinit {} fails the app-error test).

Spec backlinks moved:

ID From To
PROC-052, PROC-031, PROC-041 ReaderLoopExit.swift ReaderTeardownTests.testAppErrorEndsTheReaderWithoutAcknowledgingTheItem
PROC-041 ReaderOpeningCancellation.swift testCancellationReachesACooperativeOpen, testCloseWaitsForALateOpenToEndItsReader
PROC-044 ReaderSeams.swift testStateMonitorStopsAtClosedAndReportsTheOpeningState
EVENT-053 ReaderSeams.swift ListenerGateTests.testCallbackHeldAtItsStartDoesNotRunAfterStop

just spec-check is unchanged from the base: 0 errors, 34 warnings, 467 requirements with evidence.

Plan claims that proved wrong

  1. The object-store relay is not shared with Kotlin or Swift. Only the browser suite uses it, so it lives under sdks/browser/test/platform/object-store.
  2. Kotlin residue was only the JNA stand-ins, an empty main() and two negative consumers.
  3. Swift CallerCancellation was already handled by PR 2.
  4. There was a fourth CI double run (the real-WASM scripts).
  5. sdk_conformance_watch_text_decode and text_decode_count are not deleted. The decode-once proof needs them.
  6. The browser platform recipes stay in sdk.just. They need the Rust-shell fixture builds in crates/xmtp_sdk/dev.
  7. The Swift seam proofs did not need any runtime change.

Reviewer checks

  • Confirm that the "call after stop" form of the EVENT-053 port is equal to the old held-start hook.
  • The Swift reader ports call makeSDKMessageStream directly. StreamOwnerTests covers the owner lookup they skip.
  • test-ios and test-android do not gate merge. The Swift seam proofs and both consumer checks run only there.
  • The sdk-artifacts.py generator hash covers all of apps/xmtp_sdk_bindgen/, so a change to these tests asks for just sdk build before just sdk render.
  • nix/package/ios.nix line 1 still says "as conformance" in a comment. It is left to avoid a source-hash change.

Local verification

All passed at a6d57f74bc5bb545c0099ffc122614b32376904f:

  • cargo check -p xmtp_events -p xmtp_mls -p xmtp_sdk --tests
  • cargo check -p xmtp_sdk --features conformance and cargo check -p xmtp_mls --features conformance
  • cargo clippy -p xmtp_sdk -p xmtp_events -p xmtp_mls --features xmtp_sdk/conformance --no-deps -- -Dwarnings
  • cargo test -p xmtp-sdk-bindgen conformance_methods (2 passed)
  • just sdk build node,browser, just sdk generate node,browser, just sdk test-bridge (6 files, 761 tests)
  • NIX_DEVSHELL=js just sdk test-browser (all real-WASM and Chromium proofs, Playwright 9/9)
  • lint-generated tsc line for the three platform files
  • just ios build, then swift test --filter "XmtpSdkTests.(ReaderTeardownTests|ListenerGateTests)" (10 passed)
  • just ios check-consumer
  • just android build, then just android check-consumers
  • just spec-check, just lint-markdown, just lint-config
  • swiftformat --lint in sdks/ios; oxfmt --check on the moved TypeScript and .oxlintrc.json; oxlint . in sdks/browser
  • test-sdk-packaging.py -k test_bridge_fixture_builds_use_rust_shell

Not run locally: full just lint, just sdk lint, just test crate xmtp_sdk, and the full iOS, Android and JS suites. CI runs them.

🤖 Generated with Claude Code

Note

Remove crates/xmtp_sdk/conformance and relocate its tests into per-SDK suites

  • Removes the conformance test tree and splits it by platform: TypeScript bridge runtime tests move to apps/xmtp_sdk_bindgen/runtime-tests/ts, browser proofs move to sdks/browser/test/platform, and Swift/Kotlin negative consumers move into the iOS and Android SDK test sources.
  • Replaces just sdk conformance* recipes with test-bridge, test-browser, test-browser-storage, and test-browser-package in sdk.just. Adds new runners run-bridge-tests and run-browser-platform-tests under crates/xmtp_sdk/dev.
  • Adds iOS consumer checks (just ios check-consumer, test-seams) and Android consumer checks (just android check-consumers), including new listener-gate and reader-teardown Swift tests.
  • CI updates: test-sdk runs the new bridge/browser recipes and moves Swift seam proofs to the Swift job; test-ios skips seam proofs now run by the SDK workflow.
  • Risk: builds enabling only the conformance feature lose some exports — xmtp_events no longer declares conformance, Subscription::conformance_queue_counts and xmtp_sdk non-WASM sdk_conformance_emit/sdk_conformance_sink_error_count exports are removed, and ClientBuilder/ServerConfigurationHandle.with_application_components gates drop conformance (see Cargo.toml, bus.rs, logging.rs).

Macroscope summarized d536a0b.

neekolas and others added 4 commits October 6, 2026 17:27
Move the durable platform proofs to their owners and delete the directory.

- Bridge runtime unit tests, codec policy and projection tests move to
  apps/xmtp_sdk_bindgen/runtime-tests/ts (`just sdk test-bridge`).
- Chromium, real WASM, storage, package, logging, attachment, decode-once,
  stream-opening, pure codec and panic proofs and the object-store relay move
  to sdks/browser/test/platform (`just sdk test-browser`).
- Swift reader and listener seam proofs become XCTest cases that use the
  runtime fakes and @testable import. Delete inject_reader_gate.py and
  inject_event_start_hook.py, the Swift staging and the Swift CI job.
- Swift and Kotlin negative consumers and the missing-bundle check get
  `just ios check-consumer` and `just android check-consumers`.
- Remove the CI double runs, the conformance recipes and drivers, and the
  conformance exports and cfg arms that lost their last caller.
- Wait for an idle worker state in package.chromium.mts instead of a
  termination count that a late worker could pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- test-android.yml runs `just android check-consumers` after the unit
  tests. The old `just sdk conformance kotlin` step was the only CI run
  of the Kotlin negative consumers.
- ReaderTeardownTests puts a limit on every wait and reports a timeout:
  the first delivery, the awaits after a gate release, the end and
  failure reads, and each cancelled task.
- The Swift consumer executable exits 1 on an unexpected error instead
  of throwing at top level.
- The PROC-041 waiver names ReaderTeardownTests instead of the deleted
  conformance files.
- After the rebase onto #4425: format the merged imports, and say in the
  bindgen README that the runtime tests import the generated runtime.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#4438 made SDKClient.conversations a property and replaced
SDKClient.conversationStream with Conversations.stream(options:). Use the
new forms, and say in the doc comment that the public message streams
resolve their owner through ClientRegistry before they call
makeSDKMessageStream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#4438 committed the generated copy with // where the generator source
has ///, so every just ios check rewrote it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@neekolas
neekolas requested review from a team as code owners October 7, 2026 01:48
Comment thread .github/workflows/test-ios.yml Outdated
@macroscopeapp

macroscopeapp Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Skipped

Macroscope did not run approvability analysis for this PR. This PR modifies 1 file matched by one or more of your neverApprove patterns: .macroscope/**, docs/self-hosted/agent-context.md, docs/specs/**, .github/CODEOWNERS, apps/backend/migrations/**, crates/xmtp_db/migrations/**. Changes to these paths are never approved automatically.

You can add or adjust custom eligibility rules. Learn more.

@codecov

codecov Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.50000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 87.86%. Comparing base (98cb122) to head (d536a0b).
⚠️ Report is 2 commits behind head on self-hosted.

Files with missing lines Patch % Lines
crates/xmtp_mls/src/builder.rs 87.50% 1 Missing ⚠️
Additional details and impacted files
@@               Coverage Diff               @@
##           self-hosted    #4458      +/-   ##
===============================================
- Coverage        87.87%   87.86%   -0.01%     
===============================================
  Files              539      539              
  Lines           102659   102659              
===============================================
- Hits             90207    90204       -3     
- Misses           12452    12455       +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

The conformance removal moved the Swift seam proofs and both consumer
checks into test-ios and test-android, which do not gate merge. Before,
the Swift job and the Kotlin JVM conformance step ran them in test-sdk,
which gates the aggregate Test check.

- test-sdk.yml: restore the run_swift input and the macOS Swift job. It
  runs `just ios test-seams` (ReaderTeardownTests, ListenerGateTests)
  through `just backend ci`, then `just ios check-consumer`. The Android
  staging job runs `just android check-consumers`.
- test.yml: restore the sdk_swift filter and the run_swift wiring. The
  sdk and sdk_swift filters also list the moved proof and consumer paths
  under sdks/, which crates/** used to cover.
- test-ios.yml runs `just ios test skip-seams` and drops its consumer
  step; test-android.yml drops its consumer step. No CI run does the same
  check twice.
- ios.just: `test-seams` and `test skip-seams` share one pattern. A recipe
  test checks that they select and skip the same tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread .github/workflows/test.yml
… changes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread sdks/ios/Tests/XmtpSdkTests/ListenerGateTests.swift
The ported test only called onEvent after stopListener returned. It did not
hold a call that had started before stop. The generated trampoline starts an
unstructured task for each native onEvent call, so the started-but-not-run
window is a pending task. A test-only TaskExecutor now holds that task
before the runtime's onEvent runs. The test calls stopListener (and, for a
second held call, Client.end) while the call is held, releases it, and
checks that the callback did not run and that each call returned. The
shipped runtime is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@neekolas
neekolas added this pull request to stack #4461 October 7, 2026 02:59
@neekolas
neekolas merged commit 80e97b1 into self-hosted Oct 7, 2026
93 of 95 checks passed
@neekolas
neekolas deleted the nm/sdk-cleanup-6 branch October 7, 2026 03:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant