Skip to content

ci: improve test gates, browser auth and assembled coverage - #110

Merged
morajabi merged 4 commits into
mainfrom
codex/ci-test-confidence
Sep 30, 2026
Merged

morajabi merged 4 commits into
mainfrom
codex/ci-test-confidence

Conversation

@morajabi

@morajabi morajabi commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

CI repeated expensive Apple/Rust/protocol work while omitting browser package suites and meaningful assembled message checks. The new browser tests also exposed login credentials disappearing on reload. This PR reduces redundant validation, adds stable aggregate results and exercises built consumers against real local transport.

  • Add always-run Apple, integration and server aggregate checks that reject failures, cancellations, missing dependencies and unintended skips. Test exact workflow inventories. Conservatively skip five Apple lanes only for PRs entirely within known unrelated paths; main/manual runs remain full.
  • Consolidate AMD64 Rust validation into the all-target workspace job, retain unique ARM64 native tests plus both musl executable/installer checks, pin Rust and fix the cache. Preserve duplicate protocol workflows outside the active directory; Server Tests retains drift validation.
  • Run all CI helpers and four omitted browser package suites, cancel superseded integration runs, and bound job durations.
  • Verify the candidate source SHA and seven tarball hashes. Exercise SDK WebSocket recipient delivery, idempotency and new-connection history; real Chat SDK update dispatch, persisted replies, webhook authentication/deduplication; compiled MCP send/history and session grant revocation; existing real Hermes transport. Retain scenario receipts.
  • Add four Chromium tests against the built browser app, with external HTTP/WebSockets blocked: login routing, email challenge, reload persistence, legacy-session migration and missing-details guards. Wire the existing browser persistence adapter and await login persistence. Preserve credentials on ambiguous realtime auth failures; delete only on explicit session revocation, with regression tests.
  • Test every v2 MCP tool's missing-scope boundary and failed-send reporting. Publish MCP coverage as explicitly informational while retaining the strict default coverage command.
  • Remove five nonexistent native scheme references and validate all shared scheme targets. Fix a hosted Swift watchdog test to observe durable provider milestones instead of two 50 ms transient snapshots; retain the 50 ms product watchdog and existing Stop-during-reconnect coverage.
  • Add scheduled published OpenClaw/latest-host installation and registration qualification.

Validation: all 38 hosted PR checks passed for f2194e404 (tested merge 793fbc6dc). Both iOS/macOS Debug and Release builds, all 12 Swift packages, CodeQL, the three new aggregates, server container/migration/HTTP/WebSocket/drain qualification, both CLI architectures, installed OpenClaw/Hermes matrices and Node 20/26 packed consumers are green. Server reports account for 441 files, 3,141 executed tests and zero skips. Retained artifacts confirm all ten assembled scenarios and four Chromium cases, with zero browser skips or retries. Local validation also passed 51 CI helper tests, 231 browser-client tests, 143 MCP tests, 27 additional browser-package tests, frozen install, builds/typechecks/lints, source contracts and actionlint 1.7.10. The scheduled published OpenClaw path was smoke-tested locally: published plugin 0.0.70 installed and registered through actual host 2026.9.7 in an isolated home.

Bounds: browser APIs/OAuth introspection are controlled fixtures; Chat SDK webhook delivery is fixture-driven. Production auth/webhook behavior, real OpenClaw message handling, physical-device/native UI acceptance and deeper fault injection remain separate qualification. The browser shell has no chat composer. MCP coverage currently measures 78.46% lines and 85.42% functions; strict 95% thresholds remain unmet and unchanged, and are explicitly not enforced by CI yet. Configure the three new aggregate checks as required only after successful main runs following merge. No deployment or release is included.

@morajabi morajabi changed the title ci: improve validation efficiency and assembled message coverage ci: improve test gates, browser auth and assembled coverage Sep 30, 2026
@morajabi
morajabi marked this pull request as ready for review September 30, 2026 10:54
@morajabi
morajabi merged commit e061f6d into main Sep 30, 2026
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant