Skip to content

testing_backlog.md

Latest commit

 

History

History
51 lines (38 loc) · 2.18 KB

testing_backlog.md

File metadata and controls

51 lines (38 loc) · 2.18 KB
Raw

Testing Backlog

Purpose

Keep the remaining test work small, explicit, and prioritized.

This file tracks the highest-value gaps between:

  • the current Arbiter implementation
  • the documented SMTP and IMAP contracts
  • the tests already in place

Status legend

  • done: implemented and covered by tests
  • todo: agreed gap, not implemented yet
  • blocked: needs an implementation seam or design decision first

P1

  • Improve CI performance with dependency caching

    • Why: platform CI is now broad enough that repeated setup dominates runtime, especially Windows arm64
    • Level: CI
    • Status: todo
    • Acceptance: Windows arm64 caches vcpkg/OpenSSL setup or uses vcpkg binary caching; nox/pip editable install overhead is reduced where practical; cache keys include dependency inputs so stale toolchains are not reused silently; and CI job logs make cache hits/misses obvious.

  • Investigate macOS SMTP integration CI slowdown

    • Why: full platform CI passes, but macOS x64/arm64 spend roughly 15 minutes between server/tests/unit/test_version.py and plugins/smtp/tests/integration/test_smtp_integration.py, while the same SMTP integration file runs locally on Linux in under a second.
    • Level: CI + integration
    • Status: todo
    • Acceptance: macOS CI captures enough per-test timing or server lifecycle diagnostics to identify the slow SMTP integration test or fixture phase; the root cause is documented; and the fix either removes the macOS-only delay or marks the remaining runtime as an accepted platform cost with evidence.

  • verify_peer=true succeeds against a trusted local CA

    • Why: complete the TLS success-path contract
    • Level: integration
    • Status: blocked
    • Note: current client has no custom CA injection seam

P2

  • Logging coverage for connection attempt, submission result, and failure paths

    • Why: operational debugging and auditability
    • Level: unit
    • Status: blocked
    • Note: logging contract is documented but not implemented yet

  • Normalized error-code mapping tests

    • Why: locks down external API semantics
    • Level: unit + integration
    • Status: blocked
    • Note: docs define the error model, but the implementation still surfaces raw exceptions