Skip to content

CI — add optional live MLB integration smoke workflow #17

Description

@Mattsface

Goal

Add a separate live integration workflow that validates the application against the real MLB Stats API without making external network access part of deterministic PR CI.

Why

The normal test suite should remain offline and reproducible. At the same time, Milestone 4 now depends on behavior that only a live MLB run can verify: season-aware team discovery, full league ingestion, completed-game coverage checks, and second-run idempotency. A lightweight opt-in CI workflow gives us repeatable upstream validation without making pull requests flaky when MLB is unavailable.

Scope

  • Add a dedicated GitHub Actions workflow for live MLB smoke validation
  • Keep the existing PR test workflow unchanged and offline
  • Trigger the live workflow manually with workflow_dispatch first
  • Consider a low-frequency schedule only if it proves useful and reliable
  • Use Python 3.12 and the existing Poetry environment
  • Apply Alembic migrations to a fresh temporary SQLite database
  • Run the 2025 league import against the real MLB API
  • Prefer --format json so the workflow can validate structured results
  • Require the live run to report:
    • status == COMPLETE
    • teams_failed == 0
    • teams_succeeded == teams_discovered
  • Do not use 30 teams, 162 games/team, or 4860 team-game records as completeness rules
  • Report those familiar 2025 counts only as diagnostics/sanity information when useful
  • Run the exact 2025 import a second time and verify idempotency:
    • inserted == 0
    • updated == 0 when upstream data is unchanged
    • unchanged == team_game_records_fetched
  • Preserve useful logs for diagnosing the exact team/error if the run becomes INCOMPLETE
  • Document that a real MLB outage or upstream schema change can fail this workflow without implying the offline application test suite is broken

Load / reliability constraints

A league import performs substantially more network work than a normal unit test, so this workflow should not run on every push or pull request. Keep it manual initially and avoid aggressive scheduling or retries.

Optional follow-up

After the historical 2025 smoke workflow is stable, consider an informational current-season run to exercise in-progress-season behavior. A current-season row count must never be treated as a fixed completeness target.

Out of scope

  • Replacing offline pytest coverage with live tests
  • Making live MLB access a required merge check
  • Hardcoding current team counts or games-per-team as integrity rules
  • Adding deployment infrastructure
  • Adding M5 analytics behavior to the workflow

Completion criteria

  • A developer can manually trigger the live integration workflow from GitHub Actions
  • It provisions a fresh database, performs a real 2025 league import, validates COMPLETE coverage, reruns the import, and verifies idempotency
  • Failures expose enough context to distinguish application regressions from upstream/network failures
  • Existing PR CI remains deterministic and network-independent

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions