Skip to content

[PR-B/Phase 4] Add .github/codecov.yml with informational thresholds #161

Open

Description

@BillyOutlast

Problem

Codecov is purely informational across the board:

  • fail_ci_if_error: false ×3: ci.yml:174, server-ci.yml:125, .github/actions/rust-ci/action.yml:122
  • No .github/codecov.yml exists
  • Current baseline: 29.32% line coverage (server backend, no gates per AGENTS.md)
  • Codecov upload failures silently swallowed

Coverage drops do not block merges; Codecov infrastructure failures don't block either.

Fix

4a. Create .github/codecov.yml

Keep Codecov informational for now (baseline 29.32% can't support strict thresholds). Make coverage changes visible via patch-level thresholds.

# .github/codecov.yml
coverage:
  status:
    project:
      default:
        target: auto
        threshold: 2%
        base: auto
        informational: true   # Keep informational until coverage >50%
    patch:
      default:
        target: 80%
        informational: true   # Patch coverage shows red comment, non-blocking
  flags:
    individual:
      - name: server
        paths:
          - server/
comment:
  layout: "diff, flags, files"
  behavior: default

4b-d. KEEP fail_ci_if_error: false everywhere

Do NOT flip to true because:

  • Codecov upload failures are infrastructure (API down, token expired), not coverage
  • Flipping would block PRs when Codecov has transient issues
  • The informational: true in codecov.yml already makes coverage changes visible

Current locations to leave unchanged:

  • .github/workflows/ci.yml:174
  • .github/workflows/server-ci.yml:125
  • .github/actions/rust-ci/action.yml:122

Future Work (Out of Scope for This Issue)

When overall coverage exceeds 50%, revisit:

  • Flip informational: false on patch status (block merge on new-code coverage <80%)
  • Add informational: false on project status (block merge on >2% drop)
  • Flip fail_ci_if_error: true once Codecov SLA is verified

Track in a separate issue once coverage milestone is hit.

Verification Gate

# PR that drops coverage by 3% on a patch
git checkout -b remediation/test-coverage-drop
# Modify a covered file to skip tests
git commit -am "test: drop coverage"
git push origin remediation/test-coverage-drop
gh pr create --base develop --head remediation/test-coverage-drop \
  --title "test: coverage drop" --draft

# Expected: Codecov comment appears with "Red" on patch coverage
# Expected: PR is NOT blocked from merge
gh pr view --json mergeable
# Should be: MERGEABLE

Risk

LOW. Configuration-only change. No semantic change to gating.

Rollback

rm .github/codecov.yml
git commit -am "revert: codecov config"
git push

Effort

~45 min

Provenance

Adversarial 4-member hyperplan, 3 rounds. Phase 4 of umbrella.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions