Skip to content

fix(ci): isolate HTTP end-to-end suites and stop the SCIM readiness flake - #8544

Merged
waleedlatif1 merged 2 commits into
stagingfrom
fix/ci-scim-flake
Oct 1, 2026
Merged

waleedlatif1 merged 2 commits into
stagingfrom
fix/ci-scim-flake

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Flake root cause: the SCIM step starts next dev and waits for /api/health, whose first request cold-compiles the app under Turbopack. Across 40 recent green runs readiness took 42–93s (step totals up to 209s), and today's staging push hit >120s — the fixed 120s deadline failed on the slow tail. The server log was never uploaded, so failures were undiagnosable.
  • New End-to-end over real HTTP job holds SCIM + the four search HTTP suites, on its own Postgres (provisioned via db:migrate, the production path). Previously SCIM ran on both integration matrix legs, which are the PR critical path (~13 min avg vs 7.6 min Lint and Test); the search suites were gated by matrix.provision == 'push'. Isolation also matters: the SCIM app boots hosted, which starts background usage replay against DATABASE_URL.
  • Readiness: 300s deadline (hang catcher only), step timeout-minutes: 12, fail-fast on server exit, server log tail printed inline, and the full scim-next.log uploaded with the reports (one artifact instead of five upload steps).
  • setup-workspace composite action replaces four copies of Bun/Node/cache/install setup. Cache keys are unchanged (existing sticky disks are reused); the integration legs now also mount the lockfile-keyed node_modules disk.
  • Removed a duplicated comment paragraph in the base-ref step.

Expected effect: ~2 min off each integration leg (the PR critical path), SCIM run once instead of twice, and the readiness flake gone.

ci-score (StarSling) rates the repo 91/100 before and after; its one failing check (fetch-depth: 0 in helm.yml) is a false positive — ct diffs against the PR base.

Test plan

  • actionlint — no new findings
  • CI on this PR: new End-to-end over real HTTP job green; integration legs no longer run SCIM

…lake

The SCIM step cold-compiles the app under next dev before its first
request; readiness took 42-150s on 8 vCPU runners against a fixed 120s
deadline, so the slow tail failed. It also ran on both provisioning legs
of the integration matrix, which are the PR critical path (~13 min).

- Move SCIM and the four search HTTP suites into their own job with its
  own database, off the integration legs and run once against migrate.
- Readiness waits up to 300s, fails fast if the server exits, prints the
  server log tail, and uploads the server log with the reports.
- Factor Bun/Node/cache/install setup into a setup-workspace action.
@vercel

vercel Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Oct 1, 2026 9:33pm UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

[Medium risk] Refactors CI workflow setup into a reusable action.

The PR should not merge until the SCIM step has enough time for both a slow startup and the full suite.

Findings

  1. P1 SCIM step can time out ▶
  2. P2 SCIM CI guide is outdated ▶

Summary

The PR moves SCIM and four search HTTP suites into a dedicated, migration-provisioned job, extracts shared workspace setup into a composite action, and extends SCIM readiness while collecting failure logs. The SCIM step’s new timeout may truncate a slow suite, and the existing SCIM CI guide needs to reflect the move.

Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  CI[Test and Build workflow] --> Integration[PostgreSQL integration: push and migrate]
  CI --> HTTP[HTTP end-to-end: migrate]
  CI --> Lint[Lint and Test]
  CI --> Build[Build App]
  HTTP --> Search[Four search suites]
  Search --> SCIM[Readiness wait and SCIM suite]
Loading

Reviews (1) · Last reviewed commit: "fix(ci): isolate HTTP end-to-end suites ..."

Comment thread .github/workflows/test-build.yml Outdated
Comment thread .github/workflows/test-build.yml
@waleedlatif1
waleedlatif1 merged commit efca99c into staging Oct 1, 2026
31 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/ci-scim-flake branch October 1, 2026 21:48

This branch was successfully deployed

1 active deployment
Preview — 1c352c9a Deployed Oct 1, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant