ENH: Disable AI running --run-slow tests since take too long

[aylward]

Summary by CodeRabbit

• Documentation
  • Updated testing guidance to recommend running the fast test suite by default.
  • Clarified that slow and full test suites should be run explicitly when needed.

[coderabbitai]

Warning

Review limit reached

@aylward, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 38 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5ca9dd33-7c55-42ba-b0af-a929b36a9a9b

📥 Commits

Reviewing files that changed from the base of the PR and between a4bb612 and fe76d8d.

📒 Files selected for processing (2)

• AGENTS.md
• CLAUDE.md

Walkthrough

The agent guidance files now prohibit agents from running pytest --run-slow and pytest --run-all. They direct agents to run only the fast test suite.

Changes

Agent testing guidance

Layer / File(s)	Summary
Fast pytest suite guidance AGENTS.md, CLAUDE.md	The instructions prohibit interactive use of the slow and all-test pytest buckets. Agents should run the fast suite instead. User-run opt-in buckets remain available.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Suggested reviewers: copilot

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: preventing AI agents from running slow pytest tests.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Copilot]

Pull request overview

Updates agent-facing contributor documentation to discourage running slow pytest buckets interactively, aligning expected AI-agent behavior with practical session time constraints.

Changes:

• Adds guidance in CLAUDE.md to avoid running pytest with --run-slow / --run-all during interactive sessions.
• Adds a similar rule in AGENTS.md for agents to avoid slow/all test buckets in interactive sessions.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
CLAUDE.md	Adds a testing guideline discouraging --run-slow / --run-all during interactive sessions.
AGENTS.md	Adds an agent rule discouraging --run-slow / --run-all during interactive sessions.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 36.60%. Comparing base (ae78829) to head (fe76d8d).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #116   +/-   ##
=======================================
  Coverage   36.60%   36.60%           
=======================================
  Files          72       72           
  Lines        8510     8510           
=======================================
  Hits         3115     3115           
  Misses       5395     5395           

Flag	Coverage Δ
integration-tests	36.40% <ø> (?)
unittests	36.60% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (1)

CLAUDE.md:141

• In this doc you state the Python launcher should be py (and other examples use py -m pytest), but this new guidance calls out pytest --run-slow. To avoid mixed guidance (and to work reliably with the active venv), consider phrasing this as “running pytest with --run-slow/--run-all” rather than a bare pytest invocation.

- **Avoid `pytest --run-slow` (or `--run-all`) unless the user explicitly
  requests it.** Those runs take far too long for an interactive session.
  Default to the fast suite and let the user invoke the slow buckets.