Skip to content

Add R Statistics agent and harden deployment validation - #266

Merged
yousefi-msft merged 3 commits into
microsoft:mainfrom
mukesh-dua:main
Oct 27, 2026
Merged

yousefi-msft merged 3 commits into
microsoft:mainfrom
mukesh-dua:main

Conversation

@mukesh-dua

Copy link
Copy Markdown
Collaborator

Summary

Add a CPU-only R Statistics prompt agent with a reproducible R 4.5.3 / ggplot2 4.0.0 container, pinned dependencies, synthetic usage example, documentation, and container regression tests. Harden the Discovery agent deployer's validation so confirmation-only replies and connectivity responses cannot be mistaken for successful tool execution.

The deployer now correlates tool calls with outputs, requires measured execution receipts for code environments, detects pending approvals without self-approving, preserves the original prompt on retries, resumes the existing conversation when appropriate, and retains failed investigations and response snapshots for diagnosis.

Type of change

  • New agent (under agents/<name>/)
  • Update to an existing agent
  • New starter kit (under starter-kits/<name>/)
  • Update to an existing starter kit
  • Schema change (docs/schemas/**)
  • Workflow / script change (.github/**)
  • Documentation only
  • Other (describe below)

Related issue / tracking

n/a

Schema impact

  • No schema changes
  • Schema changes are backward compatible
  • Schema changes are breaking — add the breaking-change label and link the rollout plan below

Validation checklist

  • Documentation (README.md, schemas) is updated to match the change.
  • No hand-edits to .auto-registry/** (it is generated post-merge).
  • No OS / editor artefacts committed (.DS_Store, .env, *.swp, .idea/, .vs/, etc.).
  • If model-weight files are added, they are Git-LFS tracked, ≤5 GB each, and in an allowed format.
  • No secrets, customer data, or internal-only URLs are committed.
  • All Markdown links resolve.

Validation performed on Windows after pulling the latest fork main:

  • python -B -m unittest discover -s .github\tests -p test_agent_deployer_validation.py -v — passed, 21 tests.
  • python -B -m unittest discover -s agents\r-statistics-agent\tools\r-statistics -p "test_*.py" -v — passed, 7 Linux-container integration tests against the previously built r-statistics:1.1.0 image.
  • $Errors = $null; [System.Management.Automation.Language.Parser]::ParseFile((Join-Path (Get-Location) '.github\skills\discovery-services-agent-deployer\scripts\deploy-discovery-agent.ps1'), [ref]$null, [ref]$Errors) | Out-Null; if ($Errors.Count) { $Errors; exit 1 } — passed, no PowerShell parse errors.
  • pwsh -NoProfile -File .github\skills\discovery-services-agent-deployer\scripts\deploy-discovery-agent.ps1 r-statistics-agent -WhatIfPlan — passed.
  • python -B .github\scripts\validate_pr.py --changed-files $List --repo-root . --output $Result — passed for all staged files; $List and $Result were temporary files, removed after validation. The changed-file list was populated using git diff --cached --name-only. One nonblocking POL-019 warning: the publisher-email DNS check could not run because dnspython is not installed.
  • git diff --cached --check — passed.

The full repository test suite and a complete external Markdown-link check were not run. No model weights or binary output artifacts are included.

Reviewer notes (optional)

  • The image runs as UID/GID 10001, uses an immutable base-image digest, installs version-pinned dependencies at build time, and does not install packages at runtime.
  • Only the Dockerfile, package manifest, and installer enter the image build context. Tests and synthetic examples remain source-only; edge-case checks are part of the existing test module.
  • Human-in-the-loop and tool confirmation remain enabled in the contributed agent definition.
  • Code-environment validation now requires a DISCOVERY_VALIDATION_RESULT= receipt in actual tool stdout plus a platform output-resource URI. This intentionally rejects deployments that previously passed on assistant text alone; the format and approval/resume workflow are documented in the deployer references.
  • Local tenant/workspace configuration, deployment checkpoints, response snapshots, and generated outputs are excluded from this contribution.

Include a pinned non-root R tool image, reproducible examples and regression tests. Require real execution evidence during deployment validation, preserve diagnostics, and handle approval continuation without self-approving.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mukesh-dua
Mukesh Dua (mukesh-dua) requested a review from a team as a code owner October 27, 2026 21:00
@github-actions github-actions Bot added 1p-contribution Authored by Microsoft (party='1p' in metadata) contains-code PR includes source-code changes contains-dockerfile PR adds, modifies, renames, or removes a Dockerfile pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately. labels Oct 27, 2026
Ensure clean CI environments can collect the deployment validator regression tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mukesh-dua

Copy link
Copy Markdown
Collaborator Author

Fixed the test collection failure by declaring requests==2.32.5 in .github/requirements-ci.txt. The new deployer regression tests import requests; the package was present locally but absent from the clean CI environment. Verified in a new Python 3.12 virtual environment: pip install -r .github\requirements-ci.txt succeeded; python -B -m pytest .github\tests\test_agent_deployer_validation.py -v -ra passed (21 tests); python -B -m pytest .github\tests --collect-only -q succeeded (907 tests, no collection errors). Full-suite execution remains for the rerun in CI.

Keep the exact pinned dependency assertion aligned with the deployer test dependency. Verified the full Linux Python 3.12 suite: 907 tests and 11 subtests passed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mukesh-dua

Copy link
Copy Markdown
Collaborator Author

Updated the exact dependency-set regression guard to include requests, preserving the existing strict pinning and workflow-manifest checks. Verified the entire suite in a disposable Linux container with Python 3.12, Node.js, and all dependencies installed from .github/requirements-ci.txt: python -B -m pytest .github/tests/ -q -ra -n 4 --dist worksteal -p no:cacheprovider — 907 passed, 11 subtests passed. git diff --check also passed. This supersedes the earlier collection-only verification.

@faranm-stack faranm-stack left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve

@yousefi-msft yousefi-msft left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

@yousefi-msft
yousefi-msft merged commit b11461d into microsoft:main Oct 27, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1p-contribution Authored by Microsoft (party='1p' in metadata) contains-code PR includes source-code changes contains-dockerfile PR adds, modifies, renames, or removes a Dockerfile pr-validation-passed The pr-review workflow's validator passed. Other status checks report separately.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants