Skip to content

Fix Agent365 non-chat inference export and cross-tenant replay - #316

Merged
Nikhil Navakiran (nikhilNava) merged 3 commits into
mainfrom
copilot/a365-export-replay-parity
Nov 5, 2026
Merged

Nikhil Navakiran (nikhilNava) merged 3 commits into
mainfrom
copilot/a365-export-replay-parity

Conversation

@nikhilNava

@nikhilNava Nikhil Navakiran (nikhilNava) commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Export canonical text_completion and generate_content inference spans through Agent365; retain existing identity requirements, unrelated-operation filtering, and legacy-name behavior.
  • Continue durable replay for other identities after a retryable failure. Scan at most 100 records and attempt at most 10 per pass, releasing skipped/unattempted leases and preserving gate backoff, fatal-error handling, shutdown cleanup, and immediate draining of healthy backlogs.
  • Add regressions for inference export, cross-tenant recovery, blocked prefixes, scan/attempt bounds, periodic backoff, shutdown, and real-file backlog replay.

Gate eviction is intentionally out of scope. The 100-record scan window is bounded, not an unbounded fairness guarantee.

Validation

Executed locally on Windows with Python 3.11, using this branch's src on PYTHONPATH:

  • python -m pytest -q tests\a365 --ignore=tests\a365\integration: 711 passed, 21 skipped, 54 subtests passed.
  • Final focused run of tests\a365\test_utils.py, test_exporter.py, test_replay_coordinator.py, and test_durable_restart.py: 140 passed, 2 skipped, 17 subtests passed.
  • Confirmed the new non-chat and replay-isolation regressions fail before the corresponding production fixes.
  • python -m black --check on all six modified Python files: passed.
  • python -m pylint --output-format=parseable on both modified source files and all four modified test files (using the test allowances from tox.ini): passed, 10.00/10.
  • python -m mypy --python-version 3.10 --show-error-codes --ignore-missing-imports on modified source files, plus --check-untyped-defs --follow-imports=silent on modified test files: passed.
  • git diff --check: passed.

The full Agent365 run reports one pre-existing PytestUnhandledThreadExceptionWarning from A365EnrichingBatchSpanProcessor (cannot release un-acquired lock). Reproduced the identical warning on unchanged main: 704 passed, 21 skipped, 49 subtests passed. No unrelated processor changes are included.

Not run locally: other Python interpreters, Linux/macOS, or credential-dependent live-service integration tests.

Allow canonical non-chat inference spans through the Agent365 exporter. Continue replay after retryable failures and scan up to 100 records while limiting each pass to ten attempts, preserving backoff and lease cleanup.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a5430c45-2827-45bb-8b18-7ee88f4bec61
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a5430c45-2827-45bb-8b18-7ee88f4bec61

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the changes improving Agent365 inference-span export and durable replay isolation.

No open issues 🎉

What changed in this PR

This PR fixes two Agent365 export gaps: canonical non-chat inference spans are now accepted, and durable replay can continue processing healthy identities when another identity is retrying or blocked.

Changes:

  • Accept text_completion and generate_content while preserving legacy-name filtering.
  • Scan up to 100 queued records and attempt at most 10 deliveries per replay pass, releasing unattempted leases.
  • Add regression coverage for identity isolation, replay limits, backoff, shutdown, and persisted backlogs.

No actionable findings. No validation commands were run during this review.

File Description
tests/​a365/​test_utils.py Tests canonical inference-operation filtering.
tests/​a365/​test_replay_coordinator.py Tests cross-tenant recovery, replay bounds, backoff, and lease cleanup.
tests/​a365/​test_exporter.py Verifies canonical operations export unchanged and legacy names remain filtered.
tests/​a365/​test_durable_restart.py Tests multi-pass replay of a real-file backlog.
src/​microsoft/​opentelemetry/​a365/​core/​exporters/​utils.py Adds canonical non-chat inference operations to accepted spans.
src/​microsoft/​opentelemetry/​a365/​core/​exporters/​replay_coordinator.py Separates scan and attempt limits and continues past retryable failures.
CHANGELOG.md Documents both Agent365 fixes and replay limits.

Review effort: Balanced

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Performance comparison

Threshold: regressions >15.0% on gating scenarios fail the build. Higher ops/s is better; positive Δ means the PR is slower.

Scenario Gating Baseline (ops/s) Candidate (ops/s) Δ % Status
azure_monitor_log yes 28,198.4 28,119.1 +0.28% ✅
azure_monitor_span yes 163,692.9 162,074.6 +1.00% ✅
otel_log no 32,418.1 32,556.3 -0.42% ✅
otel_span no 47,639.5 47,076.5 +1.20% ✅

Copilot AI balanced review requested due to automatic review settings November 5, 2026 16:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the Agent365 export and durable replay changes.

No open issues 🎉

Review effort: Balanced

@nikhilNava
Nikhil Navakiran (nikhilNava) merged commit a26ac00 into main Nov 5, 2026
16 checks passed
@nikhilNava
Nikhil Navakiran (nikhilNava) deleted the copilot/a365-export-replay-parity branch November 5, 2026 16:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants