Skip to content

Preserve ASIM canary visibility diagnostics on shared deadline expiry - #15488

Draft
adbhogeGit wants to merge 1 commit into
masterfrom
adbhoge-microsoft-canary-timeout-diagnostics
Draft

adbhogeGit wants to merge 1 commit into
masterfrom
adbhoge-microsoft-canary-timeout-diagnostics

Conversation

@adbhogeGit

Copy link
Copy Markdown
Contributor

Required items, please complete

Change(s):

  • Enrich exact-row visibility shared-deadline errors with expected per-timestamp counts, the last observed per-timestamp counts, missing/undercounted timestamps, and the last query table where available.
  • Re-raise the original deadline exception with its stage, message, status, and cause intact. Keep actual HTTP/KQL and malformed-response errors unchanged.
  • Add six focused regression tests covering expiry between polls, no successful observation, settling, final-response expiry, query-error preservation, and exception/cause identity. Document the diagnostic fields.

Reason for Change(s):

  • The reported MailLog canary run for [ASIM] Add Proofpoint POD MailLog ingestion-time parser #15463 passed filter validation (25 fixture rows, 24 selected, 1 rejected), compiled its DCR, and uploaded successfully, but expired while waiting roughly 900 seconds for target visibility. Its artifact recorded 31 successful query requests followed by stage query, message Validation deadline expired., and null details.
  • The next query can hit the shared deadline before making a request, bypassing the polling loop's existing timeout diagnostics and discarding the last successful observation.
  • This PR fixes observability only. It does not resolve or claim to resolve the underlying canary ingestion timeout. It changes no parser YAML, user data, cloud resources, or catalog content.

Version Updated:

  • N/A: validation-harness code, tests, and directly related documentation only; no detection/analytic-rule templates or parser versions changed.

Testing Completed:

  • Yes: used Python 3.14.7 from the installed Azure CLI because the python command is a Windows Store alias.
  • Focused harness tests: python -m unittest discover -s ".script\tests\asimParsersTest\ingestTimeParser\tests" -p "test_asim_ingestion_harness.py" -- 27 tests passed.
  • Complete ingestion-time regression suite: python -m unittest discover -s ".script\tests\asimParsersTest\ingestTimeParser\tests" -p "test_asim_*.py" -- 158 tests passed.
  • Confirmed the between-polls regression test fails against the unchanged default-branch implementation because the deadline error has null details, and passes with this change.
  • python -m compileall -q ".script\tests\asimParsersTest\ingestTimeParser" and git diff --check passed.

Checked that the validations are passing and have addressed any issues that are present:

  • Yes: all local checks above passed. The test suite's intentional negative-path Output path must not replace the source fixture diagnostic is expected; the suite exits successfully.
  • No live canary run was performed and no cloud resources were modified. The shared-canary ingestion timeout remains unresolved; remote PR validation results are pending.
  • KQL/YAML content validation is not applicable because this PR changes no KQL/YAML content. The ingestion-time unit-test CI workflow defines no additional lint command.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
deadline=2,
)

def query_error(method, url, kwargs):
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants