Skip to content

Fix EmailEvent recipient array element types in ASIM catalog - #15487

Closed
adbhogeGit wants to merge 1 commit into
masterfrom
adbhoge-microsoft-email-recipient-metadata
Closed

adbhogeGit wants to merge 1 commit into
masterfrom
adbhoge-microsoft-email-recipient-metadata

Conversation

@adbhogeGit

Copy link
Copy Markdown
Contributor

Required items, please complete

Change(s):

  • Set ArrayValuesType=string for exactly EmailToRecipients, EmailCCRecipients, and EmailBCCRecipients in ASIM/dev/ASimTester/ASimTester.csv, scoped to EmailEvent.
  • Preserve all other catalog content and existing CRLF line endings. No parser, MailLog diagnostics, or tooling changes.

Reason for Change(s):

Version Updated:

  • N/A: catalog metadata only; no detection or analytic-rule template changed.

Testing Completed:

  • PASS: focused offline PowerShell check below verifies the exact three-row metadata correction and unchanged remaining content (run before commit):
$ErrorActionPreference = 'Stop'
$path = 'ASIM\dev\ASimTester\ASimTester.csv'
$before = @(git show 'origin/master:ASIM/dev/ASimTester/ASimTester.csv')
$after = @(Get-Content $path)
$expected = @($before | ForEach-Object { if ($_ -cmatch '^Email(To|CC|BCC)Recipients,dynamic,(Optional|Recommended),EmailEvent,,,,array,$') { $_ + 'string' } else { $_ } })
if ($before.Count -ne $after.Count -or @(Compare-Object $expected $after -SyncWindow 0 -CaseSensitive).Count) { throw 'Unexpected catalog changes' }
$recipients = @(Import-Csv $path | Where-Object { $_.Schema -ceq 'EmailEvent' -and $_.ColumnName -cmatch '^Email(To|CC|BCC)Recipients$' })
if ($recipients.Count -ne 3 -or @($recipients.ColumnName | Sort-Object -Unique).Count -ne 3 -or @($recipients | Where-Object { $_.ColumnType -cne 'dynamic' -or $_.DynamicType -cne 'array' -or $_.ArrayValuesType -cne 'string' }).Count) { throw 'Invalid recipient metadata' }
'PASS: exact three-row metadata correction; all other catalog content unchanged.'
  • PASS: git -c core.whitespace=cr-at-eol --no-pager diff --check origin/master...HEAD. Plain whitespace checks flag the existing CRLF style; git ls-files --eol -- ASIM/dev/ASimTester/ASimTester.csv confirms both index and working file use CRLF.
  • BLOCKED: & .\ASIM\dev\ASimTester\Validate-ASimCsv\Validate-ASimCsv.ps1 -FilesPath .\ASIM\dev\ASimTester\ASimTester.csv exits 1 with "The file 'ASimTester' is not valid". Its required-header list contains seven columns, while the existing catalog contains nine, including DynamicType and ArrayValuesType. Validator left unchanged; updating it requires a separate maintainer task.
  • NOT RUN: live Microsoft Sentinel/canary schema or data tests. No Azure workspace was queried or changed. KQL/YAML checks are not applicable to this CSV-only change.

Checked that the validations are passing and have addressed any issues that are present:

  • Focused offline checks pass; the documented CSV validator remains blocked by its pre-existing header mismatch as described above. No claim of live workspace validation is made.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the scoped EmailEvent recipient-array metadata change.

No open issues 🎉

What changed in this PR

Corrects recipient-array metadata in the shared ASIM validation catalog so arrays of strings can pass element-type validation. This publishes the catalog fix independently of the Proofpoint parser work in #15464.

Changes:

  • Sets ArrayValuesType=string for EmailToRecipients, EmailCCRecipients, and EmailBCCRecipients, scoped to EmailEvent.
  • Leaves parser implementations and validation tooling unchanged.
File Description
ASIM/​dev/​ASimTester/​ASimTester.csv Declares string element types for the three EmailEvent recipient arrays.

Review effort: Balanced

@melmarsezio

Chencheng Xie (Microsoft) (melmarsezio) commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Suppose the counterpart QTP of ASimEmailEventProofpointPODMessage should also failed for the same ASimDataTester's test? Derrick Lee (@yummyblabla) Could you take a look at this PR and the test of the ITP flow: https://github.com/Azure/Azure-Sentinel/actions/runs/40780599305

@adbhogeGit

Copy link
Copy Markdown
Contributor Author

Closing because the original ASimDataTester validation failure was fixed by merged PR #15498: #15498 . The tester now skips array-element type checks when ArrayValuesType is unspecified, so this separate metadata change is no longer needed to unblock the parser. Message PR #15464 subsequently passed its full canary validation: https://github.com/Azure/Azure-Sentinel/actions/runs/40981833569 .

@adbhogeGit adbhogeGit closed this Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants