Skip to content

[ASIM] Add Proofpoint TAP permitted-click ingestion-time EmailEvent parser - #15501

Draft
nithil (nithillsethupathi) wants to merge 2 commits into
masterfrom
asim/emailevent-proofpoint-tap-clicks-permitted-ingestion
Draft

nithil (nithillsethupathi) wants to merge 2 commits into
masterfrom
asim/emailevent-proofpoint-tap-clicks-permitted-ingestion

Conversation

@nithillsethupathi

Copy link
Copy Markdown
Contributor

Required items, please complete

Change(s):

  • Add the first ingestion-time Proofpoint TAP permitted-click parser under Parsers/ASimEmailEvent/IngestTimeParsers/, targeting ASimEmailEventLogs.
  • Normalize ProofPointTAPClicksPermittedV2_CL records as EmailInteraction, preserving one output per source row, optional event IDs, click-time fallback, threat context, and packed additional fields.
  • Add a dedicated ingestion-time changelog. The service filename and ParserName use the IngestTime suffix so this definition has its own filename-matched history without modifying the merged query-time parser's changelog.
  • Add the source-table-specific fixture required by ingestion-time CI. Its ten rows are an exact copy of the synthetic fixture already published with Add ASIM EmailEvent parsers for Proofpoint TAP permitted clicks #15483.
  • Leave query-time parsers, unifiers, ARM resources, connector definitions, and tooling implementation unchanged.

Reason for Change(s):

  • Provide the ingestion-time counterpart to the query-time Proofpoint TAP parser merged in Add ASIM EmailEvent parsers for Proofpoint TAP permitted clicks #15483.
  • Use the connector-backed Standard route and verified raw-table output stream Custom-ProofPointTAPClicksPermittedV2_CL.
  • Keep this PR draft while service integration and runtime behavior are reviewed. The parser expects the connector-transformed raw-table shape, including TimeGenerated; the connector currently adds that column using source | extend TimeGenerated = now(). This normalization query must not replace that preparatory transform on its own.

Version Updated:

  • New ingestion-time definition: 0.1.2, initialized from the merged query-time parser version, with EmailEvent normalization schema 1.0.0.
  • No existing parser versions were changed.

Testing Completed:

  • Passed: existing ingestion-target discovery resolves exactly one target, automatically pairs it with the published query-time parser by source/product/schema, and finds the source-specific fixture.

    python '.\.script\tests\asimParsersTest\ingestTimeParser\asim_ingest_time_validation.py' discover --repo-root . --changed-file 'Parsers\ASimEmailEvent\IngestTimeParsers\ASimEmailEventProofpointTAPClicksPermittedIngestTime.yaml' --output $ManifestResult
  • Passed: existing ingestion-time static validator; valid=true, no findings, and query-time execution metadata eligible.

    python '.\.script\tests\asimParsersTest\ingestTimeParser\asim_ingest_time_validation.py' validate 'Parsers\ASimEmailEvent\IngestTimeParsers\ASimEmailEventProofpointTAPClicksPermittedIngestTime.yaml' --repo-root . --query-time-path 'Parsers\ASimEmailEvent\Parsers\ASimEmailEventProofpointTAPClicksPermitted.yaml' --output $StaticResult
  • Passed: existing offline fixture preparation, including source-table identity, typed conversion, and timestamp rebasing. This does not execute KQL or ingest records.

    python '.\.script\tests\asimParsersTest\ingestTimeParser\asim_fixture_preparation.py' prepare '.\Sample Data\ASIM\Proofpoint_TAP_EmailEvent_ProofPointTAPClicksPermittedV2_CL_IngestedLogs.csv' --expected-source-raw-table ProofPointTAPClicksPermittedV2_CL --target-id asimemaileventproofpointtapclickspermittedingesttime --target-group local-publication --repository Azure/Azure-Sentinel --run-id local-publication-20261029 --output $FixtureResult
  • Passed: decoded publication query exactly matches the generated DCR payload. Only the service identity differs from the external authoring YAML. The synthetic CSV matches the previously published fixture byte-for-byte.

  • Passed: whitespace validation.

    git -c core.whitespace=blank-at-eol,blank-at-eof,space-before-tab,cr-at-eol diff --cached --check
  • Not run locally: workspace queries, parser-DCR execution, raw replay, ASIM runtime data checks, query-time parity, or deployment. No production/customer data or customer-derived samples were used.

  • Query-time KQL/template tests and ARM generation are not applicable to this ingestion-only change. The existing ingestion-specific checks above were used instead.

Checked that the validations are passing and have addressed any issues that are present:

  • Local discovery, static validation, offline fixture preparation, payload fidelity, and whitespace checks passed.
  • Need review before ready-for-review/publication: confirm the consuming service retains the connector's preparatory transformation before this raw-table-shape normalization, and complete synthetic-runtime validation.
  • Hosted checks are pending. Static success is not a claim of semantic equivalence, successful service composition, or deployment readiness.

nsethupathi added 2 commits October 29, 2026 19:05
Add the service definition, its dedicated changelog, and the synthetic source-specific fixture. Keep query-time content and tooling unchanged; integration and runtime checks remain outstanding.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 880d7e2b-26c6-46a3-8f1d-adfc0579f59e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 880d7e2b-26c6-46a3-8f1d-adfc0579f59e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants