Skip to content

Update SOC Handbook cost workbook and generate V3 package - #15452

Open
Dr Bill Mcilhargey (billmcilhargey) wants to merge 14 commits into
Azure:masterfrom
billmcilhargey:feat-sentinel-cost-wkb
Open

Dr Bill Mcilhargey (billmcilhargey) wants to merge 14 commits into
Azure:masterfrom
billmcilhargey:feat-sentinel-cost-wkb

Conversation

@billmcilhargey

@billmcilhargey Dr Bill Mcilhargey (billmcilhargey) commented Oct 27, 2026 •

Copy link
Copy Markdown

Repackaged release candidate (29 October 2026)

Rebuilt the existing SOC Handbook 3.0.9 / AzureSentinelCost 1.6.0 legacy V3 candidate from the reviewed source in commit d843b1f8efcf57582ecc2f7c60e390291899b492; no additional release increment or new workbook behavior was introduced.

  • Download the freshly rebuilt 3.0.9 ZIP
  • ZIP SHA-256: 157149C6AED64FF21E453E5B9199A5470B23261824AEDC63041239DC9CD41065.
  • All 13 solution workbooks, 37 corrected cost-workbook visualization queries, six independent trend/retention query contexts, and the removed licensing input were verified. Both archived templates match their generated files byte-for-byte; the generated template/parameter contents are unchanged from the reviewed head.
  • Fresh tests: 73/73 Pester regressions and 15/15 workbook-validator unit tests passed. Authenticated PR-diff checks and live Azure queries were not performed by those unit tests.
  • Canonical full local validation: 9 validators passed / 1 failed / 11 skipped. ARM-TTK has 49 checks passed / 1 failed; its 12 blank-property findings match the isolated unchanged-upstream baseline at identical locations. The local report marks this candidate not accepted. No validator, protected tooling, or unrelated workbook was changed to suppress the failure.
  • This commit changes only the regenerated ZIP and the existing 3.0.9 release-note row/date. Fresh-head GitHub CI is separate from the prior-head snapshot below. Maintainer approval and retrying the previously failed Copilot review remain outstanding.

Proposed Changes

Follow up on #14998 and its superseded duplicate #15020. GitHub rejected reopening the original PR, so this replacement includes the original SOC Handbook cost workbook contribution, the requested V3 package, and all nine subsequent automated review corrections.

The requested 13 workbook screenshots are embedded below, captioned and arranged in workbook section order, from workspace ingestion through the SOAR summary. The requested V3 solution package is also included, with a direct download link. This PR is ready for review.

Review Corrections

  1. Configurable ingestion price: the Other category now uses the selected Price, matching classified/custom data and cost totals.
  2. Data Lake normalization: trend estimates divide each bucket by its actual covered duration, including partial edges. Daily/table estimates also use the actual selected duration instead of rounding short windows up to one day. Chart output columns are preserved.
  3. UTC daily grant caps: Microsoft 365 and Defender for Servers grant use and savings cap each UTC day before summing. Unused allowance cannot transfer between days. Grant-limit tiles count intersected UTC dates, end boundaries are exclusive, and comparison charts prorate only their actual covered interval. Unknown licenses still show N/A. Documentation explains partial-day and billing limitations.
  4. Consistent Microsoft 365 eligibility: all five summary/detail/trend queries use the same risk-inclusive table set, including AADUserRiskEvents and AADRiskyUsers.
  5. Unused input removed: remove DfSP2LicensePrice; retain the protected-node input and its validation.
  6. Independent retention history: all three retention queries use Set in query, independent of the global time selection. TimeRange only scales period-cost estimates; labels/no-data guidance reflect the actual historical scan.
  7. Sub-day daily rates: the four workspace/free-data daily ingestion, cost and savings tiles now use actual selected days, with no one-day floor. Four additional tests verify 6 GB over six hours is 24 GB/day, or 48/day at a price of 2.
  8. Consistent selected end: Defender eligibility totals/detail, node context, workspace costs and free-data views now share the same half-open interval. All 16 selected-window predicates across 15 views were aligned, and a workbook-wide guard prevents inclusive-end regressions. Historical fixed-lookback queries were not changed.
  9. Hourly Usage resolution and source intervals: all five Usage trend charts now use hourly-or-coarser bins. The retained-volume and both grant-comparison trends apportion each overlapping hourly total using its StartTime/EndTime overlap with the selection, and use query-owned time context so the first partial source hour is not implicitly discarded. Numerators, rate denominators and grant lines cover the same interval. Chart output columns, billability and eligible table sets are preserved. Documentation identifies uniform partial-hour apportionment as an estimate rather than measured sub-hour activity.

Correction commits: 477fef3cc3de1266fdc39d43b2b455d5ddef821a, 070361eda8bd28adb1efdb76814e22cd58aec715, 121604912085af24f9232368abbe4a1109916eaa, and 2c567bb29a1f4ed2d09f2a79c6de86360cc87b4d.

Exact Changed-File Scope

  • Solutions/SOC Handbook/Workbooks/AzureSentinelCost.json
  • Solutions/SOC Handbook/Data/Solution_SOC Handbook.json
  • Solutions/SOC Handbook/ReleaseNotes.md
  • Solutions/SOC Handbook/Tests/AzureSentinelCost.Tests.ps1
  • Solutions/SOC Handbook/Package/mainTemplate.json
  • Solutions/SOC Handbook/Package/3.0.9.zip
  • Workbooks/WorkbooksMetadata.json - only AzureSentinelCost's metadata version and publish date.

The standalone SentinelCosts workbook is unchanged. Generated createUiDefinition.json and testParameters.json remain unchanged from the base. No shared validators, workflows, or unrelated workbooks were modified. Screenshot assets are hosted on a separate branch in the author's fork and do not add files to this PR's workbook or package scope.

Packaging and Versions

Download the V3 solution package 3.0.9 (.zip)

The requested package has been created and committed. The canonical Sentinel-only V3 tool regenerated this same unreleased candidate in documented catalog mode; no additional release increment was introduced. The download link above was checked against the recorded ZIP SHA-256.

  • Solution manifest, newest release note, and generated package: 3.0.9.
  • AzureSentinelCost metadata and packaged content: 1.6.0.
  • Notebook format: Notebook/1.0.
  • The V3 ZIP contains exactly its main and UI templates, matching the generated files byte-for-byte.
  • All 37 packaged queries match corrected source exactly; independent retention contexts, parameter removal and the three overlapping-source-hour query contexts are also verified in the package.
  • ZIP SHA-256: 157149C6AED64FF21E453E5B9199A5470B23261824AEDC63041239DC9CD41065.
  • Current-head GitHub Actions ARM-TTK - SOC Handbook checks passed. The earlier local baseline-related result is recorded separately below and is not an outstanding current-head CI failure.

Recorded Local Testing

These are the earlier implementation-validation results. Fresh 29 October repackaging and test results are recorded above; source workbook queries and test definitions remain unchanged.

  • Offline Pester regressions: 73 passed, 0 skipped. The 29 added hourly-source cases reproduced 26 failures against the previous workbook before correction. Actual one-hour Usage fixtures cover an aligned hour, five minutes inside an hour, crossing hourly and UTC-midnight boundaries, two partial hourly edges, partial six-hour buckets, and empty input. At 10 GB/day and 270 retained days the result is 2,700 GB, not 32,400 GB. All prior pricing, UTC cap, eligibility, selected-end and retention regressions remain passing.
  • Microsoft Kusto.Language 12.4.1 semantic analysis: all 36 Log Analytics queries, with populated and unset-license fixture bindings, 72 checks with zero errors. Fixtures include real Usage StartTime/EndTime datetime columns. This is offline semantic analysis, not live query execution.
  • The modified workbook metadata passed its existing repository validator. The local Mocha harness completion limitation is recorded below rather than counted as a pass.
  • The earlier local full solution run passed TypeScript compilation, JSON, all 13 workbook templates, solution validation, branding, document links, field types and hyperlink validation. Its recorded report was 9 validators passed / 1 failed / 11 skipped because of the baseline ARM-TTK finding described below. Current GitHub CI results are listed separately in the next section.
  • All 37 packaged queries exactly match source; all three affected controls retain query-owned contexts in the package. Solution/workbook versions remain 3.0.9 / 1.6.0. Both ZIP templates match generated files byte-for-byte, and scoped source whitespace validation passed.

Run the regressions from the repository root with PowerShell 7 and Pester 5+:

Invoke-Pester -Path '.\Solutions\SOC Handbook\Tests\AzureSentinelCost.Tests.ps1'

Prior-Head CI Status (28 October 2026)

Checked at 2026-10-28 15:38 UTC for head 2c567bb29a1f4ed2d09f2a79c6de86360cc87b4d:

  • 42 reported check runs: 32 successful / 9 skipped / 1 neutral / 0 failed / 0 in progress. Counts include multiple runs with the same job name. All reported check runs have completed; skipped and neutral results are not counted as passes.
  • ARM-TTK - SOC Handbook passed, including the latest reported successful ARM-TTK run.
  • Workbook and workbook-template validation, solution validation, JSON/YAML validation, KQL validation, document links, Non-ASCII validation, Detection Schema, secret scanning, and all five CodeQL language-analysis jobs succeeded.
  • The aggregate CodeQL check is neutral; its language-analysis jobs succeeded. No incomplete CI jobs or current-head failures were reported in this snapshot.

View the PR checks.

Workbook Screenshots

Here are the requested author-provided screenshots for validation, embedded directly in the description and arranged in the workbook's top-to-bottom order. The visible global selection is Last 7 days; retention and commitment-tier views retain their documented independent history windows.

The captures show populated summary tiles and source-table breakdowns, together with the visible empty/no-data states. The retained-volume chart and both grant-comparison charts show no-data messages despite populated related summaries, while the retention panel reports no billable data older than 90 days. The screenshots are visual validation evidence, not proof that every panel returned populated results. No new Azure deployment or live query execution was performed for this description refresh.

1. Workbook overview and workspace ingestion summary

Screenshot 1: Workbook overview, global parameters, and workspace ingestion summary

2. Workspace ingestion breakdown and trend

Screenshot 2: Workspace billable ingestion by log category and ingestion trend

3. Data lake scope, assumptions, and billing limitations

Screenshot 3: Data Lake Ingestion summary guidance and billing limitations

4. Data lake parameters, estimates, and table breakdown

Screenshot 4: Data lake pricing and retention parameters, summary estimates, and table breakdown

5. Retained-volume empty state and free data summary

Screenshot 5: Retained-volume chart no-data message followed by the Free Sentinel Data Ingestion Summary

6. Free data breakdown and trend

Screenshot 6: Free data by source table, estimated savings, and ingestion trend

7. Microsoft 365 benefit eligibility, seats, and summary

Screenshot 7: Microsoft 365 E7/E5/A5/F5/G5 benefit guidance, seat input, grant totals, and estimated savings

8. Microsoft 365 eligible tables and grant-comparison empty state

Screenshot 8: Eligible Microsoft 365 source tables and the grant-comparison chart no-data message

9. Defender for Servers Plan 2 benefit, protected nodes, and summary

Screenshot 9: Defender for Servers Plan 2 eligibility guidance, protected-node input, observed nodes, grant totals, and estimated savings

10. Defender for Servers eligible tables and grant-comparison empty state

Screenshot 10: Eligible Defender for Servers Plan 2 source tables and the grant-comparison chart no-data message

11. Retention summary and historical-data empty state

Screenshot 11: Retention pricing, zero-cost summary, and no billable data older than 90 days message

12. Commitment tiers

Screenshot 12: Commitment-tier guidance and usage-based suggested tier details

13. SOAR summary and Logic Apps inventory

Screenshot 13: SOAR guidance, subscription and resource-group filters, and discovered Logic Apps inventory

Earlier Local-Run Limitations

These notes describe the earlier local environment and are retained for traceability; they must not be read as unresolved failures of the now-completed current-head CI checks.

  • The canonical V3 tool generated the candidate but its earlier local validation exited on an ARM-TTK baseline finding: 49 tests passed / 1 failed. All 12 blank-array locations matched the captured unchanged upstream cc043cefcb baseline in the existing IncidentOverview/SentinelCentral payloads. Current-head GitHub ARM-TTK checks subsequently passed. No gate was bypassed, no validators were weakened, and no unrelated workbooks were modified.
  • The 15 existing local Mocha cases were attempted, but imported validator modules ran their CLI entry point and called process.exit before Mocha completed. No local pass count is claimed for that suite. This shared harness was not changed as part of the workbook correction.
  • Detection Schema and Non-ASCII checks were skipped locally because .NET Core 3.1 was unavailable; TruffleHog was also unavailable locally. Current-head GitHub Detection Schema, Non-ASCII, and secret-scanning checks passed. The solution-scoped native KQL runner had no applicable detection/parser files; the separate 72 semantic checks above cover workbook queries offline. Local skipped checks are not represented as local passes.

Review Request

v-rusraut, the requested V3 package, all nine code-review corrections, and the requested workbook screenshots for validation are now supplied. Current-head CI has completed with no reported failures or jobs in progress. Please re-review the implementation, the linked package, and the 13 screenshots embedded above. Please let me know if anything else is needed for review or validation.

Reverts Workbooks/SentinelCosts.json and Workbooks/WorkbooksMetadata.json to master. All enhancements now live in the SOC Handbook solution workbook (AzureSentinelCost.json), bumped to 3.0.8.

Structure: reorganized into 7 collapsible groups - Costing, Free Sentinel Data, E5 Benefits, Defender for Cloud Plan 2, Retention, Commitment Tiers, Data Lake. Parameters scoped per section; TimeRange/Workspace/Price remain global. Removed 10 hidden legacy placeholder items.

New sections: Free Sentinel Data (documented free tables, savings), Retention (>90 day billable retention costs), Commitment Tiers (auto-detected current tier plus usage-based suggestion), Data Lake (per-table storage cost from mirrored volume at documented 6:1 compression, plus manual-entry table for ingestion/processing/query/insights meters that are not observable from the workspace).

Behavior fixes: charts now derive bin size from the selected TimeRange (5m to 30d) instead of fixed daily bins; E5 and Defender P2 grant lines are prorated per interval rather than compared against a full-day allowance; commitment tier recommendation is withheld with a days-remaining countdown until 31 days of billable history exist; Logic App selector now emits concrete resource IDs (selected = Rank <= 10) instead of the value::all sentinel, which prevented the Azure Monitor metrics control from resolving scope.

Defect fixes in the existing workbook: removed 85 invisible U+202F characters embedded in KQL; resolved duplicate item name 'EstimateMdcPlan2Discount - Copy'; corrected 'Defedner' typo; replaced hardcoded \ Defender P2 license price with a DfSP2LicensePrice parameter; scoped the licensing savings query to TimeRange instead of scanning all data; set fallbackResourceIds to [] per workbook guidance; stripped en-us locale codes from 11 Microsoft documentation URLs.

Not included: Package/ artifacts are still at 3.0.7 and require regeneration via build-and-validate.ps1, which needs PowerShell 7 (only 5.1 is available locally).
Generate the requested V3 package candidate, synchronize solution release 3.0.9 and workbook metadata 1.6.0, and correct the release note. Runtime deployment evidence and the reproduced upstream ARM-TTK blank-array failure remain explicitly outstanding.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 27, 2026 19:18
@github-actions github-actions Bot added Community Solution Community-contributed Sentinel solution Solution Solution specialty review needed labels Oct 27, 2026
@billmcilhargey

Dr Bill Mcilhargey (billmcilhargey) commented Oct 27, 2026 •

Copy link
Copy Markdown
Author

v-rusraut, please re-review the corrections requested in #14998. GitHub rejected reopening that PR, so this is the linked replacement (also consolidating the superseded #15020).

Corrected: the V3 package outputs and ZIP are included; solution manifest, release notes and package are aligned at 3.0.9; workbook metadata/package content are aligned at 1.6.0; the description now covers every modified file. Focused workbook tests pass 15/15, the actual metadata validates, and both archived templates match their generated outputs byte-for-byte.

Still outstanding: a genuine screenshot showing the deployed workbook loading actual data is not available from this environment, and full ARM-TTK acceptance is blocked by 12 blank-array findings in unchanged IncidentOverview/SentinelCentral content. All 12 findings reproduce at identical locations on an isolated unmodified upstream checkout (cc043cefcb). No validators were weakened, no unrelated workbook content was changed, and no runtime evidence was invented.

Following your previous recommendation, this is a draft rather than an unsupported ready-for-review claim. Please review the supplied packaging/scope/version corrections and advise on the upstream validation findings. The deployment screenshot remains an explicit prerequisite before marking the contribution ready.

CI infrastructure follow-up

The YAML validation workflow failed before inspecting content because its unauthenticated GitHub client hit HTTP 403 / API rate limit exceeded. I attempted a failed-job rerun, but GitHub requires repository administrator rights. Please rerun that workflow when possible. This is an additional explicit infrastructure blocker, not a YAML change or a passing-check claim.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved grant, retention, and ingestion calculations can produce incorrect cost estimates.

6 open issues
What changed in this PR

This follow-up updates the SOC Handbook cost workbook contribution and supplies the requested V3 package, aligning solution release versions while retaining the expanded cost-analysis sections.

Changes:

  • Synchronizes the solution manifest and release records at 3.0.9.
  • Increments AzureSentinelCost workbook metadata to 1.6.0.
  • Supplies generated V3 content; deployment evidence and packaging acceptance remain outstanding.
File Description
Workbooks/​WorkbooksMetadata.json Bumps AzureSentinelCost metadata to 1.6.0.
Solutions/​SOC Handbook/​ReleaseNotes.md Records workbook enhancements and the 3.0.9 packaging update.
Solutions/​SOC Handbook/​Package/​mainTemplate.json Generated V3 template; excluded from code review by repository guidance.
Solutions/​SOC Handbook/​Data/​Solution_SOC Handbook.json Updates the solution version to 3.0.9.

Review effort: Balanced

Comment thread Solutions/SOC Handbook/Workbooks/AzureSentinelCost.json Outdated
Comment thread Solutions/SOC Handbook/Workbooks/AzureSentinelCost.json Outdated
Comment thread Solutions/SOC Handbook/Workbooks/AzureSentinelCost.json Outdated
Comment thread Solutions/SOC Handbook/Workbooks/AzureSentinelCost.json Outdated
Comment thread Solutions/SOC Handbook/Workbooks/AzureSentinelCost.json Outdated
Comment thread Solutions/SOC Handbook/Workbooks/AzureSentinelCost.json Outdated
Copilot AI balanced review requested due to automatic review settings October 27, 2026 20:57
Correct configurable ingestion pricing, covered Data Lake intervals, UTC daily grant caps and consistent M365 eligibility. Remove the unused licensing input and separate historical retention scans from the selected cost period. Add 37 offline regressions and regenerate the existing V3 3.0.9 package at workbook version 1.6.0; deployment evidence remains explicitly outstanding.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@billmcilhargey

Copy link
Copy Markdown
Author

v-rusraut, please re-review the updated implementation and V3 package in commit 477fef3.

All six automated review findings are corrected: selected-price handling, Data Lake interval normalization, UTC daily grant caps, Microsoft 365 risk-table eligibility parity, removal of the unused licensing input, and independent historical retention scans. The archive has been regenerated at the same unreleased 3.0.9 / workbook 1.6.0 versions and verified to contain all 37 corrected queries and the updated controls.

Evidence: 37/37 new offline regressions (32 failed before correction), 15/15 existing workbook tests, metadata validation, and 72 Microsoft Kusto.Language semantic checks across all 36 Log Analytics queries with populated/unset-license fixtures, with zero semantic errors. The new tests are included in this PR.

The exact seven-file scope, ZIP SHA-256 and local validation limitations are in the updated description. The local ARM-TTK finding still reproduces exactly on unchanged upstream; previous-head upstream ARM-TTK passed and the new head is being checked afresh.

I have not fabricated the requested deployment screenshot or claimed live execution. That evidence still needs an authenticated approved non-production workspace/browser session, so this remains a draft while I complete that final environment-dependent requirement. Please review the supplied code and package corrections in the meantime.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread Solutions/SOC Handbook/Workbooks/AzureSentinelCost.json Outdated
Copilot AI balanced review requested due to automatic review settings October 27, 2026 21:22
Use actual selected duration for workspace daily ingestion/cost and free-data daily ingestion/savings. Add four reviewer-example regressions (6 GB over six hours is 24 GB/day), preserve the 3.0.9 candidate version, and regenerate the V3 package. All 41 workbook regressions and 72 Microsoft KQL semantic checks pass; live deployment evidence remains outstanding.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@billmcilhargey

Dr Bill Mcilhargey (billmcilhargey) commented Oct 27, 2026 •

Copy link
Copy Markdown
Author

v-rusraut, please re-review the latest code/package update at 1216049, following 477fef3 and 070361e.

All eight current automated review findings are corrected, including the later sub-day daily-rate and inclusive-end findings. The selected interval is now consistent across all affected ingestion, cost, free-data and Defender views; historical probes retain their independent scope.

The V3 candidate is freshly regenerated at 3.0.9 / workbook 1.6.0, with all 37 packaged queries matching final source. Verification: 44/44 offline regressions, 15/15 existing workbook tests, metadata validation and 72 Microsoft Kusto.Language semantic checks with zero errors. The reviewer's six-hour rate example and next-midnight endpoint example both reproduced the original errors before correction and now pass. All addressed threads have concrete fix replies.

The preceding correction heads completed upstream checks successfully, including YAML and ARM-TTK. This update triggers fresh checks. The unchanged local ARM-TTK baseline limitation remains documented rather than suppressed.

The outstanding environment-dependent deliverable is the requested screenshot showing the deployed corrected workbook loading actual lab data. No authenticated approved lab target has been supplied, so no Azure write or fabricated runtime screenshot was made. This remains a draft pending that genuine deployment evidence; please review the completed code and package corrections in the meantime.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Defender ingestion views still disagree with benefit calculations at the selected end boundary, and required live-data deployment evidence remains outstanding.

1 open issue
1 resolved since last review

Review effort: Balanced

Comment thread Solutions/SOC Handbook/Workbooks/AzureSentinelCost.json Outdated
Copilot AI balanced review requested due to automatic review settings October 27, 2026 21:41
Use an exclusive selected end consistently across ingestion, costs, free-data summaries/trends and Defender eligibility/node views. Add midnight-boundary regressions and a workbook-wide predicate guard; document the interval convention and regenerate the existing V3 3.0.9 candidate. All 44 regressions, 15 existing workbook tests and 72 Microsoft KQL semantic checks pass. Live deployment evidence remains explicitly outstanding.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Hourly Usage totals are treated as sub-hour measurements, distorting storage projections and grant-comparison charts.

1 open issue
1 resolved since last review

Review effort: Balanced

Comment thread Solutions/SOC Handbook/Workbooks/AzureSentinelCost.json Outdated
Use hourly-or-coarser bins and apportion overlapping source hours for storage and grant estimates. Add hourly-source regressions and regenerate the existing V3 3.0.9 candidate. Live deployment evidence and reproduced upstream ARM-TTK findings remain outstanding.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 28, 2026 13:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@billmcilhargey

Copy link
Copy Markdown
Author

v-skharode, this PR is ready for your review.

  • Screenshots for validation: all 13 workbook screenshots are added below, captioned and arranged in the same top-to-bottom order as the workbook.
  • Package created as requested: the V3 solution package (3.0.9) is included in this PR.

The captures retain the visible empty/no-data states in some panels; the existing validation limitations documented in the PR description still apply.

Please let me know if anything else is needed for review or validation. Thanks!

Workbook screenshots (all 13, in workbook order)

1. Workbook overview and workspace ingestion summary

Screenshot 1: Workbook overview, global parameters, and workspace ingestion summary

2. Workspace ingestion breakdown and trend

Screenshot 2: Workspace billable ingestion by log category and ingestion trend

3. Data lake scope, assumptions, and billing limitations

Screenshot 3: Data Lake Ingestion summary guidance and billing limitations

4. Data lake parameters, estimates, and table breakdown

Screenshot 4: Data lake pricing and retention parameters, summary estimates, and table breakdown

5. Retained-volume empty state and free data summary

Screenshot 5: Retained-volume chart no-data message followed by the Free Sentinel Data Ingestion Summary

6. Free data breakdown and trend

Screenshot 6: Free data by source table, estimated savings, and ingestion trend

7. Microsoft 365 benefit eligibility, seats, and summary

Screenshot 7: Microsoft 365 E7/E5/A5/F5/G5 benefit guidance, seat input, grant totals, and estimated savings

8. Microsoft 365 eligible tables and grant-comparison empty state

Screenshot 8: Eligible Microsoft 365 source tables and the grant-comparison chart no-data message

9. Defender for Servers Plan 2 benefit, protected nodes, and summary

Screenshot 9: Defender for Servers Plan 2 eligibility guidance, protected-node input, observed nodes, grant totals, and estimated savings

10. Defender for Servers eligible tables and grant-comparison empty state

Screenshot 10: Eligible Defender for Servers Plan 2 source tables and the grant-comparison chart no-data message

11. Retention summary and historical-data empty state

Screenshot 11: Retention pricing, zero-cost summary, and no billable data older than 90 days message

12. Commitment tiers

Screenshot 12: Commitment-tier guidance and usage-based suggested tier details

13. SOAR summary and Logic Apps inventory

Screenshot 13: SOAR guidance, subscription and resource-group filters, and discovered Logic Apps inventory

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up review — 29 October 2026

Reviewed current head 2c567bb29a1f4ed2d09f2a79c6de86360cc87b4d, the seven-file PR scope, all inline feedback, and the review/discussion history. All nine existing inline threads are already resolved and have corresponding fixes and replies. I found no additional high-confidence defect requiring a source change, so I have not created an unnecessary commit or regenerated the unchanged release candidate.

Fresh verification

  • 73/73 Pester calculation regressions passed, with zero skipped tests.
  • 15/15 existing workbook-template/metadata validator unit tests passed. The local harness reports that GitHub App authentication/PR context is unavailable; these unit-test results are not a claim that authenticated PR-diff checks or live Azure queries ran locally.
  • All 37 packaged visualization queries exactly match source. All six query-owned time contexts (three hourly-overlap trends and three retention views) also match, and the removed licensing-price input is absent from both notebooks.
  • Both ZIP entries, mainTemplate.json and createUiDefinition.json, match their generated files byte-for-byte. ZIP SHA-256 remains ABC6C964F3EBA8D2304B53AC22C019480D5B9A4765494584D11DAB225BAA5CD2.
  • Solution/workbook versions remain 3.0.9 / 1.6.0, and the reviewed source changes pass git diff --check.
  • Current GitHub checks report no failures; skipped/neutral checks are not counted as passes. A human approval is still required.

Outstanding review action

The latest Copilot review on this head failed on 28 October rather than returning a new finding. I attempted to request another review, but GitHub rejects the API request because the authenticated account lacks upstream write/reviewer-request permissions. The available browser is not signed in, so I could not retry through the PR UI either. A maintainer should re-request the Copilot review.

No new Azure deployment or live query execution was performed. The existing screenshot/no-data limitations remain applicable; these offline checks do not establish that every panel loads live data.

Regenerate the existing V3 candidate and refresh its release note. All 37 cost-workbook queries and both archive entries match source; 88 focused tests pass. Full local validation still reports the 12 pre-existing upstream ARM-TTK blank-property findings, with no validator changes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 29, 2026 16:43
@billmcilhargey

Copy link
Copy Markdown
Author

Repackaged the corrected SOC Handbook solution and pushed commit d843b1f.

Download the freshly rebuilt V3 3.0.9 package

  • Retained solution 3.0.9 and cost-workbook 1.6.0; this is the same unreleased candidate, not another feature release.
  • The new commit changes only the ZIP and the existing 3.0.9 release-note row/date. Generated template and parameter contents are unchanged from the reviewed corrected source; the archive was recreated by the supported legacy V3 tool.
  • Verified all 13 packaged workbooks, all 37 corrected cost-workbook visualization queries, all six query-owned time contexts, and absence of the removed licensing input. Both ZIP entries match their generated files byte-for-byte.
  • Fresh tests: 73/73 Pester regressions and 15/15 workbook-validator unit tests passed. Unit tests do not establish authenticated PR-diff or live Azure validation.
  • ZIP SHA-256: 157149C6AED64FF21E453E5B9199A5470B23261824AEDC63041239DC9CD41065.

Local acceptance is still blocked: the packager's ARM-TTK check and the independent canonical full validation runner both reproduce Template Should Not Contain Blanks: 49 ARM checks pass, 1 fails, reporting the same 12 locations as the isolated unchanged-upstream baseline. Full-run totals are 9 validators passed / 1 failed / 11 skipped. This is a generated candidate, not a locally accepted build. No validation gate, shared tooling, or unrelated workbook was changed to suppress the result.

The PR description now points to the new archive and checksum, preserves all 13 screenshot links, and clearly separates prior-head CI evidence from this new head's checks. Fresh GitHub checks have been triggered; their results are not assumed from the old head. No Azure deployment was performed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The workbook unnecessarily scans raw events across all selected workspaces on each refresh, while six free-data queries include billable records and inflate estimated savings.

2 open issues

Review effort: Balanced

Comment thread Solutions/SOC Handbook/Workbooks/AzureSentinelCost.json Outdated
Comment thread Solutions/SOC Handbook/Workbooks/AzureSentinelCost.json Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Empty trend charts despite populated related summaries require representative live-data validation and maintainer review, since offline checks cannot establish deployed behavior.

2 open issues

Review effort: Balanced

Exclude billable records from all six free-data panels and calculate commitment-tier volume and readiness from hourly Usage aggregates. Add regression coverage and regenerate the existing SOC Handbook 3.0.9 candidate.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 29, 2026 19:00

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review recommendations addressed

Pushed e4d59c8, addressing both outstanding recommendations:

  1. All six free-data volume/savings queries exclude billable Usage records before aggregation. The reviewer's mixed SecurityAlert example is covered by regressions.
  2. Commitment-tier volume and history readiness now use hourly billable Usage aggregates instead of union * over raw events. MB-to-GB conversion, the 31-day gate, existing tier-selection/output logic, and independence from the global time selection are covered.

Workbook guidance and the existing release-note row were updated. The same unreleased SOC Handbook 3.0.9 / AzureSentinelCost 1.6.0 candidate was regenerated; no unrelated content or protected validator/tooling implementation was changed.

Verification

  • 90/90 Pester calculation regressions passed, including 17 new cases; the original 73 remain passing.
  • 15/15 existing workbook-template/metadata validator unit tests passed. These unit tests do not establish authenticated PR-diff or live Azure validation.
  • All 13 packaged workbooks, 37 cost-workbook visualization queries, and 7 query-owned historical/trend contexts were verified; both archived templates match their generated files byte-for-byte.
  • git diff --check passes, and the pushed branch matches the verified commit.

Download the updated V3 3.0.9 package — this supersedes the earlier candidate download links in this discussion.

SHA-256: 30F1E675A7EE8435A8B971DDF1A2820B660F659361DE0A81B2EFA20295A8521D.

Acceptance limitation retained

The canonical full local validation reports 9 validators passed / 1 failed / 11 skipped and explicitly marks the package not accepted. ARM-TTK has 49 checks passed / 1 failed (Template Should Not Contain Blanks), reporting the same 12 blank-property locations documented previously. Both affected generated payload lines (750 and 1318) are byte-for-byte unchanged from the previous head. No validator was weakened and no unrelated workbook was changed to suppress this failure.

The local KQL validator does not cover these workbook queries, and no new live Azure queries or deployment were performed. Fresh-head GitHub CI and maintainer acceptance are separate from these offline results.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The query still recommends a concrete commitment tier from combined ingestion across multiple workspaces, which can mislead users even when the status is 'Not evaluated'.

1 open issue
2 resolved since last review

Review effort: Balanced

"type": 3,
"content": {
"version": "KqlItem/1.0",
"query": "let reviewDays = 31.0; let reviewStart = ago(31d); let historyProbeStart = ago(32d); let currentTier = '{CurrentCommitmentTier}'; let tierLevels = dynamic([50,100,200,300,400,500,1000,2000,5000]); Usage | where TimeGenerated >= historyProbeStart and TimeGenerated < now() | where IsBillable == true | summarize TotalBilledMB = sumif(Quantity, TimeGenerated >= reviewStart), FirstObserved = min(TimeGenerated) | extend ObservedHistoryDays = iif(isnull(FirstObserved), 0.0, min_of(reviewDays, datetime_diff('second', now(), FirstObserved) / 86400.0)) | extend DaysRemaining = toint(ceiling(max_of(reviewDays - ObservedHistoryDays, 0.0))) | extend AvgDailyGB = iif(DaysRemaining == 0, TotalBilledMB / 1024.0 / reviewDays, real(null)), TierLevels = tierLevels | mv-expand TierLevel = TierLevels to typeof(int) | extend IsEligible = TierLevel >= AvgDailyGB | summarize SuggestedTierGBPerDay = minif(TierLevel, IsEligible), AvgDailyGB = any(AvgDailyGB), ObservedHistoryDays = any(ObservedHistoryDays), DaysRemaining = any(DaysRemaining) | extend suggestedTier = case(DaysRemaining > 0, strcat('Ready in ', tostring(DaysRemaining), iif(DaysRemaining == 1, ' day', ' days')), AvgDailyGB < 50.0, 'Pay-As-You-Go', isnotempty(SuggestedTierGBPerDay), strcat(tostring(SuggestedTierGBPerDay), ' GB/day commitment'), 'Above 5000 GB/day - review Azure portal options') | extend alignmentStatus = case(DaysRemaining > 0, 'Not Ready', currentTier == 'Not available' or currentTier == 'Unknown' or currentTier == 'Multiple workspaces', 'Not evaluated', suggestedTier startswith 'Above 5000 GB/day', 'Review in portal', currentTier == suggestedTier, 'Aligned', 'Review tier') | project Current = currentTier, Suggested = suggestedTier, Status = alignmentStatus, ['Avg daily GB'] = round(AvgDailyGB, 2)",
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Community Solution Community-contributed Sentinel solution Solution Solution specialty review needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants