Skip to content

Update Sentinel Costs workbook - #14998

Closed
Dr Bill Mcilhargey (billmcilhargey) wants to merge 6 commits into
Azure:masterfrom
billmcilhargey:feat-sentinel-cost-wkb
Closed

Update Sentinel Costs workbook#14998
Dr Bill Mcilhargey (billmcilhargey) wants to merge 6 commits into
Azure:masterfrom
billmcilhargey:feat-sentinel-cost-wkb

Conversation

@billmcilhargey

@billmcilhargey Dr Bill Mcilhargey (billmcilhargey) commented Aug 27, 2026 •
edited
Loading

Copy link
Copy Markdown

Status

This historical PR remains closed and is superseded by #15452.

The maintainer invited reopening this PR, but GitHub rejected both the REST and dedicated reopen operations. The replacement PR carries the original SOC Handbook cost workbook contribution, the requested V3 package outputs, synchronized solution/package release 3.0.9, workbook metadata version 1.6.0, and a corrected complete scope description.

Historical Scope of This Closed PR

The historical diff here contains these three files; the generated package and other corrections are in the replacement PR:

  • Solutions/SOC Handbook/Workbooks/AzureSentinelCost.json
  • Solutions/SOC Handbook/Data/Solution_SOC Handbook.json
  • Solutions/SOC Handbook/ReleaseNotes.md

The workbook reorganizes ingestion, free-data, retention, Microsoft 365 and Defender for Servers Plan 2 benefit, commitment tier, Data Lake, and SOAR views. The standalone Workbooks/SentinelCosts.json is not part of this historical diff.

Follow-up Review and Evidence

Please continue review in #15452. It is intentionally a draft under the maintainer's earlier guidance: a genuine post-deployment data-loading screenshot is still outstanding, and ARM-TTK has the same 12 blank-array findings in unchanged IncidentOverview/SentinelCentral payloads on both the contribution and an isolated unmodified upstream baseline. No live deployment, passing ARM-TTK result, or screenshot is claimed.

The replacement documents the passing TypeScript compilation, 15 workbook regression tests, standalone metadata validation, JSON/workbook/solution/link checks, and byte-for-byte V3 ZIP verification, together with all skipped checks and remaining blockers.

@billmcilhargey

Dr Bill Mcilhargey (billmcilhargey) commented Aug 27, 2026

Copy link
Copy Markdown
Author

WIP - working on getting Data lake cost into here and fixing up SOAR (Playbooks - Logic Apps) and a few other things

Not ready for Review

@billmcilhargey

Dr Bill Mcilhargey (billmcilhargey) commented Aug 27, 2026

Copy link
Copy Markdown
Author

WIP - Not ready for review

This workbook continues to be a work in progress. It is still being drafted and tested and is not ready for review.

Summary of Changes

  • Release metadata was increased exactly once from 1.5.1 to 1.6.0 in WorkbooksMetadata.json. The workbook format remains Notebook/1.0.
  • Workbook items expanded from 20 to 53.
  • Added structured sections for ingestion, free Sentinel data, Microsoft 365 benefits, Defender for Servers Plan 2, retention, Data Lake, and SOAR.
  • Added and refined parameters for time range, workspace, ingestion price, retention price, Logic App execution cost, Microsoft 365 eligible seats, and Defender for Servers Plan 2 protected servers or nodes.
  • Added validation rules for positive license counts and non-negative pricing inputs.
  • Reworked ingestion summaries into average size, average cost, total size, and total cost tiles.
  • Added source-table ingestion detail and improved GB and currency formatting.
  • Added free-data summaries for total data, estimated savings, average daily savings, and selected-period savings.
  • Added a free-versus-billable daily ingestion visualization.
  • Added Microsoft 365 eligible-table detail and daily ingestion-versus-grant comparison.
  • Added Microsoft 365 grant and estimated savings calculations based on eligible seat counts.
  • Added Defender for Servers Plan 2 table detail, pooled trend analysis, node identification, daily grant, period grant, grant usage, and estimated savings.
  • Added DfSP2 daily ingestion-versus-grant visualization.
  • Added DfSP2 entitlement guidance and clarified that licensing, protection, and eligibility must be verified separately.
  • Added retention summary tiles for average daily and selected-period costs.
  • Added retained-data-by-source-table detail with data volume and cost columns.
  • Added Microsoft Sentinel Data Lake guidance and links to official cost-management documentation.
  • Added SOAR documentation covering Logic Apps, billable executions, execution pricing, and cost-management validation.
  • Added Logic App subscription, resource-group, and resource selectors.
  • Added an Azure Metrics-based automation cost summary.
  • Added Logic App execution metrics visualization and estimated automation cost tile.
  • Added no-data messages and consistent empty-state styling.
  • Added conditional visibility for optional sections when required inputs are unset.
  • Reorganized and repositioned existing workbook content around the new sections.
  • Expanded titles, section descriptions, assumptions, warnings, and documentation links.
  • Some legacy hidden items remain and still require cleanup, including a duplicate DfSP2 savings item.

Manual Testing

Testing has been performed manually in the Caldova demo tenant:

  1. Open https://security.microsoft.com.
  2. Go to Sentinel → Workbooks → Add workbook.
  3. Select Edit → Advanced Editor.
  4. Press Ctrl+A and remove the existing workbook content.
  5. Copy and paste the complete SentinelCosts.json into the Advanced Editor.
  6. Confirm that the editor reports no errors.
  7. Select Apply first.
  8. Select Done editing second.
  9. Test the workbook sections, parameters, queries, charts, tables, and cost calculations.

Testing is ongoing. The workbook remains in draft and is not ready for review.

Known Limitations

  • The SOAR summary requires the AzureMetrics table to use the Analytics plan. It cannot run when the table is configured for Basic Logs.
  • Data Lake costs are not calculated from the Log Analytics Usage table and must be validated through Microsoft Sentinel Cost Management or Azure Cost Management.
  • Retention, Microsoft 365, DfSP2, and SOAR values are estimates and should not be treated as invoices.

@v-atulyadav v-atulyadav added the Workbook Workbook specialty review needed label Aug 28, 2026
@v-atulyadav
v-atulyadav requested a lite review from Copilot August 28, 2026 04:36
Copilot AI reviewed Aug 28, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Updates the Sentinel Costs workbook metadata to reflect a new release version.

Changes:

  • Bump Sentinel Costs workbook metadata version from 1.5.1 to 1.6.0.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@billmcilhargey
Enhance Sentinel Costs workbook with optional SOAR automation inputs …
0b93b33 
…and resource selectors
@billmcilhargey
Move Sentinel cost workbook enhancements into SOC Handbook solution
a2957e8 
Reverts Workbooks/SentinelCosts.json and Workbooks/WorkbooksMetadata.json to master. All enhancements now live in the SOC Handbook solution workbook (AzureSentinelCost.json), bumped to 3.0.8.

Structure: reorganized into 7 collapsible groups - Costing, Free Sentinel Data, E5 Benefits, Defender for Cloud Plan 2, Retention, Commitment Tiers, Data Lake. Parameters scoped per section; TimeRange/Workspace/Price remain global. Removed 10 hidden legacy placeholder items.

New sections: Free Sentinel Data (documented free tables, savings), Retention (>90 day billable retention costs), Commitment Tiers (auto-detected current tier plus usage-based suggestion), Data Lake (per-table storage cost from mirrored volume at documented 6:1 compression, plus manual-entry table for ingestion/processing/query/insights meters that are not observable from the workspace).

Behavior fixes: charts now derive bin size from the selected TimeRange (5m to 30d) instead of fixed daily bins; E5 and Defender P2 grant lines are prorated per interval rather than compared against a full-day allowance; commitment tier recommendation is withheld with a days-remaining countdown until 31 days of billable history exist; Logic App selector now emits concrete resource IDs (selected = Rank <= 10) instead of the value::all sentinel, which prevented the Azure Monitor metrics control from resolving scope.

Defect fixes in the existing workbook: removed 85 invisible U+202F characters embedded in KQL; resolved duplicate item name 'EstimateMdcPlan2Discount - Copy'; corrected 'Defedner' typo; replaced hardcoded \ Defender P2 license price with a DfSP2LicensePrice parameter; scoped the licensing savings query to TimeRange instead of scanning all data; set fallbackResourceIds to [] per workbook guidance; stripped en-us locale codes from 11 Microsoft documentation URLs.

Not included: Package/ artifacts are still at 3.0.7 and require regeneration via build-and-validate.ps1, which needs PowerShell 7 (only 5.1 is available locally).
@billmcilhargey
Update Sentinel cost workbook package
04cb636
@billmcilhargey
Refine Defender Plan 2 workbook details
1d7ef2d
@billmcilhargey
Dr Bill Mcilhargey (billmcilhargey) marked this pull request as ready for review August 31, 2026 22:16
@billmcilhargey

Dr Bill Mcilhargey (billmcilhargey) commented Aug 31, 2026

Copy link
Copy Markdown
Author

Ready for Review and updated

Closed
Copilot AI reviewed Sep 1, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Copilot was unable to run its full agentic suite in this review.

Pull request overview

Copilot reviewed 2 out of 3 changed files in this pull request and generated 2 comments.

Comment thread Solutions/SOC Handbook/Data/Solution_SOC Handbook.json
Comment thread Solutions/SOC Handbook/Data/Solution_SOC Handbook.json
@v-rusraut

v-rusraut commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Hi Dr Bill Mcilhargey (@billmcilhargey)
After updating the workbook, please package the solution using the V3 tool - https://github.com/Azure/Azure-Sentinel/blob/master/Tools/Create-Azure-Sentinel-Solution/V3/README.md . Additionally, provide a screenshot demonstrating that the workbook is loading data successfully after deployment.

@v-rusraut

v-rusraut commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Hi Dr Bill Mcilhargey (@billmcilhargey)

Please create package of solution using the V3 tool - https://github.com/Azure/Azure-Sentinel/blob/master/Tools/Create-Azure-Sentinel-Solution/V3/README.md . Additionally, provide a screenshot demonstrating that the workbook is loading data successfully after deployment.

@v-rusraut

v-rusraut commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Hi Dr Bill Mcilhargey (@billmcilhargey) ,
Please create package of solution.

@v-rusraut

v-rusraut commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Hi Dr Bill Mcilhargey (@billmcilhargey) ,
Please create package for solution.

@v-rusraut

v-rusraut commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Hi Dr Bill Mcilhargey (@billmcilhargey)

Please create package for solution.

@v-rusraut

v-rusraut commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Hi Dr Bill Mcilhargey (@billmcilhargey) ,
Please work on above comment.

@v-rusraut

v-rusraut commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Hi Dr Bill Mcilhargey (@billmcilhargey) , Please create the package for this solution. If you need more time, you can convert this PR to a draft. Once the package is ready for review, you can mark the PR as ready for review again.

@v-rusraut

v-rusraut commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Hi Dr Bill Mcilhargey (@billmcilhargey) ,
Please create package for the solution.

@v-rusraut

v-rusraut commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

Hi Dr Bill Mcilhargey (@billmcilhargey),
Please work on above comment.

@v-rusraut

v-rusraut commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

Hi Dr Bill Mcilhargey (@billmcilhargey), are you facing any difficulties creating the package for this solution? Please let us know, and we’ll be happy to assist you.

@v-rusraut

v-rusraut commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Hi Dr Bill Mcilhargey (@billmcilhargey) , we wanted to check on the status of PR #14998. PR is pending for more than 30 days. Please let us know if you need any assistance to review this PR. Per our standard operating procedures if no response is received in the next 7 business days, we will close this PR.
Thank you for your cooperation.

@v-rusraut

v-rusraut commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Hi Dr Bill Mcilhargey (@billmcilhargey),
Since we have not received a response in the last 7 days, we are closing your PR #14998 per our standard operating procedures. If you still need support for this issue, you can re-open the PR at any time.
If you do re-open, we simply request that you ensure the PR has response to the last request. Thank you for your cooperation

@v-rusraut v-rusraut closed this Oct 14, 2026
Open
@billmcilhargey

Dr Bill Mcilhargey (billmcilhargey) commented Oct 27, 2026

Copy link
Copy Markdown
Author

The scope/version feedback and requested V3 package outputs have been addressed in #15452 (commit c089744553ffde7f189724b84b7e2028a97934d6). GitHub would not reopen this PR, so the replacement carries the corrected six-file contribution. The replacement remains a draft: the live deployed-workbook screenshot is still pending, and the same ARM-TTK blank-array failure was reproduced on unmodified upstream. Review has been requested from the maintainer there; please continue the discussion in the replacement PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments

Assignees

@v-rusraut v-rusraut

Labels

Workbook Workbook specialty review needed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@billmcilhargey @v-rusraut @v-atulyadav