From 1466cd36d56f532d63d7d999d007a85ad18143eb Mon Sep 17 00:00:00 2001 From: "D. Dessy" Date: Fri, 30 Oct 2026 10:23:44 +0100 Subject: [PATCH 1/2] Adjust the slide narrative to the slower Eventgen pace Eventgen now writes ~300 events/min (attack stream ~150/min), so the results are smaller samples than the slides assumed. The narrative now says what that looks like, with figures measured on the running stack: after 10 min about 3,000 events, 150 purchases of which 43% fail, about $1,400 lost, platforms 34-43% on 140-930 events each, and failed purchases spread over ~50 cities; in threat mode the Pyongyang IPs show up in the validate tool after ~2 min and by ~5 min 175.45.* is ~90% of failed purchases and ~95% of lost revenue. Slides: Step 2 expected line, the Buttercup Insights table, the Demo 2 expected line (ask 2-5 min after enabling, because a late ask makes the 15-minute window nearly all attack and fades the 503/404 baseline), the warm-stack line (10 min), the NK troubleshooting row, and speaker notes on sampling noise and on the NK skew of the international and platform answers. S4R-DEMO.md gets a data-pace table and matching notes, the make demo-prep reminder mentions the Eventgen warm-up, and the HTML is rebuilt. Rendered the edited slides to check they still fit. Co-Authored-By: Claude Sonnet 5.5 --- Makefile | 2 +- demo-slides/S4R-DEMO.md | 17 ++++++++++++++--- demo-slides/s4r-demo-slides.html | 19 +++++++++++-------- demo-slides/s4r-demo-slides.md | 19 +++++++++++-------- 4 files changed, 37 insertions(+), 20 deletions(-) diff --git a/Makefile b/Makefile index ea8c167..19c458f 100644 --- a/Makefile +++ b/Makefile @@ -117,7 +117,7 @@ check-splunkbase-pins: ## Compare compose.yml Splunkbase pins to current default demo-prep: ## Pre-demo check: Splunkbase pins + status + MCP verify + warm-stack reminder @echo "=== Splunk MCP demo prep ===" - @echo "Tip: warm the stack before a live meeting — cold 'make up' can take many minutes." + @echo "Tip: warm the stack before a live meeting — cold 'make up' can take many minutes, and Eventgen needs ~10 more minutes (~300 events/min) for the numbers to settle." @echo "" @$(MAKE) check-splunkbase-pins @echo "" diff --git a/demo-slides/S4R-DEMO.md b/demo-slides/S4R-DEMO.md index 0c71c5e..3b9f9ac 100644 --- a/demo-slides/S4R-DEMO.md +++ b/demo-slides/S4R-DEMO.md @@ -26,7 +26,16 @@ make s4r-attack-nk-status # expect: disabled (infrastructure mode) | Data flowing | Search: `index=main sourcetype=access_combined \| head 5` | | Tabs pre-open | `docs/s4r/SPL-CATALOG.md`, `.cursor/agents/s4r-power-user.md`, `.cursor/agents/s4r-it-ops.md`, `.cursor/mcp.json` | -**Warm stack:** Cold `make up` can take many minutes — start Splunk before the session. +**Warm stack:** Cold `make up` can take many minutes — start Splunk before the session, and give Eventgen at least 10 more minutes: it writes ~300 events/min (~150 more while the attack runs), and about 3,000 events make the percentages settle. + +**Data pace (what to expect at ~300 events/min):** + +| After | Default mode | +| ----- | ------------ | +| ~5 min | ~1,500 events, ~75 purchases: usable, but noisy | +| ~10 min | ~3,000 events, ~150 purchases, ~43% failing, ~$1,400 lost; platforms 34–43%; failed purchases on ~50 cities (about 1 each) | + +Threat mode adds ~150 attack events/min (all purchases, ~94% failing): the validate tool lists the Pyongyang IPs after ~2 min, and by ~5 min `175.45.*` is ~90% of failed purchases and ~95% of lost revenue. Ask the Demo 2 question 2–5 min after enabling: later, the 15-minute window is nearly all attack and the 503/404 baseline fades. **After threat segment:** `make s4r-attack-nk-disable` to restore default mode. @@ -321,12 +330,14 @@ Optional: `make verify-mcp-remote MCP_VERIFY_CLIENT=cursor` in terminal (fast). > *Start the North Korean attack simulation for the Buttercup workshop.* -Wait ~2 min; optional **`SA-S4R_validate_nk_attack_traffic`**. Then Step 3 Demo 2 prompt (**last 15 minutes**). +Wait ~2 min; optional **`SA-S4R_validate_nk_attack_traffic`** (lists the Pyongyang IPs by then). Then Step 3 Demo 2 prompt (**last 15 minutes**), ideally within 5 min of enabling. **Terminal fallback:** `make s4r-attack-nk-enable` (see Slide 27). **Narrate:** *“Security should surface North Korea on failed purchases (`SA-S4R_geo_failed_purchases` / validate NK tool); DevOps should see scripted user agents; IT Ops still sees 503 from baseline traffic.”* +**Expect:** `175.45.*` is the dominant source of failed purchases and lost revenue (~90% and ~95% by ~5 min), scripted user agents fail ~94%, real platforms stay flat, and IT Ops still reports 503/404 as background. + **Say:** *“Verdict: mixed — infrastructure still broken, but Security has a lead for investigation.”* --- @@ -364,7 +375,7 @@ On-screen table: **deck slide 32**. Expanded detail below for presenters. | ------- | ---------------- | | MCP tools missing | `make update-mcp-client MCP_CLIENT=cursor`; restart Cursor | | No events | `make status`; `docker logs splunk-init` | -| NK mode no signal | **`SA-S4R_query_nk_demo_state`**; **`SA-S4R_validate_nk_attack_traffic`**; `make restart`; search **last 15m** | +| NK mode no signal | **`SA-S4R_query_nk_demo_state`**; **`SA-S4R_validate_nk_attack_traffic`**; wait ≥2 min (attack ≈ 150 events/min); `make restart`; search **last 15m** | | Concurrency limit | Wait; run one team at a time | | Token / 401 | `make up` or `make update-mcp-client MCP_CLIENT=cursor` | diff --git a/demo-slides/s4r-demo-slides.html b/demo-slides/s4r-demo-slides.html index 5be9485..816e033 100644 --- a/demo-slides/s4r-demo-slides.html +++ b/demo-slides/s4r-demo-slides.html @@ -450,7 +450,7 @@

Step 2 — Live demo prompt

Watch: SA-S4R_geo_failed_purchases

-

Expected (infrastructure): Yes — high purchase failure; lost revenue from failed checkouts.

+

Expected (infrastructure): Yes — about 40% of purchases fail; lost revenue from failed checkouts (hundreds of dollars after ~10 min).

@@ -650,17 +650,17 @@

Buttercup Insights

DevOps -Flat ~40% all platforms +Roughly flat ~40% (34–43%) High, escalate IT Ops Business Analytics -Revenue lost via lookup +Revenue lost via lookup (hundreds of $) Critical Security -No geo concentration +No geo concentration (~1 failure per city) Low @@ -727,7 +727,7 @@

Step 3 — Demo 2: North Korea attack

As Buttercup Power User: is the money loss due to bad infrastructure or an active threat? Delegate to all four teams. Use the last 15 minutes.

-

Expected: Security leads on NK geo; IT Ops still sees 503/404 from baseline.

+

Expected: Security leads on NK geo; IT Ops still sees 503/404 from baseline. Ask 2–5 min after enabling: by ~5 min 175.45.* is ~90% of failed purchases (~95% of lost revenue).

@@ -774,7 +774,7 @@

Appendix — before you start

-

Warm stack: start Splunk before the session (make up can take many minutes).

+

Warm stack: start Splunk at least 10 min before the session — Eventgen writes ~300 events/min, and ~3,000 events make the percentages settle.

@@ -825,7 +825,7 @@

Appendix — troubleshooting

NK mode no signal -SA-S4R_query_nk_demo_state / validate NK; search last 15m +SA-S4R_query_nk_demo_state / validate NK; wait ≥2 min (attack ≈ 150 events/min); search last 15m Concurrency limit @@ -842,5 +842,8 @@

Appendix — troubleshooting

This deck follows that sequence. Step 1 = dashboard build. Step 2 = single natural-language question via MCP. Step 3 = Power User + four specialists + executive synthesis.

Pattern demo, not a product pitch. Splunk stays source of truth; the assistant calls governed tools. Same logs as Labs 3–7 dashboard build.

The catalog is the runbook — the LLM reads layout from S4R-DASHBOARD.md and SPL from the catalog, not from memory. Same panels the agentic demo narrates later.

After the workshop, attendees have a pinned dashboard. Next: how Splunk MCP exposes that same app as tools — then Steps 2 and 3 move the questions into chat.

Product contract: https://help.splunk.com/en/splunk-cloud-platform/mcp-server-for-splunk-platform/2.0/managing-custom-tools-in-splunk-mcp-server -make up POSTs s4r_mcp_tools.json and enables each SA-S4R tool.

PDF slide 6: native + Splunkbase public + customer private. SA-S4R is the private-app column (their Minecraft demo analog). Collision detection and rate limiting are MCP Server features.

Managing custom tools: batch replace does not enable. Enable is a second POST. SPL and API execution still need the saved search or REST handler.

Create leaves tools disabled. This PoC enables with override true so tools/list shows SA-S4R_*. Then each tool gets a Tool Role Policy limiting it to role s4r_user, which splunker holds: other MCP users do not see the tools and get HTTP 403 if they call them.

Write tool only on explicit user request. Makefile s4r-attack-nk-* is the operator fallback. After make up, tools should already appear in Cursor.

Step 2 can be a single assistant — no delegation required. Good for executives and workshop attendees who finished the dashboard and want answers, not panels.

Read-only demo discipline; token minted at make up. The SA-S4R_* tools are limited to role s4r_user (Tool Role Policies), so another MCP user would not even see them. Cursor spawns npx mcp-remote over HTTPS as splunker, not admin. Steps 2 and 3 both use this bridge — Step 3 adds agent roles on top.

Default data mode = infrastructure (~40% 503/404 everywhere). This is the lightweight path before introducing the orchestrator in Step 3.

Step 3 is for builders who want teachable, git-owned personas — mirrors how real platform teams delegate.

Point at each box in the repo — nothing is hidden inside one mega-prompt. Power User = orchestrator; Executive synthesis = final stakeholder table.

SPL is not duplicated in every agent file — agents reference the catalog. Same catalog Steps 1 and 2 already used.

Default data mode = infrastructure (~40% 503/404 everywhere). If slow, call one team first: IT Ops only from catalog IT Ops section.

Numbers come from live MCP — do not read fixed amounts from slides. Step 2 gave one answer; Step 3 shows how each team contributed.

Step 2: ask directly. Step 3: prefix with As Buttercup Power User and delegate to all four teams.

Use case 5 / Demo 2: make s4r-attack-nk-enable before ask. Attendees can swap questions without changing agents or runbook.

Verdict: mixed — infrastructure still broken, but Security has a lead. Cleanup after demo: SA-S4R_apply_nk_demo_state (infrastructure) or make s4r-attack-nk-disable.

\ No newline at end of file diff --git a/demo-slides/s4r-demo-slides.md b/demo-slides/s4r-demo-slides.md index 86bfaa0..d6a3575 100644 --- a/demo-slides/s4r-demo-slides.md +++ b/demo-slides/s4r-demo-slides.md @@ -371,10 +371,11 @@ Watch: **`SA-S4R_geo_failed_purchases`** -**Expected (infrastructure):** Yes — high purchase failure; lost revenue from failed checkouts. +**Expected (infrastructure):** Yes — about 40% of purchases fail; lost revenue from failed checkouts (hundreds of dollars after ~10 min). --- @@ -565,14 +566,15 @@ flowchart TB | Team | Finding | Severity | | ------ | --------- | -------- | | IT Ops | 503/404 dominate | Critical | -| DevOps | Flat ~40% all platforms | High, escalate IT Ops | -| Business Analytics | Revenue lost via lookup | Critical | -| Security | No geo concentration | Low | +| DevOps | Roughly flat ~40% (34–43%) | High, escalate IT Ops | +| Business Analytics | Revenue lost via lookup (hundreds of $) | Critical | +| Security | No geo concentration (~1 failure per city) | Low | **Root-cause hypothesis:** Server-side failure, not fraud --- @@ -600,6 +602,7 @@ Step 2: ask directly. Step 3: prefix with As Buttercup Power User and delegate t --- @@ -618,10 +621,10 @@ Watch: **`SA-S4R_apply_nk_demo_state`** (`threat`) → **`SA-S4R_validate_nk_att > *As Buttercup Power User: is the money loss due to bad infrastructure or an active threat? Delegate to all four teams. Use the last 15 minutes.* -**Expected:** Security leads on NK geo; IT Ops still sees 503/404 from baseline. +**Expected:** Security leads on NK geo; IT Ops still sees 503/404 from baseline. Ask 2–5 min after enabling: by ~5 min `175.45.*` is ~90% of failed purchases (~95% of lost revenue). --- @@ -659,7 +662,7 @@ make s4r-attack-nk-status # expect: disabled (or SA-S4R_query_nk_demo_state | MCP in Cursor | Restart Cursor; Splunk MCP tools visible | | Data flowing | `index=main sourcetype=access_combined \| head 5` | -**Warm stack:** start Splunk before the session (`make up` can take many minutes). +**Warm stack:** start Splunk at least 10 min before the session — Eventgen writes ~300 events/min, and ~3,000 events make the percentages settle. --- @@ -704,7 +707,7 @@ make s4r-attack-nk-status # expect: disabled (or SA-S4R_query_nk_demo_state | ------- | ---------------- | | MCP tools missing | `make update-mcp-client MCP_CLIENT=cursor`; restart Cursor | | No events | `make status`; `docker logs splunk-init` | -| NK mode no signal | `SA-S4R_query_nk_demo_state` / validate NK; search **last 15m** | +| NK mode no signal | `SA-S4R_query_nk_demo_state` / validate NK; wait ≥2 min (attack ≈ 150 events/min); search **last 15m** | | Concurrency limit | Wait; run one team at a time | | Token / 401 | `make up` or `make update-mcp-client MCP_CLIENT=cursor` | From 69bc50681b6073d121f5a66fd9e8d20da20962cb Mon Sep 17 00:00:00 2001 From: "D. Dessy" Date: Fri, 30 Oct 2026 10:27:04 +0100 Subject: [PATCH 2/2] Align the slide severities with the role scale and define the scale The Buttercup Insights slide showed "Critical" for IT Ops and Business Analytics, but the Power User template only allows low, med and high, and every agent run reported High. Both cells now say High, so the slide shows what a live run produces. The table still tells the story through DevOps ("High, escalate IT Ops") and the server-side root cause. The Power User template gets one line defining the three levels, so runs stay consistent: high = money or checkout is being lost now, med = degraded but not losing money, low = nothing to act on. The slide HTML is rebuilt (only these two cells differ). Co-Authored-By: Claude Sonnet 5.5 --- .cursor/agents/s4r-power-user.md | 2 ++ demo-slides/s4r-demo-slides.html | 4 ++-- demo-slides/s4r-demo-slides.md | 4 ++-- 3 files changed, 6 insertions(+), 4 deletions(-) diff --git a/.cursor/agents/s4r-power-user.md b/.cursor/agents/s4r-power-user.md index cfeb44b..3c2805e 100644 --- a/.cursor/agents/s4r-power-user.md +++ b/.cursor/agents/s4r-power-user.md @@ -66,6 +66,8 @@ Data generation and workshop modes: `docs/s4r/SA-S4R-APP.md`. Orchestration desi **Dashboard panels:** IT Ops ✓/✗ · DevOps ✓/✗ · Business ✓/✗ · Security ✓/✗ ``` +**Severity scale (use only these):** **high** = money or checkout is being lost now, **med** = degraded but not losing money, **low** = nothing to act on. + ## Guardrails - **Delegation:** User says delegate → four Task subagents (parallel). Parent agent runs **synthesis only**, not team SPL. diff --git a/demo-slides/s4r-demo-slides.html b/demo-slides/s4r-demo-slides.html index 816e033..53284c7 100644 --- a/demo-slides/s4r-demo-slides.html +++ b/demo-slides/s4r-demo-slides.html @@ -646,7 +646,7 @@

Buttercup Insights

IT Ops 503/404 dominate -Critical +High DevOps @@ -656,7 +656,7 @@

Buttercup Insights

Business Analytics Revenue lost via lookup (hundreds of $) -Critical +High Security diff --git a/demo-slides/s4r-demo-slides.md b/demo-slides/s4r-demo-slides.md index d6a3575..4998860 100644 --- a/demo-slides/s4r-demo-slides.md +++ b/demo-slides/s4r-demo-slides.md @@ -565,9 +565,9 @@ flowchart TB | Team | Finding | Severity | | ------ | --------- | -------- | -| IT Ops | 503/404 dominate | Critical | +| IT Ops | 503/404 dominate | High | | DevOps | Roughly flat ~40% (34–43%) | High, escalate IT Ops | -| Business Analytics | Revenue lost via lookup (hundreds of $) | Critical | +| Business Analytics | Revenue lost via lookup (hundreds of $) | High | | Security | No geo concentration (~1 failure per city) | Low | **Root-cause hypothesis:** Server-side failure, not fraud