From 2121dbecad969809669d52d43fc895f906401d9f Mon Sep 17 00:00:00 2001 From: Phil Merrell Date: Fri, 10 Jul 2026 09:35:52 -0600 Subject: [PATCH] chore(kaizen): weekly research scan 2026-07-10 Generated by the kaizen-research skill. Top 5 ideas appended to docs/kaizen/review-queue.md for the kaizen-review-prep run later this morning. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/kaizen/research/2026-07-10.md | 213 +++++++++++++++++++++++++++++ docs/kaizen/review-queue.md | 39 ++++++ 2 files changed, 252 insertions(+) create mode 100644 docs/kaizen/research/2026-07-10.md diff --git a/docs/kaizen/research/2026-07-10.md b/docs/kaizen/research/2026-07-10.md new file mode 100644 index 00000000..4409fe0c --- /dev/null +++ b/docs/kaizen/research/2026-07-10.md @@ -0,0 +1,213 @@ +# Kaizen Research — Friday, July 10, 2026 +> Scan window: July 3 – July 10, 2026 (7 days) +> Web budget: ~58/50 used (modest overage — 13 subagents + version-pin registry churn; see Web Budget block). + +## TL;DR + +**Quiet external week, loud internal one — and they point the same direction.** The ecosystem is in a lull before the July 28 MCP RC GA, so the week's real signal is a *second, independent* reliability bug in the AgentCore SDK we haven't upgraded: **#571 — `AgentCoreMemorySessionManager` reorders events across processes (emits `tool_result` before `tool_use`), a class-level-counter regression that corrupts multi-replica Runtime deployments** (fixed upstream, pending a release after 1.17.0). It lands next to the still-unadopted **#482 SSE-deadlock fix (in 1.17.0)** — and both bite the exact surfaces the product *just* leaned harder on: 1.1.0/1.2.0 shipped **Scheduled Runs + Memory Spaces**, which make AgentCore Memory and multi-replica Runtime load-bearing. **The #1 idea is unchanged from last week and now doubly-forced: bump `bedrock-agentcore` off 1.9.1.** The most pressing internal signal is that last week's two recommended-Ship dep bumps (agentcore, Strands) **did not land** while two feature-dense releases did — the exposure grew as the feature surface grew. + +## External Scan + +### What's moving this week + +The shape of the week is **a pre-RC lull with two concrete reliability tremors.** No new frontier model reached Bedrock (OpenAI GPT-5.6 and Gemini 3.5 Pro's slip to July 17 are both off-Bedrock; Anthropic's week was governance/culture posts), no pricing change, no reference-repo commits, and the MCP blog is frozen until the July 28 final. Against that quiet backdrop, three things actually moved: **Strands shipped 1.46 then 1.47 (today)** — the headline is `continue_on_error` on the MCP client, a direct resilience lever for our flaky external/Gateway MCP servers; **FastMCP flipped its Host/Origin validation twice in four days** (3.4.3 hardened SSRF but broke serverless/reverse-proxy — our exact Lambda-behind-Gateway topology — and 3.4.4 restored it), making ≥3.4.4 the safe floor for our externally-hosted servers; and the **AgentCore SDK surfaced #571**, a fresh multi-replica Memory-corruption bug distinct from the known #564. On UX, Vercel's AI SDK matured tool-approval from last week's basic human-in-the-loop into a **server-side policy layer with cryptographically-signed approvals**, and MCP Apps' host matrix expanded (M365 Copilot, Goose, Postman, Archestra) — validating our SEP-1865 host implementation as on-spec. The surprise is the convergence: independent sources (agentcore #571, Strands 1.47 `continue_on_error`, FastMCP compat) all point at **MCP + Memory resilience** — the same paths Scheduled Runs and Memory Spaces just promoted to production-critical. + +### Notable items by source + +> **Annotation conventions:** +> - `*relevance*:` — impact-on-existing-code lens. +> - `*unlocks*:` — capability-unlock lens (new primitive / new product surface). + +#### AWS Bedrock / AgentCore +- **AgentCore Runtime now publishes an `ActiveSessionCount` CloudWatch metric** — Per-minute gauge under `AWS/Bedrock-AgentCore` (dimensioned by `Service`), the only net-new July release-note entry. Lets you alarm on the concurrent-session quota you're consuming. — https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/release-notes.html — *relevance*: our inference-api runtime; pair with the SSE 600s-timeout debugging note. Low-effort ops win, no code change. — *unlocks*: early-warning on session-leak/exhaustion — a defensive pairing with the #482 deadlock exposure (a hung container shows session pileup before the 429). +- **New Bedrock console: model-compare + eval projects** — Summit-era console refresh; console-only, no API/SDK surface. — https://aws.amazon.com/blogs/aws/top-announcements-of-the-aws-summit-in-new-york-2026/ — *relevance*: marginal (humans picking model ids); nothing to integrate. +- Everything else on the AgentCore release-notes page (Gateway stateful MCP sessions, Identity Secrets Manager ARNs, runtime quota increases, managed Harness GA, Managed KB GA) sits under **June 2026** — already logged. No new model/region/quota in-window. + +#### Strands Agents +**Latest `strands-agents==1.47.0` (Jul 10 — today); we pin 1.40.0 (May 14) — now 7 minors behind.** The dual-language monorepo lumps TS + Python commits into auto-drafted release notes; `CHANGELOG.md` no longer exists at repo root (use the Releases API). Only breaking change 1.45→1.47 is an irrelevant in-process memory-store rename (`LocalMemoryStore`→`TestMemoryStore`, not AgentCore Memory). +- **1.47.0 — `continue_on_error` on the MCP client (#3101)** — A flaky MCP server no longer aborts the whole turn. — https://github.com/strands-agents/sdk-python/releases/tag/python%2Fv1.47.0 — *relevance*: our external/Gateway MCP resilience (`FilteredMCPClient` / gateway targets) — a flaky server currently kills the turn. — *unlocks*: graceful degradation across multi-source tool turns; the most directly useful new capability in the 1.40→1.47 jump. +- **1.47.0 — durable message identifiers (#2836)** — *relevance*: touches session-persistence / interrupted-turn machinery (stable ids across rehydration). +- **1.46.0 — surface `cache_read`/`cache_write` input tokens in the metadata chunk (#2302)** (Anthropic provider) + **OTel `tool_trace` on interrupted tool calls (#3031)** — *relevance*: the cache-token accounting touches the same surface as `CountTokensBedrockModel` / the context-attribution badge; the interrupted-tool trace maps to our interrupt-resume work. +- **Open issue #3144 — `CacheConfig(strategy="auto")` never caches the system prompt** — *relevance*: **directly questions whether our cache-point config in `to_bedrock_config` actually engages** — a silent full-input-token cost regression if it doesn't. (See idea #3.) Also #3076 (nested agent-as-tool interrupts don't resume across rehydration) and #3120 (`serve_a2a` never hits AgentCore idle timeout — the CLAUDE.md A2A-server note) map to our roadmap. + +#### Reference repo (aws-samples/sample-strands-agent-with-agentcore) +- **No commits in the July 3–10 window** (verified via GitHub API `since/until` → empty). Latest activity is still July 1 (Sonnet 5 `NO_TEMPERATURE_MODELS` + the React/Tailwind frontend modernization) — all captured last week. — https://github.com/aws-samples/sample-strands-agent-with-agentcore/commits/main — *applicability*: nothing to port. If it stays dormant next Friday, drop this source to a **bi-weekly cadence**. + +#### MCP ecosystem +- **No new blog post or spec change July 3–10.** The 2026-07-28 RC stays locked until the July 28 final publish; the beta SDKs (Jun 29), EMA (Jun 18), and the RC itself (May 21) are all pre-window. — https://blog.modelcontextprotocol.io — *relevance*: next week is the GA-cutover scan to watch. Standing watch-item unchanged: the RC's stateless core routes on `Mcp-Method` and lets clients cache `tools/list` to a server-set `ttlMs` — affects how our Gateway MCP targets are fronted (round-robin vs sticky) once we adopt the July-28 SDKs. + +#### FastMCP +**Two in-window releases (was 3.4.2 / Jun 6); latest 3.4.4 (Jul 9).** Not pinned in our repo — lives in the external Lambda MCP server repos behind Gateway, so any auto-tracking build picks these up. +- **v3.4.3 (Jul 5) — SSRF + OAuth hardening** — Streamable HTTP now validates **Host and Origin before session handling** (blocks DNS rebinding + NAT64/IPv6 SSRF bypasses), rejects unsafe OAuth redirect schemes, fixes proxy session-teardown races. — https://github.com/jlowin/fastmcp/releases — *implications*: the stricter Host/Origin guard is the one to verify against the Gateway/Lambda-URL front — it caused a compat regression fixed four days later. +- **v3.4.4 (Jul 9) — regression fix + HF OAuth provider** — Relaxes the 3.4.3 host-origin defaults to **restore compatibility with ASGI/serverless/reverse-proxy deployments** (i.e. exactly our Lambda-behind-Gateway topology). No breaking changes. — https://github.com/jlowin/fastmcp/releases — *implications*: **if any of our external servers briefly ran 3.4.3, the Host/Origin guard could have rejected Gateway-fronted requests. ≥3.4.4 is the safe floor** — recommend pinning externally-hosted servers to `>=3.4.4` rather than tracking latest, given two guard flips in four days. (See Risks.) + +#### Agentic UI/UX patterns +- **Vercel AI SDK — policy-based tool approvals + signed approvals** — Beyond last week's basic human-in-the-loop: a 4-state model (`not-applicable`/`approved`/`denied`/`user-approval`) driven by **per-tool input-inspecting policy functions** (gate on `amount`/`recipient`/`role`) deciding auto-approve vs prompt server-side, plus **cryptographically-signed approvals** (`experimental_toolApprovalSecret`) so tampered approval history is rejected. — https://ai-sdk.dev/docs/agents/tool-approvals — *fit*: pattern-only (Angular signal store) — *where it'd land*: a policy check in the approval hook (`apis/shared`) + a new approval-state on the `tool_use` SSE card. The **policy layer** (not the raw per-call prompt) is the new idea. Enriches the queued [2026-07-03] tool-approval item. +- **MCP Apps — host matrix expanded** — Client list now spans Claude/Desktop, VS Code Copilot, **M365 Copilot, Goose, Postman, MCPJam, Archestra.AI**; the overview reinforces UI-preload-before-tool-call — exactly our early-mount `ui_resource` + `ui_tool_input_partial` design. — https://modelcontextprotocol.io/extensions/apps/overview — *fit*: **direct validation, no port** — our SEP-1865 host implementation is on-spec and interoperable with a growing host set. +- **assistant-ui 0.14.26 (Jul 4) — roving-tabindex keyboard nav** for thread/tool lists (arrow-key nav, Right→"More"); companion `assistant-stream@0.3.25` added server-side MCP connection timeouts. — https://github.com/Yonom/assistant-ui/releases — *fit*: pattern-only — *where it'd land*: a keyboard-nav directive on the chat message-list / session sidebar (signals focus index); a cheap WCAG 2.1 AA win. +- **assistant-ui `JSONGenerativeUI` compiler (0.0.8, Jul 4)** — streamed JSON → rendered components; a "model streams a UI spec → host compiles" pattern maturing as a discrete package. — https://github.com/Yonom/assistant-ui/releases — *fit*: pattern-only (**watch, don't build**) — an alternative to iframe-sandboxed MCP Apps for *trusted first-party* generative UI; a design option for the Agent Designer epic, not a near-term port. + +#### Frontier model announcements +- **Anthropic/Bedrock: quiet week, no capability delta.** July 3–10 posts are governance/culture (Bernanke to the LTBT, "reflect on your Claude usage," "The Making of Claude Code," Fable 5 cyber-safeguards). Opus 4.8 on Bedrock unchanged. — https://www.anthropic.com/news — *relevance*: none to our harness. +- **OpenAI GPT-5.6 (Sol/Terra/Luna) GA + Gemini 3.5 Pro slipped to July 17 (2M context, "Deep Think")** — both **off-Bedrock**; watchlist only. — *relevance*: no change to our Bedrock model set today. Gemini's 2M context is the one delta to re-check if it reaches Vertex/Bedrock. GPT-5.6's split of generation-number from capability-tier naming is a model-selector UX note. + +#### Agent harness patterns +- **Claude Code 2.1.200–206 — loop-resilience + MCP timeout** — `2.1.206` fixes **MCP servers ignoring per-server `request_timeout_ms`** (timing out at the 60s default); `2.1.205` fixes a **user turn silently lost at `--max-turns`** and stale terminal state on resumed background agents; `2.1.203` fixes background sessions hanging on a **stale daemon token**; `2.1.200` flips the permission default to **Manual**. — https://github.com/anthropics/claude-code/blob/main/CHANGELOG.md — *relevance*: the per-tool timeout is a direct resilience item for our slow Gateway/MCP targets (configurable per-tool vs one global). The max-turns loss + stale-token hang reinforce the "surface as error, don't hang" pattern we apply to SSE — audit that our loop flushes pending input at a quota/max-iteration cutoff and refreshes OAuth/token-vault creds mid-stream rather than hanging. +- **(Idea-only) LangChain v3 content-block streaming + async background subagents + improved tool-error propagation** — https://docs.langchain.com/oss/python/releases/changelog — *relevance*: the typed per-channel streaming projection echoes our SSE event-type contract; confirms the industry direction (async subagents + tool-error propagation) our Claude Code baseline already leads. No Anthropic engineering post this week (index tops at Apr 23). + +#### Pricing / quota +- **No change this window.** Sonnet 5 promo still $2/$10 through Aug 31 (reverts $3/$15); Opus 4.8 unchanged. The runtime quota increase (concurrent sessions → 5,000 us-east-1/us-west-2, 200 interactions/s + 25 new-sessions/s) is dated **Jul 1** — pre-window, richer numbers than last week's note but no architectural impact. — https://aws.amazon.com/bedrock/pricing/ + +#### Community + GitHub issues +- **AgentCore SDK #571 — `AgentCoreMemorySessionManager` reorders events across processes** (emits `tool_result` before `tool_use`, model rejects the turn). Root cause: `_get_monotonic_timestamp()` inflates by a full 1s on collision and holds counter state at *class* level, so separate processes corrupt each other's ordering (regression from PR #83). **Fixed via PR #572 (updated Jul 9); lands in a version after 1.17.0.** — https://github.com/aws/bedrock-agentcore-sdk-python/issues/571 — **NEW, high relevance:** distinct from #564; the class-level counter directly bites multi-replica Runtime deployments like ours. Reinforces the upgrade case beyond the SSE fix — but note the fix is **not yet released** (latest is still 1.17.0), so watch for 1.18. +- **AgentCore SDK #567 — `AgentCoreToolSearchPlugin` should accept `tool_filters`** (OPEN, Jul 6) — https://github.com/aws/bedrock-agentcore-sdk-python/issues/567 — *relevance*: upstream converging on the filtering primitive we hand-rolled in `FilteredMCPClient` — watch for a native replacement (subtraction candidate). #564 (eventually-consistent `ListEvents` drops history) still OPEN, unresolved even in 1.17.0. +- **Anthropic cookbook — "plan-big-execute-small" coordinator cost pattern (PR #754, merged Jul 6)** — large model plans, small models execute, with an architecture diagram. — https://github.com/anthropics/anthropic-cookbook/commits/main — *relevance*: maps onto our multi-model Strands/Converse routing and the Nova-Micro-for-titles pattern; a candidate reference for a cost-tiered coordinator (cf. tool-search token-bloat). +- **HN/community quiet on our stack this week** — no in-window threads; third-party "when to skip the managed Harness" writeups continue to validate the #570 GO-with-boundary stance. — https://clawaws.com/blog/agentcore-harness-explained/ + +#### opencode (harness reference) +- **v1.17.14–18 (Jul 6–9); latest v1.17.18.** — https://github.com/anomalyco/opencode/releases — *Tooling*: v1.17.14 added a **"code mode MCP adapter for confined orchestration scripts"** (sandboxed script-driven tool orchestration beyond one-tool-per-call) + fixed paginated MCP catalogs losing tool metadata — worth a glance vs our multi-protocol ToolRegistry. *Cost*: continued per-model reasoning-effort + prompt-cache routing (xAI/Grok, OpenRouter small-model variants) — reinforces the cheap-vs-capable lever. *Context*: better context-overflow error classification (error-surfacing only, no new compaction machinery). + +#### LibreChat +- **No in-window release; latest confirmed v0.8.7 (Jun 24, 2026 — a genuine 2026 release, last week's "stale" worry resolved).** — https://github.com/danny-avila/LibreChat/releases — carry-over (pattern-only): the v0.8.7 **inline Context Usage gauge** confirms "surface context consumption inline" is now table-stakes (parallels our PR #433 badge); **on-the-fly skill authoring with GitHub sync** is a user/agent-self-authoring divergence from our admin-curated Skills model — worth watching. + +#### Seasonal +- Out of window — none scanned this week. + +### Patterns worth considering + +- **MCP + Memory resilience as the load-bearing surface** — Independent sources this week (agentcore #571 cross-process Memory reorder, Strands 1.47 `continue_on_error`, FastMCP 3.4.4 Gateway compat) all harden the MCP/Memory paths. Scheduled Runs + Memory Spaces (1.1.0) just made those paths production-critical. + - **Where**: Strands 1.47, agentcore SDK #571/#564, FastMCP 3.4.4. + - **Fit**: the two dep bumps we've queued for weeks now sit on top of *concrete, in-the-wild corruption/deadlock bugs* that the new features amplify. `continue_on_error` retires any hand-rolled MCP-abort handling. + - **Verdict**: Worth trying (ship the bumps). +- **Server-side approval policy + integrity, not just a prompt** — Vercel's approval model decides auto-vs-prompt server-side via input-inspecting policy functions, and signs approvals so history can't be forged. + - **Where**: Vercel AI SDK tool-approvals. + - **Fit**: enriches the queued tool-approval item with a policy layer + integrity check; lands in our approval hook (`apis/shared`), not the raw per-call prompt. + - **Verdict**: Worth trying (fold into the existing tool-approval item). + +## Internal Audit + +### Activity (last 7 days) +- **Commits on develop**: ~30 across the 1.1.0 + 1.2.0 release train — Agent Designer (skill/tool/model binding, live preview, param governance, KB-in-designer), Scheduled Runs (dispatcher + worker + interval cadence + Run-now), Memory Spaces (data layer → SPA panel → sharing → export → consolidation), KB Sync. High feature velocity. +- **PRs opened**: 1 open (#615 — target Agents instead of Assistants on scheduled runs) — **merged**: 1.1.0 (#604) + 1.2.0 (#612) release trains — **reverted**: 0. +- **Issues opened (7d)**: 1 — #559 (Epic: Agentic platform primitives — Harness + Registry). **closed**: n/a. +- **CI failures (workflow → count)**: **Nightly Build & Test → 2** (Jul 6, Jul 7 on `main`); Backend Deploy → 1 (Jul 6, develop push); per-PR CI failures on the Agent-Designer-Phase-4 and Memory-A5-SPA branches (both since merged — likely transient). + +### Repeated friction signals +- **Recommended-Ship dep bumps still haven't landed while two feature releases did** (the recurring pattern, now escalating) — the [2026-07-03] review marked **agentcore 1.17 (#482)** and **Strands 1.45** as *Ship first*; a week later the pins are **byte-identical** (strands 1.40.0, bedrock-agentcore 1.9.1), while 1.1.0/1.2.0 shipped Scheduled Runs + Memory Spaces + Agent Designer + KB Sync. + - **Hypothesis**: the kaizen bumps still lack a forcing function that competes with feature work — same as the "hygiene ships under a security label, not a kaizen one" read from 07-03. But the reliability case *strengthened* this week: **#571** is a second Memory-corruption bug in an unadopted version, and the new features increase exposure to exactly the Memory/multi-replica-Runtime surfaces #482/#571 corrupt. + - **Fix candidate**: ship the **agentcore bump** as an incident-prevention PR framed against Scheduled Runs/Memory Spaces (not hygiene), then the Strands 1.47 keystone with `continue_on_error`. Both now have concrete, feature-tied forcing functions. +- **Nightly still not confidently green** — failed Jul 6 and Jul 7 on `main`. The [2026-06-19] nightly item stays open; #518 was incomplete per last week. Still the dep-bump gate. + +### Version-pin lag +| Dep | Pinned | Latest | Release date | Lag | Notes | +|---|---|---|---|---|---| +| strands-agents | 1.40.0 | **1.47.0** | 2026-07-10 | 7 minors / 57d | 🔴 moved today. `continue_on_error` (#3101), durable msg ids; only breaking = in-process memory-store rename (N/A) | +| bedrock-agentcore | 1.9.1 | 1.17.0 | 2026-07-02 | 8 minors / 50d | **#482 SSE-deadlock fixed in 1.17.0; #571 Memory-reorder fix pending a post-1.17.0 release** — we're exposed to both | +| boto3 | 1.43.9 | 1.43.45 | 2026-07-09 | 36 patches / 55d | daily API-model patches; non-breaking | +| fastapi | 0.136.1 | 0.139.0 | 2026-07-01 | 3 minors / 68d | unchanged vs last week; 0.x minors can carry breaks | +| docling | 2.81.0 | **2.111.0** | 2026-07-08 | 33 minors / 109d | 🔴 moved (2.109→2.111); still blocks #405 `.txt` uploads | +| @angular/core | 21.2.17 | **22.0.6** | 2026-07-08 | **1 major** | 🔴 moved (22.0.5→22.0.6); major migration, not a kaizen bump — hold | +| vitest | 4.1.5 | 4.1.10 | 2026-07-06 | 5 patches | 🔴 moved (4.1.9→4.1.10); patch-only, safe | +| @analogjs/vitest-angular | 3.0.0-alpha.30 | 3.0.0-alpha.61 | 2026-07-08 | 31 alphas | `latest` tag is 2.6.3 (older track); we track 3.x alpha deliberately | +| aws-cdk-lib | 2.260.0 | 2.261.0 | 2026-07-02 | 1 release | no new release this week; non-breaking | +| constructs | 10.6.0 | 10.6.0 | 2026-03-23 | 0 | ✅ up to date | +| fastmcp (unpinned) | — | 3.4.4 | 2026-07-09 | — | external servers only; **≥3.4.4 is the safe floor** (3.4.3 Host/Origin guard breaks Gateway-fronted deployments) | + +### Retirement candidates +- **Reference repo (aws-samples/sample-strands-agent-with-agentcore) → consider bi-weekly cadence** — zero commits this window; dormant since July 1. Not a source to drop, but a weekly full-subagent scan is over-provisioned while it's quiet. Re-check next Friday. +- **`AgentCoreToolSearchPlugin` `tool_filters` (SDK #567)** — if this native filtering primitive ships, it's a subtraction candidate against our hand-rolled `FilteredMCPClient`. Watch, don't act. +- **`angualar-best-practices` skill (typo'd, 185+ days)** — carried from prior reviews; still **not** a confirmed retirement (verify usage before acting). No new dormant skills this week. + +### Risks introduced this week + +- **We are now exposed to TWO unadopted AgentCore SDK reliability bugs** — #482 (SSE deadlock, fixed in 1.17.0) and **#571 (cross-process Memory event-reorder corruption, fix pending)** — https://github.com/aws/bedrock-agentcore-sdk-python/issues/571 — Scheduled Runs (multi-replica Runtime) and Memory Spaces (heavier Memory use) amplify exactly these failure classes. #571's fix isn't released yet, so 1.17.0 closes #482 but not #571 — watch for 1.18. +- **FastMCP 3.4.3 Host/Origin guard can reject Gateway-fronted requests** — https://github.com/jlowin/fastmcp/releases — any externally-hosted MCP server that auto-tracked latest between Jul 5–9 could have started 403-ing Gateway traffic; **≥3.4.4 is the safe floor.** Verify our external server pins. +- **Prompt caching may silently not engage** — Strands #3144 (`CacheConfig(strategy="auto")` never caches the system prompt) — https://github.com/strands-agents/sdk-python/issues — if our `to_bedrock_config` cache-point config is silently inert, every turn pays full input-token cost. Auditable now that 1.46 surfaces `cache_read`/`cache_write` tokens. (Idea #3.) +- **Nightly green unconfirmed** — failed Jul 6–7 on `main`; the dep bumps (#1/#2) would land on a suite whose status isn't trustworthy. +- **Angular 22 major lag widening** (now 22.0.6) — carried watch item; schedule a migration spike, don't bump reactively. + +## Ideas — Top 5 (ranked) + +| # | Idea | Surface | Effort | Impact | Subtracts? | Unlocks? | +|---|---|---|---|---|---|---| +| 1 | Bump `bedrock-agentcore` off 1.9.1 — now double-forced (#482 SSE deadlock + NEW #571 Memory-reorder), exposure grown by Scheduled Runs/Memory Spaces | backend | M | H | Retires queued [2026-05-22] #482 guard | — | +| 2 | Bump Strands 1.40 → 1.47; adopt `continue_on_error` MCP resilience (#3101) + hook ordering | backend | M–H | H | Candidate: hand-rolled MCP-abort handling; custom cache-point plumbing | Flaky Gateway/external MCP no longer aborts the turn; `Limits` cost caps | +| 3 | Audit whether prompt caching actually engages in `to_bedrock_config` (Strands #3144) | backend | L–M | M–H | — (cost-correctness audit) | Potentially large per-turn cost cut if caching is silently off | +| 4 | Tool-approval **policy layer** + signed approvals (Vercel AI SDK) — evolve the queued approval item | frontend + backend | M | M | Replaces ad-hoc synthetic-error approval handling | Server-side auto-vs-prompt policy; tamper-proof approval history | +| 5 | Wire a CloudWatch `ActiveSessionCount` alarm on the inference-api runtime | infra | L | M | — (ops addition — justified: early-warning for the #482 failure class) | Catches session-leak/exhaustion before a 429 | + +### 1. Bump `bedrock-agentcore` off 1.9.1 — now double-forced (#482 + #571) +- **Source**: https://github.com/aws/bedrock-agentcore-sdk-python/pull/563 (#482, in 1.17.0) + **NEW** https://github.com/aws/bedrock-agentcore-sdk-python/issues/571 (Memory reorder, fix pending); internal exposure grown by Scheduled Runs + Memory Spaces. +- **Surface area**: `backend/pyproject.toml`, inference-api chat router, `AgentCoreMemorySessionManager` usage; full local pytest suite. +- **Change**: bump to 1.17.0 to close #482 now; **track 1.18** for the #571 fix (not yet released) and bump again when it lands. Verify the async→sync streaming-bridge fix is active on `/invocations`; drop any planned hand-written #482 guard. +- **Subtracts**: retires the queued [2026-05-22] hand-written #482 guard — library-native subtraction. +- **Effort × Impact**: Med × High. +- **Verdict**: Worth trying — **highest priority; two Memory/Runtime corruption bugs sit in versions we skip, and the week's new features amplify them.** Supersedes/updates the [2026-07-03] agentcore queue item with the #571 evidence + the feature-surface-growth forcing function. + +### 2. Bump Strands 1.40 → 1.47; adopt `continue_on_error` (#3101) + hook ordering +- **Source**: Strands 1.46–1.47 (https://github.com/strands-agents/sdk-python/releases); supersedes the [2026-07-03] 1.45 queue item. +- **Surface area**: `backend/src/agents/main_agent/` (hooks, `to_bedrock_config`, compaction), `FilteredMCPClient`/gateway targets, `CountTokensBedrockModel`. +- **Change**: bump to 1.47.0; **adopt `continue_on_error` on the MCP client** so a flaky external/Gateway MCP server degrades gracefully instead of aborting the turn; adopt optional hook ordering (#2559) for the OAuth-consent + tool-approval `BeforeToolCall` sequence through the tool-fold; audit `cache_tools_ttl` / `context_manager="auto"` / `Limits` (per decisions.md 2026-05-18 — not a bare compaction swap). Only breaking change is the N/A memory-store rename. +- **Subtracts**: candidate — hand-rolled MCP-abort handling; custom cache-point plumbing; runaway-guard via `Limits`. +- **Unlocks**: graceful multi-source MCP degradation; `Limits` per-invocation cost caps; deterministic hook ordering (the enabler for idea #4). +- **Effort × Impact**: Med–High × High. +- **Verdict**: Worth trying — after #1. `continue_on_error` is newly relevant given Scheduled Runs lean on external tools unattended. + +### 3. Audit whether prompt caching actually engages in `to_bedrock_config` (Strands #3144) +- **Source**: **NEW** — Strands open issue #3144 (`CacheConfig(strategy="auto")` never caches the system prompt); 1.46 now surfaces `cache_read`/`cache_write` tokens in the metadata chunk (#2302). +- **Surface area**: `backend/src/agents/main_agent/` cache-point config in `to_bedrock_config`; the metadata/usage path feeding `CountTokensBedrockModel` + the context-attribution badge. +- **Change**: assert cache points are actually written and read — inspect `cache_read`/`cache_write` token counts across a multi-turn session; if the system prompt (or tool schema) isn't caching, fix the cache-point placement. Cheap to measure now that the tokens are surfaced. +- **Subtracts**: no — a cost-correctness audit (may confirm we're fine, or expose a silent full-input-token regression). +- **Unlocks**: potentially large per-turn cost reduction if caching is silently off; a verifiable caching invariant. +- **Effort × Impact**: Low–Med × Med–High. +- **Verdict**: Worth trying — cheapest high-upside item this week; the measurement is nearly free. + +### 4. Tool-approval policy layer + signed approvals (evolve the queued approval item) +- **Source**: Vercel AI SDK tool-approvals (https://ai-sdk.dev/docs/agents/tool-approvals); builds on the queued [2026-07-03] tool-approval item. +- **Surface area**: tool-approval `BeforeToolCall` hook (`apis/shared`), `tool_use`/`tool_result` SSE contract, frontend tool-call card + signal store; reuses `beginContinuationStreaming`. +- **Change**: add a **server-side policy layer** deciding auto-approve vs `user-approval` via per-tool input-inspecting functions (gate on the tool's args, not just its identity); render the approval-requested state on the tool card with input args shown; optionally sign approvals so history can't be forged. Auto-resume once all approvals in a turn resolve. +- **Subtracts**: replaces ad-hoc synthetic-error approval handling with explicit lifecycle states. +- **Unlocks**: fine-grained auto-vs-prompt policy; tamper-proof approval audit; closes the "approval hook can't see through the tool-fold" hole (pairs with #2's hook ordering). +- **Effort × Impact**: Med × Med. +- **Verdict**: Worth trying — sequence after #2's hook ordering lands. Fold into the existing queued approval item rather than a separate track. + +### 5. Wire a CloudWatch `ActiveSessionCount` alarm on the inference-api runtime +- **Source**: **NEW** — AgentCore Runtime `ActiveSessionCount` metric (https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/release-notes.html). +- **Surface area**: infrastructure — a CloudWatch alarm on the inference-api runtime's `AWS/Bedrock-AgentCore` `ActiveSessionCount` gauge (PlatformStack observability). +- **Change**: alarm when concurrent sessions approach the (raised) quota; a hung container from the #482 deadlock manifests as session pileup — this catches it before a 429 and before users on unrelated sessions report stalls. +- **Subtracts**: no — an ops addition; justified as cheap early-warning for the exact failure class idea #1 fixes (defense-in-depth while the bump is pending). +- **Unlocks**: proactive detection of session-leak/exhaustion and the #482 hang. +- **Effort × Impact**: Low × Med. +- **Verdict**: Worth trying — low-effort ops win that pairs with #1. + +## Take + +The system is trending *with* the ecosystem, but this week the story is discipline, not discovery: the external world was quiet, and the loudest signal is that **the reliability bumps we've recommended for weeks still haven't shipped — while the product added exactly the features (Scheduled Runs, Memory Spaces) that make the un-fixed bugs more dangerous.** The single change that matters most is still the **`bedrock-agentcore` bump**, now doubly-forced by a second Memory-corruption bug (#571) landing next to the #482 deadlock. Phil would notice idea #3 first as an operator if it fires — silent caching-off would be a real bill — but idea #1 is the one to ship before the next incident, and it's cheaper to frame it against Scheduled Runs than as hygiene. If only one thing lands this cycle, land the agentcore bump; if two, add the Strands 1.47 `continue_on_error` keystone, because unattended scheduled runs against flaky external tools are a turn-aborting hazard we can now configure away. + +--- + +## Sources Scanned + +| # | Source | URL | Accessed | Items | +|---|---|---|---|---| +| 1 | AWS Bedrock/AgentCore What's New + release notes | https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/release-notes.html | 2026-07-10 | 2 | +| 2 | Strands Agents releases (monorepo) + PyPI | https://github.com/strands-agents/sdk-python/releases | 2026-07-10 | 5 | +| 3 | Reference repo commits | https://github.com/aws-samples/sample-strands-agent-with-agentcore/commits/main | 2026-07-10 | 0 (dormant) | +| 4 | MCP blog / spec | https://blog.modelcontextprotocol.io | 2026-07-10 | 1 (quiet) | +| 4a | FastMCP releases + PyPI | https://github.com/jlowin/fastmcp/releases | 2026-07-10 | 2 | +| 4b | Agentic UI/UX (Vercel AI SDK, assistant-ui, MCP Apps) | https://ai-sdk.dev/docs/agents/tool-approvals | 2026-07-10 | 4 | +| 5 | Frontier models (Anthropic/OpenAI/Google/Meta) | https://www.anthropic.com/news | 2026-07-10 | 3 | +| 6 | Agent harness (Claude Code CHANGELOG + LangChain) | https://github.com/anthropics/claude-code/blob/main/CHANGELOG.md | 2026-07-10 | 2 | +| 6a | opencode releases | https://github.com/anomalyco/opencode/releases | 2026-07-10 | 3 | +| 7 | Bedrock pricing | https://aws.amazon.com/bedrock/pricing/ | 2026-07-10 | 0 (no change) | +| 8 | AgentCore SDK / toolkit issues | https://github.com/aws/bedrock-agentcore-sdk-python/issues/571 | 2026-07-10 | 4 | +| 9 | Community (HN) + Anthropic cookbook | https://github.com/anthropics/anthropic-cookbook/commits/main | 2026-07-10 | 2 | +| 12 | LibreChat releases | https://github.com/danny-avila/LibreChat/releases | 2026-07-10 | 0 (no in-window) | +| 19 | Version pins (PyPI + npm registry JSON) | https://pypi.org/pypi/strands-agents/json | 2026-07-10 | 10 deps | + +## Web Budget + +Used: **~58 / 50 requests** (modest overage). +Skipped (unreachable / rate-limited): Reddit (domain-blocked for WebFetch — WebSearch summaries only); one Anthropic "reflect on your Claude usage" post body (guessed URL 404'd — flagged title-only, not verified). +Skipped (other): none material. +Notes: overage driven by the **13-subagent fan-out** + the version-pin subagent fighting registry JSON parsing (an early WebFetch summarizer misparsed truncated dates; a curl+jq fallback produced authoritative numbers) and the Strands subagent disambiguating the dual-language monorepo release tags. Signal density was moderate — the material items (agentcore #571, Strands 1.47 `continue_on_error`, FastMCP 3.4.4 Gateway compat, Strands #3144 caching) justified the fan-out; frontier/pricing/reference were dry and could be lighter next quiet week. diff --git a/docs/kaizen/review-queue.md b/docs/kaizen/review-queue.md index 270b8f2d..a3b9ebd7 100644 --- a/docs/kaizen/review-queue.md +++ b/docs/kaizen/review-queue.md @@ -5,6 +5,45 @@ Items added by `kaizen-research`, consumed by `kaizen-review-prep`. ## Open +### [2026-07-10] Bump `bedrock-agentcore` off 1.9.1 — now double-forced (#482 SSE deadlock + NEW #571 Memory-reorder) +- **Source**: research/2026-07-10.md ▸ Top 5 #1 — https://github.com/aws/bedrock-agentcore-sdk-python/pull/563 (#482, in 1.17.0) + **NEW** https://github.com/aws/bedrock-agentcore-sdk-python/issues/571 (cross-process Memory event-reorder corruption, fix pending a post-1.17.0 release). +- **Surface**: backend (`backend/pyproject.toml`, inference-api chat router, `AgentCoreMemorySessionManager` usage; full local pytest suite) +- **Effort × Impact**: M × H +- **Subtracts**: yes — retires the queued [2026-05-22] #482 hand-written guard (library-native subtraction) +- **Status**: open — **updates/supersedes the [2026-07-03] agentcore bump item with #571 evidence + a feature-tied forcing function.** 1.1.0/1.2.0 shipped Scheduled Runs (multi-replica Runtime) + Memory Spaces (heavier Memory use), amplifying exactly the failure classes #482/#571 corrupt. Bump to 1.17.0 now (closes #482); **track 1.18 for the #571 fix** (not yet released — latest is still 1.17.0). Frame the PR as incident-prevention against the new features, not hygiene. + +### [2026-07-10] Bump Strands 1.40 → 1.47; adopt `continue_on_error` MCP resilience (#3101) + hook ordering +- **Source**: research/2026-07-10.md ▸ Top 5 #2 — Strands 1.46–1.47 (https://github.com/strands-agents/sdk-python/releases). **Supersedes the [2026-07-03] 1.45 item** (now 7 minors behind). +- **Surface**: backend (`agents/main_agent/` hooks + `to_bedrock_config` + compaction, `FilteredMCPClient`/gateway targets, `CountTokensBedrockModel`) +- **Effort × Impact**: M–H × H +- **Subtracts**: candidate — hand-rolled MCP-abort handling (`continue_on_error`), custom cache-point plumbing (`cache_tools_ttl`), runaway guard (`Limits`) +- **Unlocks**: a flaky external/Gateway MCP server no longer aborts the turn (newly relevant — Scheduled Runs use external tools unattended); `Limits` per-invocation cost caps; deterministic hook ordering (the enabler for the tool-approval fix) +- **Status**: open — adopt `continue_on_error` on the MCP client + optional hook ordering (#2559); audit `cache_tools_ttl`/`context_manager="auto"`/`Limits` (decisions.md 2026-05-18 bars a bare compaction swap). Only breaking change 1.45→1.47 is the N/A in-process memory-store rename. Run full local pytest; watch compaction + context-attribution. + +### [2026-07-10] Audit whether prompt caching actually engages in `to_bedrock_config` (Strands #3144) +- **Source**: research/2026-07-10.md ▸ Top 5 #3 — **NEW** Strands open issue #3144 (`CacheConfig(strategy="auto")` never caches the system prompt); Strands 1.46 now surfaces `cache_read`/`cache_write` tokens in the metadata chunk (#2302). https://github.com/strands-agents/sdk-python/issues +- **Surface**: backend (`agents/main_agent/` cache-point config in `to_bedrock_config`; the metadata/usage path feeding `CountTokensBedrockModel` + the context-attribution badge) +- **Effort × Impact**: L–M × M–H +- **Subtracts**: no — a cost-correctness audit (may confirm we're fine, or expose a silent full-input-token regression) +- **Unlocks**: potentially large per-turn cost cut if caching is silently off; a verifiable caching invariant +- **Status**: open — cheapest high-upside item this week. Assert cache points are written *and* read by inspecting `cache_read`/`cache_write` counts across a multi-turn session; fix cache-point placement if the system prompt/tool schema isn't caching. Measurement is nearly free now that 1.46 surfaces the tokens. + +### [2026-07-10] Tool-approval policy layer + signed approvals (Vercel AI SDK) — evolve the queued approval item +- **Source**: research/2026-07-10.md ▸ Top 5 #4 — Vercel AI SDK tool-approvals (https://ai-sdk.dev/docs/agents/tool-approvals). Builds on the [2026-07-03] tool-approval item. +- **Surface**: frontend + backend (tool-approval `BeforeToolCall` hook in `apis/shared`, `tool_use`/`tool_result` SSE contract, frontend tool-call card + signal store; reuses `beginContinuationStreaming`) +- **Effort × Impact**: M × M +- **Subtracts**: yes — replaces ad-hoc synthetic-error approval handling with explicit approve/deny/user-approval lifecycle states +- **Unlocks**: server-side **policy layer** deciding auto-approve vs prompt via per-tool input-inspecting functions (gate on the tool's args, not just identity); cryptographically-signed, tamper-proof approval history; closes the "approval hook can't see through the tool-fold" hole (pairs with the Strands hook-ordering bump) +- **Status**: open — **fold into the queued [2026-07-03] tool-approval item** rather than run a separate track; sequence after the Strands hook-ordering bump lands. The new-this-week piece is the policy layer + integrity check, beyond last week's basic human-in-the-loop. + +### [2026-07-10] Wire a CloudWatch `ActiveSessionCount` alarm on the inference-api runtime +- **Source**: research/2026-07-10.md ▸ Top 5 #5 — **NEW** AgentCore Runtime `ActiveSessionCount` metric (https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/release-notes.html) +- **Surface**: infrastructure (CloudWatch alarm on the inference-api runtime's `AWS/Bedrock-AgentCore` `ActiveSessionCount` gauge; PlatformStack observability) +- **Effort × Impact**: L × M +- **Subtracts**: no — ops addition; justified as cheap early-warning for the exact failure class the agentcore bump fixes (defense-in-depth while the bump is pending) +- **Unlocks**: proactive detection of session-leak/exhaustion and the #482 hang (a hung container manifests as session pileup) before a 429 +- **Status**: open — low-effort ops win that pairs with the agentcore bump. Alarm when concurrent sessions approach the raised quota (5,000 us-west-2). + ### [2026-07-06] Spike: managed AgentCore Harness as the headless/scheduled run engine — ✅ SPIKE + Q2 LIVE PROBE COMPLETE, recommend Ship (headless-only, GO-with-boundary) - **Source**: `scoping/2026-07-06-managed-harness-build-vs-adopt.md` (brief) + `scoping/2026-07-06-managed-harness-spike-findings.md` (**findings — 3 gating questions answered**). Surfaced while dogfooding scheduled runs (Phil asked whether we use the AWS Harness feature; we use the lower-level Runtime). AWS **managed Harness** is now GA. - **Surface**: backend (`apis/shared/harness/run_agent_headless` — swap the Runtime `/invocations` target for an `InvokeHarness` endpoint on the headless lane only; swap `sse.py` accumulator for a Converse-stream one → same `RunResult`) + infra (a managed-Harness resource + OAuth-inbound JWT authorizer — a 1:1 port of our existing Runtime `customJwtAuthorizer`, `inference-agentcore-construct.ts:275`). Interactive `inference-api` untouched.