Skip to content

fix: security scan remediation, eval tier-comparison UI, dependency upgrades, agent architecture docs - #3

Merged
khanrubd merged 1 commit into
mainfrom
develop
Nov 8, 2026
Merged

khanrubd merged 1 commit into
mainfrom
develop

Conversation

@khanrubd

@khanrubd khanrubd commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Summary

This branch carries four workstreams:

  1. Security remediation. Every finding from an automated security code review was verified against the code. The confirmed ones are fixed, each with tests.
  2. Eval results UI. Sweeps started from the UI now show the same cross-arm, per-tier summary tables as results/matrix.md, and the page offers a matrix.md download.
  3. Dependency upgrades. This includes mcp / mcp-types 2.3.0, which was previously blocked by strands-agents.
  4. Documentation. There is a new docs/agent-architecture.md with three figures. The existing architecture figures are realigned with the deployed infrastructure.

Security remediation

Gateway and identity

  • The Gateway REQUEST interceptor fails closed on malformed or unparseable bodies.
  • The interceptor keeps the detail of configuration errors, and it no longer caches an empty allowlist.
  • The SPA client allowlist is enforced on the API channel. It is read from an SSM parameter and wired only to the routed tools.
  • principal_type is forwarded on the agent channel.

Pagination

  • Cursors are now opaque and bound to the caller. They are encrypted with a dedicated KMS key under an encryption context naming the caller's tenant, the rep and the issuing operation.
  • A cursor that is malformed, tampered with, or presented by another caller or to another operation is rejected as a validation error.
  • A KMS refusal returns a 5xx and is no longer misreported as an auth or validation failure.
  • Paging resumes after the last row returned.
  • Search metadata is filtered by the caller's scope, and listing-scope checks are enforced.

Writes and proposals

  • update_* writes are conditional. A lost race reports fields_changed, and log_activity now does the same.
  • Approve and rebase re-check:
    • the primary-contact reference;
    • cascades;
    • the stage machine.
  • Revert re-checks stage moves.
  • Revert refuses to restore a primary contact that is dangling or no longer referenced.
  • Store transport errors map to ServiceBusy. Stalled batch reads back off.
  • Handler denials are logged.

Agents and runtime

  • Runtime session events are bounded.
  • pipeline_aggregate is bounded.
  • Free-text fields and chat prompts have length caps.
  • The vendor SDK transport and its loggers are locked down. The logger lock-down now lives in a stdlib-only module, which keeps httpx2 and pyyaml out of the Lambda zip import closures.

Browser and session

  • CloudFront enforces a Content-Security-Policy. A shared CSP-violation trap is added to the browser journeys.
  • Sign-out revokes the refresh token and resets the user's settings. A settings save that lands after sign-out is discarded.
  • Reading an eval run requires full visibility.

Infrastructure (Terraform)

  • A pagination-cursor CMK, with grants to the paginating action Lambdas.
  • An SSM parameter for the API client allowlist.
  • A CloudFront response-headers policy carrying the CSP.
  • New grant rows in the action-role matrix, including an index-only Query for approve_proposal.

Not changed, by decision

  • No finding was addressed by requiring MFA. MFA stays optional.
  • One finding was reviewed and declined as not warranting a behaviour change.

Eval results UI

  • TierComparisonSummary shows the cross-arm tier comparison tables from the backend matrix renderer. Inline code spans render as <code> elements; no raw HTML is injected. The stored tier-comparison input is validated before rendering.
  • The Download matrix.md button serves the same rendered markdown as the CLI artifact.
  • New browser journey 21-tier-comparison. Journey 08 is updated for the download control.

Dependency upgrades

Area Changes
Python mcp / mcp-types 2.1.1 → 2.3.0, strands-agents 1.62.0 → 1.64.0, boto3/botocore 1.43.121 → 1.43.125, anthropic 2.3.0 → 2.6.0, openai 3.41.0 → 3.46.0, ruff 0.17.3 → 0.17.4, types-PyYAML
Frontend vite 8.3.6, tailwindcss / @tailwindcss/postcss 4.3.5, @vitejs/plugin-react 6.2.1, vitest / @vitest/coverage-v8 5.1.0, eslint 10.14.0, jsdom 31.0.0, globals 17.14.0
Relay @aws-sdk/client-bedrock-agentcore / client-ssm 3.1163.0, vitest 5.1.0
Browser journeys @playwright/test 1.65.0
  • requirements.txt and every npm lockfile are regenerated alongside these.
  • mcp 2.3.0 omits the empty _meta object from tools/call requests that 2.2.0 and earlier sent. The MCP compatibility test's recorded request shape is updated to match.
  • The plan fixtures are regenerated.
  • The iam_roles and lambda_actions smoke expectations are updated for the new grant rows.

Smoke, scripts and runbook

  • IAM simulation of KMS key policies. These policies use Resource: "*", so the smoke harness substitutes the key ARN before simulating them.
  • Gateway array-body check. The Gateway itself answers a JSON-RPC array body with a tool error (isError: true), without invoking the interceptor or any target. The smoke row now passes on a JSON-RPC error or isError: true. It fails on any tool output without isError.
  • MCP secret rotation. rotate_mcp_client_secret.py rolls back a newly added secret whose add response cannot be read. It deletes every secret except the old one and writes no file.
  • Canary gate, demo seeder and smoke harness. All three treat ServiceBusy wrapping a connection error as unreachable.
  • Runbook additions:
    • resetting a lost MCP client secret through Cognito;
    • the command for the remote canary-tenant emptiness check.

Documentation

  • docs/agent-architecture.md covers:
    • the AgentCore deployment;
    • the supervisor → subagents → tools workflow;
    • a single chat turn, with a figure for each of these.
  • The architecture figures are cross-checked against the deployed stack and updated:
    • the deployment figure shows both interceptors;
    • the network figure shows the cursor and state-backend CMKs, and the CSP;
    • the user-plane figure shows the allowlist SSM parameter;
    • the test ladder shows the make verify targets.
  • docs/security-baseline.md adds:
    • the allowlist parameter;
    • refresh-token revocation;
    • the length caps;
    • the eval-read visibility rule;
    • the KMS row for the cursor key.
  • README.md and docs/development.md are refreshed to match.

Testing

  • make verify (lint, format, type-check, unit, stub, integration, gates, shellcheck): green. The unit suite is 15,790 passed.
  • Local reproduction of the GitLab and GitHub CI jobs, plus a fresh-clone run: green. GitLab CI on the pushed branch: passed.
  • Against a deployed dev stack:
    • the full post-deploy verification (Layers 4–6) had 0 failures, with only previously documented deferrals;
    • all browser journeys 01–21 passed;
    • the MCP client probe passed on mcp 2.3.0;
    • the canary tenant was confirmed empty.
  • No live model sweep was run. Eval changes were exercised with the stub sweep.

Upgrade notes

  • Run terraform apply, which creates the cursor KMS key, the allowlist SSM parameter and the CSP headers policy.
  • Pagination cursors issued before the upgrade are not accepted. Clients should restart the listing.

This branch carries four workstreams:

1. **Security remediation.** Every finding from an automated security code review was verified against the code. The confirmed ones are fixed, each with tests.
2. **Eval results UI.** Sweeps started from the UI now show the same cross-arm, per-tier summary tables as `results/matrix.md`, and the page offers a `matrix.md` download.
3. **Dependency upgrades.** This includes `mcp` / `mcp-types` 2.3.0, which was previously blocked by `strands-agents`.
4. **Documentation.** There is a new `docs/agent-architecture.md` with three figures. The existing architecture figures are realigned with the deployed infrastructure.

## Security remediation

### Gateway and identity
- The Gateway REQUEST interceptor fails closed on malformed or unparseable bodies.
- The interceptor keeps the detail of configuration errors, and it no longer caches an empty allowlist.
- The SPA client allowlist is enforced on the API channel. It is read from an SSM parameter and wired only to the routed tools.
- `principal_type` is forwarded on the agent channel.

### Pagination
- Cursors are now opaque and bound to the caller. They are encrypted with a dedicated KMS key under an encryption context naming the caller's tenant, the rep and the issuing operation.
- A cursor that is malformed, tampered with, or presented by another caller or to another operation is rejected as a validation error.
- A KMS refusal returns a 5xx and is no longer misreported as an auth or validation failure.
- Paging resumes after the last row returned.
- Search metadata is filtered by the caller's scope, and listing-scope checks are enforced.

### Writes and proposals
- `update_*` writes are conditional. A lost race reports `fields_changed`, and `log_activity` now does the same.
- Approve and rebase re-check:
  - the primary-contact reference;
  - cascades;
  - the stage machine.
- Revert re-checks stage moves.
- Revert refuses to restore a primary contact that is dangling or no longer referenced.
- Store transport errors map to `ServiceBusy`. Stalled batch reads back off.
- Handler denials are logged.

### Agents and runtime
- Runtime session events are bounded.
- `pipeline_aggregate` is bounded.
- Free-text fields and chat prompts have length caps.
- The vendor SDK transport and its loggers are locked down. The logger lock-down now lives in a stdlib-only module, which keeps `httpx2` and `pyyaml` out of the Lambda zip import closures.

### Browser and session
- CloudFront enforces a Content-Security-Policy. A shared CSP-violation trap is added to the browser journeys.
- Sign-out revokes the refresh token and resets the user's settings. A settings save that lands after sign-out is discarded.
- Reading an eval run requires full visibility.

### Infrastructure (Terraform)
- A pagination-cursor CMK, with grants to the paginating action Lambdas.
- An SSM parameter for the API client allowlist.
- A CloudFront response-headers policy carrying the CSP.
- New grant rows in the action-role matrix, including an index-only `Query` for `approve_proposal`.

### Not changed, by decision
- No finding was addressed by requiring MFA. MFA stays optional.
- One finding was reviewed and declined as not warranting a behaviour change.

## Eval results UI

- `TierComparisonSummary` shows the cross-arm tier comparison tables from the backend matrix renderer. Inline code spans render as `<code>` elements; no raw HTML is injected. The stored tier-comparison input is validated before rendering.
- The **Download matrix.md** button serves the same rendered markdown as the CLI artifact.
- New browser journey `21-tier-comparison`. Journey 08 is updated for the download control.

## Dependency upgrades

| Area | Changes |
|---|---|
| Python | `mcp` / `mcp-types` 2.1.1 → 2.3.0, `strands-agents` 1.62.0 → 1.64.0, `boto3`/`botocore` 1.43.121 → 1.43.125, `anthropic` 2.3.0 → 2.6.0, `openai` 3.41.0 → 3.46.0, `ruff` 0.17.3 → 0.17.4, `types-PyYAML` |
| Frontend | `vite` 8.3.6, `tailwindcss` / `@tailwindcss/postcss` 4.3.5, `@vitejs/plugin-react` 6.2.1, `vitest` / `@vitest/coverage-v8` 5.1.0, `eslint` 10.14.0, `jsdom` 31.0.0, `globals` 17.14.0 |
| Relay | `@aws-sdk/client-bedrock-agentcore` / `client-ssm` 3.1163.0, `vitest` 5.1.0 |
| Browser journeys | `@playwright/test` 1.65.0 |

- `requirements.txt` and every npm lockfile are regenerated alongside these.
- `mcp` 2.3.0 omits the empty `_meta` object from `tools/call` requests that 2.2.0 and earlier sent. The MCP compatibility test's recorded request shape is updated to match.
- The plan fixtures are regenerated.
- The `iam_roles` and `lambda_actions` smoke expectations are updated for the new grant rows.

## Smoke, scripts and runbook

- **IAM simulation of KMS key policies.** These policies use `Resource: "*"`, so the smoke harness substitutes the key ARN before simulating them.
- **Gateway array-body check.** The Gateway itself answers a JSON-RPC array body with a tool error (`isError: true`), without invoking the interceptor or any target. The smoke row now passes on a JSON-RPC error or `isError: true`. It fails on any tool output without `isError`.
- **MCP secret rotation.** `rotate_mcp_client_secret.py` rolls back a newly added secret whose add response cannot be read. It deletes every secret except the old one and writes no file.
- **Canary gate, demo seeder and smoke harness.** All three treat `ServiceBusy` wrapping a connection error as unreachable.
- **Runbook additions:**
  - resetting a lost MCP client secret through Cognito;
  - the command for the remote canary-tenant emptiness check.

## Documentation

- `docs/agent-architecture.md` covers:
  - the AgentCore deployment;
  - the supervisor → subagents → tools workflow;
  - a single chat turn, with a figure for each of these.
- The architecture figures are cross-checked against the deployed stack and updated:
  - the deployment figure shows both interceptors;
  - the network figure shows the cursor and state-backend CMKs, and the CSP;
  - the user-plane figure shows the allowlist SSM parameter;
  - the test ladder shows the `make verify` targets.
- `docs/security-baseline.md` adds:
  - the allowlist parameter;
  - refresh-token revocation;
  - the length caps;
  - the eval-read visibility rule;
  - the KMS row for the cursor key.
- `README.md` and `docs/development.md` are refreshed to match.

## Testing

- `make verify` (lint, format, type-check, unit, stub, integration, gates, shellcheck): green. The unit suite is 15,790 passed.
- Local reproduction of the GitLab and GitHub CI jobs, plus a fresh-clone run: green. GitLab CI on the pushed branch: passed.
- Against a deployed dev stack:
  - the full post-deploy verification (Layers 4–6) had 0 failures, with only previously documented deferrals;
  - all browser journeys 01–21 passed;
  - the MCP client probe passed on `mcp` 2.3.0;
  - the canary tenant was confirmed empty.
- No live model sweep was run. Eval changes were exercised with the stub sweep.

## Upgrade notes

- Run `terraform apply`, which creates the cursor KMS key, the allowlist SSM parameter and the CSP headers policy.
- Pagination cursors issued before the upgrade are not accepted. Clients should restart the listing.
Comment thread scripts/smoke/cloudfront.py Dismissed
Comment thread tests/unit/test_infra_csp.py Dismissed
Comment thread tests/unit/test_infra_cursor_key.py Dismissed
@khanrubd
khanrubd merged commit fc527e1 into main Nov 8, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants