Skip to content

fix(byok): provider route binding (superseded by #10) - #8

Closed
knowlet wants to merge 26 commits into
mainfrom
fix/byok-provider-route-binding
Closed

knowlet wants to merge 26 commits into
mainfrom
fix/byok-provider-route-binding

Conversation

@knowlet

@knowlet knowlet commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Superseded by #10. The AI configuration fixes are consolidated in #10. Demo seeding, case UUID, ledger and replay repairs were deliberately excluded from the consolidated patch and remain available on this branch for separate work. The evidence below applies to this original branch.

Closed as a superseded validation draft; no merge was performed. The source branch is preserved.


Summary

Bind tenant BYOK credentials to their selected provider endpoint so a direct provider or custom server cannot accidentally receive the deployment gateway's route or key. Align runtime consumers, connection probes and status, preserve intentional local endpoints without authentication, and clarify provider-specific model guidance. Repair the seeded CORE ledger defects exposed by the required real-console walkthrough; this is a validation-only draft in the contributor fork, not a merge request for fork main.

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactor / tech debt
  • CI / infra change

Related issues

Related contribution requirements: beenuar#353. No issue is closed by this validation draft.

Changes

  • Resolve endpoint, model and credential source together in the API and separately packaged agents service. Inherit deployment credentials only for the same complete route; prevent ambient SDK credentials from filling an explicitly unauthenticated custom route.
  • Apply routing consistently to triage, Explain, connection probes and authenticated tenant status; redact URL-carried secrets and align air-gap policy. Other AI workloads retain the deployment gateway.
  • Use a supported direct Anthropic default and compatible request parameters. Clarify hosted/default versus custom endpoint model names in Settings and update the generated API description.
  • Document native Bedrock through the gateway and distinguish the limited AWS OpenAI-compatible API. Preserve native IAM/profile/bearer configuration outside UI BYOK.
  • Add real HTTP consumer tests and nine authenticated PostgreSQL/vault/API cases with locked dependencies and zero-skip enforcement.
  • Repair only recognizable synthetic demo cases/runs: persist provenance, use canonical case UUIDs, preserve existing events/artifacts, and exclude private operator investigations from public replay. Add six live PostgreSQL regressions and a broken-linkage negative control.
  • Add real CORE browser acceptance after smoke/demo setup, with labelled synthetic screenshots and no mocked API/session injection. Keep existing assertions and database-role boundaries intact; fetch canonical release tags read-only for fork validation.

Testing

Evidence is for final head 4ef8ee2fa28b67a18d54bf075533c5d4e8ba0bde, unless labelled before-fix or local.

  • 15 live PostgreSQL tests passed, zero skipped: nine authenticated BYOK/vault/API/provider-HTTP tests and six demo-seed regressions. Both deliberate regressions failed their intended assertions (wrong probe Authorization and noncanonical case linkage); the entire migrations job and role scanner passed (job).
  • Two real browser tests passed against the actual CORE API and Next console, covering visible synthetic provenance, the persisted investigation ledger and provider model guidance. Golden pipeline and literal make smoke each passed all 10 stages (workflow).
  • Full Integration and Compose smoke workflows passed, including both historical upgrade matrices. Compose smoke executes literal make up → make doctor → make smoke on already booted, verified images with pulling/building disabled; this does not prove a fresh public-registry installation. Doctor reported one pre-smoke warning (no alerts yet) and no failures; smoke then passed all 10 stages.
  • Whole-tree Python lint/type checks, web lint/type checks, web unit tests and OpenAPI export passed. Complete CI passed all 32 jobs. The full Python job recorded API/GraphQL 5,455 passed / 254 skipped / 1 xfailed, agents 2,613 passed / 20 skipped / 219 subtests, and repository tests 2,013 passed / 10 skipped / 15 subtests. All follow-on reachability, discovery and contract gates passed. Optional/live-suite skips are reported separately from the dedicated 15-test live acceptance, which had zero skips.
  • All 11 eval axes passed with no regression at the console's reported precision; MITRE delta is +0.00 pp. Full Before/After summaries and their exact commits are below. Both retain the same two pre-existing failing templates.
  • CodeQL succeeded; the remaining medium finding in scripts/check_session_refresh_live.mjs is unchanged from the target upstream base. Newly introduced lint/type/CodeQL findings were fixed, one resolved mypy baseline entry was removed, and no mypy baseline allowance or CodeQL suppression was added.
  • Unexecuted blockers: iOS job 131107615630 and macOS published-install job 131107609626 were cancelled before receiving a runner, with no executed steps. They are not passes; the complete check set is not green.
  • Before-fix real-stack acceptance reproduced a missing synthetic label and zero ledger runs through the console's UUID lookup (job); the repaired journey above passes.
  • Local helper suite: nine cases and 15 subcases passed; changed Python files passed pinned Ruff lint/format. Local Docker socket access is unavailable. Local integration/full-suite and manual boxes stay unchecked; mocked preview tests are not used as CORE evidence.
  • Self-review and independent code review completed; all authored commits have DCO sign-offs. No real paid provider/AWS account access was tested.

This fork's main is 17 upstream commits behind target base 4d8790e21489bffdb33bdbd1ea9ea712899efaf9. The exact-base comparison contains the proposed 34-file patch; the fork PR comparison also includes those historical upstream commits. Upstream PR creation is blocked by the integration's HTTP 403.

  • Unit tests added / updated
  • Integration tests pass locally
  • Manually tested (describe steps below)
Manual test steps

The following paths were automated in CI; they are not a claim of manual local execution.

  1. Run authenticated API/vault/provider requests using the locked API environment and a real PostgreSQL database; require both positive assertions and failing negative controls.
  2. Start the CORE spine, run doctor/bootstrap/login, pipeline positive/negative controls, make smoke and make demo, then sign in through the actual console. Verify the LockBit case, synthetic marker, persisted ledger and Settings model guidance.
  3. Inspect final lint/type/schema/eval results and retained browser captures. The separate Compose workflow exercises literal make up on verified images; native AWS permissions and cancelled macOS jobs remain unverified.

Screenshots (if UI changes)

The browser's download security gate denied artifact download because it could not establish source authority. The original CI artifacts are retained, but no pixel-level visual inspection is claimed. Artifacts expire after the workflow's 14-day retention period.

Checklist

  • My code follows the style guidelines of this project (ruff, eslint, gofmt)
  • I have performed a self-review of my own code
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally
  • I have updated documentation as needed (README, CHANGELOG, CONTRIBUTING)
  • My changes do not introduce new linter warnings
  • I have checked for sensitive data / credentials in my diff

Eval harness (required for substrate / playbook / detection changes)

Required if this PR touches: services/agents/, services/api/app/orchestrator/,
playbooks/packs/, detections/, services/agents/tests/eval_data/,
scripts/generate_eval_incidents.py, scripts/run_evals.py, or any prompt /
RAG corpus / response template under those trees.

Run python3 scripts/run_evals.py --json --out /tmp/report.json before and
after your change and paste both summary blocks below. Any axis that
regresses must be called out explicitly. Playbook-touching PRs must include
the playbook_completion_rate block — orphan playbooks / templates fail CI.

Before (paste scripts/run_evals.py output on base branch)
Base: 4d8790e21489bffdb33bdbd1ea9ea712899efaf9
Source: https://github.com/beenuar/AiSOC/actions/runs/43128241220/job/130885681787
Recorded command: python3 scripts/run_evals.py --ci --out eval_report.json
Run completed successfully on 2026-11-12; this is retrieved CI evidence, not a local rerun.

==============================================================================
  AiSOC Pillar-1 Eval - 200-incident synthetic benchmark
==============================================================================
  [PASS] mitre_accuracy               accuracy                     0.970  (target >= 0.80)
         per-template macro       0.964  (target >= 0.80, n=55 templates) [PASS] (2 failing templates)
           regressions: outlook-auto-forward-rule, compromised-ci-runner
  [PASS] alert_reduction              reduction_ratio              0.753  (target >= 0.70)
  [PASS] investigation_completeness   mean_keyword_coverage        0.943  (target >= 0.85)
         per-template macro       0.943  (target >= 0.80, n=55 templates) [PASS]
  [PASS] response_quality             mean_rubric_score            1.000  (target >= 0.80)
         per-template macro       1.000  (target >= 0.75, n=55 templates) [PASS]
  [PASS] hunt_corpus                  positive_scenario_catch_rate 1.000  (target >= 1.00)
  [PASS] adversary_eval               graceful_degradation_catch_rate 0.475  (target >= 0.40)
  [PASS] confidence_calibration       investigation_brier_score    0.065  (target <= 0.18)
  [PASS] memory_recall                memory_recall_accuracy       1.000  (target >= 1.00)
  [PASS] override_accuracy            override_accuracy            1.000  (target >= 1.00)
  [PASS] playbook_completion_rate     completion_rate              0.735  (target >= 0.50)
  [PASS] detection_fp_rate            worst_per_rule_fp_rate       0.003  (target <= 0.05)
------------------------------------------------------------------------------
  Synthetic telemetry: 361 events across 14 sources, 200 incidents wired up (services/agents/tests/eval_data/synthetic_telemetry.jsonl)
------------------------------------------------------------------------------
  Per-investigation budget (deterministic substrate, model=gpt-4o)
    tokens   mean=2186  median=2114  p95=2452  p99=2478
    USD      mean=$0.01469  median=$0.01368  p95=$0.01693  p99=$0.01699
    latency  p50=0.0093 ms  p95=0.0141 ms  p99=0.0183 ms  (substrate path)
==============================================================================
  PASS — ALL GATES GREEN
  Report written to: eval_report.json
After (paste scripts/run_evals.py output on this branch)
Head: 4ef8ee2fa28b67a18d54bf075533c5d4e8ba0bde
Source: https://github.com/knowlet/AiSOC/actions/runs/43196173098/job/131107744098
Reported console precision only; raw artifact JSON was not downloaded.

Recorded command: python3 scripts/run_evals.py --ci --out eval_report.json

==============================================================================
  AiSOC Pillar-1 Eval - 200-incident synthetic benchmark
==============================================================================
  [PASS] mitre_accuracy               accuracy                     0.970  (target >= 0.80)
         per-template macro       0.964  (target >= 0.80, n=55 templates) [PASS] (2 failing templates)
           regressions: outlook-auto-forward-rule, compromised-ci-runner
  [PASS] alert_reduction              reduction_ratio              0.753  (target >= 0.70)
  [PASS] investigation_completeness   mean_keyword_coverage        0.943  (target >= 0.85)
         per-template macro       0.943  (target >= 0.80, n=55 templates) [PASS]
  [PASS] response_quality             mean_rubric_score            1.000  (target >= 0.80)
         per-template macro       1.000  (target >= 0.75, n=55 templates) [PASS]
  [PASS] hunt_corpus                  positive_scenario_catch_rate 1.000  (target >= 1.00)
  [PASS] adversary_eval               graceful_degradation_catch_rate 0.475  (target >= 0.40)
  [PASS] confidence_calibration       investigation_brier_score    0.065  (target <= 0.18)
  [PASS] memory_recall                memory_recall_accuracy       1.000  (target >= 1.00)
  [PASS] override_accuracy            override_accuracy            1.000  (target >= 1.00)
  [PASS] playbook_completion_rate     completion_rate              0.735  (target >= 0.50)
  [PASS] detection_fp_rate            worst_per_rule_fp_rate       0.003  (target <= 0.05)
------------------------------------------------------------------------------
  Synthetic telemetry: 361 events across 14 sources, 200 incidents wired up (services/agents/tests/eval_data/synthetic_telemetry.jsonl)
------------------------------------------------------------------------------
  Per-investigation budget (deterministic substrate, model=gpt-4o)
    tokens   mean=2186  median=2114  p95=2452  p99=2478
    USD      mean=$0.01469  median=$0.01368  p95=$0.01693  p99=$0.01699
    latency  p50=0.0073 ms  p95=0.0114 ms  p99=0.0306 ms  (substrate path)
==============================================================================
  PASS — ALL GATES GREEN
  Baseline compare: MITRE Δ = +0.00 pp (allowed drop ≤ 1.00 pp) [OK]
  • Not applicable — this PR does not touch substrate, playbooks, detections, or eval inputs
  • No axis regressed
  • An axis regressed; rationale and follow-up are described in the Summary above

knowlet and others added 18 commits November 11, 2026 00:29
## Summary

Fix Settings deep links such as /settings?tab=deployment opening
Profile, plus related Case and Responder navigation defects.

## Root causes and changes

- Settings ignored the query string via useState('profile'). All 15
sections now read the URL.
- CaseWorkspace read the tab only at mount. Tab clicks, query changes,
investigation transitions and browser history now stay synchronized.
- Responder redirected every legacy shortcut to Triage. New shortcuts
use canonical routes; existing Cases and On-call links redirect
correctly.

A shared useUrlTab hook validates tabs, preserves other query parameters
and fragments, supports Reload/Back/Forward, and avoids duplicate
history entries. Settings adds Suspense and active controls expose
aria-current.

## Testing

Verified on 7b5788d:
- Lint: passed, 0 errors / 95 existing warnings; no new warnings.
- TypeScript: passed.
- Production and Storybook builds: passed.
- Vitest: 121 files / 1,199 tests passed; coverage gate passed.
- E2E: 13 passed / 1 pre-existing skipped. All 6 new browser tests and 5
new HTTP tests passed without retries.
- CodeQL: passed, no new alerts in changed code.
- Ruff lint/format and mypy baseline: passed.

Negative controls reproduced the original failures on the unchanged
base. HTTP assertions use the existing HTML parser; both initial CodeQL
test-regex findings are resolved.

Validation logs and CI links: #5

## Checklist

- [x] Bug fix with regression tests and research notes
- [x] No new lint warnings, dependencies or workflow changes
- [x] DCO sign-off included
- [x] Contribution guidance in beenuar#353 followed; this does not close beenuar#353
- [x] Eval harness not applicable (frontend navigation only)

---------

Signed-off-by: Albert Hsieh <knowlet@users.noreply.github.com>
)

## Summary
The browser stores a refresh token but never calls POST
/api/v1/auth/refresh. The first protected 401 deletes both tokens, so
the default 30-minute access expiry discards a usable seven-day refresh
credential.

## Type of change
- [x] Bug fix
- [x] Documentation update

## Related issues
None filed.

## Changes
- Share refresh and one-time replay across JSON, streams, downloads and
authenticated fetch callers.
- Preserve method, body, headers, tenant and cancellation. Keep
recoverable credentials on transient failures.
- Prevent pending requests from restoring a logged-out session or
replaying under a different login.
- Align responder guards with shared JWT validity and refresh
availability.
- Restore the missing Explain proxy rewrite exposed by the route gate.
- Add regression tests, real HTTP/API/PostgreSQL checks and CHANGELOG.
Token lifetimes, backend security checks and dependencies are unchanged.

## Testing
- [x] Unit tests added and updated
- [x] Local integration tests passed
- [ ] Rendered-browser manual test

| Check | Local result |
| --- | --- |
| Web suite | 1,194 passed across 122 files |
| TypeScript and production build | Passed |
| ESLint | 0 errors, 95 warnings identical to base |
| Live API/PostgreSQL session suite | 10 passed, 0 skipped |
| Production client, real HTTP | 4 passed; base client fails; restored
fix passes |
| Ruff check/format and mypy baseline | Passed |
| Console route/wire/auth-header gates | Passed |
| Attribution, naming and whitespace gates | Passed |

HTTP verification uses real API authentication and PostgreSQL, simulated
browser storage/navigation, and disabled unrelated lifespan workers. It
is not rendered-browser, complete-stack or operator-deployment
validation. Remote CI has not run.

Full root cause, design and verification details:
docs/research/session-refresh.md.

## Screenshots
Not applicable: session transport behavior only.

## Checklist
- [x] Style, self-review and regression coverage complete
- [x] Documentation and CHANGELOG updated
- [x] No new lint warnings
- [x] Diff checked for credentials
- [x] DCO signed off

## Eval harness
- [x] Not applicable: no substrate, playbook packs, detections, prompts
or eval inputs changed

Signed-off-by: Albert Hsieh <knowlet@users.noreply.github.com>
…r#1822) (beenuar#1823)

Closes beenuar#1822.

Verified before changing anything: `services/agents/app/api/metrics.py`
`_authorise` compares against `METRICS_TOKEN` and answers 401 with
'metrics endpoint requires METRICS_TOKEN outside development' when it is
unset — and in `docker-compose.yml` only the **api** service received
the variable; the agents block had no occurrence. So on a default
deployment every scrape of `:15002/metrics` got 401, exactly as @pitimon
reported (their compose override + agents recreate confirmed the fix
shape in their deployment).

One line plus a comment: the agents environment now carries the same
`METRICS_TOKEN: ${METRICS_TOKEN:-}` passthrough the api block already
had. `docker compose config` validates locally. The changelog entry
lands in the consolidated changelog-integrity PR alongside this wave's
other entries.

(Branch created via the API: git-protocol pushes are currently refused
with 403 on this machine while REST writes work — same constraint as
previously recorded.)
…#1821) (beenuar#1824)

Closes beenuar#1821.

Verified before changing anything: `record_poll_failure` does
`error_count + 1` and `record_poll_success`'s values dict never touched
the column — @pitimon's reported case (a static '19 errors' for days
beside a fresh `last_sync` and +237k ingested events) is exactly what a
lifetime counter produces and what a consecutive counter cannot.

Took the reporter's **option 1**: `error_count` now means *consecutive
failed polls since the last success* — success writes 0, failure
increments. That matches how the fleet panel presents the number
(current condition beside `last_sync`), and the zero-when-healthy
rendering already hides the badge on healthy rows. Two tests pin the
semantics at the only two writers (success resets to 0; failure still
increments), asserted against the compiled UPDATE statements. 49
connector tests pass locally including the full scheduler suite; ruff
format clean.

Changelog entry lands in the consolidated changelog-integrity PR for
this wave.
…loses beenuar#1820) (beenuar#1825)

Closes beenuar#1820.

@pitimon asked the right question — deliberate exclusion or not yet
reached? — and the honest answer is **neither defensible**: the gate
listed wazuh among the unclassified debt ('requests a capped page and
never follows a continuation'), and the docstring's deliberate-cap
rationale ('drop the polling interval, not chase a single huge batch')
is the exact argument this repo has rejected elsewhere, because the cut
is silent and the abutting windows mean the overflow is never
re-requested by anything.

**The fix**: `_search` — the one executor both fetch paths share — now
follows `search_after` (OpenSearch's documented continuation for sorted
queries) to the end of the window, appending a `_doc` tiebreaker so
equal timestamps cannot skip or repeat rows across pages. Properties,
matching the paging doctrine: the opening page keeps the existing
never-raise contract exactly (request error / non-200 / bad JSON → `[]`
+ the existing wazuh.* warnings); a failed later page **keeps what was
read** (`connector.paging.next_page_*`); a full page whose last hit
carries no sort values stops rather than looping on an uncontinuable
read; and the 25-page bound announces itself (`page_cap_reached`) with
the data-loss consequence stated. Both alert and vulnerability-findings
reads page identically through the one loop. Docstrings now say what is
true; the old rationale is kept as history rather than deleted.

**Gate**: `MAX_UNCLASSIFIED` falls 49 → 48 and the audit now counts
wazuh among the followers (43 follow, 1 reviewed, 48 unclassified —
verified locally).

**Tests**: 9 new transport-level cases (multi-page read with
`search_after` echo verification, tiebreaker presence, short-page end,
failed/transport-error later pages keeping partials, opening failure
still `[]`, sortless full page stopping safely, page-cap bound, and the
vulnerability path paging) plus the existing wazuh connector suite and
gate tests — 46 pass locally; ruff format clean.

Changelog entry lands in the consolidated changelog-integrity PR for
this wave. (Branch created via the API — git-protocol pushes currently
403 on this machine while REST works.)
## Summary

The Agentmetry inbox template drops optional source context needed to
investigate a finding. Preserve selected account provenance,
supporting-event references, recorder coverage and MCP schema
observations under `finding.evidence.agentmetry`, keeping reported
account claims separate from `actor.user` and excluding full account
email and opaque account/user IDs.

Existing field paths, the activity/finding split, the severity ladder
and the OCSF 1.1.0 declaration remain unchanged.

## Type of change

- [x] Bug fix
- [x] Documentation update

## Related discussion

beenuar#1577

## Changes

- Extend `agentmetry.yaml` with selected optional context mappings
rather than copying the raw payload.
- Add `agentmetry_context_test.go` regressions for supporting-event
references, host-wide correlation, reported account provenance,
partial/unknown claims, zero/false/empty coverage values and MCP schema
states.
- Document field destinations, privacy boundaries and investigation
limits; extend the existing claim-matrix row with the mapping coverage.

## Testing and scope

The committed regressions use the inbox JSON decoder, embedded template
and normalized JSON serialization. They cover field selection and
routing, without claiming that a supporting event arrived or that a
reported account identifies the person who made a call. Source-event
references must be scoped by tenant and host; host-wide findings can
reference multiple sessions.

The required upstream CI checks must complete before merge. These
mapping regressions do not establish current-release full-stack
compatibility, finding persistence, retrieval, UI pivots or source-trail
verification. The existing dated manual-run caveat is retained.

## Checklist

- [x] Unit regression tests added
- [x] Documentation updated
- [x] Self-review completed
- [x] Sensitive data and credentials checked in the diff

## Eval harness

- [x] Not applicable: this PR does not touch substrate, playbooks,
detections or eval inputs.

Signed-off-by: Ioannis L. <44038245+blitzcrieg1@users.noreply.github.com>
…nuar#1819) (beenuar#1828)

Carries beenuar#1819 by @knowlet — the capability is exactly right and reviewed
well; the carry exists because two required checks failed on content a
fork branch can't be amended from here.

**What the PR does (as reviewed):** copilot's chat transports bind three
read-only tools (alerts, one record by UUID, Fusion entity risk) that
forward the **caller's own** authorization and tenant to the core API —
the API remains the authority, no LLM argument or page context can
choose a URL, credential or tenant; `kind` is enum-bound and `record_id`
round-trips `uuid.UUID` with a response-echo check; entity paths are
percent-encoded with `safe=''`; redirects disabled, reads bounded,
failures answer as explicit unavailability. The text cache is disabled
when tools are bound (a cached prose answer must not impersonate a live
read), a truncated tool loop no longer publishes the last raw lookup as
a conclusion, and the Fusion risk endpoints return **503 instead of a
fabricated empty queue** — the exact honest-failure doctrine this repo
enforces. Big test surface: read boundaries (223 lines), live reads, API
filter tests, provenance.

**Corrections relative to beenuar#1819 as submitted (the two failing required
checks, reproduced locally before fixing):**
1. *No competitor product named*: seven vendor-name mentions in
prose/docstrings/test strings reworded neutrally (the gate's own
suggested phrasing); test semantics unchanged — the renamed source value
appears only at its insert site.
2. *Python — Lint & Type-check*: the import-boundary test's finder now
subclasses `importlib.abc.MetaPathFinder` and carries the ABC's own
`find_spec` signature — the two `arg-type` findings (and the `override`
finding the first fix shape would have left) against the mypy baseline.

**Verified locally:** competitor gate 0 · ruff lint + format clean over
services/agents + services/api · `check_mypy_baseline` OK (no tree got
worse) · 62 passed / 1 skipped across the five agents suites
(import-boundary, read-boundaries, fallback-honesty, agent-identity,
vaid). The DB-backed live suites get their grade in CI.

Authorship preserved via co-author trailer; the contribution credit
stays with @knowlet.

---------

Co-authored-by: knowlet <knowlet@users.noreply.github.com>
…t that shipped four times (beenuar#1827)

**The defect class:** a fix PR writes its entry under an empty
`[Unreleased]`, a release promotes the section, and when the PR merges
after the tag, git's three-way merge anchors the entry by context —
inside the released section. Main then claims a fix the tagged artifact
does not contain. Four measured instances: Datadog into `[23.0.0]`
(hand-corrected in beenuar#1805), Cortex and Trend into `[23.1.0]` (corrected
here), and — found by running the new gate — a post-tag credit line in
`[22.1.0]`.

**The gate:** `scripts/check_changelog_frozen.py` compares the newest
five released sections byte-for-byte against the `CHANGELOG.md`
committed at each section's own tag. Mismatch prints a unified diff; an
unresolvable tag is an explicit failure, not a skip; zero comparisons
refuses as vacuous. `--self-test` injects drift in every direction; nine
unit tests cover the parser and auditor. **Run against the pre-fix tree
it exits 1 naming `[23.1.0]` and showing both drifted entries** — the
gate found real drift before it ever went green, including the
`[22.1.0]` instance nobody knew about.

**Wiring:** unconditional workflow (no `paths:` filter — the drift lands
via merges whose own diffs touch only `[Unreleased]`, so a filter would
never fire), self-test before gate, and the context added to
`.github/required-checks.json`. Per the documented order, branch
protection gains the context **after** this merges (manifest first; a
manifest briefly ahead of protection reds nothing).

**The corrections, by doctrine rather than convenience:**
- Cortex/Trend entries move to `[Unreleased]`, where the next release
will truthfully carry them.
- `[22.2.0]` and `[22.1.0]` are spliced verbatim from their tags; the
`[22.1.0]` post-tag credit line relocates to `.github/CREDITS.md`
(@rudycelekli keeps the credit on the surface built to stay current —
released sections are frozen).
- `[Unreleased]` gains this wave's reconciled entries: beenuar#1817 (tab deep
links), beenuar#1823 (agents METRICS_TOKEN), beenuar#1824 (error_count reset), beenuar#1825
(Wazuh search_after), each crediting its reporter/author.
- CREDITS rows: @pitimon (three high-signal reports) and @knowlet's row
extended (beenuar#1817, beenuar#1818).

**Local verification:** gate self-test 6/6; gate OK on the corrected
tree (5 of 5 sections verified); 9/9 unit tests; ruff format clean;
attribution, competitor-name and required-check-substance gates all pass
locally.

Merges **after** beenuar#1823/beenuar#1824/beenuar#1825 so its `[Unreleased]` entries
describe merged reality.
…pendabot alerts) (beenuar#1829)

Resolves Dependabot alerts beenuar#232 beenuar#233 beenuar#234 beenuar#235 — all four advisories
(Ed25519 small-order acceptance, two TLS-client crash paths, ReDoS in
header trimming) name **1.4.3** as the first patched version, and
`apps/mobile` resolved 1.4.2 through its existing `>=1.4.1` override
floor.

The floor moves to `>=1.4.3` and the mobile lock re-resolves: delta
audited, **exactly one package moved** (`node-forge` 1.4.2 → 1.4.3 at
both import sites). The override remains scoped to the mobile app's own
lockfile (excluded from the root workspace), per the scoped-floor
convention.

Alert handling after merge: the four alerts should close on the next
dependency-graph sync — if they lag, they stay open as 'fixed and
awaiting the SBOM', not dismissed.
…eck (CodeQL beenuar#1129) (beenuar#1830)

Addresses the one open code-scanning alert,
[beenuar#1129](https://github.com/beenuar/AiSOC/security/code-scanning/1129)
(`js/file-access-to-http`, warning/medium) on
`scripts/check_session_refresh_live.mjs:50`.

**What the alert sees:** fixture-file data (the expired access token)
flowing into an outbound request. **What it cannot see:** that flow is
the harness's negative control — the token is signed by the isolated API
precisely so the live 401 can be asserted — and the destination is
already pinned to loopback by an assertion on `fixture.base_url` before
any request is made (file data cannot choose a non-loopback URL).

**The change:** the token is now validated as a structurally well-formed
JWT (three base64url segments) before any request — a genuine harness
improvement (a malformed fixture fails at the door with a named reason
instead of as a confusing 401-assertion three calls later), and the
strongest shape constraint available for a value that must flow.

**Disposition if the alert persists:** this repo has measured that
inline guards and `codeql[...]` markers frequently do not satisfy this
tracker (documented in `tests/isolation/test_sso_live.py` and
`services/agents/app/hunt/from_report.py`). If beenuar#1129 is still open after
this merges, it gets dismissed as a false positive with this reasoning
on the alert — the documented disposition — rather than contorting the
harness.
## Summary

Adds a **Sophos Central** connector (`sophos_central`, category `edr`),
one of the most widely deployed endpoint and server protection platforms
and absent from the catalogue. It reads Sophos Central alerts through
the SIEM Integration API (`GET /siem/v1/alerts`), the same feed Sophos's
own SIEM integration reads. The operator enters a client ID and secret
and nothing else: the tenant and its data region are discovered from the
credential, and partner or organization credentials work by naming one
tenant. The window is read to the end, the vendor's 24-hour lookback
limit is respected, and Sophos's three severities are kept as published.

## Type of change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Documentation update
- [ ] Refactor / tech debt
- [ ] CI / infra change

## Related issues

Part of beenuar#356 (connector wishlist). This doesn't close it, because the
wishlist stays open by design.

## Changes

- **The vendor contract.** It comes from Sophos's developer portal:
[getting started
(tenant)](https://developer.sophos.com/getting-started-tenant), [getting
started (partner)](https://developer.sophos.com/getting-started) and the
[alerts route
reference](https://developer.sophos.com/reference/siem-v1/alerts/get-alerts/).
Sophos's own `Sophos-Central-SIEM-Integration` client resolves tenants
and pages the same way.
- `services/connectors/app/connectors/sophos_central.py` (new):
- **Auth.** OAuth2 client credentials at
`https://id.sophos.com/api/v2/oauth2/token` with `scope=token`. The
token is cached with a minute of headroom and renewed once on a 401.
- **Tenant discovery.** `GET https://api.central.sophos.com/whoami/v1`
tells the connector what kind of credential it has.
- A credential with `idType: tenant` reads the `apiHosts.dataRegion`
that whoami names. If a configured `tenant_id` names a different tenant,
the connector refuses and says why.
- A partner or organization credential has no data region of its own, so
`tenant_id` is required. The connector looks the tenant up through
`/{idType}/v1/tenants/{id}` with `X-Partner-ID` or `X-Organization-ID`
and reads its `apiHost`.
- `tenant_id` is validated as a UUID in the constructor, because it is
interpolated into a request path. A bad value becomes a 422 with the
reason.
- **The bearer token stays with Sophos.** The data region host comes
from a response, so the egress guard, which vets configured values,
never sees it. The host must be `https` under `sophos.com`, and it is
rebuilt from the scheme, host and port that were checked before the
token is sent there.
- **Reading the window to the end.** Pages are read at `limit=1000` (the
documented maximum; the minimum is 200) through `read_token_pages` with
`more_field="has_more"`. Following pages send `cursor` without
`from_date`, as Sophos's client does. A refused opening request raises,
so the scheduler records a failed poll rather than a quiet one. A later
page failing keeps the pages already read, and the 25-page bound is
announced when it binds.
- **The 24-hour ceiling.** `from_date` "must be within last 24 hours".
The scheduler's outage backfill asks for up to exactly 24 hours, which
the API would refuse outright. A longer window is shortened to just
under the limit and logged as `sophos_central.window_shortened`.
- **Normalization.** The canonical envelope carries the vendor record
under `raw_event`. `description` becomes the title, `location` the
hostname, `source` the actor (`n/a` is no user), `data.source_info.ip`
the source IP and `data.ips_threat.remote_ip` the destination IP.
`threat`, `type`, the endpoint id and the endpoint type are kept.
- **Severity.** Sophos publishes `LOW | MEDIUM | HIGH` and nothing
above. These map straight through, so nothing is ever `critical`. Any
other value carries no severity rather than a guess.
- `services/connectors/app/connectors/_paging.py`: `read_token_pages`
gains an opt-in `more_field`. Sophos returns `next_cursor` on its last
page too, so ending on the token alone would re-request the final page
up to the page cap. When `more_field` is set, the read ends as soon as
that flag is not `True`. Existing callers are unchanged.
- `services/ingest/internal/normalizer/ocsf_classes.go`:
`sophos_central` is filed at 2001 Security Finding, so its alerts
promote, and `check_ocsf_class_coverage.py --check` stays at 42 generic
connectors.
- `plugins/sophos-central/plugin.yaml` plus `pnpm marketplace:sync`
output.
- `apps/docs/docs/connectors/sophos-central.md`, generated by
`generate_connector_docs.py`. The human-notes section covers:
  - getting the credential
  - partner and organization tenants
  - what is read and the 24-hour limit
  - the severity table
  - a troubleshooting table keyed on the connector's own error messages
- Regenerated derived artefacts:
  - connector counts (`generate_connector_count.py`)
  - the `ConnectorType` union
  - the catalog fallback
  - the conformance matrix
  - `llms.txt`
  - corpus stats
- The connector figures in `AGENTS.md` and the count row in
`docs/audit/CLAIM_TO_GATE_MATRIX.md` were updated by hand.
- `CHANGELOG.md`: entry under Unreleased → Added.

## Testing

- [x] Unit tests added / updated
- [ ] Integration tests pass locally
- [x] Manually tested (describe steps below)

<details>
<summary>Manual test steps</summary>

1. `services/connectors/tests/connectors/test_sophos_central.py` (54
tests) drives the real connector through an `httpx.MockTransport` that
serves Sophos ID, the global API and a data region. It covers:
   - registration and schema
   - the token request's form body
   - whoami-to-data-region routing with `X-Tenant-ID`
   - partner and organization lookups with their scope headers
   - the missing and mismatched tenant messages
- refusal of non-Sophos, plain-http and look-alike data region hosts,
and the host being rebuilt from its checked parts
   - all pages arriving, with `cursor` replacing `from_date`
   - `has_more: false` ending the read beside a cursor
   - a failed later page keeping earlier pages
   - the page cap being announced
   - the 24-hour clamp being logged
   - a refused opening request raising
   - a lapsed token being renewed once
   - the normalized envelope and the three-tier severity ladder

   Direct tests of `more_field` on the shared reader are included.
2. Negative control: with the `more_field` check removed from
`read_token_pages`, 11 of the tests fail.
3. `pytest tests/` in `services/connectors`: 1596 passed and 3 failed in
`test_ticket_sync_loop.py`. Those 3 fail identically on an untouched
`main` checkout, because the machine I tested on has no DNS for the
hostname they resolve.
4. These all report OK:
- `check_connector_paging.py`, `check_connector_profiles.py`,
`check_ocsf_class_coverage.py --check` and
`check_connector_404_conflation.py`
- `connector_conformance.py --check` and the `--check` mode of every
connector generator (count, types, docs, catalog fallback)
- `build_marketplace.py --check`, `curate_detections.py --check`,
`generate_llms_txt.py --check` and `generate_corpus_stats.py --check`
- `check_changelog_frozen.py`, `check_repo_self_links.py`,
`check_competitor_names.py` and `check_attribution.py --all`
5. `ruff check` and `ruff format --check .` with ruff 0.16.10, and
`check_mypy_baseline.py` with mypy alone on Python 3.11: no tree got
worse.

I didn't run the Go change locally. It is one entry in the class table,
and `TestEveryDeclaredConnectorHasAnOCSFDecision` in CI checks it, along
with the Python mirror `check_ocsf_class_coverage.py`, which passes. No
live Sophos Central tenant was used. The contract is the vendor's
published reference and its own SIEM client.

</details>

## Screenshots (if UI changes)

Not applicable. The connector appears in the existing catalog grid and
wizard through the registry.

## Checklist

- [x] My code follows the style guidelines of this project (`ruff`,
`eslint`, `gofmt`)
- [x] I have performed a self-review of my own code
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally
- [x] I have updated documentation as needed (README, CHANGELOG,
CONTRIBUTING)
- [x] My changes do not introduce new linter warnings
- [x] I have checked for sensitive data / credentials in my diff

## Eval harness (required for substrate / playbook / detection changes)

- [x] Not applicable — this PR does not touch substrate, playbooks,
detections, or eval inputs
- [ ] No axis regressed
- [ ] An axis regressed; rationale and follow-up are described in the
Summary above

Signed-off-by: Asish Kumar <officialasishkumar@gmail.com>
…g and investigation

The confidence scorer already recognises feed-sourced TI hits — _ti_contribution
counts misp/otx/taxii/kev/virustotal, and alert_enricher._SOURCE_TO_TI_KEY maps
those source names into the keys it reads. But nothing populates the feed slots:
services/threatintel's poll loop ingests TAXII/MISP/OTX/KEV into
OpenSearch/Qdrant/Neo4j (read by the TI page, attribution and semantic memory),
while the alert scorer reads fused.enrichments, which only the enrichment service
and the tenant IOC store write. So a feed match never moves an alert's confidence
and never surfaces in an investigation — the otx/misp/taxii scorer slots are a
dead scaffold.

Per the system design, services/enrichment is the per-IOC lookup layer and
services/threatintel is the feed + IOC/actor search API. This keeps OTX a
threatintel feed and adds one enrichment provider that asks threatintel per IOC,
rather than a second OTX client or a feed->store bridge:

- enrichment gets a ThreatIntelClient that calls threatintel's existing
  GET /api/v1/iocs/search?value=<ioc> (service-token + X-AiSOC-Tenant-ID auth) on
  IP/domain/url/hash. A hit is emitted under the matching feed's own source name
  (otx/misp/...), so the existing _SOURCE_TO_TI_KEY lifts it into the scorer with
  no scorer change. One provider covers every feed.

Two propagation gaps had to be closed for the provider to fire — both pre-existing
"tenant was available but not passed" cases:

- fusion AlertEnricher now carries the alert's tenant on each /enrich/bulk item
  (the provider needs it for threatintel's service auth; the enrichment cache is
  tenant-scoped anyway).
- agents enrich_ioc now forwards the run's tenant, bound by default_registry's
  closure exactly like lookup_file_hash — it is not an LLM-visible argument (the
  tool schema still lists only ioc_value/ioc_type), so "the tenant is never a
  parameter a model can set" holds. Without this the feed-lookup fired in scoring
  but no-oped in investigations.

Tests: threatintel_test.go drives the client through httptest (a feed hit emits
the source names + classification, a no-match returns nil, an empty token or
absent tenant no-ops); test_alert_enricher.py pins that every bulk item carries
the tenant; test_investigator_enrichment_tool.py pins that enrich_ioc forwards
the bound tenant.

Signed-off-by: Kyrylo N <98964984+43n51c@users.noreply.github.com>
…enuar#1833, closes beenuar#1832) (beenuar#1836)

Carries beenuar#1833 by @officialasishkumar, which reviewed **sound** on every
security axis — the RFC 7396-at-depth-one merge under a row lock, stored
secret leaves kept as vault tokens with only newly-typed values
encrypted (pinned by
`test_merged_secrets_keep_their_tokens_and_only_new_values_are_encrypted`),
and the retirement of the documented delete-and-re-add footgun that
loses the id, checkpoint, counters and push token.

**Why the carry:** the required check *Verify docs/openapi.yaml is up to
date* fails deterministically on beenuar#1833 — the request-model change alters
the published spec and the committed `docs/openapi.yaml` wasn't
regenerated — and fork branches can't be pushed to from this repository.
This branch is beenuar#1833 byte-identical plus the regenerated spec (23
insertions, verified with `export_openapi.py --check`). The other ten
failures on beenuar#1833 were the runner-fleet DNS outage, not the diff.

**Verified locally:** ruff lint + format clean · mypy baseline OK ·
merge-semantics tests 10 passed / 3 skipped (DB-backed skips per the
probe convention) · spec check green.

Closes beenuar#1832. Authorship preserved via co-author trailer.

Co-authored-by: Asish Kumar <officialasishkumar@gmail.com>
…rries beenuar#1831) (beenuar#1835)

Carries beenuar#1831 by @43n51c — a textbook exists-but-unwired find: the
enrichment service read all three API keys in its config and did nothing
with any of them, so an operator setting
`SHODAN_API_KEY`/`URLSCAN_API_KEY`/`IPINFO_API_KEY` got silent no-ops
while `PortInfo` sat documented as 'detected by Shodan' with nothing
filling it. The three clients, their wiring, compose/env docs, and
transport-level tests are his work, byte-identical except one correction
below.

**The security correction (why the carry exists):** Go's `*url.Error`
stringifies the full URL it failed on, so a credential in the query
string reaches the logs through any `%w`-wrapped transport error — a
timeout would have logged `?token=<secret>` / `?key=<secret>`.
- **IPInfo**: the token moves to the vendor-supported `Authorization:
Bearer` header (the same choice as the Cyble and Mandiant clients in the
same package); the URL carries no secret at all. Test updated to pin the
header.
- **Shodan**: the vendor contract requires `?key=` in the URL, so the
error path now redacts the query before wrapping, pinned by a new test
asserting a transport error never carries the key and does carry the
`key=REDACTED` marker.
- **URLScan** already used its `API-Key` header correctly — untouched.

**Verified locally:** `go build`, `go vet`, full enricher test package
green; `gofmt` clean. Fork branches can't be pushed to from this
repository, hence the carry; authorship preserved via co-author trailer
so the credit stays with @43n51c. CHANGELOG entry lands at the next
reconciliation alongside this wave's others.

Co-authored-by: 43n51c <43n51c@users.noreply.github.com>
…quest-forgery)

CodeQL flagged the threatintel lookup as request forgery: the IOC value
is attacker-influenced alert data and fmt.Sprintf folds it into one
tainted string, which the taint tracker cannot see QueryEscape confining
to the query. Constructing through url.URL with url.Values keeps the
scheme, host and path from operator config and a package constant, so
the tainted value provably never reaches the authority. Also resolves
the docker-compose env-block conflict with the enrichers merged in beenuar#1835
and records the feature under [Unreleased].
feat(enrichment): per-IOC threat-intel feed lookup, wired into scoring and investigation (carries beenuar#1838)
Cuts **v23.2.0** — the community-capability release. Everything in it
merged to `main` since the v23.1.0 tag; this PR is the release surfaces
plus two honesty fixes found while cutting it.

## Release content (already on main)
- **Sophos Central connector** (beenuar#1834, @officialasishkumar), **console
connector editing** (beenuar#1836 carrying beenuar#1833, same contributor, closes
beenuar#1832), **Shodan/URLScan/IPInfo enrichers** (beenuar#1835 carrying beenuar#1831,
@43n51c).
- Paging-to-the-end-of-the-window fixes for Cortex XDR/XSIAM (beenuar#1815),
Trend Vision One (beenuar#1816 carrying beenuar#1814) and the Wazuh indexer (beenuar#1825,
reported in beenuar#1820 by @pitimon).
- Copilot grounding (beenuar#1828 carrying beenuar#1819, @knowlet), session renewal
(beenuar#1818), tab deep links (beenuar#1817), `METRICS_TOKEN` passthrough (beenuar#1823,
closes beenuar#1822), `error_count` semantics (beenuar#1824, closes beenuar#1821), Agentmetry
context (beenuar#1826, @blitzcrieg1), changelog-freeze gate (beenuar#1827), node-forge
floor (beenuar#1829), session-check hardening (beenuar#1830).

## What this PR changes
- **CHANGELOG**: `[Unreleased]` promoted to `[23.2.0] - 2026-11-11`;
five entries added for merged work that had none (enrichers, copilot
grounding, Agentmetry, node-forge, session-check script).
- **Freeze-gate fix**: `check_changelog_frozen.py` failed its own
release PR by design — the promoted section's tag cannot exist until
after the merge. The newest section, only when it equals `VERSION` and
only until its tag exists, now reports an explicit `PENDING` state;
every other untagged section stays fatal, and the moment the tag exists
the section is compared like any other. Four new tests pin each bound;
self-test extended to ten cases.
- **Honest plugin floor**: the Sophos manifest claimed
`min_aisoc_version: "23.1.0"`, an artifact that does not contain the
connector — corrected to `23.2.0`, marketplace mirrors re-synced.
- **Surfaces**: `VERSION` 23.2.0, web `package.json`, chart `7.21.0` /
`appVersion v23.2.0`, README badge, `RELEASES.md` TL;DR + What's new (11
defects fixed here, 50 across the last three releases, dependency
advisories excluded), `docs/openapi.yaml` re-exported (version string
only), `llms.txt`/`llms-full.txt` regenerated, scoreboard row re-graded
live at `d1bbafa8` (0.97).

After merge: tag `v23.2.0` on the merge commit; `release.yml` publishes
the GitHub release from the `[23.2.0]` section and pushes images.
Neither `stable` channel moves (v22 generation; a minor may not cross a
major without the dispatch).
Resolve hosted provider defaults consistently across API, agents and status.
Do not forward deployment credentials to a different tenant endpoint, and
keep public provider defaults blocked by air-gapped mode.

Signed-off-by: Albert Hsieh <knowlet@pm.me>
@coderabbitai

coderabbitai Bot commented Nov 12, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

const login = await authApi.login(fixture.email, fixture.password);
assert.equal(login.user.id, fixture.user_id);
const expired = await realFetch(`${origin}/api/v1/auth/me`, {
headers: { Authorization: `Bearer ${fixture.expired_access_token}` },
Keep unauthenticated local routes usable without forwarding deployment
secrets. Use supported direct Anthropic defaults and request parameters,
and show the authenticated tenant route in Settings. Exercise real HTTP
and PostgreSQL paths; fetch release tags for unchanged fork CI gates.

Signed-off-by: Albert Hsieh <knowlet@pm.me>
Honor the same explicit allowlist in runtime, status and connection probes.
Redact URL userinfo and query fragments from the authenticated status view
without changing the runtime destination. Exercise low-scope access through
the real database-backed API and keep the provider policy mirrors in parity.

Signed-off-by: Albert Hsieh <knowlet@pm.me>
Document the conditional custom Bedrock Chat Completions endpoint without
implying native SigV4 support or support for Anthropic models on that API.

Signed-off-by: Albert Hsieh <knowlet@pm.me>
Add CORE demo and Settings acceptance with retained synthetic screenshots,
and prove the BYOK gateway credential regression with a real negative control.
Fix new type diagnostics, shrink the resolved baseline entry, refresh the
status schema description, and fetch canonical upgrade tags for fork checks.

Signed-off-by: Albert Hsieh <knowlet@pm.me>
Exercise make up, doctor and smoke after clean Compose convergence.
Keep the checked images with pull policy never and no rebuild, and
redact the bootstrap password while preserving command failures.

Signed-off-by: Albert Hsieh <knowlet@pm.me>
Use the pinned pnpm action, compare the exact browser destination, remove
a redundant import, and locate the Model field by its accessible name.
Verify provider-specific guidance when switching providers.

Signed-off-by: Albert Hsieh <knowlet@pm.me>
Make the documented CORE showcase visible through the console's actual
case UUID query and label known synthetic cases. Repair legacy seed runs
without replacing immutable events or operator data; only seed-owned
runs may populate the public replay.

Cover fresh seeds, repeated repairs, tenant scope and private operator
runs against PostgreSQL, with a negative control for the former case key.

Signed-off-by: Albert Hsieh <knowlet@pm.me>
@knowlet knowlet changed the title fix(llm): bind tenant BYOK credentials to provider routes [validation only] fix(byok): bind provider routes and validate the CORE walkthrough [validation only] Nov 12, 2026
The migration job already declares its administrative connection.
Describe the seed step without an orphan scanner marker; keep its actual
role, assertions and negative control unchanged.

Signed-off-by: Albert Hsieh <knowlet@pm.me>
@knowlet knowlet changed the title fix(byok): bind provider routes and validate the CORE walkthrough [validation only] fix(byok): provider route binding (superseded by #10) Nov 13, 2026
@knowlet knowlet closed this Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants