Skip to content

fix(llm): gateway native auth and reload (superseded by #10) - #7

Closed
knowlet wants to merge 26 commits into
mainfrom
fix/gateway-native-auth-reload
Closed

knowlet wants to merge 26 commits into
mainfrom
fix/gateway-native-auth-reload

Conversation

@knowlet

@knowlet knowlet commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Superseded by #10. The gateway fixes are consolidated in #10 together with tenant BYOK routing. Review and follow the consolidated draft; the evidence below remains a historical record for this branch.

Closed as a superseded validation draft; no merge was performed. The source branch is preserved.


Summary

Validation-only draft in the contributor fork; intended target: beenuar/AiSOC:main. Branch base: 4d8790e. Use the upstream three-dot comparison to review the proposed patch; this fork PR also contains inherited upstream history. Upstream creation via the integration previously returned HTTP 403.

Restore provider-native authentication and configuration reloads for the Compose LiteLLM gateway, preserving default CORE/Ollama startup and existing task aliases. This includes Bedrock bearer and AWS credential-chain support, plus whitespace normalization that preserves keyless compatible endpoints.

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactor / tech debt
  • CI / infra change

Related issues

Contribution guidance: beenuar#353. No issue is auto-closed.

Changes

  • Remove generic OPENAI_API_KEY binding from all nine aliases so native provider authentication resolves correctly.
  • Forward AWS settings and add an optional read-only host profile mount. Unset blank endpoint/AWS placeholders; normalize present whitespace-only OpenAI/Anthropic keys to "", preserving missing and nonblank values.
  • Add make llm-reload and recreate LiteLLM during upgrade, preserving Compose overlays/profile and propagating failure.
  • Correct CORE startup and hosted-provider instructions; document Marketplace authorization and the actual tenant BYOK scope.
  • Add image HTTP/reload regression coverage and fork-only canonical release-tag discovery without relaxing CI gates.

Deployment scope

Runtime changes cover docker-compose.yml and its overlays; docker-compose.prod.yml inherits the wrapper. infra/helm/aisoc is unchanged: its entrypoint, environment/configuration and model defaults have not been ported. Passing Helm render checks is not Helm runtime acceptance of this fix.

Testing

  • Unit tests added / updated
  • Integration tests pass locally
  • Manually tested (describe steps below)

Current head: f9abe7d.
Fork CI checkout: 39e2908, merging that head into fork base 8a06f2e. These results do not establish acceptance of a new merge with upstream main.

Check Result at this head
Gateway suite 32 passed, zero skipped: 14 startup/static + 17 image cases (16 real proxy HTTP, 1 endpoint-resolver check) + 1 real Compose reload.
Negative controls All three expected failures reproduced: generic key masks Bedrock bearer; unsetting an empty key breaks keyless OpenAI compatibility; omitted recreation retains old startup state.
CI Passed, including lint/type-check and full API/agents/repository suites.
Integration Passed, including data-preserving upgrades from v7.3.1 and v7.5.0.
Browser E2E 13 passed, 1 skipped; backend responses are intercepted.

Evidence:

The gateway job pulled ghcr.io/berriai/litellm:v1.102.1 and executed the wrapper's /app/docker/prod_entrypoint.sh through real proxy/reload tests. No separate image-filesystem listing is claimed. It also ran base docker compose config --quiet. AWS profiles were exercised through a real read-only Docker mount; a combined base + aws-profile overlay config command is not asserted by that job.

Local evidence: 14 passed, 18 Docker-gated skips; Ruff 0.16.8 passed; mypy 2.4.1 full baseline passed with no new findings. Reverting whitespace normalization reproduced the expected failures. Local Docker absence is not reported as a pass.

Fixtures use synthetic credentials and no external network. Live AWS/model access, production deployment and real tenant BYOK acceptance remain pending.

Manual test steps

Pending deployment acceptance:

  1. Start CORE with make up and verify local inference after weights are available.
  2. Configure FAST/DEEP and native credentials; run make llm-reload with the startup overlays/profile.
  3. Compare host/container config hashes and complete both aliases; do not count FAST fallback as DEEP success.
  4. Verify the chosen AWS identity and a real tenant Explain/automatic-triage request where applicable.

Screenshots (if UI changes)

No visual UI changes.

Checklist

  • My code follows the style guidelines of this project (ruff, eslint, gofmt)
  • I have performed a self-review of my own code
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally
  • I have updated documentation as needed (README, CHANGELOG, CONTRIBUTING)
  • My changes do not introduce new linter warnings
  • I have checked for sensitive data / credentials in my diff

Eval harness (required for substrate / playbook / detection changes)

Before (paste scripts/run_evals.py output on base branch)
Not applicable: no substrate, playbook, detection, prompt or eval-input paths changed.
After (paste scripts/run_evals.py output on this branch)
Not applicable: no substrate, playbook, detection, prompt or eval-input paths changed.
  • Not applicable — this PR does not touch substrate, playbooks, detections, or eval inputs
  • No axis regressed
  • An axis regressed; rationale and follow-up are described in the Summary above

Operational risk and rollback

Recreation briefly interrupts in-flight requests. Image-tag changes require rerunning the gateway image/reload suite to revalidate /app/docker/prod_entrypoint.sh. The AWS profile mount is opt-in. Revert the patch and recreate the gateway to restore the previous behavior, including its credential/configuration defects.

knowlet and others added 18 commits November 11, 2026 00:29
## Summary

Fix Settings deep links such as /settings?tab=deployment opening
Profile, plus related Case and Responder navigation defects.

## Root causes and changes

- Settings ignored the query string via useState('profile'). All 15
sections now read the URL.
- CaseWorkspace read the tab only at mount. Tab clicks, query changes,
investigation transitions and browser history now stay synchronized.
- Responder redirected every legacy shortcut to Triage. New shortcuts
use canonical routes; existing Cases and On-call links redirect
correctly.

A shared useUrlTab hook validates tabs, preserves other query parameters
and fragments, supports Reload/Back/Forward, and avoids duplicate
history entries. Settings adds Suspense and active controls expose
aria-current.

## Testing

Verified on 7b5788d:
- Lint: passed, 0 errors / 95 existing warnings; no new warnings.
- TypeScript: passed.
- Production and Storybook builds: passed.
- Vitest: 121 files / 1,199 tests passed; coverage gate passed.
- E2E: 13 passed / 1 pre-existing skipped. All 6 new browser tests and 5
new HTTP tests passed without retries.
- CodeQL: passed, no new alerts in changed code.
- Ruff lint/format and mypy baseline: passed.

Negative controls reproduced the original failures on the unchanged
base. HTTP assertions use the existing HTML parser; both initial CodeQL
test-regex findings are resolved.

Validation logs and CI links: #5

## Checklist

- [x] Bug fix with regression tests and research notes
- [x] No new lint warnings, dependencies or workflow changes
- [x] DCO sign-off included
- [x] Contribution guidance in beenuar#353 followed; this does not close beenuar#353
- [x] Eval harness not applicable (frontend navigation only)

---------

Signed-off-by: Albert Hsieh <knowlet@users.noreply.github.com>
)

## Summary
The browser stores a refresh token but never calls POST
/api/v1/auth/refresh. The first protected 401 deletes both tokens, so
the default 30-minute access expiry discards a usable seven-day refresh
credential.

## Type of change
- [x] Bug fix
- [x] Documentation update

## Related issues
None filed.

## Changes
- Share refresh and one-time replay across JSON, streams, downloads and
authenticated fetch callers.
- Preserve method, body, headers, tenant and cancellation. Keep
recoverable credentials on transient failures.
- Prevent pending requests from restoring a logged-out session or
replaying under a different login.
- Align responder guards with shared JWT validity and refresh
availability.
- Restore the missing Explain proxy rewrite exposed by the route gate.
- Add regression tests, real HTTP/API/PostgreSQL checks and CHANGELOG.
Token lifetimes, backend security checks and dependencies are unchanged.

## Testing
- [x] Unit tests added and updated
- [x] Local integration tests passed
- [ ] Rendered-browser manual test

| Check | Local result |
| --- | --- |
| Web suite | 1,194 passed across 122 files |
| TypeScript and production build | Passed |
| ESLint | 0 errors, 95 warnings identical to base |
| Live API/PostgreSQL session suite | 10 passed, 0 skipped |
| Production client, real HTTP | 4 passed; base client fails; restored
fix passes |
| Ruff check/format and mypy baseline | Passed |
| Console route/wire/auth-header gates | Passed |
| Attribution, naming and whitespace gates | Passed |

HTTP verification uses real API authentication and PostgreSQL, simulated
browser storage/navigation, and disabled unrelated lifespan workers. It
is not rendered-browser, complete-stack or operator-deployment
validation. Remote CI has not run.

Full root cause, design and verification details:
docs/research/session-refresh.md.

## Screenshots
Not applicable: session transport behavior only.

## Checklist
- [x] Style, self-review and regression coverage complete
- [x] Documentation and CHANGELOG updated
- [x] No new lint warnings
- [x] Diff checked for credentials
- [x] DCO signed off

## Eval harness
- [x] Not applicable: no substrate, playbook packs, detections, prompts
or eval inputs changed

Signed-off-by: Albert Hsieh <knowlet@users.noreply.github.com>
…r#1822) (beenuar#1823)

Closes beenuar#1822.

Verified before changing anything: `services/agents/app/api/metrics.py`
`_authorise` compares against `METRICS_TOKEN` and answers 401 with
'metrics endpoint requires METRICS_TOKEN outside development' when it is
unset — and in `docker-compose.yml` only the **api** service received
the variable; the agents block had no occurrence. So on a default
deployment every scrape of `:15002/metrics` got 401, exactly as @pitimon
reported (their compose override + agents recreate confirmed the fix
shape in their deployment).

One line plus a comment: the agents environment now carries the same
`METRICS_TOKEN: ${METRICS_TOKEN:-}` passthrough the api block already
had. `docker compose config` validates locally. The changelog entry
lands in the consolidated changelog-integrity PR alongside this wave's
other entries.

(Branch created via the API: git-protocol pushes are currently refused
with 403 on this machine while REST writes work — same constraint as
previously recorded.)
…#1821) (beenuar#1824)

Closes beenuar#1821.

Verified before changing anything: `record_poll_failure` does
`error_count + 1` and `record_poll_success`'s values dict never touched
the column — @pitimon's reported case (a static '19 errors' for days
beside a fresh `last_sync` and +237k ingested events) is exactly what a
lifetime counter produces and what a consecutive counter cannot.

Took the reporter's **option 1**: `error_count` now means *consecutive
failed polls since the last success* — success writes 0, failure
increments. That matches how the fleet panel presents the number
(current condition beside `last_sync`), and the zero-when-healthy
rendering already hides the badge on healthy rows. Two tests pin the
semantics at the only two writers (success resets to 0; failure still
increments), asserted against the compiled UPDATE statements. 49
connector tests pass locally including the full scheduler suite; ruff
format clean.

Changelog entry lands in the consolidated changelog-integrity PR for
this wave.
…loses beenuar#1820) (beenuar#1825)

Closes beenuar#1820.

@pitimon asked the right question — deliberate exclusion or not yet
reached? — and the honest answer is **neither defensible**: the gate
listed wazuh among the unclassified debt ('requests a capped page and
never follows a continuation'), and the docstring's deliberate-cap
rationale ('drop the polling interval, not chase a single huge batch')
is the exact argument this repo has rejected elsewhere, because the cut
is silent and the abutting windows mean the overflow is never
re-requested by anything.

**The fix**: `_search` — the one executor both fetch paths share — now
follows `search_after` (OpenSearch's documented continuation for sorted
queries) to the end of the window, appending a `_doc` tiebreaker so
equal timestamps cannot skip or repeat rows across pages. Properties,
matching the paging doctrine: the opening page keeps the existing
never-raise contract exactly (request error / non-200 / bad JSON → `[]`
+ the existing wazuh.* warnings); a failed later page **keeps what was
read** (`connector.paging.next_page_*`); a full page whose last hit
carries no sort values stops rather than looping on an uncontinuable
read; and the 25-page bound announces itself (`page_cap_reached`) with
the data-loss consequence stated. Both alert and vulnerability-findings
reads page identically through the one loop. Docstrings now say what is
true; the old rationale is kept as history rather than deleted.

**Gate**: `MAX_UNCLASSIFIED` falls 49 → 48 and the audit now counts
wazuh among the followers (43 follow, 1 reviewed, 48 unclassified —
verified locally).

**Tests**: 9 new transport-level cases (multi-page read with
`search_after` echo verification, tiebreaker presence, short-page end,
failed/transport-error later pages keeping partials, opening failure
still `[]`, sortless full page stopping safely, page-cap bound, and the
vulnerability path paging) plus the existing wazuh connector suite and
gate tests — 46 pass locally; ruff format clean.

Changelog entry lands in the consolidated changelog-integrity PR for
this wave. (Branch created via the API — git-protocol pushes currently
403 on this machine while REST works.)
## Summary

The Agentmetry inbox template drops optional source context needed to
investigate a finding. Preserve selected account provenance,
supporting-event references, recorder coverage and MCP schema
observations under `finding.evidence.agentmetry`, keeping reported
account claims separate from `actor.user` and excluding full account
email and opaque account/user IDs.

Existing field paths, the activity/finding split, the severity ladder
and the OCSF 1.1.0 declaration remain unchanged.

## Type of change

- [x] Bug fix
- [x] Documentation update

## Related discussion

beenuar#1577

## Changes

- Extend `agentmetry.yaml` with selected optional context mappings
rather than copying the raw payload.
- Add `agentmetry_context_test.go` regressions for supporting-event
references, host-wide correlation, reported account provenance,
partial/unknown claims, zero/false/empty coverage values and MCP schema
states.
- Document field destinations, privacy boundaries and investigation
limits; extend the existing claim-matrix row with the mapping coverage.

## Testing and scope

The committed regressions use the inbox JSON decoder, embedded template
and normalized JSON serialization. They cover field selection and
routing, without claiming that a supporting event arrived or that a
reported account identifies the person who made a call. Source-event
references must be scoped by tenant and host; host-wide findings can
reference multiple sessions.

The required upstream CI checks must complete before merge. These
mapping regressions do not establish current-release full-stack
compatibility, finding persistence, retrieval, UI pivots or source-trail
verification. The existing dated manual-run caveat is retained.

## Checklist

- [x] Unit regression tests added
- [x] Documentation updated
- [x] Self-review completed
- [x] Sensitive data and credentials checked in the diff

## Eval harness

- [x] Not applicable: this PR does not touch substrate, playbooks,
detections or eval inputs.

Signed-off-by: Ioannis L. <44038245+blitzcrieg1@users.noreply.github.com>
…nuar#1819) (beenuar#1828)

Carries beenuar#1819 by @knowlet — the capability is exactly right and reviewed
well; the carry exists because two required checks failed on content a
fork branch can't be amended from here.

**What the PR does (as reviewed):** copilot's chat transports bind three
read-only tools (alerts, one record by UUID, Fusion entity risk) that
forward the **caller's own** authorization and tenant to the core API —
the API remains the authority, no LLM argument or page context can
choose a URL, credential or tenant; `kind` is enum-bound and `record_id`
round-trips `uuid.UUID` with a response-echo check; entity paths are
percent-encoded with `safe=''`; redirects disabled, reads bounded,
failures answer as explicit unavailability. The text cache is disabled
when tools are bound (a cached prose answer must not impersonate a live
read), a truncated tool loop no longer publishes the last raw lookup as
a conclusion, and the Fusion risk endpoints return **503 instead of a
fabricated empty queue** — the exact honest-failure doctrine this repo
enforces. Big test surface: read boundaries (223 lines), live reads, API
filter tests, provenance.

**Corrections relative to beenuar#1819 as submitted (the two failing required
checks, reproduced locally before fixing):**
1. *No competitor product named*: seven vendor-name mentions in
prose/docstrings/test strings reworded neutrally (the gate's own
suggested phrasing); test semantics unchanged — the renamed source value
appears only at its insert site.
2. *Python — Lint & Type-check*: the import-boundary test's finder now
subclasses `importlib.abc.MetaPathFinder` and carries the ABC's own
`find_spec` signature — the two `arg-type` findings (and the `override`
finding the first fix shape would have left) against the mypy baseline.

**Verified locally:** competitor gate 0 · ruff lint + format clean over
services/agents + services/api · `check_mypy_baseline` OK (no tree got
worse) · 62 passed / 1 skipped across the five agents suites
(import-boundary, read-boundaries, fallback-honesty, agent-identity,
vaid). The DB-backed live suites get their grade in CI.

Authorship preserved via co-author trailer; the contribution credit
stays with @knowlet.

---------

Co-authored-by: knowlet <knowlet@users.noreply.github.com>
…t that shipped four times (beenuar#1827)

**The defect class:** a fix PR writes its entry under an empty
`[Unreleased]`, a release promotes the section, and when the PR merges
after the tag, git's three-way merge anchors the entry by context —
inside the released section. Main then claims a fix the tagged artifact
does not contain. Four measured instances: Datadog into `[23.0.0]`
(hand-corrected in beenuar#1805), Cortex and Trend into `[23.1.0]` (corrected
here), and — found by running the new gate — a post-tag credit line in
`[22.1.0]`.

**The gate:** `scripts/check_changelog_frozen.py` compares the newest
five released sections byte-for-byte against the `CHANGELOG.md`
committed at each section's own tag. Mismatch prints a unified diff; an
unresolvable tag is an explicit failure, not a skip; zero comparisons
refuses as vacuous. `--self-test` injects drift in every direction; nine
unit tests cover the parser and auditor. **Run against the pre-fix tree
it exits 1 naming `[23.1.0]` and showing both drifted entries** — the
gate found real drift before it ever went green, including the
`[22.1.0]` instance nobody knew about.

**Wiring:** unconditional workflow (no `paths:` filter — the drift lands
via merges whose own diffs touch only `[Unreleased]`, so a filter would
never fire), self-test before gate, and the context added to
`.github/required-checks.json`. Per the documented order, branch
protection gains the context **after** this merges (manifest first; a
manifest briefly ahead of protection reds nothing).

**The corrections, by doctrine rather than convenience:**
- Cortex/Trend entries move to `[Unreleased]`, where the next release
will truthfully carry them.
- `[22.2.0]` and `[22.1.0]` are spliced verbatim from their tags; the
`[22.1.0]` post-tag credit line relocates to `.github/CREDITS.md`
(@rudycelekli keeps the credit on the surface built to stay current —
released sections are frozen).
- `[Unreleased]` gains this wave's reconciled entries: beenuar#1817 (tab deep
links), beenuar#1823 (agents METRICS_TOKEN), beenuar#1824 (error_count reset), beenuar#1825
(Wazuh search_after), each crediting its reporter/author.
- CREDITS rows: @pitimon (three high-signal reports) and @knowlet's row
extended (beenuar#1817, beenuar#1818).

**Local verification:** gate self-test 6/6; gate OK on the corrected
tree (5 of 5 sections verified); 9/9 unit tests; ruff format clean;
attribution, competitor-name and required-check-substance gates all pass
locally.

Merges **after** beenuar#1823/beenuar#1824/beenuar#1825 so its `[Unreleased]` entries
describe merged reality.
…pendabot alerts) (beenuar#1829)

Resolves Dependabot alerts beenuar#232 beenuar#233 beenuar#234 beenuar#235 — all four advisories
(Ed25519 small-order acceptance, two TLS-client crash paths, ReDoS in
header trimming) name **1.4.3** as the first patched version, and
`apps/mobile` resolved 1.4.2 through its existing `>=1.4.1` override
floor.

The floor moves to `>=1.4.3` and the mobile lock re-resolves: delta
audited, **exactly one package moved** (`node-forge` 1.4.2 → 1.4.3 at
both import sites). The override remains scoped to the mobile app's own
lockfile (excluded from the root workspace), per the scoped-floor
convention.

Alert handling after merge: the four alerts should close on the next
dependency-graph sync — if they lag, they stay open as 'fixed and
awaiting the SBOM', not dismissed.
…eck (CodeQL beenuar#1129) (beenuar#1830)

Addresses the one open code-scanning alert,
[beenuar#1129](https://github.com/beenuar/AiSOC/security/code-scanning/1129)
(`js/file-access-to-http`, warning/medium) on
`scripts/check_session_refresh_live.mjs:50`.

**What the alert sees:** fixture-file data (the expired access token)
flowing into an outbound request. **What it cannot see:** that flow is
the harness's negative control — the token is signed by the isolated API
precisely so the live 401 can be asserted — and the destination is
already pinned to loopback by an assertion on `fixture.base_url` before
any request is made (file data cannot choose a non-loopback URL).

**The change:** the token is now validated as a structurally well-formed
JWT (three base64url segments) before any request — a genuine harness
improvement (a malformed fixture fails at the door with a named reason
instead of as a confusing 401-assertion three calls later), and the
strongest shape constraint available for a value that must flow.

**Disposition if the alert persists:** this repo has measured that
inline guards and `codeql[...]` markers frequently do not satisfy this
tracker (documented in `tests/isolation/test_sso_live.py` and
`services/agents/app/hunt/from_report.py`). If beenuar#1129 is still open after
this merges, it gets dismissed as a false positive with this reasoning
on the alert — the documented disposition — rather than contorting the
harness.
## Summary

Adds a **Sophos Central** connector (`sophos_central`, category `edr`),
one of the most widely deployed endpoint and server protection platforms
and absent from the catalogue. It reads Sophos Central alerts through
the SIEM Integration API (`GET /siem/v1/alerts`), the same feed Sophos's
own SIEM integration reads. The operator enters a client ID and secret
and nothing else: the tenant and its data region are discovered from the
credential, and partner or organization credentials work by naming one
tenant. The window is read to the end, the vendor's 24-hour lookback
limit is respected, and Sophos's three severities are kept as published.

## Type of change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Documentation update
- [ ] Refactor / tech debt
- [ ] CI / infra change

## Related issues

Part of beenuar#356 (connector wishlist). This doesn't close it, because the
wishlist stays open by design.

## Changes

- **The vendor contract.** It comes from Sophos's developer portal:
[getting started
(tenant)](https://developer.sophos.com/getting-started-tenant), [getting
started (partner)](https://developer.sophos.com/getting-started) and the
[alerts route
reference](https://developer.sophos.com/reference/siem-v1/alerts/get-alerts/).
Sophos's own `Sophos-Central-SIEM-Integration` client resolves tenants
and pages the same way.
- `services/connectors/app/connectors/sophos_central.py` (new):
- **Auth.** OAuth2 client credentials at
`https://id.sophos.com/api/v2/oauth2/token` with `scope=token`. The
token is cached with a minute of headroom and renewed once on a 401.
- **Tenant discovery.** `GET https://api.central.sophos.com/whoami/v1`
tells the connector what kind of credential it has.
- A credential with `idType: tenant` reads the `apiHosts.dataRegion`
that whoami names. If a configured `tenant_id` names a different tenant,
the connector refuses and says why.
- A partner or organization credential has no data region of its own, so
`tenant_id` is required. The connector looks the tenant up through
`/{idType}/v1/tenants/{id}` with `X-Partner-ID` or `X-Organization-ID`
and reads its `apiHost`.
- `tenant_id` is validated as a UUID in the constructor, because it is
interpolated into a request path. A bad value becomes a 422 with the
reason.
- **The bearer token stays with Sophos.** The data region host comes
from a response, so the egress guard, which vets configured values,
never sees it. The host must be `https` under `sophos.com`, and it is
rebuilt from the scheme, host and port that were checked before the
token is sent there.
- **Reading the window to the end.** Pages are read at `limit=1000` (the
documented maximum; the minimum is 200) through `read_token_pages` with
`more_field="has_more"`. Following pages send `cursor` without
`from_date`, as Sophos's client does. A refused opening request raises,
so the scheduler records a failed poll rather than a quiet one. A later
page failing keeps the pages already read, and the 25-page bound is
announced when it binds.
- **The 24-hour ceiling.** `from_date` "must be within last 24 hours".
The scheduler's outage backfill asks for up to exactly 24 hours, which
the API would refuse outright. A longer window is shortened to just
under the limit and logged as `sophos_central.window_shortened`.
- **Normalization.** The canonical envelope carries the vendor record
under `raw_event`. `description` becomes the title, `location` the
hostname, `source` the actor (`n/a` is no user), `data.source_info.ip`
the source IP and `data.ips_threat.remote_ip` the destination IP.
`threat`, `type`, the endpoint id and the endpoint type are kept.
- **Severity.** Sophos publishes `LOW | MEDIUM | HIGH` and nothing
above. These map straight through, so nothing is ever `critical`. Any
other value carries no severity rather than a guess.
- `services/connectors/app/connectors/_paging.py`: `read_token_pages`
gains an opt-in `more_field`. Sophos returns `next_cursor` on its last
page too, so ending on the token alone would re-request the final page
up to the page cap. When `more_field` is set, the read ends as soon as
that flag is not `True`. Existing callers are unchanged.
- `services/ingest/internal/normalizer/ocsf_classes.go`:
`sophos_central` is filed at 2001 Security Finding, so its alerts
promote, and `check_ocsf_class_coverage.py --check` stays at 42 generic
connectors.
- `plugins/sophos-central/plugin.yaml` plus `pnpm marketplace:sync`
output.
- `apps/docs/docs/connectors/sophos-central.md`, generated by
`generate_connector_docs.py`. The human-notes section covers:
  - getting the credential
  - partner and organization tenants
  - what is read and the 24-hour limit
  - the severity table
  - a troubleshooting table keyed on the connector's own error messages
- Regenerated derived artefacts:
  - connector counts (`generate_connector_count.py`)
  - the `ConnectorType` union
  - the catalog fallback
  - the conformance matrix
  - `llms.txt`
  - corpus stats
- The connector figures in `AGENTS.md` and the count row in
`docs/audit/CLAIM_TO_GATE_MATRIX.md` were updated by hand.
- `CHANGELOG.md`: entry under Unreleased → Added.

## Testing

- [x] Unit tests added / updated
- [ ] Integration tests pass locally
- [x] Manually tested (describe steps below)

<details>
<summary>Manual test steps</summary>

1. `services/connectors/tests/connectors/test_sophos_central.py` (54
tests) drives the real connector through an `httpx.MockTransport` that
serves Sophos ID, the global API and a data region. It covers:
   - registration and schema
   - the token request's form body
   - whoami-to-data-region routing with `X-Tenant-ID`
   - partner and organization lookups with their scope headers
   - the missing and mismatched tenant messages
- refusal of non-Sophos, plain-http and look-alike data region hosts,
and the host being rebuilt from its checked parts
   - all pages arriving, with `cursor` replacing `from_date`
   - `has_more: false` ending the read beside a cursor
   - a failed later page keeping earlier pages
   - the page cap being announced
   - the 24-hour clamp being logged
   - a refused opening request raising
   - a lapsed token being renewed once
   - the normalized envelope and the three-tier severity ladder

   Direct tests of `more_field` on the shared reader are included.
2. Negative control: with the `more_field` check removed from
`read_token_pages`, 11 of the tests fail.
3. `pytest tests/` in `services/connectors`: 1596 passed and 3 failed in
`test_ticket_sync_loop.py`. Those 3 fail identically on an untouched
`main` checkout, because the machine I tested on has no DNS for the
hostname they resolve.
4. These all report OK:
- `check_connector_paging.py`, `check_connector_profiles.py`,
`check_ocsf_class_coverage.py --check` and
`check_connector_404_conflation.py`
- `connector_conformance.py --check` and the `--check` mode of every
connector generator (count, types, docs, catalog fallback)
- `build_marketplace.py --check`, `curate_detections.py --check`,
`generate_llms_txt.py --check` and `generate_corpus_stats.py --check`
- `check_changelog_frozen.py`, `check_repo_self_links.py`,
`check_competitor_names.py` and `check_attribution.py --all`
5. `ruff check` and `ruff format --check .` with ruff 0.16.10, and
`check_mypy_baseline.py` with mypy alone on Python 3.11: no tree got
worse.

I didn't run the Go change locally. It is one entry in the class table,
and `TestEveryDeclaredConnectorHasAnOCSFDecision` in CI checks it, along
with the Python mirror `check_ocsf_class_coverage.py`, which passes. No
live Sophos Central tenant was used. The contract is the vendor's
published reference and its own SIEM client.

</details>

## Screenshots (if UI changes)

Not applicable. The connector appears in the existing catalog grid and
wizard through the registry.

## Checklist

- [x] My code follows the style guidelines of this project (`ruff`,
`eslint`, `gofmt`)
- [x] I have performed a self-review of my own code
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally
- [x] I have updated documentation as needed (README, CHANGELOG,
CONTRIBUTING)
- [x] My changes do not introduce new linter warnings
- [x] I have checked for sensitive data / credentials in my diff

## Eval harness (required for substrate / playbook / detection changes)

- [x] Not applicable — this PR does not touch substrate, playbooks,
detections, or eval inputs
- [ ] No axis regressed
- [ ] An axis regressed; rationale and follow-up are described in the
Summary above

Signed-off-by: Asish Kumar <officialasishkumar@gmail.com>
…g and investigation

The confidence scorer already recognises feed-sourced TI hits — _ti_contribution
counts misp/otx/taxii/kev/virustotal, and alert_enricher._SOURCE_TO_TI_KEY maps
those source names into the keys it reads. But nothing populates the feed slots:
services/threatintel's poll loop ingests TAXII/MISP/OTX/KEV into
OpenSearch/Qdrant/Neo4j (read by the TI page, attribution and semantic memory),
while the alert scorer reads fused.enrichments, which only the enrichment service
and the tenant IOC store write. So a feed match never moves an alert's confidence
and never surfaces in an investigation — the otx/misp/taxii scorer slots are a
dead scaffold.

Per the system design, services/enrichment is the per-IOC lookup layer and
services/threatintel is the feed + IOC/actor search API. This keeps OTX a
threatintel feed and adds one enrichment provider that asks threatintel per IOC,
rather than a second OTX client or a feed->store bridge:

- enrichment gets a ThreatIntelClient that calls threatintel's existing
  GET /api/v1/iocs/search?value=<ioc> (service-token + X-AiSOC-Tenant-ID auth) on
  IP/domain/url/hash. A hit is emitted under the matching feed's own source name
  (otx/misp/...), so the existing _SOURCE_TO_TI_KEY lifts it into the scorer with
  no scorer change. One provider covers every feed.

Two propagation gaps had to be closed for the provider to fire — both pre-existing
"tenant was available but not passed" cases:

- fusion AlertEnricher now carries the alert's tenant on each /enrich/bulk item
  (the provider needs it for threatintel's service auth; the enrichment cache is
  tenant-scoped anyway).
- agents enrich_ioc now forwards the run's tenant, bound by default_registry's
  closure exactly like lookup_file_hash — it is not an LLM-visible argument (the
  tool schema still lists only ioc_value/ioc_type), so "the tenant is never a
  parameter a model can set" holds. Without this the feed-lookup fired in scoring
  but no-oped in investigations.

Tests: threatintel_test.go drives the client through httptest (a feed hit emits
the source names + classification, a no-match returns nil, an empty token or
absent tenant no-ops); test_alert_enricher.py pins that every bulk item carries
the tenant; test_investigator_enrichment_tool.py pins that enrich_ioc forwards
the bound tenant.

Signed-off-by: Kyrylo N <98964984+43n51c@users.noreply.github.com>
…enuar#1833, closes beenuar#1832) (beenuar#1836)

Carries beenuar#1833 by @officialasishkumar, which reviewed **sound** on every
security axis — the RFC 7396-at-depth-one merge under a row lock, stored
secret leaves kept as vault tokens with only newly-typed values
encrypted (pinned by
`test_merged_secrets_keep_their_tokens_and_only_new_values_are_encrypted`),
and the retirement of the documented delete-and-re-add footgun that
loses the id, checkpoint, counters and push token.

**Why the carry:** the required check *Verify docs/openapi.yaml is up to
date* fails deterministically on beenuar#1833 — the request-model change alters
the published spec and the committed `docs/openapi.yaml` wasn't
regenerated — and fork branches can't be pushed to from this repository.
This branch is beenuar#1833 byte-identical plus the regenerated spec (23
insertions, verified with `export_openapi.py --check`). The other ten
failures on beenuar#1833 were the runner-fleet DNS outage, not the diff.

**Verified locally:** ruff lint + format clean · mypy baseline OK ·
merge-semantics tests 10 passed / 3 skipped (DB-backed skips per the
probe convention) · spec check green.

Closes beenuar#1832. Authorship preserved via co-author trailer.

Co-authored-by: Asish Kumar <officialasishkumar@gmail.com>
…rries beenuar#1831) (beenuar#1835)

Carries beenuar#1831 by @43n51c — a textbook exists-but-unwired find: the
enrichment service read all three API keys in its config and did nothing
with any of them, so an operator setting
`SHODAN_API_KEY`/`URLSCAN_API_KEY`/`IPINFO_API_KEY` got silent no-ops
while `PortInfo` sat documented as 'detected by Shodan' with nothing
filling it. The three clients, their wiring, compose/env docs, and
transport-level tests are his work, byte-identical except one correction
below.

**The security correction (why the carry exists):** Go's `*url.Error`
stringifies the full URL it failed on, so a credential in the query
string reaches the logs through any `%w`-wrapped transport error — a
timeout would have logged `?token=<secret>` / `?key=<secret>`.
- **IPInfo**: the token moves to the vendor-supported `Authorization:
Bearer` header (the same choice as the Cyble and Mandiant clients in the
same package); the URL carries no secret at all. Test updated to pin the
header.
- **Shodan**: the vendor contract requires `?key=` in the URL, so the
error path now redacts the query before wrapping, pinned by a new test
asserting a transport error never carries the key and does carry the
`key=REDACTED` marker.
- **URLScan** already used its `API-Key` header correctly — untouched.

**Verified locally:** `go build`, `go vet`, full enricher test package
green; `gofmt` clean. Fork branches can't be pushed to from this
repository, hence the carry; authorship preserved via co-author trailer
so the credit stays with @43n51c. CHANGELOG entry lands at the next
reconciliation alongside this wave's others.

Co-authored-by: 43n51c <43n51c@users.noreply.github.com>
…quest-forgery)

CodeQL flagged the threatintel lookup as request forgery: the IOC value
is attacker-influenced alert data and fmt.Sprintf folds it into one
tainted string, which the taint tracker cannot see QueryEscape confining
to the query. Constructing through url.URL with url.Values keeps the
scheme, host and path from operator config and a package constant, so
the tainted value provably never reaches the authority. Also resolves
the docker-compose env-block conflict with the enrichers merged in beenuar#1835
and records the feature under [Unreleased].
feat(enrichment): per-IOC threat-intel feed lookup, wired into scoring and investigation (carries beenuar#1838)
Cuts **v23.2.0** — the community-capability release. Everything in it
merged to `main` since the v23.1.0 tag; this PR is the release surfaces
plus two honesty fixes found while cutting it.

## Release content (already on main)
- **Sophos Central connector** (beenuar#1834, @officialasishkumar), **console
connector editing** (beenuar#1836 carrying beenuar#1833, same contributor, closes
beenuar#1832), **Shodan/URLScan/IPInfo enrichers** (beenuar#1835 carrying beenuar#1831,
@43n51c).
- Paging-to-the-end-of-the-window fixes for Cortex XDR/XSIAM (beenuar#1815),
Trend Vision One (beenuar#1816 carrying beenuar#1814) and the Wazuh indexer (beenuar#1825,
reported in beenuar#1820 by @pitimon).
- Copilot grounding (beenuar#1828 carrying beenuar#1819, @knowlet), session renewal
(beenuar#1818), tab deep links (beenuar#1817), `METRICS_TOKEN` passthrough (beenuar#1823,
closes beenuar#1822), `error_count` semantics (beenuar#1824, closes beenuar#1821), Agentmetry
context (beenuar#1826, @blitzcrieg1), changelog-freeze gate (beenuar#1827), node-forge
floor (beenuar#1829), session-check hardening (beenuar#1830).

## What this PR changes
- **CHANGELOG**: `[Unreleased]` promoted to `[23.2.0] - 2026-11-11`;
five entries added for merged work that had none (enrichers, copilot
grounding, Agentmetry, node-forge, session-check script).
- **Freeze-gate fix**: `check_changelog_frozen.py` failed its own
release PR by design — the promoted section's tag cannot exist until
after the merge. The newest section, only when it equals `VERSION` and
only until its tag exists, now reports an explicit `PENDING` state;
every other untagged section stays fatal, and the moment the tag exists
the section is compared like any other. Four new tests pin each bound;
self-test extended to ten cases.
- **Honest plugin floor**: the Sophos manifest claimed
`min_aisoc_version: "23.1.0"`, an artifact that does not contain the
connector — corrected to `23.2.0`, marketplace mirrors re-synced.
- **Surfaces**: `VERSION` 23.2.0, web `package.json`, chart `7.21.0` /
`appVersion v23.2.0`, README badge, `RELEASES.md` TL;DR + What's new (11
defects fixed here, 50 across the last three releases, dependency
advisories excluded), `docs/openapi.yaml` re-exported (version string
only), `llms.txt`/`llms-full.txt` regenerated, scoreboard row re-graded
live at `d1bbafa8` (0.97).

After merge: tag `v23.2.0` on the merge commit; `release.yml` publishes
the GitHub release from the `[23.2.0]` section and pushes images.
Neither `stable` channel moves (v22 generation; a minor may not cross a
major without the dispatch).
Keep the bundled Ollama defaults while allowing provider-native OpenAI,
Anthropic and Bedrock credentials. Normalize blank optional settings,
forward AWS authentication options, and recreate the gateway on upgrade.

Signed-off-by: Albert Hsieh <knowlet@pm.me>
@coderabbitai

coderabbitai Bot commented Nov 12, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

const login = await authApi.login(fixture.email, fixture.password);
assert.equal(login.user.id, fixture.user_id);
const expired = await realFetch(`${origin}/api/v1/auth/me`, {
headers: { Authorization: `Bearer ${fixture.expired_access_token}` },
Cover the pinned gateway's actual HTTP transport with isolated profile,
IMDSv2, metadata-disabled, bearer-denied, and blank endpoint cases.

Signed-off-by: Albert Hsieh <knowlet@pm.me>
Exercise the loaded model and mounted file separately, then prove that
restoring generic credentials or removing recreation reproduces each fault.

Signed-off-by: Albert Hsieh <knowlet@pm.me>
Record pinned LiteLLM evidence and distinguish deployment gateway routing
from the Explain and automatic-triage BYOK overrides. Document inherited
fields, probe limitations, and separate gateway/provider credentials.

Signed-off-by: Albert Hsieh <knowlet@pm.me>
Populate ephemeral runner refs for unchanged changelog and chart-version
gates. Forks do not inherit upstream tags; no remote refs are modified.

Signed-off-by: Albert Hsieh <knowlet@pm.me>
Replace the untyped heterogeneous capture dictionary with a typed call record
and assert that the wrapper delegates exactly once. This resolves the new
var-annotated finding without widening the mypy baseline.

Signed-off-by: Albert Hsieh <knowlet@pm.me>
Document that custom BYOK can use AWS's explicit OpenAI-compatible endpoint
for supported models, while Anthropic still requires the native gateway
route. Keep bearer-key compatibility separate from native SigV4 support
and label live-account acceptance as pending.

Signed-off-by: Albert Hsieh <knowlet@pm.me>
self.assertEqual(result.returncode, 0, result.stdout + result.stderr)
return result.stdout.strip()

def state(expected_hash: str) -> dict:
Fork checkout only fetches tags from the contributor repository. Load the
canonical release tags in the upgrade job so both historical schemas are
available before exercising the existing migration and data-survival checks.
Keep all assertions and remote refs unchanged.

Signed-off-by: Albert Hsieh <knowlet@pm.me>
Document LiteLLM in default CORE and require explicit hosted model routing.
Normalize whitespace-only provider keys while preserving explicit empty keys
for unauthenticated compatible endpoints. Cover both providers through the
pinned gateway and guard against unsetting a working keyless configuration.

Signed-off-by: Albert Hsieh <knowlet@pm.me>
@knowlet knowlet changed the title fix(llm): restore native gateway auth and reload configuration [validation only] fix(llm): restore native auth and reload the Compose gateway [validation only] Nov 13, 2026
@knowlet knowlet changed the title fix(llm): restore native auth and reload the Compose gateway [validation only] fix(llm): gateway native auth and reload (superseded by #10) Nov 13, 2026
@knowlet knowlet closed this Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants