Repository navigation
feat(models): per-organization access to unlisted models - #1237
Conversation
Review: per-organization access to unlisted modelsNo critical issues found. I checked these areas:
Non-blocking notes:
✅ |
Review · Status🟩 CompletedIronLoop completed the review and posted it to GitHub. ResultRun detailsAutomatic trigger · attempt 1 of 3 · completed in 6m 40s |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5a858e008f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
Review · Summary
🟢 No actionable findings
No actionable defects found in the reviewed pull-request change.
Validation
- ✅ Access-control paths — Static tracing found caller-aware model resolution and fail-closed grant checks across the changed inference, response, attestation, and administrative access paths.
- ✅ Change coverage — The changed grant lifecycle, cache invalidation, internal access document, and route-level coverage were inspected.
Review details
- Run:
d54c39c1-957d-40e4-aa07-2a8893455da6 - Attempts: 1
lloydmak99
left a comment
There was a problem hiding this comment.
Per-organization access to unlisted models looks sound. Every request-path model lookup funnels through resolve_model_cached → caller_can_use_model (services/models/mod.rs): listed models short-circuit to allowed, unlisted models allow only Trusted or a granted Organization, failed grant lookups error rather than allow, and refused models return the same NotFound as an unknown name. All inference call sites pass Organization(api_key.org); public routes pass Anonymous; platform/title/auto-redact paths use Trusted on constants. Grant/revoke write the grant and audit rows atomically, the hot-path check filters o.is_active AND m.is_active, org deletion cascades with audit logging, and migration V0094 is additive (safe rolling deploy).
No blocking issues found. Minor notes, none blocking:
- The attestation report path resolves for the caller before the no-nonce cache lookup, and the deprecation middleware resolves per-request — worth a glance to confirm these close the cross-org cache/header leaks as intended.
Checks: git diff --check and cargo fmt --all -- --check passed; local cargo build/tests couldn't run (environment lacks the cc linker), but CI is fully green on head 5a858e0 — unit, integration, E2E, lint, cargo audit/deny all passing.
Storage for per-organization access to unlisted models. Nothing reads it for a request yet: the next commit enforces it. - organization_model_access (organization_id, model_id, created_at, created_by_user_id), primary key (organization_id, model_id), index on model_id (V0097). A grant is a relationship between two rows, written by admins, so no existing column can carry it. - ModelsRepository::organization_has_model_access: whether a grant is in effect. A grant counts only while the organization and the model are both active. The trait default refuses, so an implementor that keeps no grants cannot open a model. - ModelsRepository::list_restricted_model_access: every active unlisted model with the active organizations that hold a grant. The trait default fails instead of returning an empty list, which a reader would take to mean that no model is restricted. - AdminRepository: look a model up for access management, grant, revoke, list by model and by organization. - Deleting an organization removes its grants in the same transaction, as it already revokes its reporting tokens. Part of nearai/infra#264.
…rant Until now `isListed: false` only kept a model out of the public catalogs: anyone who knew its name could call it. An unlisted model is now a restricted model. An organization may use it only while it holds a grant, and everyone else gets exactly the answer a model that does not exist gets. Every lookup of a model for a request states who is asking: - ModelCaller::Anonymous: listed models only. - ModelCaller::Organization(id): listed models, plus the unlisted models the organization holds a grant for. - ModelCaller::Trusted: every model (the platform itself). ModelsServiceTrait::resolve_and_get_model, get_model_by_name and resolve_alias_cached, CompletionServiceTrait::get_model and AttestationServiceTrait::get_attestation_report take the caller, so a call site that does not say who is asking does not compile. They all end in resolve_model_cached, which returns None both for an unknown name and for a model the caller may not use. Each route therefore answers with the unknown-model response it already had. Callers: inference routes and the completion service use the API key's organization, the attestation report too, the public ITA token route is anonymous, the deprecation-header middleware and the billing of the auto-redact classify pass are trusted. Usage recording and admin flows are not changed. Paths that needed more than the new argument: - Attestation report: nonce-less reports are cached by model name and a cache hit skips the build where the model was resolved. The model is now resolved for the caller before the cache lookup, so a report cached for a granted organization is not served to anyone else. The service shares the resolve cache of ModelsServiceImpl for this. - Chat completions reads the catalog list before the completion service resolves the model and can answer from it (input modalities). The row is read only if the organization may use the model. - Strict alias mode and the alias warning name the canonical model behind an alias; both resolve for the caller. - The Responses API looks the model up and calls the provider pool directly for image models; that lookup takes the organization. - GET /v1/attestation/report runs under the API-key middleware variant that also hands the handler the key's organization. It is the same credential check. Cost: a listed model pays nothing new (no database query, no lock): the check is `is_listed` on the row already resolved. An unlisted model costs one grant lookup per (organization, model), cached with the resolve cache (same TTL, positive and negative answers) and cleared by invalidate_models_cache(). Part of nearai/infra#264.
Admin routes, with the authentication, token permissions and error
shapes of the other admin model routes:
- GET /v1/admin/models/{model_name}/access: the model (id, modelName,
isListed, isActive) and the organizations that hold a grant. Read.
- PUT /v1/admin/models/{model_name}/access/{org_id}: grant. Idempotent:
a second PUT returns the existing grant unchanged. Write.
- DELETE /v1/admin/models/{model_name}/access/{org_id}: revoke.
Idempotent. Write.
- GET /v1/admin/organizations/{org_id}/model-access: the models an
organization holds grants for. Read.
Unknown model or organization: 404. A model is looked up by canonical
name, active or not, so a grant can be reviewed or revoked while its
model is inactive; the organization must be active. The grant keeps the
admin who made it; a revocation deletes the row and is logged with the
admin's id.
Granting or revoking clears the cached access decisions, so it applies
to the next request on the instance that served it. Other instances
follow within the resolve cache TTL (30 seconds).
Internal route for gateways, behind the service token of
POST /v1/internal/usage and answered with Cache-Control: no-store:
GET /v1/internal/model-access
{"object":"model_access",
"models":[{"id":"<model name>","organization_ids":["<uuid>", ...]}]}
It lists every active unlisted model with the active organizations that
hold a grant (an empty list when there are none). A model that is not
in the document is unrestricted. It is read from the database on every
call.
Part of nearai/infra#264.
- A table over every inference route that takes a model name. For an unlisted model that has a provider, an organization without a grant gets a response equal to the unknown-model response of that route: status, headers and body. With a grant the request is served. - A guard that fails when a route is added to the inference routers without being added to that table, or to the list of routes that take no model name. - The pre-dispatch checks (input modality, strict alias mode, alias header on the attestation report) do not disclose an unlisted model. - Grant, then revoke: both apply at once on the same instance, also over a cached decision. - Listing a model opens it, unlisting restricts it, and grants prepared while it was listed apply from then on. A grant on an inactive model has no effect. - Admin routes: happy paths, idempotency, 404s, token permissions. - GET /v1/internal/model-access: content and authentication. - The public ITA token route treats an unlisted model as unknown. - Usage is recorded for an organization that holds no grant. Part of nearai/infra#264.
- Deprecation headers leaked an unlisted model. A streaming POST /v1/responses answers 200 before its model is resolved, and the header middleware then looked the model up as a trusted caller, so a caller without a grant got x-model-deprecation-date and x-model-successor. The middleware now resolves for the request's own caller: the API key's organization, or anonymous when there is none. - Grant and revoke cleared the model list cache and the resolved models, which reloads the list from every backend. They now drop the one cached access decision (ModelsServiceTrait::invalidate_model_access), and only when the call changed something. - The title model depended on the caller's grants: the platform chooses it, but it was resolved for the request's organization. It now runs through CompletionServiceTrait::create_platform_chat_completion, which resolves a PlatformModel as trusted. A PlatformModel cannot be built from a string and replaces the model the request names, so a model name from a request cannot take that path. - GET /v1/internal/model-access lists every unlisted model, active or not. A lane can still be serving a model that was deactivated here, and a model missing from the document is read as unrestricted. - Grants and revocations are audited: organization_model_access_log, one row per change with the acting user, written in the statement that makes the change and only when it changed something. Deleting an organization logs its removed grants as revocations with no acting user. Same migration file (V0097, not applied anywhere yet). The log's action column is registered as approved plaintext for the database encryption checks. - Tests: every fixture model of the access table has a planned deprecation, so the table compares response headers too, and a dedicated test covers the headers with and without a grant. The route guard reads every router builder by method and path. Aliases are covered on every text route. Part of nearai/infra#264.
The route guard fails for any registered route without a decision on record. Since the branch was cut, main added: - GET /v1/attestation/pins (#1263): public, the attested providers' trust pins. It takes no model name and reads no catalog row. - The credit transfer and organization credential routes (#1271). They take no model name. Their routers are built in their own modules, so the guard now scans those two files as well as src/lib.rs and the attestation router; otherwise a model route added there later would not be seen. Part of nearai/infra#264.
5a858e0 to
e392cc3
Compare
- A grant could outlive its organization. The admin service checked that the organization was active and then inserted the grant; a deletion in between removed the other grants and left this one. The grant now runs in one transaction that holds the organization row (FOR SHARE) and inserts only while it is active. Deletion updates the same row and removes the grants in its own transaction, so either it waits and removes this grant too, or the grant waits and answers organization_not_found. - A listed model could name an unlisted successor, and its name reached callers that may not use it: x-model-successor and the successor of the public catalogs. The deprecation middleware now names the successor only if it resolves for the request's own caller (both lookups share the time limit), and the public catalogs keep a successor only if it is a listed model. The trusted catalog keeps it. - A failed access check during alias resolution is logged; the alias still stays unresolved. - Docs: the four admin access routes document 403; revoke states that the organization must be active; the gateway document says that its entries are canonical names, so a gateway checks the model an alias stands for. Part of nearai/infra#264.
|
On the non-blocking notes from the reviews of 10-12:
|
lloydmak99
left a comment
There was a problem hiding this comment.
This PR turns isListed: false into a real access boundary: unlisted models are usable only by organizations with an explicit grant, and refusals are indistinguishable from "model not found." The access-control logic is sound and fail-closed, and all previously raised blocking items are addressed.
services/models/mod.rs: resolution funnels through a single choke point (resolve_model_cached → caller_can_use_model) — listed/Trustedallowed,Anonymousdenied,Organizationgrant lookups fail-closed (a failed lookup returnsErr, never an allow), and refusals return the sameNotFoundas unknown names.- Grant/revoke vs. org-deletion race is handled:
grant_model_accesstakesFOR SHAREon the org row and org deletion removes grants inside theFOR UPDATEdeletion transaction, so the two serialize; grant/revoke+audit are atomic single-statement CTEs. - Attestation report cache no longer leaks — the model is resolved for the caller before the no-nonce cache lookup, so a disallowed caller errors out before any cache hit.
- Deprecation/successor disclosure only names a successor the caller may actually use, including the streaming-200 path.
- Migration V0097 is additive (two tables + indexes), safe for rolling deploy. Note: flipping an existing model to
isListed: falseis a hard access cutover — an operational rollout matter the author verified against prod (one unlisted model, no direct API inference traffic), not a code defect.
Checks: git diff --check and cargo fmt --all -- --check both clean; local compilation unavailable here (no cc linker), relied on reported-green CI (unit, integration, lint, audit/deny) on head 5a858e0.
Part of nearai/infra#264. Base:
main(it builds on #1233, which is merged).Seven commits:
feat(models): store which organizations may use a model: the table and its repository.feat(models): restrict unlisted models to organizations that hold a grant: the rules and every call site.feat(admin): manage model access grants and publish them for gateways: the admin routes and the internal document.test(e2e): cover per-organization model access.fix(models): close the review findings on model access: see "Review round" below.test(e2e): give the routes added on main a model access decision: see "Rebase onto main" below.fix(models): close the second review round on model access: see "Second review round" at the end.What and why
#1233 added
isListed. It keeps a model out of the public catalogs, but anyone who knows the model's name can still call it if it has a provider.This PR makes an unlisted model a restricted model. An organization can use it through this API only while it holds a grant. Everyone else gets exactly the answer a model that does not exist gets. Grants are (organization, model) pairs, managed by admins.
Example: model X is unlisted and organization A holds a grant. A can call X. Nobody else can, and nobody else can tell that X exists.
Access rules
POST /v1/internal/usagestill records usage on any active model for any organization: usage that was served is always billed.GET /v1/models,GET /v1/model/list,GET /v1/model/{name}) are not affected. They stay public-only.How it is enforced
Every lookup of a model for a request now states who is asking:
ModelsServiceTrait::resolve_and_get_model,get_model_by_nameandresolve_alias_cached,CompletionServiceTrait::get_modelandAttestationServiceTrait::get_attestation_reporttake the caller. A call site that does not say who is asking does not compile.All of them end in one function,
services::models::resolve_model_cached. It returnsNoneboth for an unknown name and for a model the caller may not use. The call sites cannot tell the two apart, so each route answers with the unknown-model response it already had. No new error variant, status, type or message.Who asks what:
create_chat_completion,create_chat_completion_stream)GET /v1/attestation/reportGET /v1/attestation/ita-token(public)GET /v1/model/{name}(public)resolve_public_model)PlatformModel(below)Paths that needed more than a new argument:
x-no-aliasing) and the alias warning name the canonical model behind an alias. Both now resolve for the caller.x-model-deprecation-dateandx-model-successorto successful responses. A streamingPOST /v1/responsesanswers 200 before its model is resolved and fails inside the stream, so "the response is a success" does not mean the caller may use the model. The middleware now resolves the model for the request's own caller.TITLE_GENERATION_MODEL), billed to the organization. It must not depend on that organization's grants.CompletionServiceTrait::create_platform_chat_completionruns aPlatformModeland resolves it as trusted. APlatformModelcannot be built from a string: its constructors are a closed list that reads platform configuration, and it replaces whatever model the request names. A model name from a request cannot take that path.GET /v1/attestation/reportnow runs under the API-key middleware variant that also hands the handler the key's organization. It is the same credential check (the two middlewares share one function).Cost
model.is_listedon the row that was already resolved.invalidate_models_cache()(catalog writes) clears the grant cache with the others.Admin API
Same authentication, token permissions and error shapes as the other admin model routes. Model names with a slash must be URL-encoded. Unknown model or organization: 404 (
model_not_found,organization_not_found).GET /v1/admin/models/{model_name}/accessid,modelName,isListed,isActive) and the organizations that hold a grantPUT /v1/admin/models/{model_name}/access/{org_id}DELETE /v1/admin/models/{model_name}/access/{org_id}GET /v1/admin/organizations/{org_id}/model-accessA model is looked up by canonical name, active or not, so a grant can be reviewed or revoked while its model is inactive. The organization must be active.
Internal API for gateways
GET /v1/internal/model-access, behind the service token ofPOST /v1/internal/usage(CLOUD_API_USAGE_TOKEN), answered withCache-Control: no-store. 503 until the token is configured.{ "object": "model_access", "models": [ { "id": "vendor/model", "organization_ids": ["5f1c…", "9a2e…"] }, { "id": "vendor/other-model", "organization_ids": [] } ] }It lists every unlisted model, active or not, with the ids of the active organizations that hold a grant (an empty list when there are none). A model that is not in the document is unrestricted. That is why a model that was deactivated here stays in it: a lane may still be serving it. A gateway polls the document and enforces access per request. It is read from the database on every call, so a change shows on the next poll of any instance.
Storage
One migration file, two tables. It takes the next free number at merge time; it is
V0097__add_organization_model_access.sqlon the branch (main's highest is V0096).Why a migration and not an existing column: a grant is a relationship between two rows, written by admins.
organizations.settingsis customer-editable, so it must not hold access state.Audit. A revocation deletes its grant row, so the grants table cannot say who had access when. The log is append-only: one row per grant and one per revocation, with the organization, the model, the acting admin and the time. The row is written in the same statement as the change, and only when the change happened: granting a grant that exists, or revoking one that does not, writes nothing. Nothing in the API reads it. Its one text column (
action) is registered as approved plaintext in the database-encryption field registry.Deleting an organization removes its grants in the same transaction, like it revokes its reporting tokens, and logs each one as a revocation with no acting user.
User erasure needs no change. Both tables hold ids only, and the admin's email is read from
userswhen grants are listed. The log's foreign keys carry noON DELETEaction, like the role audit log (V0077): erasure rewrites theusersrow in place and keeps its id.Rollout
POST /v1/internal/usageand stays open. That lane's organization needs its grant before the gateway turns its own enforcement on.isListed: false, enforcement also follows on the other instances within those 30 seconds. But the pre-dispatch metadata read from the 5 minute list cache (the alias name, the input modality answer) can linger there for a model that was public minutes earlier.Behaviour changes for existing clients
isListed: falsethrough the existingPATCH /v1/admin/modelsnow cuts off callers without a grant. Before, it only hid the model from the catalogs.x-model-successoris left out when the successor does not resolve for the caller, and the public catalogs keep a successor only if it is a listed model. A successor that is no longer active is left out the same way. The trustedGET /v1/internal/modelskeeps it.Known limit
An unknown name and a refused name do different database work: the unknown name misses on every request, the refused one is answered from caches. A caller close enough to the database to measure that could tell the two apart by timing. The responses themselves are equal.
Tests
Unit:
services::models: the public views name only a listed successor; the caller rules, the grant cache (both answers, TTL, full invalidation), access-only invalidation (one decision dropped, the list and the resolved model not read again), a failed grant lookup is an error and never an allow, the trait defaults fail closed.organization_not_found.E2E (
organization_model_access.rs):src/lib.rs, and over the attestation, credit transfer and organization credential routers it merges: each registered method and path is either in the table or listed with the reason it is not. A new route fails the test until that decision is made.GET /v1/internal/model-access: content and authentication.Run locally on the last commit, against a fresh PostgreSQL database each:
cargo fmt --all -- --checkcargo clippy --all-targets --all-features -- -D warningscargo nextest run --lib --binscargo nextest run --test e2e_allcargo nextest run --test integration_testscargo test --locked -p database --test hot_path_typed_queries --test billing_typed_queries --test credit_allocation --test session_rotation --test staking_wallet_binding -- --test-threads=4cargo test --locked -p database --test credit_transfers -- --test-threads=1cargo check --all-targetson each commitThe existing suite is unchanged apart from test doubles that implement the changed traits. No existing assertion was weakened.
I also broke three checks on purpose to see the new tests fail: the embeddings handler asking as a trusted caller, the chat pre-dispatch gate disabled, and the deprecation middleware resolving as trusted again. Each time the matching tests failed.
Review round
The fifth commit closes the findings of an independent review:
Rebase onto main
Rebased on 2026-10-21 onto main at 71bbb0b (24 commits). The merge commit of 10-12 is gone.
legacy_credit_reconciliations), V0095 and V0096, so the file is nowV0097__add_organization_model_access.sql. Same content.routes/completions.rswas split into one module per endpoint (refactor(api): split routes/completions.rs into per-endpoint modules #1265). Each change moved with its code:reject_if_aliasedincompletions/mod.rs, the chat pre-dispatch checks inchat.rs, text completions intext.rs, the trusted lookup for the auto-redact classify billing inchat_auto_redact.rs, the per-route lookups inaudio.rs,images.rs,embeddings.rs,rerank.rs,score.rsandprivacy.rs, andGET /v1/internal/model-accessinmodels.rs. No change in behaviour.build_attestation_routesnow also takes the admin settings forGET /v1/attestation/pins. The report route keeps the workspace-context middleware from this PR.x-model-pricing-change-date(feat(models): announce scheduled pricing changes #1247) is added by the same middleware as the deprecation headers, after the lookup for the request's own caller. So a caller without a grant does not get it either; the table test compares every response header. The two new e2e lookups inmodel_pricing_notices.rsread the model asModelCaller::Trusted.GET /v1/attestation/pins(feat(attestation): admin-added Sigstore-verified Tinfoil model pins; provider-neutral pins listing #1263) is public and takes no model name. The credit transfer and organization credential routes (feat: add single-hop credit transfers and scoped organization credentials #1271) take no model name either; their routers live in their own modules, so the guard now reads those files too. That is the sixth commit.Second review round
The seventh commit closes the review threads of 10-12:
FOR SHARE) and inserts only while the organization is active, in one transaction; otherwise it answers 404organization_not_found.x-model-successorand in the public catalogs. See "Behaviour changes" above.