Agent Hosts and Apps request inference through a typed Runtime resource. They do not receive hosted-model credentials, provider endpoints, local model process access, or authority to choose a hidden alternate backend.
This document defines the target contract needed by a durable Agent Host. Current implementation truth stays in GitHub issues.
Agent Host or App
-> Runtime capability check
-> model provider operation
-> configured local or hosted backend
-> typed stream events and terminal result
Runtime authorizes the caller, resource, action, model policy, and session. Runtime stores hosted credentials in its owner-only secret store, selects the approved HTTPS destination, and brokers each hosted effect. The confined model provider owns protocol adaptation, request serialization, streaming, rate-limit translation, and provider-specific error handling. The model backend supplies output; it does not gain Runtime authority.
A tool call or effect proposal returned by a model is untrusted input. The Agent Host must submit it as a new typed Runtime operation under the active agent principal and session.
ElastOS has one typed model-provider contract: offers_list, runs_create,
runs_get, runs_events, and runs_cancel. A Runtime can use a configured
offer backed by a local engine or a hosted API. Backend placement and consumer
placement are independent:
| Backend | Same-Runtime use | Granted cross-Runtime use |
|---|---|---|
| Local engine | Operator-configured local inference | Share the selected local capability after remote acceptance |
| Hosted API | Home owner creates named hosted instances on the model-provider capsule. Default is private. | Owner-funded share after explicit Share, grant, and exact-model terms check |
Both consumption paths use the existing model operations and service-offer
contract. Carrier admits the model target through the destination Runtime's
remote-model authorization path. The destination verifies the authenticated
source endpoint and service grant, then constructs its own local provider binding
for the consumer principal, capsule and run. RuntimeCreateBinding and
RuntimeAccessBinding remain local-channel authority records. Installed
acceptance and remaining remote-service work are recorded in GitHub issues.
An operator explicitly selects a configured capability for publication under
Runtime policy. The owning Runtime publishes it as an
elastos.service.offer/v1 service and owns grants, quotas, selection, audit,
and routing. For remote model work, the
destination Runtime verifies the signed offer and grant, then maps the
authenticated source Runtime and consumer principal, capsule, and run into
destination-owned authority. Adding an allowlist entry alone cannot establish
this binding. Carrier authenticates and transports the route that Runtime
selects; the destination grants model authority. Publication advertises the
selected capability. Destination-owned grants authorize its use. The host Home,
workspace, and other runs retain their separate access rules. Hosted credentials
stay in the owning Runtime's secret store; local model artifacts stay in their
owning boundaries.
Marketplace compares each signed model's declared minimum memory with the device's effective physical or cgroup memory capacity before Get. This check describes the minimum requirement. Runtime checks the device's memory again before starting the engine for a reply.
On Linux, Runtime verifies the visible cgroup v2 hierarchy and reads every
ancestor's memory limit and current usage. Missing or unreadable required files
make the observation unavailable. File cache credit is bounded by current
usage; an unreadable cache statistic gives zero credit. An unavailable memory
observation returns model_memory_observation_unavailable and asks the Home
owner to check the memory limits before trying again.
A local engine is admitted right before it starts: the device's room must cover
the model weights, its KV cache at the configured context, batch scratch and
headroom of max(10% of effective memory capacity, 1 GiB); otherwise the run fails with
model_memory_unavailable before anything is spawned, and its message names the
room Runtime saw and the model's need in GB. On Linux the room is MemAvailable
capped by the cgroup limits above. On macOS it is a ceiling, not free memory:
total memory less wired and compressor-occupied pages, which macOS fills by
compressing other apps' memory or dropping file cache. A macOS critical memory
pressure level, or an unreadable one, also refuses with model_memory_unavailable;
warning pressure does not. While a model loads or generates, the provider reads
the macOS pressure level about once a second; at critical it stops the run and
its engine, which releases the weights Metal keeps wired, and the run fails with
model_memory_critical. Each model-provider
process runs one local engine at a time, stops it after 60 s without requests
and starts it again on the next request. Runtime's host lock protects its Home
data root. Distinct data roots use separate locks and providers, including
under the same OS account. For supported Local AI operation, run one active
Home per device. Each provider checks the device's memory when it starts its
engine.
The owning Runtime DID signs the service offer, which names the admitted provider capability and contains only bounded capability and policy facts. The provider identity remains an internal execution binding. Hosted URLs and credentials stay in Runtime's private route; provider process details and topology stay behind the model-provider contract. A model artifact is separate immutable content. Its canonical package identity is the CID of the complete manifest-and-payload closure. Engine, component, and payload hashes are verification facts rather than package identities. Runtime prepares and admits the package through the Content provider path in Content capsule distribution.
A hosted web API is a Runtime-owned HTTPS interoperability edge on the Home that stores the credential. Runtime checks the exact owner route and sends the selected destination and credential through its broker. The installed Mac path confines the provider's direct external sockets; Linux product confinement still needs proof. Runtime requires the owner's exact Inbox decision for each hosted connection, including saved operator connections. End saves the owner's decision before Runtime cancels admitted sends. Saved connections and their credentials remain available for owner management.
Mac Runtime to Carrier to a Jetson or seed Runtime, then
to that Runtime's model provider and backend, is service use. Ordinary capsules
see only typed elastos://model/* resources. Publishing a paid hosted offer
requires operator-owned quota, accounting, and data-policy facts.
The OpenAI Responses API is one provider-internal hosted model adapter. It creates model responses behind the same typed model-provider contract. It does not provide local Codex agent execution.
Codex integration is a separate, later agent-execution adapter. It controls
local Codex agent threads behind typed agent operations. Runtime admits each
operation through explicit filesystem, network, tool, and approval grants.
Codex can use a model internally, but it is not a model offer and does not
appear in offers_list.
A request uses one explicit selection mode:
| Mode | Meaning | Required record |
|---|---|---|
| Pinned | Use one provider and model identifier | Requested and resolved provider/model |
| Provider auto | Let one named provider choose a model according to its documented policy | Provider, requested auto selector, and resolved model when reported |
| Runtime policy | Let Runtime select from an explicit allowlist under a named policy | Policy revision and selected provider/model |
The selection mode is part of the request and session facts. auto is not a
model identity and must not be displayed later as if it were the model that ran.
If the upstream provider reports the resolved model, the provider records it.
If it does not, the result says that the resolved model is unknown.
Fallback is opt-in. A policy must name the allowed providers or models, trigger conditions, ordering, data-handling constraints, and cost or rate limits. A timeout, refusal, context error, or malformed response must not silently switch providers. The result records every attempted backend without exposing secret configuration.
Changing provider or model changes inference behavior, not the agent principal, Agent Host artifact, conversation identity, memory ownership, capabilities, or mandates.
Runtime assigns or validates a durable request ID unique within the agent task.
New preparation and inference requests use model-v1:<issued_ms>:<nonce>.
The client creates a random nonce for each new action and retains that complete
ID during recovery. Runtime and the provider look up retained bindings first,
including legacy IDs. An unseen ID needs a valid timestamp within five minutes,
with at most one minute of future clock skew. An expired or missing legacy ID
receives the fixed request_expired refusal before new work starts.
Retained bindings survive their complete freshness window. Before pruning a terminal result, its owner persists the maximum pruned issuance time. This floor also refuses the old ID after restart or a backwards clock change. Unresolved runs retain their binding. A new action needs a new request ID.
The provider binds these facts before dispatch:
- agent and Runtime session identifiers;
- caller capsule instance;
- requested selection mode and policy revision;
- resolved provider and model when known;
- sampling and token limits;
- context or attachment references permitted for the call;
- creation, dispatch, first-event, and terminal timestamps;
- cancellation state;
- usage and cost facts reported by the backend; and
- one terminal outcome.
Secrets, raw provider credentials, and private endpoint details do not belong in the session journal or App-visible result.
For a local llama offer, offers_list includes context.context_window_tokens
for one engine slot and context.max_output_tokens for the selected profile.
Assistant packs against these values. For local llama offers only, a text/v1 or
text/v2 input can include max_output_tokens; Runtime binds that value in the
input hash. Hosted offers reject this field with context_rejected. An omitted
value uses the local profile limit. Before generation, the provider asks the pinned
engine to count the exact chat request with its template and special tokens.
It sends generation only when the counted input and requested answer fit the
slot. The offer's input and output byte policies remain separate transport
bounds.
A model request follows one observable lifecycle:
created -> dispatched -> streaming -> completed
| -> failed
| -> cancelled
| -> reconciling -> completed
| -> failed
| -> cancelled
| -> settlement_unknown
-> failed
Every request reaches one terminal state. A provider must not leave the Agent
Host or UI in responding after the backend stream has completed, failed, or
closed. A terminal event contains the request ID, outcome, final usage when
known, and enough error classification for the caller to decide whether a new
request is safe.
Stream events use monotonically increasing sequence numbers. Reconnection asks for events after the last durable sequence. The provider may replay already emitted text events with their original sequence numbers, but the consumer must deduplicate them. Reconnection cannot dispatch the model request again.
New model runs use compact journals of at most 1 MiB, with a 32 KiB header
and 4 KiB backend state. Admission allows at most 256 journal entries and
64 MiB of charged growth. Active runs reserve at least 1 MiB each. Frozen
terminal records, including unresolved outcomes, use their actual bounded
size and still count toward the entry limit. Every terminal outcome, including
settlement_unknown, stays truthful through its configured history window and
the complete request replay window. Runtime commits a monotonic refusal floor
before expired history is removed, so clock rollback cannot dispatch it again.
Crash transaction files count toward the limits. Runtime checks
these limits before it records a new run or dispatches inference.
Recovery reads one record at a time, with ceilings of 512 entries, 32 MiB per legacy record and 128 MiB for the complete scan. Existing runs can settle with their original binding when new admission is full. An atomic replacement temporarily owns one extra file; failed writes remove only that owned file. At startup, the coordinator validates the complete bounded journal and committed request floor before it removes an abandoned transaction. Removal requires the exact private file and parent identities and a confirmed absent creator PID. Files with a live or unproven creator, invalid metadata, or changed identity remain in place for operator recovery.
When legacy history exceeds the entry or aggregate ceiling, startup moves the
exact private runs directory into one protected recovery archive and opens a
fresh bounded journal. Startup preserves the archive without decoding every
record. The existing run operations validate and read an exact archived record
for its owner; an archived active run reports settlement_unknown without
contacting its backend. Retained legacy and future-issued request identities
still resolve before fresh request validation. Home starts with the archived
facts available for owner inspection. Fresh inference stays gated while the
owner uses Reclaim in System to retire the admitted local models, then uses them
again. The active serialized coordinator saves an acknowledgement bound to the
archive only after its owned execution, artifacts and workers are absent.
Operator-configured hosted offers keep their configuration when an active
coordinator receives trusted Init with the existing owner_reclaim: true flag;
the same ownership checks apply. That provider operation remains an operator
contract. A second recovery preserves the existing archive and requires operator
recovery instead of replacing it.
Cancellation is a request to stop backend work and settle the request as
cancelled. The provider records the request before closing the backend stream.
If the backend cannot confirm cancellation, the request enters reconciliation
rather than being reported as safely cancelled.
The managed local engine holds its execution slot while Runtime closes its
owned engine and process group. Confirmed closure settles the request as
cancelled; an unresolved close settles as settlement_unknown and retains
the engine ownership. Hosted Chat Completions and Responses cancellation settles
as settlement_unknown after dispatch because stream closure alone cannot
confirm backend stop. Terminal results and events keep their original identity
across replay and restart. Installed acceptance stays with the owning issue.
Interrupting the presentation layer does not cancel provider work by itself. The Agent Host can reconnect to the same request ID and recover durable events. After an Agent Host or Runtime restart, the session journal must distinguish:
- a completed request whose terminal event can be replayed;
- a request still running and eligible for reattachment;
- a confirmed cancelled or failed request; and
- a request with unknown backend settlement that requires reconciliation.
Resuming a conversation creates a new inference request unless it is only reattaching to an existing request. The UI must show which case occurred.
Provider results use stable error classes while retaining a redacted backend code for diagnosis:
- selection unavailable;
- credentials unavailable;
- authentication rejected;
- rate limited;
- context or input rejected;
- backend timeout;
- transport interrupted;
- backend failed;
- response malformed;
- cancelled; and
- settlement unknown.
Errors identify the failed request and whether any output was emitted. They do not expose API keys, authorization headers, private URLs, or raw backend logs to ordinary capsules.
For a remote create, Runtime can return a request-bound
elastos.model.invocation-refusal/v1 assertion. It states that this invocation
stopped before provider dispatch (dispatch: not_started). A closed provider
request_expired fault has a separate phase, dispatch: provider_refused:
provider validation ran and declined new inference. The destination emits it, the consumer checks
the exact request, offer, input hash and remote binding, and the gateway returns
HTTP 409. The local gateway binds the same provider expiry fact to its normalized
request. Assistant checks its fresh request identity, offer, operation and phase.
A retained or pending request, generic provider error, malformed assertion or lost reply keeps its
recovery or unknown status. Carrier tries another address after connection
failure, but stops after a model create invocation error. This assertion settles
the current invocation only; it does not settle earlier uncertain attempts.
Hosted gateways such as OpenRouter and local engines such as llama.cpp or Ollama are provider implementations, not architecture. Their model identifiers and availability can change independently of ElastOS.
The owner of this Home enters hosted credentials in Home Settings. Assistant and Marketplace open that Settings tab. The Home stores the key in the existing owner-only model-provider config. After an explicit run, the selected hosted processor receives the prompt. Product UI names the Home that stores the key, the processor that receives prompts, the requested selector, the resolved model when known, and whether explicit fallback is enabled. Assistant keeps one model selector. The composer placeholder is Message Assistant. The trigger shows the user-chosen instance name. Each selector row shows that name and a short route subtitle. Expanded detail names the processor, prompt destination, payer, limits, privacy, and availability. Cost appears in a completed run receipt when the backend reports it. The UI never shows the secret.
Runtime validates the credential separately from a consented paid test.
Validate requires Runtime-owned HTTPS authority to the pinned public host.
Validation needs the same exact owner Inbox approval as every other hosted
request, including saved operator connections; End stops its sends. The Home
can also use an owner-scoped
providers/model-provider/validate-fixtures.json. That file, when present,
may bind validate HTTP only to http://127.0.0.1 with an explicit port.
Save stays private and publishes no service offer. Share uses grants, Carrier
routing, and run journals. In 0.8.0 the Services app, where Share lived, is
hidden: a hosted model shared by an earlier version stays shared, and each
person's access ends when its grant expires, 6 hours after approval, or when
the owner selects Revoke access in Inbox. The key stays on the provider Home. Consumers see the
intermediary Home, the selected upstream processor (OpenRouter or Venice), and
the payer. One
connection budget covers that provider's private and shared offers, including
unresolved spend. Provider-side key and account caps cover activity outside
this Runtime. Qualify the exact model and provider terms before Share.
OpenRouter Terms 5.1–5.2 and Venice TOS 7.3 End User API terms are starting
points, not blanket model qualification. Catalogues, browser persistence,
logs, and API responses omit keys and private URLs. The product surface is
the generic model-provider capsule with repeatable owner-bound instances.
Each instance has a stable offer identity, display name, adapter, selected
model, Runtime secret reference, privacy and share policy, limits, and
lifecycle. One Home can keep several instances of the same provider, for
example private DeepSeek and Qwen via OpenRouter, and shared Venice. Private is
the default. Changing or disconnecting one instance leaves
the others. Existing model:openrouter and model:venice offers migrate into
named instances without losing secrets.
The on-disk operator form remains providers/model-provider/config.json with
mode 0600 under parent mode 0700. Secrets live in Runtime-owned secret
storage beside that provider root. That file is the provider boundary, not a
user-facing editor. Terminal, JSON, and key-file workarounds are not the
product path. System supplies generic installed-capsule and secret
management. The capsule UI is Add hosted model, Name, Provider, API key,
Model, Test, and Save, then Use in Assistant, Share as service, Edit, and
Disconnect. An authenticated edit with a blank key retains the stored key of
that same provider instance. A new instance requires a key. Save retries keep
one instance identity, including while activation is pending.
An exact key-only update takes effect for the next request. A running request
retains its original credential and journal binding. A failed provider refresh
returns a pending state after saving configuration; the Home must retry activation
or restart before claiming that configuration is active.
A Decisions offer binds the exact validated catalog entry's canonical model
ID to the offer revision. This binding is part of the execution hash. The
adapter checks the response against that ID before projecting output.model
as the selected contract model. The terminal backend report retains the actual
provider-reported canonical ID and accounting. These are provider reports,
not attestations of upstream computation. Missing or changed identity fails
validation; model prefixes and date suffixes do not establish equivalence.
Canonical architecture documents do not freeze a commercial model name or claim that a catalog entry will remain available.
Hosted configuration records privacy policy, cost and rate limits, requested selector, resolved model when the backend reports it, and explicit fallback policy. The secret stays in Runtime-owned storage on the provider Home. It does not enter catalogues, offers, logs, or a consumer Home.
System offers no decision models and no Approval Lens evaluator: OpenRouter discovery lists text models only, no screen selects an evaluator, and an instance's display name never selects one. Runtime asks no evaluator: an evaluator choice saved by an earlier release is ignored, and an Assistant hosted request needs only the hosted connection approval in Inbox. A decision model saved by an earlier release stays in System's Hosted list, without Use in Assistant, so it can be disconnected.
Runtime verifies bounded signed catalogue metadata, prepares content, checks the complete package closure and binds an admitted artifact to a compatible model offer. Marketplace owns discovery and details; System owns model configuration and storage controls. Assistant selects an exact admitted or granted offer and keeps its typed run lifecycle. These surfaces share Runtime state.
Marketplace separates verified local content and exact granted shared models. In 0.8.0 the Services app is hidden, so Marketplace no longer lists contacts' AI services and no new access can be requested. A model approved earlier stays listed until its grant expires, 6 hours after approval, and opens in Assistant. Provider Home, processor, prompt destination, payer and reported limits use the same facts as Assistant. Discovery gives Marketplace read authority; Runtime retains grant and inference authority.
The installed small-model journey and its exact evidence scope are recorded in GitHub issues. Acquisition, readiness, inference and warm reuse are separate checks. A local pin establishes a retained local copy; it does not establish global availability or exclusive custody.
Selecting a model may ask Runtime to prepare it under current authority. The person sees availability, Preparing/progress, Ready or actionable failure, offline and incompatibility states. Preparation can be cancelled or retried. Use records the caller's Keep claim with the new preparation reservation. Opening an admitted model or selecting it explicitly in Assistant waits for a matching Keep acknowledgement; restoring a workspace reads existing state. Assistant requires a verified catalog mapping for admitted local model selection. System owns explicit removal. It checks other people's claims before withdrawing the offer, preserves the caller's Keep on failure, and releases that claim only after successful removal. Recovery uses the recorded Remove caller. Active-run protection, recoverability evidence and warning/consent for possible sole-copy loss remain part of the removal contract. Installed proof covers a new pending Use claim and idempotent Open/selection of an already-kept SmolLM2 model, including reload and preserved model files. Removal failure and recovery cases use bounded source fixtures. Remote inference is a separately selected service. The same inventory distinguishes retention, admission and readiness, while catalog identity stays visible. A local pin alone proves neither trust nor engine readiness. There is no user GGUF download, file-picker or private path configuration step in this product flow.
Runtime validates signed publisher/catalog and package facts, resource policy, engine compatibility and content integrity before deriving a private verified artifact descriptor for the model provider. Source supports operator offers and admitted-content binding through the existing ProviderRegistry and run journal, while preserving other configured capabilities. An idle-safe provider refresh can expose the newly admitted local offer; a busy run or retained unknown outcome prevents destructive reconfiguration and removal. The explicit owner Reclaim operation can retire a model while its unknown history stays retained, after owned execution closes. Expired history releases its model binding; live engine and worker ownership remain separate barriers. Content preparation status is distinct from a dispatched model run.
Existing offers_list, runs_create, runs_get, runs_events and runs_cancel
remain the inference contract. Runtime binds the selected package and offer to
the exact admitted descriptor; the provider revalidates it before inference.
Preparation failure never changes the selected model to an available substitute.
Drafts and existing runs survive selection, progress, cancel, retry and restart.
The ordinary explicit Send/run action dispatches once only when that selected
model and provider are ready.
Bounded transfer and manifest/inventory limits are in Content capsule distribution. The current execution queue and proof dependencies are in GitHub issues. Fresh-install acceptance must prove the compatible engine and its verified libraries are available, then select the real signed catalog entry with no pre-existing GGUF or private offer setup and obtain a real Qwen reply. Exact package size, publisher trust, license/provenance and availability deployment are prerequisites. Small signed fixtures prove rejection and lifecycle behavior, not a production catalog or real inference. Code, tests, manifests and docs go to Git review; model bytes and private publisher keys stay outside Git.
Local-engine and hosted-API acceptance are separate tracks. Each backend must pass installed lifecycle tests before it can be shared:
- Evaluate Qwen3.5-9B Q4_K_M as the stable Mac baseline and PrismML Bonsai 8B Q1 as an experimental low-memory comparison on an M5 Mac with 24 GB of memory. Qwen3.8-27B and Bonsai 27B remain later benchmark candidates rather than initial defaults.
- Use llama.cpp as the common first engine for macOS Metal and later Jetson CUDA. Pin engine and model provenance. Runtime verifies installed artifacts; the model provider owns start, health, limits, streaming, cancellation, shutdown, restart, and orphan cleanup. Consider MLX only if the common engine path proves insufficient.
- Deliver owner-bound hosted instances through the existing model-provider capsule and the OpenAI-compatible Chat Completions seam. OpenRouter and Venice remain required adapters. A user can create several named instances of the same provider. Use the seam only where official docs confirm the request. Private setup publishes no offer. Prove a live authorized private hosted run for each provider after the owner enters that provider's key in Home. Prove the existing provider-internal OpenAI Responses API adapter separately. xAI/Grok stays later.
- Prove J3 sharing of the accepted Mac local model with another Runtime. This requires signed offer and grant admission plus bounded Carrier ingress. A Jetson deployment is a separate acceptance track.
- After each private connection exists, add an explicit owner-funded Share of that exact provider and model. Services is hidden in 0.8.0; sharing returns inside People. Qualify OpenRouter Terms 5.1–5.2 or Venice TOS 7.3, plus the selected model terms, before Share. Keep one connection budget across that provider's private and shared offers, with per-consumer limits. Commercial billing, staking, and wider providers stay Later. Later, prove a full destination Runtime on Jetson. Jev stays on TypeSafe/OpenRouter until a separate proof shows it on another processor.
Installed acceptance covers Brave inference, ordered streaming, reconnect, cancellation, one terminal result, restart, engine crash and orphan cleanup, secret and endpoint redaction, and explicit paid provider choice. Remote proof uses two Runtime identities and two principals. It covers:
- signed offer admission and explicit host-owned grant, revoke, expiry, and denial before backend dispatch;
- rejection of forged
runtime_bindingfields and isolation of identical principal strings issued by different Runtimes; - scoped create, get, events, and cancel, including access denial for other runs and the host Home and workspace;
- prompt and event privacy, bounded capacity, queues, tokens, time, and usage;
- disconnect and reconnect without redispatch, restart with unknown settlement, and service withdrawal; and
- consumer consent to prompt transfer, separately from operator consent to compute use or credential charges, with honest provenance and backend facts.
These checks extend the existing service offer and model operations rather than introducing a separate sharing API, store, or journal. A content CID identifies an immutable package; a signed service offer identifies an available capability. Model output and tool proposals remain untrusted and cannot authorize effects.
A conforming model provider must prove:
- no upstream credential or private endpoint reaches the caller;
- caller, session, capability, and model policy are checked before dispatch;
- selection and fallback behavior are explicit and recorded;
- stream events are ordered and reconnectable;
- each request has one terminal state;
- cancellation and unknown settlement are distinguishable;
- reconnecting presentation does not repeat inference or effects;
- model output cannot authorize a Runtime operation; and
- provider and model changes do not change agent identity or authority.