Skip to content

docs: add Python SDK documentation and separate language guides #59

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
think-in-universe merged 2 commits into from
Oct 1, 2026
Merged

docs: add Python SDK documentation and separate language guides #59

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cbf2057
Document Python SDK and split SDK guides by language
think-in-universe Oct 1, 2026
9eff439
Use version-neutral SDK docs and JavaScript examples
think-in-universe Oct 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 27 additions & 78 deletions cloud/inference-sdk/index.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,103 +1,52 @@
---
title: "SDK"
icon: "braces"
description: "Use the TypeScript SDK for verified, end-to-end encrypted Chat Completions."
description: "Use the JavaScript and Python SDKs for verified, encrypted Chat Completions."
---

## Install
NEAR AI Inference SDK is available for JavaScript and Python. Both provide an OpenAI-compatible Chat Completions client, attestation verification, optional encryption, and explicit response-signature verification.

<CardGroup cols={2}>
<Card title="JavaScript" icon="braces" href="/cloud/inference-sdk/javascript">
Install from npm and get started with Node.js 24 or later.
</Card>
<Card title="Python" icon="code" href="/cloud/inference-sdk/python">
Install from PyPI and get started with Python 3.12 or later.
</Card>
</CardGroup>

The package requires Node.js 24 or later.
## Install

```bash
<CodeGroup>
```bash JavaScript
npm install @nearai/inference-sdk
```

## E2EE

End-to-end encryption (E2EE) is disabled by default in `InferenceClient`. Set `e2ee: true` to enable it for a model with supported model attestation. When enabled, the SDK verifies the required attestation evidence, encrypts supported Chat fields to a key from the verified model evidence, and decrypts the response automatically.

```ts
import { InferenceClient } from '@nearai/inference-sdk/node';

const baseUrl = 'https://cloud-api.near.ai/v1';
const model = 'z-ai/glm-5.3-flash';
const apiKey = process.env.NEARAI_API_KEY;
if (!apiKey) throw new Error('NEARAI_API_KEY is required');

const client = new InferenceClient({
baseUrl,
apiKey,
e2ee: true,
});

const completion = await client.chat.completions.create({
model,
messages: [{ role: 'user', content: 'Reply with the word ok.' }],
});

console.log(completion.choices[0]?.message.content);
```bash Python
pip install nearai-inference-sdk
```
</CodeGroup>

## Verification

`InferenceClient` verifies the Gateway and every returned NEAR model-attestation candidate before sending an E2EE Chat Completion. Call `verifyResponse()` after a completion to verify its response signature with the request and response bytes captured by the client.
Packages: [npm](https://www.npmjs.com/package/@nearai/inference-sdk) · [PyPI](https://pypi.org/project/nearai-inference-sdk/).

```ts
const receipt = await client.verifyResponse(completion.id);
console.log(`Verified ${receipt.signatureKind} response.`);
```

<Note>
`verifyResponse()` is explicit. For a streaming completion, consume the entire stream first, then call it with the completion ID so the SDK can verify the exact response bytes.
</Note>

### Set a TCB policy
## E2EE

By default, the SDK accepts the Intel TCB statuses `UpToDate` and `OutOfDate`. To require `UpToDate`, apply a policy to both the Gateway and model evidence:
End-to-end encryption is disabled by default. Enable it with `e2ee: true` in JavaScript or `e2ee=True` in Python to encrypt supported Chat fields to a verified model key. E2EE requires a supported NEAR TEE model and rejects models without supported model attestation.

```ts
const strict = {
policy: { acceptedTcbStatuses: ['UpToDate'] as const },
};
## Verification

const client = new InferenceClient({
baseUrl,
apiKey,
e2ee: true,
gatewayVerification: strict,
modelVerification: strict,
});
```
The clients verify Gateway evidence before sending Chat and verify model evidence where supported. In Python, model verification requires a NEAR vLLM deployment with attestation enabled; other providers use Gateway-only verification. The JavaScript SDK also supports Chutes model attestation.

### Runtime behavior
After receiving a completion, call `verifyResponse(completion.id)` in JavaScript or `await client.verify_response(completion.id)` in Python before using the answer as verified. For streaming, consume the entire stream first. Deployment verification alone does not verify a particular response.

For Node, import from `@nearai/inference-sdk/node`. It verifies the Gateway TLS peer and binds later requests to the verified Gateway identity. For a browser or application proxy, import from `@nearai/inference-sdk`; browser Fetch cannot read the TLS peer certificate, so that entry point does not perform Gateway TLS binding.
A `provider_tee` signature binds the exact request and response bytes to a verified model signer. A `gateway` signature binds those bytes to a verified Gateway signer; it does not prove that an attested model generated them.

For a manual verification flow or a custom policy, see [Verification](/cloud/verification).
See [Verification](/cloud/verification) for evidence, policies, and verification limits, or follow the [JavaScript](/cloud/inference-sdk/javascript#verification) and [Python](/cloud/inference-sdk/python#verification) guides.

## OHTTP

Set `ohttp: true` to encapsulate Chat requests and responses with OHTTP. OHTTP uses the SDK's default Ed25519 signing algorithm; it is not compatible with `signingAlgo: 'ecdsa'`.

```ts
const client = new InferenceClient({
baseUrl,
apiKey,
e2ee: true,
ohttp: true,
});
```

The Chat and `verifyResponse()` calls stay the same. Before sending Chat, the SDK verifies the Gateway's signed OHTTP configuration; a missing or invalid configuration blocks the request. OHTTP encapsulates the exchange to the Gateway, while E2EE encrypts supported Chat fields to the verified model key.

<Note>
OHTTP applies only to Chat. Attestation and signature requests remain regular HTTPS. The configured endpoint or proxy must expose `/ohttp` at the same origin. Authorization and custom headers sent on the outer request, along with the client's network address, are not hidden by OHTTP.
</Note>
Both SDKs support optional OHTTP to encapsulate the Chat HTTP exchange to the attested Gateway. It is disabled by default and requires Ed25519. E2EE protects supported fields to the model independently. See the [JavaScript](/cloud/inference-sdk/javascript#ohttp) or [Python](/cloud/inference-sdk/python#ohttp) configuration.

## Examples

See the [Inference SDK examples and setup instructions](https://github.com/nearai/inference-sdk/tree/main/examples) for runnable projects.

- [InferenceClient](https://github.com/nearai/inference-sdk/blob/main/examples/example-js/gateway/client.ts): streaming and non-streaming Chat, response verification, and Gateway image provenance checks.
- [OpenAI SDK integration](https://github.com/nearai/inference-sdk/blob/main/examples/example-js/gateway/client-openai-sdk.ts): use `InferenceClient.fetch` with the OpenAI client.
- [Manual verification and E2EE](https://github.com/nearai/inference-sdk/blob/main/examples/example-js/gateway/bare.ts): work directly with attestation, encryption, and signature-verification helpers.
See the language guides for runnable examples, or explore the [SDK source and examples](https://github.com/nearai/inference-sdk).
114 changes: 114 additions & 0 deletions cloud/inference-sdk/javascript.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
---
title: "JavaScript"
icon: "braces"
description: "Use the JavaScript SDK for verified, end-to-end encrypted Chat Completions."
---

## Install

The SDK is available on [npm](https://www.npmjs.com/package/@nearai/inference-sdk). It requires Node.js 24 or later for Node usage.

```bash
npm install @nearai/inference-sdk
```

Set your API key in the environment:

```bash
export NEARAI_API_KEY="your-api-key"
```

Save the example below as `chat.mjs` and run it with `node chat.mjs`.

## E2EE

End-to-end encryption (E2EE) is disabled by default in `InferenceClient`. Set `e2ee: true` to enable it for a model with supported model attestation. When enabled, the SDK verifies the required attestation evidence, encrypts supported Chat fields to a key from the verified model evidence, and decrypts the response automatically.

```js
import { InferenceClient } from '@nearai/inference-sdk/node';

const baseUrl = 'https://cloud-api.near.ai/v1';
const model = 'z-ai/glm-5.3-flash';
const apiKey = process.env.NEARAI_API_KEY;
if (!apiKey) throw new Error('NEARAI_API_KEY is required');

const client = new InferenceClient({
baseUrl,
apiKey,
e2ee: true,
});

const completion = await client.chat.completions.create({
model,
messages: [{ role: 'user', content: 'Reply with the word ok.' }],
});

// Verify the response signature before displaying the answer.
const verified = await client.verifyResponse(completion.id);
console.log(`Verified ${verified.signatureKind} response.`);
console.log(completion.choices[0]?.message.content);
```

## Verification

`InferenceClient` verifies the Gateway and every returned NEAR model-attestation candidate before sending an E2EE Chat Completion. Call `verifyResponse()` after a completion to verify its response signature with the request and response bytes captured by the client.

```js
const receipt = await client.verifyResponse(completion.id);
console.log(`Verified ${receipt.signatureKind} response.`);
```

<Note>
`verifyResponse()` is explicit. For a streaming completion, consume the entire stream first, then call it with the completion ID so the SDK can verify the exact response bytes.
</Note>

### Set a TCB policy

By default, the SDK accepts the Intel TCB statuses `UpToDate` and `OutOfDate`. To require `UpToDate`, apply a policy to both the Gateway and model evidence:

```js
const strict = {
policy: { acceptedTcbStatuses: ['UpToDate'] },
};

const client = new InferenceClient({
baseUrl,
apiKey,
e2ee: true,
gatewayVerification: strict,
modelVerification: strict,
});
```

### Runtime behavior

For Node, import from `@nearai/inference-sdk/node`. It verifies the Gateway TLS peer and binds later requests to the verified Gateway identity. For a browser or application proxy, import from `@nearai/inference-sdk`; browser Fetch cannot read the TLS peer certificate, so that entry point does not perform Gateway TLS binding.

For a manual verification flow or a custom policy, see [Verification](/cloud/verification).

## OHTTP

Set `ohttp: true` to encapsulate Chat requests and responses with OHTTP. OHTTP uses the SDK's default Ed25519 signing algorithm; it is not compatible with `signingAlgo: 'ecdsa'`.

```js
const client = new InferenceClient({
baseUrl,
apiKey,
e2ee: true,
ohttp: true,
});
```

The Chat and `verifyResponse()` calls stay the same. Before sending Chat, the SDK verifies the Gateway's signed OHTTP configuration; a missing or invalid configuration blocks the request. OHTTP encapsulates the exchange to the Gateway, while E2EE encrypts supported Chat fields to the verified model key.

<Note>
OHTTP applies only to Chat. Attestation and signature requests remain regular HTTPS. The configured endpoint or proxy must expose `/ohttp` at the same origin. Authorization and custom headers sent on the outer request, along with the client's network address, are not hidden by OHTTP.
</Note>

## Examples

See the [Inference SDK examples and setup instructions](https://github.com/nearai/inference-sdk/tree/main/examples) for runnable projects.

- [InferenceClient](https://github.com/nearai/inference-sdk/blob/main/examples/example-js/gateway/client.ts): streaming and non-streaming Chat, response verification, and Gateway image provenance checks.
- [OpenAI SDK integration](https://github.com/nearai/inference-sdk/blob/main/examples/example-js/gateway/client-openai-sdk.ts): use `InferenceClient.fetch` with the OpenAI client.
- [Manual verification and E2EE](https://github.com/nearai/inference-sdk/blob/main/examples/example-js/gateway/bare.ts): work directly with attestation, encryption, and signature-verification helpers.
151 changes: 151 additions & 0 deletions cloud/inference-sdk/python.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
---
title: "Python"
icon: "code"
description: "Use the asynchronous Python SDK for verified, encrypted Chat Completions."
---

## Install

The SDK is available on [PyPI](https://pypi.org/project/nearai-inference-sdk/). The package requires Python 3.12 or later.

```bash
pip install nearai-inference-sdk
export NEARAI_API_KEY="your-api-key"
```

## Send and verify a completion

Save this example as `chat.py` and run it with `python chat.py`:

```python chat.py
import asyncio
import os

from nearai_inference_sdk import InferenceClient


async def main():
async with InferenceClient(
os.environ["NEARAI_API_KEY"],
e2ee=True,
) as client:
completion = await client.chat.completions.create(
model="z-ai/glm-5.3-flash",
messages=[{"role": "user", "content": "Reply with the word ok."}],
)
# Verify the exact request and response bytes before using the answer.
verified = await client.verify_response(completion.id)
print(f"Verified {verified.signature_kind} response.")
print(completion.choices[0].message.content or "")


asyncio.run(main())
```

The client connects to `https://cloud-api.near.ai/v1` by default. Reuse a client for multiple requests; the async context manager closes its HTTP connections when finished.

## E2EE

End-to-end encryption is disabled by default. `e2ee=True` encrypts supported Chat fields to a verified model key and decrypts the response automatically.

In Python, model attestation requires catalog metadata with `providerType="vllm"` and `attestationSupported=True`. Other providers, including Chutes, use Incognito mode: Gateway verification only. E2EE and model deployment policies reject Incognito models before sending Chat. Use `e2ee=False` for Gateway-only Chat without a claim about model TEE execution.

## Verification

Before sending Chat, the client verifies the Gateway and, for supported NEAR TEE models, every returned model report. Gateway TLS identity verification is enabled by default and subsequent requests are pinned to that identity. Failed required checks stop the request.

Response-signature verification is explicit: call `await client.verify_response(completion.id)` after receiving the completion. It verifies the exact request and response bytes retained by the client against the relevant verified evidence.

- `provider_tee`: a verified model signer signed the exact request and response bytes.
- `gateway`: a verified Gateway signer signed those bytes; this does not prove that an attested model generated them.

Successful deployment checks are cached for 60 minutes by default. Set `attestation_cache_time_to_live_ms=0` to verify deployments for every request. Response records also expire after 60 minutes by default; verify each response before its record expires.

To verify a deployment before the first Chat request, call `await client.verify(model)`. This shares Chat's attestation cache but does not verify a future response signature. For application-owned deployment policies and standalone evidence verification, see the [Python verification guide](https://github.com/nearai/inference-sdk/blob/main/py/docs/verification-guide.md) and [Verification](/cloud/verification).

### Streaming

Consume the entire stream before calling `verify_response()`. This example buffers the text until verification succeeds; use it inside the client context above:

```python
stream = await client.chat.completions.create(
model="z-ai/glm-5.3-flash",
messages=[{"role": "user", "content": "Hello!"}],
stream=True,
)
completion_id = None
parts = []
async for chunk in stream:
if chunk.id:
completion_id = chunk.id
if chunk.choices:
parts.append(chunk.choices[0].delta.content or "")

if completion_id is None:
raise RuntimeError("Stream returned no completion ID")
await client.verify_response(completion_id)
print("".join(parts))
```

## Use the OpenAI SDK

For an existing `AsyncOpenAI` integration, use `InferenceClient.http_client` as its transport. Configure the API key on the inference client, which authenticates Chat, evidence, and signature requests.

```bash
pip install nearai-inference-sdk openai
```

```python chat_openai.py
import asyncio
import os

from openai import AsyncOpenAI
from nearai_inference_sdk import InferenceClient


async def main():
api_key = os.environ["NEARAI_API_KEY"]
async with InferenceClient(api_key, e2ee=True) as inference_client:
async with AsyncOpenAI(
api_key=api_key,
base_url="https://cloud-api.near.ai/v1/",
http_client=inference_client.http_client,
) as client:
completion = await client.chat.completions.create(
model="z-ai/glm-5.3-flash",
messages=[{"role": "user", "content": "Hello!"}],
)
await inference_client.verify_response(completion.id)
print(completion.choices[0].message.content or "")


asyncio.run(main())
```

Run it with `python chat_openai.py`. This transport supports Chat Completions, including streaming; it does not support the Responses API.

## OHTTP

Set `ohttp=True` when constructing the client to encapsulate the Chat HTTP exchange to the attested Gateway:

```python
async with InferenceClient(api_key, e2ee=True, ohttp=True) as client:
completion = await client.chat.completions.create(
model="z-ai/glm-5.3-flash",
messages=[{"role": "user", "content": "Hello!"}],
)
await client.verify_response(completion.id)
print(completion.choices[0].message.content or "")
```

OHTTP requires the default Ed25519 algorithm and cannot be used with `signing_algo="ecdsa"`. E2EE is configured independently. Missing or invalid Gateway OHTTP evidence blocks the Chat request.

<Note>
OHTTP applies only to Chat. Attestation and signature requests remain regular HTTPS. The endpoint must expose `/ohttp` at the same origin. Outer authorization headers and the client's network address are not hidden by OHTTP.
</Note>

## Examples and reference

- [Python verification guide](https://github.com/nearai/inference-sdk/blob/main/py/docs/verification-guide.md): policies, encryption, streaming, and errors.
- [Python API reference](https://github.com/nearai/inference-sdk/blob/main/py/docs/api-reference.md): client options, verification functions, and result fields.
- [Runnable Python examples](https://github.com/nearai/inference-sdk/tree/main/examples/example-py).
Loading
Loading