Add Sandbox API support - #67
Conversation
Adds isolated microVM sandboxes (create/exec/fs/list/catalog/lifecycle), ported from deepinfra-python's Sandbox API: a new unified DeepInfraClient (GET/POST/PUT/DELETE, retries, NDJSON streaming) backing a typed Sandbox class, plus a matching error hierarchy. Includes unit tests, a runnable example, and a new README section. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Missed these when fixing the same issue in README/example — the doc comment on Sandbox.exec() and the fs test fixtures still referenced the invalid /work path. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
| ): AsyncGenerator<NdjsonEvent> { | ||
| let buffer = ""; | ||
| for await (const chunk of stream) { | ||
| buffer += chunk.toString("utf8"); |
There was a problem hiding this comment.
chunk.toString('utf8') decodes every chunk on its own, so a multi-byte char split across two chunks comes out as replacement chars. reproduced by splitting {"stdout":"héllo"} inside the é, stdout is "h��llo". the prod unicode runs passed only because the chunk boundary never landed inside a char. use string_decoder: const decoder = new StringDecoder('utf8'), buffer += decoder.write(chunk) in the loop and buffer += decoder.end() after it.
| const response = await this._client.stream( | ||
| this.execSpec(command, options.timeout), | ||
| ); | ||
| return foldExecEvents(iterNdjson(response.data)); |
There was a problem hiding this comment.
stream() maps transport errors only up to the response headers, the body iteration here runs outside any mapping. a connection that stalls mid-stream does get aborted by the axios idle timeout, but the caller sees a bare Error('aborted') instead of APITimeoutError (reproduced with a local server that writes one line and hangs). wrap this in try/catch and map like mapTransportError does, 'aborted' / ECONNABORTED to APITimeoutError, everything else to APIConnectionError.
| timeout: Duration | undefined, | ||
| ): RequestSpec { | ||
| const timeoutSeconds = timeout === undefined ? 0 : parseDuration(timeout); | ||
| // Read timeout outlives the server-side command timeout so the server's |
There was a problem hiding this comment.
this reads as a total read timeout, but the axios timeout on a stream response is an idle timeout on the socket, it resets on every chunk. so it is not a cap on exec duration, a command that keeps printing stays alive until the server kills it. the value is fine, just say idle timeout in the comment so nobody relies on it as a hard limit.
| } | ||
|
|
||
| export interface CreateOptions extends WaitOptions { | ||
| image?: string; |
There was a problem hiding this comment.
I'm guessing you used the 0.2.0 python SDK version as an example for Claude, but you missed that in 0.3.0 we removed the user supplied image id :)
| headers, | ||
| params: spec.params, | ||
| data: spec.json === undefined ? spec.content : spec.json, | ||
| timeout: spec.timeout === undefined ? this.timeout : spec.timeout * 1000, |
There was a problem hiding this comment.
units are mixed in the same class: the constructor timeout is ms, RequestSpec.timeout is seconds, Duration is seconds. the python client is seconds everywhere. make the constructor seconds too (DEFAULT_TIMEOUT = 60) and convert once here.
| sandbox_id: string; | ||
| plan: string; | ||
| image: string; | ||
| state: string; |
There was a problem hiding this comment.
type this as 'running' | 'stopped' | 'failed' | 'deleted' | (string & {}) and reuse it for TERMINAL_STATES and the wait targets in sandbox.ts, a typo in a state string then fails to compile instead of waiting 300s for a state that never comes.
| } | ||
| } | ||
|
|
||
| async terminate(): Promise<void> { |
There was a problem hiding this comment.
consider adding Symbol.asyncDispose that calls terminate() and swallows NotFoundError, same as the python exit. gives ts 5.2+ users await using sb = await Sandbox.create(...) in place of the try/finally from the readme.
| // Large scripts: upload, then run | ||
| await sb.fs.write( | ||
| "/workspace/script.py", | ||
| await fs.promises.readFile("script.py", "utf8"), |
There was a problem hiding this comment.
fs is not imported in this snippet. please copy and paste every readme snippet into a file and run it, same as we did for the docs.
| const out = await sb.runPython("print(21 * 2)"); | ||
| out.check(); // throws CommandFailedError on a non-zero exit code | ||
|
|
||
| await sb.fs.write("/workspace/in.csv", "a,b\n1,2\n"); |
There was a problem hiding this comment.
port the paragraph from the python readme here: /workspace is the only path fs accepts and the only one that survives stop/start, everything else comes back from the base image so runtime pip installs are gone after a restart, and the idle timeout stops the sandbox with the same effect. that is exactly the /work bug you hit, worth stating.
| * and eventually the inference wrappers) funnels through this client. | ||
| */ | ||
| export class DeepInfraClient { | ||
| private apiKeyValue?: string; |
There was a problem hiding this comment.
this is an ordinary enumerable property at runtime, ts private is compile time only, and Sandbox holds the client as an enumerable _client field. so console.log(sandbox) prints the api key in full (util.inspect reaches _client.apiKeyValue at depth 2), same for console.log(client) and JSON.stringify(client). reproduced against a local server with a fake key. users log the sandbox object all the time and pino/winston/sentry object serializers capture it the same way, the python sdk is safe only because repr is explicit.
keep the key in a real #apiKey private field (or a closure), add util.inspect.custom { return this.toString(); } on both DeepInfraClient and Sandbox with a toString like DeepInfraClient(baseUrl=...), and consider Object.defineProperty for _client so it is non-enumerable. then a regression test that util.inspect(sb) and JSON.stringify(client) do not contain the key.
the error objects are fine, console.log(err), inspect with depth null, JSON.stringify(err) and err.response.config.headers do not contain the key.
|
please bump the runtime deps in this PR, it is a package.json only change and it is the difference between "dependencies": {
- "@swc/core": "^1.4.6",
- "@swc/wasm": "^1.4.6",
- "axios": "^1.6.7",
- "form-data": "^4.0.0"
+ "axios": "^1.20.0",
+ "form-data": "^4.0.6"
},@swc/core and @swc/wasm are imported nowhere, they only pull a native binary onto every consumer. I tried it on this branch: zero source changes, Also ran the sandbox flow against prod on axios 1.20.0, 15/15: catalog, create, list by tags, exec via the ndjson stream (560KB unicode output, server side timeout, rc 3 + check()), 1MB binary fs round-trip, 400/401/404/409 mapping including the stream error path, stop/start/terminate. pnpm-lock.yaml is also committed and would go stale, delete it or regenerate it in the same commit. The rest of the cleanup (node 20/22/24 in ci, eslint vs prettier, build.config.js) can stay a follow-up, it does not affect what ships to users. |
- Security: apiKeyValue is now a true #private field (was TS-private,
i.e. enumerable at runtime), and DeepInfraClient/Sandbox get
[util.inspect.custom]() so console.log/util.inspect/JSON.stringify
never expose the API key. Also fixes a related circular reference
(sandbox.fs.sandbox === sandbox) that broke JSON.stringify(sandbox)
unconditionally, found while adding the regression test for this.
- Correctness: decode NDJSON chunks with StringDecoder instead of
per-chunk toString('utf8'), so a multi-byte UTF-8 character split
across two chunks decodes correctly instead of producing replacement
characters; map stream-body errors (e.g. a stalled connection) to
typed APITimeoutError/APIConnectionError instead of leaking a raw
Error out of exec().
- API: DeepInfraClient's constructor timeout is now seconds everywhere
(was silently milliseconds, inconsistent with RequestSpec/Duration);
removed the create-time `image` parameter to match the current
upstream Sandbox API (server already ignored it); exec()'s variadic
signature is now properly overloaded so a misplaced options object no
longer type-checks; SandboxInfo.state is a typed SandboxState instead
of a bare string; added [Symbol.asyncDispose] for `await using`
parity with Python's __exit__.
- API surface: src/index.ts no longer re-exports internal helpers
(parseJsonBody, defaultClient, extractErrorMessage,
exceptionFromResponse) that Python keeps private — only the classes
Python's __all__ exposes are public now.
- Cleanup: removed the AuthenticationError constructor cast by giving
it the same (message, opts) shape as the other error classes.
- Deps: bumped axios to ^1.20.0 and form-data to ^4.0.6 (fixes all
`npm audit --omit=dev` findings, including a critical form-data
advisory), dropped @swc/core and @swc/wasm (unused, native-binary
weight for every consumer), removed the stale pnpm-lock.yaml (CI only
ever ran npm; a second lockfile just goes stale silently).
All changes verified against the real API (52+ scenarios across prior
rounds, plus targeted re-verification of every fix here and every
README Sandboxes snippet run verbatim) in addition to the unit suite.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
| @@ -0,0 +1,11 @@ | |||
| export { Sandbox } from "@/lib/sandbox/sandbox"; | |||
| export { SandboxFS } from "@/lib/sandbox/sandbox-fs"; | |||
| export type { SandboxInfo } from "@/lib/sandbox/sandbox-info"; | |||
There was a problem hiding this comment.
SandboxState is not exported. sandbox.state is typed as SandboxState in the shipped d.ts, but const s: SandboxState = sb.state in a consumer project fails with TS2305, no exported member. tried it against the packed tarball in a fresh project with strict tsc. add it here:
export type { SandboxInfo, SandboxState } from "@/lib/sandbox/sandbox-info";
It was used as Sandbox.state's return type in the shipped d.ts but never re-exported, so a consumer writing `const s: SandboxState = ...` hit TS2305. Verified by packing the tarball and type-checking against it from a fresh project with strict tsc, same repro ats3v used. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2 participants
Summary
deepinfra-python'sSandboxAPI and adapted to idiomatic TypeScript/Promises (no sync/async duality, snake_case DTOs matching the wire format, camelCase accessors elsewhere).DeepInfraClient(GET/POST/PUT/DELETE, retry/backoff, NDJSON exec streaming, binary-safe fs reads) and a typed error hierarchy (AuthenticationError,NotFoundError,ConflictError,RateLimitError/TooManySandboxesError,SandboxTimeoutError,CommandFailedError, etc.), all newly and publicly exported.DeepInfraClient→LegacyModelClient) to free up the name; it was never part of the public API (src/index.tsnever re-exported it), so this isn't a breaking change.examples/sandbox-quickstart.ts.Test plan
npm run build(tsc + tsc-alias) — passesnpx jest— 58/58 tests pass across 16 suites, no regressions to existing inference-wrapper testsprettier --check— clean/work/...paths (copied from Python's README) are rejected by the live API, which requires/workspace/...— fixed in this repo's README/example🤖 Generated with Claude Code