Skip to content

feat: add the OpenShell (Kubernetes) template - #1233

Open
bpmct wants to merge 1 commit into
mainfrom
kubernetes-openshell
Open

bpmct wants to merge 1 commit into
mainfrom
kubernetes-openshell

Conversation

@bpmct

@bpmct bpmct commented Nov 4, 2026

Copy link
Copy Markdown
Member

Description

Adds a coder-labs template that runs each workspace in an NVIDIA OpenShell sandbox on Kubernetes. It is the third sibling of agent-sandbox (Kubernetes) (#1230) and Plain Sandbox (Kubernetes) (#1232).

  • Firewall: the OpenShell proxy, by host name and program. Profiles: Fully isolated (default), GitHub only, GitHub and package registries, and Custom (host names and ports 443, 80, 22 typed in the form). There is no Public internet, because OpenShell 0.1.5 has no rule for any public host.
  • File limits: Landlock keeps /usr and /etc read-only and denies paths outside the policy.
  • Kernel isolation: the node's container runtime with Landlock and seccomp by default, the fastest. List kata-qemu or kata-clh in runtimes to offer a microVM. gVisor does not work, because it has no Landlock.
  • Launcher: Terraform cannot call the gateway, so a small launcher Pod (build/launcher/Dockerfile, Alpine and the openshell CLI, 35 MB) creates the sandbox on start and deletes it on stop. Coder owns the home disk, mounted at /sandbox/home.
  • Cluster setup: cluster/ installs the gateway with the Helm provider and creates the namespaces, the gateway credentials, the RuntimeClass labels, and the RBAC. Its coder_variables output is the --variables-file.
  • Shared with the siblings: build/Dockerfile, the scoped AI Gateway token, the code-server, git-clone, and Claude Code setup, the presets, and the README layout.
  • Icon: .icons/openshell.svg, the OpenShell mark from NVIDIA/OpenShell (Apache-2.0).

Startup on EKS, create to connected agent: about 2 s from a prebuild, 14 s cold, 17 s cold on Kata.

Type of Change

  • New module
  • New template
  • Bug fix
  • Feature/enhancement
  • Documentation
  • Other

Template Information

Path: registry/coder-labs/templates/kubernetes-openshell

Testing & Validation

  • Tests pass locally (terraform test: 24 template runs and 5 cluster/ runs, with mock providers). CI runs .tftest.hcl only for modules, so these do not run in CI.
  • Code formatted (prettier --check, terraform fmt), README validator and shellcheck pass
  • Changes tested locally (EKS, VPC CNI, OpenShell 0.1.5, Kata Cloud Hypervisor)
Live test results
Case Result
cluster/ apply (a second gateway release) 8 resources, gateway ready in 19 s
Claim from a prebuild build 1.6 s, agent ready 2.0 s
No prebuild (Custom) ready 14.4 s
Kata Cloud Hypervisor, no prebuild ready 16.8 s, VM kernel 6.18.35
Stop, then start stop 37 s, ready 24 s
Fully isolated GitHub blocked, Coder 200
GitHub and package registries GitHub, npm, PyPI 200; example.com blocked; prebuilt clone present
Custom (example.com, *.wikipedia.org, 443) example.com and en.wikipedia.org reachable; port 80, GitHub, npm, metadata blocked; wikipedia.org itself blocked
Landlock writing /usr and reading /etc/shadow denied
AI Gateway scoped token set, claude -p answers
Delete sandbox and home disk removed
Decisions
  • Two namespaces: the gateway in its own namespace, sandboxes and launchers in the workspace namespace. The provisioner gets Secrets access only in the workspace namespace, not next to the gateway's server key and JWT keys. cluster/ copies the CA (for supervisors) and the client bundle (for launchers) across.
  • Agent token from a Secret in the launcher Pod. OpenShell still puts it in plain text in the sandbox Pod and the Sandbox, which the README lists under Limits.
  • The launcher runs create in the background, so SIGTERM deletes the sandbox at once, even while a create waits for compute. The old prototype ran it in the foreground, and a stop or a delete during a Pending create left a suspended Sandbox behind.
  • restart_policy = "Always" on the launcher: a restarted launcher replaces the sandbox. A crash costs the running session but not the home, instead of a workspace whose agent never comes back.
  • cpu and memory are template variables (1 and 2Gi), because OpenShell sets them as both the request and the limit, and 2 CPU left new sandboxes Pending on a busy cluster.
  • Presets set runtime = "none" when runtimes is set, like feat(registry/coder-labs/templates): add the Plain Sandbox (Kubernetes) template #1232, so a claim never changes an immutable parameter.
Review

A Claude (Fable) review found no blockers. Fixed: the Helm provider constraint (>= 3.0 for the attribute syntax), the TERM trap during a create, the openshell policy update example (prettier's sh plugin had mangled it), the Landlock and wildcard wording, the agent token note, a removed redundant curl rule, and more tests. Not changed: polling for a sandbox that the gateway removes, because a gateway blip would then restart a healthy sandbox.

Related Issues

None


🤖 This PR was generated by Coder Agents on behalf of @bpmct.

@bpmct
bpmct marked this pull request as ready for review November 4, 2026 19:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant