feat: add the OpenShell (Kubernetes) template

registry/coder-labs/templates/kubernetes-openshell/README.md

@@ -0,0 +1,148 @@
+---
+display_name: OpenShell (Kubernetes)
+description: Run each workspace in an NVIDIA OpenShell sandbox, with a firewall by host name and program
+icon: ../../../../.icons/openshell.svg
+verified: false
+tags: [kubernetes, openshell, sandbox, firewall, kata, ai]
+---
+
+# OpenShell (Kubernetes)
+
+Run each workspace in an [NVIDIA OpenShell](https://docs.nvidia.com/openshell/latest/about/overview) sandbox. You and your AI agent work in the same sandbox, with one Coder agent named `dev`. OpenShell blocks all network access by default. Its proxy checks each connection against a policy that names the host and the program that opens it. It also limits file access with Landlock. By default the workspace can reach only the Coder server.
+
+## Why OpenShell
+
+- **Rules for each program:** a rule can let `git` reach `github.com` and still block `python3` from the same host. OpenShell logs each denied connection with the program name.
+- **Rules you can change live:** `openshell policy update` opens a host for a running sandbox, with no restart.
+- **File limits:** Landlock makes `/usr` and `/etc` read-only and denies every path that is not in the policy, such as `/home` and `/root`, even to code that runs as the workspace user.
+- **Kernel isolation:** with Kata, each sandbox is a microVM with its own kernel.
+
+If host names are enough, use [Plain Sandbox (Kubernetes)](https://registry.coder.com/templates/coder-labs/kubernetes-sandboxed). It has fewer parts and faster stops. OpenShell is at version 0.1, so pin the version and test each upgrade.
+
+## Startup speed
+
+With a prebuild, workspaces often start in about 2 seconds, which matters for agents that start a new workspace for each task. To see where the time goes, see [Speed tuning](#speed-tuning).
+
+## Prerequisites
+
+Required:
+
+- **A Kubernetes cluster** with a network plugin that enforces NetworkPolicy, a StorageClass for the home disk, and Linux 6.2 or later on the nodes, for Landlock.
+- **[agent-sandbox](https://github.com/kubernetes-sigs/agent-sandbox)** v1.0.4 or later, which the OpenShell Kubernetes driver uses.
+- **The cluster setup** in [`cluster/`](https://github.com/coder/registry/blob/main/registry/coder-labs/templates/kubernetes-openshell/cluster): a Terraform configuration that installs the OpenShell gateway and creates the namespace, the RBAC, and the gateway credentials. See [`cluster/README.md`](https://github.com/coder/registry/blob/main/registry/coder-labs/templates/kubernetes-openshell/cluster/README.md).
+- **The workspace image**, built from [`build/Dockerfile`](https://github.com/coder/registry/blob/main/registry/coder-labs/templates/kubernetes-openshell/build/Dockerfile).
+- **The launcher image**, built from [`build/launcher/Dockerfile`](https://github.com/coder/registry/blob/main/registry/coder-labs/templates/kubernetes-openshell/build/launcher/Dockerfile). It has only the `openshell` CLI.
+- **Access to the cluster** for the Coder provisioner. The template uses the in-cluster ServiceAccount, or `~/.kube/config` with `use_kubeconfig = true`. With the AI Gateway on, the provisioner also calls the Coder API at the access URL to create the workspace's token.
+
+Optional:
+
+- **Kata nodes**, for a microVM for each workspace: a node group with `/dev/kvm`. List them in `runtimes`. gVisor does not work, because it has no Landlock.
+- **Coder Premium**, for prebuilds and the AI Gateway. Without it, set `prebuild_instances = 0` and `enable_ai_gateway = false`, and give Claude Code another way to sign in.
+- **A DaemonSet that pulls the image** on every node.
+
+## Quick start
+
+1. Build both images and push them to your registry. See [Build the images](#build-the-images).
+2. Install agent-sandbox, and set up the cluster with the Terraform configuration in [`cluster/`](https://github.com/coder/registry/blob/main/registry/coder-labs/templates/kubernetes-openshell/cluster).
+3. Write the variables that `cluster/` prints to a file outside the template directory, add the images, and push the template:
+
+   ```sh
+   cd cluster
+   terraform output -raw coder_variables > ~/kubernetes-openshell.yaml
+   cd ..
+   cat >> ~/kubernetes-openshell.yaml << 'EOF'
+   workspace_image: registry.example.com/coder/workspace:v2.30.0
+   launcher_image: registry.example.com/coder/openshell-launcher:0.1.5
+   EOF
+   coder templates push kubernetes-openshell -d . --variables-file ~/kubernetes-openshell.yaml
+   ```
+
+Users pick the network profile and, if you set `runtimes`, the runtime when they create a workspace. They cannot change them later. Each profile except Custom is a preset with `prebuild_instances` prebuilds on the node's container runtime.
+
+## Build the images
+
+```sh
+docker buildx build --platform linux/amd64 \
+  --build-arg CODER_VERSION=2.30.0 \
+  -t registry.example.com/coder/workspace:v2.30.0 --push build
+docker buildx build --platform linux/amd64 \
+  --build-arg OPENSHELL_VERSION=0.1.5 \
+  -t registry.example.com/coder/openshell-launcher:0.1.5 --push build/launcher
+```
+
+- Match `CODER_VERSION` to your Coder server, and `OPENSHELL_VERSION` to your gateway. Rebuild when you upgrade either. `CODER_BINARY_URL`, such as `https://coder.example.com/bin/coder-linux-amd64`, copies the binary from your server instead of GitHub.
+- `INSTALL_CODE_SERVER=false` and `INSTALL_CLAUDE_CODE=false` leave out the tools. With `INSTALL_CLAUDE_CODE=false`, set `enable_claude_code = false` in the template.
+- Use a fixed tag or a digest. The template rejects `:latest`, because with `imagePullPolicy: IfNotPresent` the nodes keep an old copy.
+
+## Architecture
+
+Terraform cannot call the OpenShell gateway, so a small launcher Pod does it:
+
+1. **Start:** Terraform creates the home disk, a Secret with the agent token, and the launcher Pod. The launcher runs `openshell sandbox create` with the profile's policy, and the Coder agent is the main process of the sandbox.
+2. **Sandbox:** the gateway creates an agent-sandbox Sandbox, a supervisor Pod, and NetworkPolicies. The sandbox can talk only to its supervisor, so traffic cannot go around the proxy.
+3. **Stop:** Kubernetes stops the launcher, and the launcher deletes the sandbox. The home disk stays.
+
+- **Home:** Coder owns the disk, which is mounted at `/sandbox/home`. Files there survive a stop.
+- **Pods:** the sandbox runs as user 1000. OpenShell sets `cpu` and `memory` (1 CPU and 2 GiB by default) as both the request and the limit.
+- **Apps:** code-server, and Claude Code, which continues the last conversation in `/sandbox/home`.
+
+## Speed tuning
+
+In our Amazon EKS test cluster (`m7i.2xlarge` nodes, VPC CNI), from create to a connected agent:
+
+| Workspace                            | Ready in   |
+| ------------------------------------ | ---------- |
+| From a prebuild                      | about 2 s  |
+| No prebuild, Custom                  | about 14 s |
+| No prebuild, Kata (Cloud Hypervisor) | about 17 s |
+| Start after stop                     | about 24 s |
+| Stop                                 | about 37 s |
+
+These all add to the speed:
+
+1. **Prebuilds** (Premium). A claim changes no Pod, so the sandbox keeps running, and the build takes about 1.5 s. GitHub profiles have already cloned `git_repo_url`.
+2. **The image.** [`build/Dockerfile`](https://github.com/coder/registry/blob/main/registry/coder-labs/templates/kubernetes-openshell/build/Dockerfile) adds the Coder agent, code-server, and Claude Code, so a start downloads nothing, and it also works in Fully isolated. The image is about 800 MB compressed, so pull it ahead of time with a DaemonSet.
+3. **The disks and the supervisor.** A cold start waits for the home disk to attach and for the supervisor Pod. Stop waits until OpenShell deletes the sandbox.
+4. **Free CPU.** OpenShell reserves the full `cpu` for each sandbox, so a full cluster leaves new sandboxes Pending. Lower `cpu`, or add nodes.
+
+## Firewall
+
+The OpenShell proxy is the firewall. It runs in the supervisor, outside the workspace, so code in the workspace cannot change it. Each rule names a host, a port, and the programs that can use them. Everything else is denied and logged.
+
+- **Coder:** the Coder agent and Claude Code can reach the access URL, and so can every program they start, such as the shells in your terminal.
+- **Profile hosts:** every program can reach them.
+
+To open a host for a running workspace only:
+
+```sh
+openshell policy update "$SANDBOX" --rule-name extra \
+  --binary /usr/bin/curl --add-endpoint example.com:443 --wait
+```
+
+`$SANDBOX` is the sandbox name: `coder-` and the first 13 characters of the workspace ID without dashes. The change is gone after the next stop. To see blocked requests, run `openshell logs "$SANDBOX" --since 5m --source sandbox`.
+
+### Network ingress
+
+The sandbox accepts connections only from its supervisor. The Coder agent dials out to the Coder server, so the web terminal, code-server, `coder ssh`, port forwarding, and desktop IDEs all relay through Coder.
+
+### Network egress
+
+| Profile                       | Can reach, in addition to Coder                                     |
+| ----------------------------- | ------------------------------------------------------------------- |
+| Fully isolated (default)      | Nothing                                                             |
+| GitHub only                   | GitHub host names on 443                                            |
+| GitHub and package registries | GitHub, npm, the Go module proxy, and PyPI on 443                   |
+| Custom                        | The host names and ports that you type. No preset, so a cold start. |
+
+There is no Public internet profile, because OpenShell 0.1.5 has no rule for any public host. With Custom, the form shows a list of host names and a choice of ports 443, 80, and 22. `*.example.com` matches one label, such as `api.example.com`, and `**.example.com` matches any depth. Neither matches `example.com` itself, so add it too if you need it. The form rejects IP addresses, names without a dot, `*.com`, and cluster names such as `.svc` or `.local`, and the build checks them again.
+
+## Limits
+
+- OpenShell decrypts HTTPS by default. It adds its own CA through `SSL_CERT_FILE` and similar variables, so it can read the traffic it allows.
+- The launcher uses the gateway's client certificate, which can manage every sandbox. Anyone who can read Secrets in the namespace can use it. Use OIDC in production.
+- OpenShell puts the agent token in plain text in the sandbox Pod and its Sandbox, so keep read access to the namespace tight.
+- If the launcher container exits, Kubernetes restarts it, and it replaces the sandbox. The home stays, but running programs stop.
+- Without Kata, the sandbox shares the node kernel, with Landlock and seccomp only.
+- An allowed host that stores user content, such as GitHub, can receive data from the workspace.
+- With the AI Gateway on, each start gives the workspace a new Coder token with only the `user:read_personal` scope, which the AI Gateway accepts. Code in the workspace can use it for AI requests. Coder cannot revoke it on stop, so it stays valid until the default token lifetime ends, and it shows in the owner's token list as `ai-gateway-<workspace>-<time>`.
+- No root, no Docker, IPv4 only.

registry/coder-labs/templates/kubernetes-openshell/build/Dockerfile

@@ -0,0 +1,47 @@
+FROM codercom/enterprise-base:ubuntu
+
+ARG TARGETARCH=amd64
+ARG CODER_VERSION=
+ARG CODER_BINARY_URL=
+ARG INSTALL_CODE_SERVER=true
+ARG CODE_SERVER_VERSION=4.105.1
+ARG INSTALL_CLAUDE_CODE=true
+
+USER root
+
+# The Coder agent.
+RUN set -eu; \
+	if [ -n "${CODER_BINARY_URL}" ]; then \
+		curl -fsSL --retry 5 -o /usr/local/bin/coder "${CODER_BINARY_URL}"; \
+	elif [ -n "${CODER_VERSION}" ]; then \
+		v="${CODER_VERSION#v}"; \
+		curl -fsSL --retry 5 "https://github.com/coder/coder/releases/download/v${v}/coder_${v}_linux_${TARGETARCH}.tar.gz" \
+			| tar -xz -C /usr/local/bin ./coder; \
+	else \
+		echo "Set CODER_VERSION or CODER_BINARY_URL" >&2; exit 1; \
+	fi; \
+	chmod 755 /usr/local/bin/coder; \
+	/usr/local/bin/coder version
+
+# code-server, outside /home so a home volume does not hide it. The
+# code-server module uses it with offline = true.
+RUN set -eu; \
+	if [ "${INSTALL_CODE_SERVER}" = true ]; then \
+		curl -fsSL https://code-server.dev/install.sh \
+			| sh -s -- --method=standalone --prefix=/opt/code-server --version "${CODE_SERVER_VERSION}"; \
+	fi
+
+# Claude Code, outside /home for the same reason. The template runs
+# /opt/claude/bin/claude.
+RUN set -eu; \
+	if [ "${INSTALL_CLAUDE_CODE}" = true ]; then \
+		export HOME=/opt/claude; \
+		mkdir -p /opt/claude/bin; \
+		curl -fsSL https://claude.ai/install.sh | bash; \
+		ln -sf "$(readlink -f /opt/claude/.local/bin/claude)" /opt/claude/bin/claude; \
+		chmod -R a+rX /opt/claude; \
+		/opt/claude/bin/claude --version; \
+	fi
+
+USER coder
+WORKDIR /home/coder

registry/coder-labs/templates/kubernetes-openshell/build/launcher/Dockerfile

@@ -0,0 +1,14 @@
+FROM alpine:3.23@sha256:85fe1e81d6758c208f3e1eed4338a1997e19d4be002d4dd32d3100c9a8c010a0
+
+ARG TARGETARCH=amd64
+ARG OPENSHELL_VERSION=0.1.5
+
+# The openshell CLI. Use the version of your gateway.
+RUN set -eu; \
+	case "${TARGETARCH}" in amd64) arch=x86_64 ;; arm64) arch=aarch64 ;; *) echo "unsupported arch ${TARGETARCH}" >&2; exit 1 ;; esac; \
+	wget -qO- "https://github.com/NVIDIA/OpenShell/releases/download/v${OPENSHELL_VERSION}/openshell-${arch}-unknown-linux-musl.tar.gz" \
+		| tar -xz -C /usr/local/bin openshell; \
+	chmod 755 /usr/local/bin/openshell; \
+	/usr/local/bin/openshell --version
+
+USER 1000:1000

registry/coder-labs/templates/kubernetes-openshell/cluster/README.md

@@ -0,0 +1,41 @@
+# Cluster setup for kubernetes-openshell
+
+A cluster administrator sets up the cluster once. The Terraform configuration in this directory creates everything the template needs in the cluster:
+
+- The OpenShell gateway, from its Helm chart, in its own namespace.
+- The workspace namespace, where the gateway creates the sandboxes.
+- The gateway CA for the supervisor Pods, and the gateway client bundle for the launcher Pods, in the workspace namespace.
+- A Role and RoleBinding for the Coder provisioner. It can manage Pods, disks, and Secrets in the workspace namespace only.
+- The `openshell.ai/sandbox-attachable=true` label on the Kata RuntimeClasses in `runtimes`, so that OpenShell admits them.
+
+## 1. Install agent-sandbox
+
+The OpenShell Kubernetes driver creates agent-sandbox Sandboxes. Install the controller and the extensions, v1.0.4 or later:
+
+```sh
+kubectl apply -f https://github.com/kubernetes-sigs/agent-sandbox/releases/download/v1.0.4/sandbox-with-extensions.yaml
+```
+
+For Kata, see [Install the runtimes](https://github.com/coder/registry/blob/main/registry/coder-labs/templates/kubernetes-agent-sandbox/cluster/README.md#2-install-the-runtimes) in the agent-sandbox template.
+
+## 2. Apply
+
+Run Terraform from this directory with an account that can create namespaces, RBAC, and Helm releases:
+
+```sh
+terraform init
+terraform apply -var namespace=coder-workspaces
+```
+
+Set `runtimes`, for example `["kata-clh"]`, to offer Kata. Use the same value in the template. If the Coder provisioner does not run as the `coder` ServiceAccount in the `coder` namespace, set `provisioner_service_account`. `openshell_version` pins the Helm chart. Use the same version for the CLI in `build/launcher/Dockerfile`.
+
+Keep the Terraform state. Terraform needs it to change or remove the setup later.
+
+## 3. Push the template with the same values
+
+The `coder_variables` output has the namespace, the gateway URL, and the runtimes as a variables file for `coder templates push`. See the Quick start in the [template README](https://github.com/coder/registry/blob/main/registry/coder-labs/templates/kubernetes-openshell/README.md#quick-start).
+
+## Changes and removal
+
+- `terraform destroy` removes the gateway, both namespaces, and every workspace in them. It also removes the label from the RuntimeClasses in `runtimes`.
+- Removing a runtime from `runtimes` removes its label, so OpenShell no longer admits it. Remove it from the template first.