Skip to content

Add userdata support for runner VMs - #77

Closed
relwell wants to merge 1 commit into
mainfrom
userdata-support
Closed

relwell wants to merge 1 commit into
mainfrom
userdata-support

Conversation

@relwell

@relwell relwell commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

What

Adds optional ORKA_VM_USERDATA / ORKA_VM_USERDATA_FILE_PATH settings (OK-6038, part of epic OK-6037): a bootstrap script executed as root inside each runner VM on its first boot, before the GitHub runner starts. One base image can then serve many runner pools - toolchain installs, MDM enrollment, and service registration no longer need to be baked into the VM image.

Note: This prepares the integration for Orka 3.7, which introduces VM userdata. The feature requires an Orka 3.7.0+ cluster and orka3 CLI; the integration fails fast at startup with a clear error if userdata is configured but the installed CLI predates the --userdata flag.

How

  • Config mirrors the existing GitHub App private key pattern: ORKA_VM_USERDATA holds the script content directly, or ORKA_VM_USERDATA_FILE_PATH points at a script file read at startup (content wins when both are set). The 64KiB encoded size cap is validated at startup for fail-fast behavior.
  • The script is written to a file readable only by the current user, and deploys pass the file path to orka3 vm deploy --userdata (the CLI reads and base64-encodes it) - so the script content never appears in command lines, and therefore never in the integration's error logs, which embed command args.
  • Userdata never delays or gates runner VM boot: it runs in the background and the runner connects on the normal timeline regardless of the script's outcome. Output is written to /var/log/orka/userdata.log inside the VM.
  • README and examples/.env document the new settings, semantics, and version requirements.

Testing

  • Unit tests (ginkgo, matching the existing suites): deploy args include the userdata file path (and never the script content), metadata and userdata compose, the userdata file is created 0600 with exact content, and env validation accepts/rejects scripts on both sides of the 64KiB encoded boundary.
  • go build ./..., go vet ./..., gofmt -l, and the full go test ./... suite are green.
  • Live-validated against a lab Orka cluster running the userdata stack: the integration's real deploy path (writeUserdataFile → deployArgs → orka3 vm deploy --userdata) provisioned a VM whose script ran on first boot - confirmed via the VM's userdataPhase=success, the marker file the script wrote in the guest, and the guest's userdata.log (finished with status "success" (exit code 0)).

🤖 Generated with Claude Code

Adds optional ORKA_VM_USERDATA / ORKA_VM_USERDATA_FILE_PATH settings: a
bootstrap script executed as root inside each runner VM on its first
boot, before the GitHub runner starts. This prepares the integration
for Orka 3.7, which introduces VM userdata - one base image can serve
many runner pools without baking bootstrap steps into the image.

- The script (from env content or a file, mirroring the GitHub App
  private key pattern) is validated against the 64KiB encoded size cap
  at startup and stored in a file readable only by the current user;
  deploys pass the file path to orka3 --userdata, so the script content
  never appears in command lines or error logs
- Startup fails fast with a clear error when the installed orka3 CLI
  predates the --userdata flag (requires orka3 3.7.0+)
- Userdata never delays or gates runner VM boot; output is written to
  /var/log/orka/userdata.log inside the VM

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@relwell
relwell requested a review from a team as a code owner September 1, 2026 18:22
@ispasov

ispasov commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

I wouldn't add user data just yet (not in this form)

I want to use userdata to replace the SSH configuration of runners. Rn the whole agent provisioning depends on the connectivity to the VMs. And people need to provide VM credentials in addition to that.
Ideally we use user data for it and get rid of ssh altogether.
However, if we are to do that, then the provisioning user data is going to "fight" with the one provided by users (with the changes in this PR).

So I would take it slowly here - test the SSH replacement first and then determine if/how we want to expose user data to end users.

WDYT?

@relwell relwell closed this Sep 30, 2026
@relwell

relwell commented Sep 30, 2026

Copy link
Copy Markdown
Contributor Author

Closing this one for now -- we can reference it in the future if necessary

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants