Repository navigation
Fall back to userspace WireGuard when the kernel lacks it [murmur:murmuration/vprox-oidc-claim-allowlists] - #5
Merged
Merged
Conversation
Modal VM sandboxes run kernels without the WireGuard module, so creating the client interface fails with EOPNOTSUPP before registration. On that error the client now runs wireguard-go on a TUN interface of the same name and serves the standard UAPI socket, so the existing wgctrl configuration, health checks, reconnects, and /disconnect work unchanged. Closing the device removes the interface and the socket. The health check also falls back to a raw ICMP socket when unprivileged ping sockets are forbidden by the kernel's default ping_group_range. CI runs the lib tests as root too, covering an end-to-end test that tunnels through a userspace client to an in-memory WireGuard gateway. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ApprovabilityVerdict: Approved at Macroscope's review found this PR approvable — This adds a per-client userspace WireGuard fallback for kernels without WireGuard support, plus coverage and documentation for that path. The change is bounded to local tunnel setup and does not introduce a shared production, billing, access, data durability, or external-action concern. You can add or adjust custom eligibility rules. Learn more. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Modal VM sandboxes (
runtime="vm", kernel 7.2.9) have no WireGuard kernel module. The published releasesha-9ad4489…therefore fails before registration witherror creating vprox interface: operation not supported. This PR adds a userspace WireGuard fallback to the client and keeps verified control-plane TLS, Modal OIDC authentication, reconnects and cleanup unchanged. The gateway still uses kernel WireGuard.Change
CreateInterface: tries kernel WireGuard first, as before. If netlink returnsEOPNOTSUPP(whatip link add … type wireguardreports asUnknown device type), the client logs that it is falling back. It then runs wireguard-go on a TUN interface with the same name and the default 1420 MTU./var/run/wireguard/<ifname>.sock.wgctrlconfiguration path (peer, endpoint, keepalive, AllowedIPs), netlink address/link-up handling, the/connectand/disconnectrequests, and the reconnect loop all work unchanged.wg showalso works with it.DeleteInterface: closes the userspace device. That removes the TUN interface and unlinks the UAPI socket. If the process is killed hard, the kernel destroys the non-persistent TUN anyway, and the next run'sUAPIOpenreplaces the stale socket.EACCES, it retries with a raw socket. The kernel's defaultnet.ipv4.ping_group_range(1 0) forbids unprivileged ICMP sockets, and minimal VMs without distribution sysctl defaults keep it. In that state every health check would fail and the client would loop on reconnect. I hit exactly this on the test VM. I haven't confirmed the value inside Modal VM sandboxes, but the fallback is a no-op where unprivileged ping already works.golang.zx2c4.com/wireguardmoves from an indirect to a direct dependency at the same version. The static binary grows by about 0.75 MB (20.9 MB → 21.7 MB). The existing release publisher builds amd64 and arm64 as before.Tests
TestUserspaceClient_TunnelsReconnectsAndCleansUp(root only) runs this sequence:CheckConnectionover the tunnel.wgctrlsees a userspace device with the gateway peer and a recent handshake./disconnect.CI: a new step in the
go test -racejob compiles thelibtests and runs them undersudo. Tests that need root skip themselves elsewhere.Regression check: before I added the raw-socket ping fallback, this test failed on the test VM (
ping_group_range=1 0) withsocket: permission denied. With the fallback it passes.Test evidence
Kernel without WireGuard, reproduced locally: I blocked and unloaded the
wireguardmodule on a Debian 6.1 VM, andip link add wgprobe type wireguardprintedError: Unknown device type., matching Modal. In that state,Client.CreateInterfaceloggedkernel WireGuard is unavailable; using userspace WireGuard for vproxfb0, andwgctrlreporteddevice type: userspace. Deleting it left no interface and no socket. With the module reloaded, the same call created aLinux kerneldevice. This was a one-off check, not a committed test.Local checks:
go test -race ./...passes, thelibsuite passes as root, gofmt and actionlint are clean, and the linux/arm64 build succeeds.Earlier live tests (from the request, not run against this PR): a standalone wireguard-go client in two fresh nonprod Modal VM sandboxes did all of the following:
MODAL_IDENTITY_TOKEN;104.197.117.12:50227, TLS namemurmur-vprox.nonprod.prasso.ai;This PR builds the same wireguard-go client into vprox itself. Those live handshakes also needed the gateway's UDP DNAT reply fix, which is tracked separately in prassoai/murmuration#12043.
Not done
sha-<merge commit>release to pin and test.Created via Murmur
🤖 Generated with Claude Code
Note
Fall back to userspace WireGuard (
wireguard-go) when kernel lacks itClient.CreateInterfacenow detectsEOPNOTSUPPfrom netlink and builds a userspace WireGuard device on a TUN interface instead, with a UAPI socket at the standard per-interface location.DeleteInterfacecloses the retained userspace device rather than calling netlink. See client.go and wg_userspace.go.CheckConnectionretries its three-packet ICMP health check with a privileged socket when the unprivileged attempt fails withEACCES, via a newClient.pinghelper./dev/net/tunand leaves a TUN interface plus UAPI socket in$RUNDIR/wireguard/<iface>.sockuntilDeleteInterfaceruns; the server side still uses kernel WireGuard.📊 Macroscope summarized 66f15c9. 4 files reviewed, 1 issue evaluated, 1 issue filtered, 0 comments posted
🗂️ Filtered Issues
lib/wg_userspace.go — 0 comments posted, 1 evaluated, 1 filtered
newUserspaceDevicepermanently exits its UAPI accept loop on anyAccepterror, but leaves the WireGuard device andClientrunning. In the pinnedipc.UAPIListen, deleting or replacing/var/run/wireguard/<ifname>.socktriggers an inotify error delivered byAccept; subsequentwgctrl.ConfigureDevicecalls inClient.Connectthen cannot access the device, so a live client cannot reconnect until it is restarted. Propagate the listener failure or recreate the socket rather than silently abandoning UAPI service. [ Failed validation ]🐞