Repository navigation
Description
Goal
Make self-hosted K3s a practical execution backend for T3 Code, using T3's existing remote-environment and delegation architecture. One persistent T3 control environment should be able to launch, monitor, resume, cancel, and clean up isolated agent environments on a local K3s cluster, with web/desktop/mobile clients staying usable.
Scope / updated authorization (October 28, 2026): User explicitly authorized exactly one upstream design issue, pingdotgg/t3code#26535, which maintainers converted to an Ideas discussion on 2026-10-28; the original issue remains a read-only provenance link. All continuing engineering and coordination remain downstream in kvnloo/t3code; no additional upstream issues, PRs, comments, default-branch writes or merges without a new human authorization. Preserve contributor credit. This is a capability proposal, not a claim of native K3s support.
Problem / distinctions
There are two independent questions to validate:
- Host T3 on K3s: headless server, stable public/private endpoint, persistence, pairing/auth, upgrades and restart recovery.
- Use K3s for worker execution: provision a separate sandbox/T3 environment per approved task, route a remote agent invocation to it, retain task state, and safely reclaim resources.
Packaging a container does not automatically provide scheduling; T3 remote delegation does not automatically create or manage pods.
Relevant upstream work (recheck exact heads/status before coding)
- Remote delegation PR #16719 merged into feature branch
t3code/peer/links-uion Oct 27, not proof of availability onmainor in a stable release. Use only after checking integration. - Remote launch recovery #17051 documents a record-before-launch durability gap; PR #20937 closed unmerged on Oct 27.
- External sandbox broker #14793 and managed workers #14620 describe nearby approaches. Their issue dispositions do not establish that provisioning shipped.
- Docker/Kubernetes-oriented container PRs #5287 and #5284 closed unmerged; avoid repeating broad auth/UI changes.
- Pairing/public URL #13582 includes a Kubernetes ingress/pod-IP use case.
- Container host identity #11849 remains relevant; avoid per-restart identity drift. Nested rootless kind/Podman issue #21730 is related but separate; PR #21700 closed unmerged as of Oct 28.
- Existing factory kind → K3s CPU Job pilot is about verified-work jobs, not T3 remote-environment lifecycle. Reuse evidence/fixtures if useful; do not create a second scheduler.
Proposed minimal architecture
Web / Desktop / Mobile
|
persistent T3 owner (threads, approvals, durable orchestration)
|
optional external K3s environment broker (idempotent request/lease)
|
K3s API --> per-task isolated T3 server + workspace/PVC + agent runtime
|
connected environment / scoped remote delegation
|
result, logs, status, artifacts --> owning T3 thread
- Owner authority stays singular. No distributed SQLite, shared writable checkout, or cross-pod takeover of an owning thread.
- External adapter first. Implement a narrow K3s broker/sidecar and declarative manifests before changing core T3 contracts/UI. Avoid a second scheduler and unnecessary hosted-cloud dependencies.
- Task identity is durable. Stable idempotency key, exact repo/commit/branch, pod/environment ID, TTL, cancellation/restart states and recovered receipts. Never treat pod completion alone as an accepted agent result.
- Security by default. Namespace/service-account/RBAC least privilege; pod security, network policies and CPU/RAM/GPU limits; scoped Git/provider credentials; no public ingress by default; explicit pairing/approval. Do not log secrets.
- Data lifecycle. Separate owner state from per-task workspace/PVC. Define stop/resume/archive/delete explicitly. Cancellation and idle cleanup must not delete user work without consent.
- Performance. No extra work when disabled, bounded polling/WS updates, report cold-start/provisioning latency and resource overhead. Local-only K3s must work without cloud GPUs.
Acceptance / staged proof
- Inventory: check current upstream
mainand relevant feature branches; document existing remote/server, container, pairing and authentication capabilities, blockers and non-goals with links. - P0 — hosting: on a real single-node local K3s cluster, deploy a persistent headless T3 server with a stable reachable endpoint and storage; pair a client; restart/upgrade the pod and verify thread/state survives. Provide manifests and rollback instructions. Record exact image SHA/tag, K3s version and host.
- P1 — one worker: from the owning T3 environment, submit an approved test task against a pinned repo revision; provision exactly one isolated K3s worker; connect/execute; deliver its actual result to the owner; show status from at least web or mobile.
- Failure tests: duplicate request, lost launch response, pod eviction/restart, owner restart, unreachable worker, cancel, expired lease and credential revocation each have defined recovery and no duplicate accepted completion.
- Isolation: demonstrate resource caps, scoped auth/network access, no secret leakage in logs, and no shared mutable owner DB/workspace.
- Evidence: capture exact-head checks, focused tests, runtime transcripts, observed provisioning/cold-start and cleanup times, remaining unsupported modes and unresolved upstream dependencies.
Execution / return
- Start with read-only architecture and overlap audit; classify any dependency blocked by unmerged/rebased upstream work.
- Make the smallest independently testable fork-only implementation slice, ideally out-of-tree K3s adapter/manifests before core UI changes.
- Report tests and artifacts on this issue, with
PASS / FAIL / NOT RUN, tested commit and exact K3s environment. Keep any eventual fork PR separate and in draft; do not merge or post upstream. - Add only proven, qualified implementation to the existing T3Code promotion queue #42. A READY classification is not upstream authorization.
Expansion: PAIR-informed inference and private multi-host mesh (2026-10-28)
Human objective: expand this program with prior NVIDIA Personal AI Router (PAIR), heterogeneous/local GPU affinity and eventually scale a bounded T3/K3s worker fabric across Tailscale or other authenticated private VPNs. This is deferred roadmap scope; it does not relax the P0/P1 proof gates or authorize new upstream writes, live deployment, host changes or competing task schedulers. Keep Draft PR #46 strictly P0.
Ownership boundaries and minimal interfaces
T3 owner (task state, approvals, delegation, result verification)
|
optional K3s environment broker (idempotent provision/lease/delete)
|
private address + attested T3 environment identity + user-approved /mcp peer link
|
T3 worker environments on approved hosts or independent K3s clusters
| (optional; inference only)
PAIR-compatible inference gateway -> eligible model-serving host
|
verified owner result + bounded telemetry -> existing evaluation/policy systems
- T3 remains the only agent/task authority; T3's saved client connection, peer-link OAuth grant and runtime provisioning lease are three distinct identities/authorizations.
- K3s/Kubernetes schedules pods; do not stretch one K3s control plane across intermittent WAN hosts by default. Begin with separate clusters and expose only approved workloads/endpoints across the private network. No Karmada/Nomad/new scheduler in the baseline.
- PAIR routes independent inference requests to one compatible serving node (model inventory, available GPU/RAM, workload, observed affinity); it is not a K3s worker scheduler and does not pool 12 GB + 8 GB into 20 GB or shard a live model across hosts. It must be optional: preserve the existing provider endpoint when disabled.
- Network is a replaceable transport: a private HTTPS endpoint, stable T3 environment ID, verified peer identity, approval and authorization boundaries. Tailscale Kubernetes Operator ingress/egress is the first supported candidate. Later independently assess Headscale + Tailscale clients, NetBird, and configured WireGuard/other private VPN routes; do not assume Tailscale Operator OAuth/Ingress automatically works with those replacements. Manual DNS/endpoint discovery is acceptable; multicast is not assumed. No public Funnel or unrestricted cluster API.
- Isolation and learning remain separate: no provider secrets in model/worker telemetry; the overlay network alone does not authorize a T3 peer or PAIR node; no automatic scaling/placement changes without deterministic budget/security limits and human opt-in.
P2 — multi-host/private-network expansion (after verified P0 and P1)
- Test two explicitly approved independent environments over LAN then tailnet, then two independent K3s clusters via Tailscale Operator cross-cluster Service ingress/egress (stable private FQDN, HTTPS, no exposed kube API). Record packet route (direct vs relay), DNS behavior, RTT, throughput, handshake/reconnect and negative ACL proof. Do not mistake connected for authorized.
- Define a minimal VPN-neutral
PrivateEnvironmentEndpointrecord:environmentId, authenticatedhttpsOrigin, identity verification, transport label, tested reachability status, last probe and lease reference. Keep all credentials out of record, Git and logs. - Demonstrate remote owner->worker link/delegation with parent/worker restarts, duplicate request, route churn, approval revocation, offline/disconnected cluster, cancellation, exact result/receipt identity and no duplicate accepted completion. No automatic enrollment of transient workers in client auto-balance.
- Compare private options only on real supported setups (Tailscale first; Headscale/NetBird/WireGuard after matching gateway/identity capabilities are verified). Leave unsupported transport
NOT RUN, not implicitly compatible.
P3 — PAIR-informed inference gateway (separate from task placement)
- Inventory
kvnloo/Personal-AI-Routerand the official NVIDIA/Personal-AI-Router, read PAIR's current runtime/engine and actual gateway/protocol capabilities, and find a compatible T3-controlled harness endpoint (e.g. explicitly configured OpenAI-compatible model gateway if supported). Never assume all T3 harnesses accept a PAIR endpoint. - Explicitly account for PAIR's existing cross-VPN discovery limitations: #85, #91, #92; PAIR uses mDNS on LAN and must have a verified unicast/manual-node or fixed implementation for VPN-only peers. Origin PR #10 proposes address-based overlay support but was still open in this review. No claim it shipped.
- Reuse the already researched node/engine/model eligibility, warm-model affinity, GPU class/VRAM headroom, cold-start penalty, current queue and tailnet RTT; route each independent request to a single eligible node, never pool VRAM or interrupt/reschedule a streaming response.
- Verify which backend actually served a request. Previous downstream PAIR PR #2 proposes authoritative
X-PAIR-Served-Byafter failover but is closed/unmerged; do not rely on this header without confirming upstream behavior. Collect verified scheduler/engine receipts or mark attribution unproven. - Compare explicit local-only baseline against two-node eligible routing using p50/p95 TTFT, tokens/sec, p95 tail latency, model load/cold start, tokens/J or watt-hours when available, GPU memory/power, request failures, and network/relay overhead. Record tested model, engine, node, network and exact versions. Compare aggregate throughput and utilization, not claim faster single-request inference.
P3b — optional KAI GPU fractioning (gated, not part of P0/P1)
Decision: HOLD / read-only compatibility research until a measured need exists. GPU fractioning is a node-level enforcement adapter for KAI-assigned, same-physical-GPU fractions, not a T3 placement system, PAIR model router, cross-host memory pool, or new scheduler. Keep existing CPU-only P0 and single-worker P1 manifests unchanged; do not install the operator, touch the NVIDIA driver/container runtime, evict active GPU tenants, or create GPU pods on a working host without a separate explicit owner-approved rollback/maintenance plan.
- Ownership and contract: T3 owns the user-approved task/peer link and verified result; z0intelligence validates semantic eligibility and policies; Kerdoios may offer an opt-in bounded capacity ranking; Kubernetes/Kueue owns resource admission; KAI Scheduler chooses the GPU and fractional memory allocation;
gpu-fractioningmanages node NRI/fractiondand MPS/mpsdmemory enforcement plusmetricsdper-pod observation; PAIR routes independent inference calls only. Existingz0.capacity.snapshot.v1z0 #131, #133, #134 is the optional read-only capacity bridge;unknownis not zero and a capacity offer is not runtime authorization. - Compatibility gates (verify live, do not assume): current source at
dbf2d4erequires Kubernetes >=1.28; containerd >=2.0 with NRI enabled (or supported CRI-O); NVIDIA GPU Operator >=26.7.0, toolkit >=1.20.1, device plugin >=0.20.1, and NVIDIA driver r615+ / CUDA 13.4. GPU Operator's NRI mode changes RuntimeClass and requires management CDI namespace allowlisting. Check actual K3s-bundled containerd/CRI/NRI, driver, toolkit, device-plugin versions and GPU support on each node before even drafting an install command. The working RTX 3080 Ti 12 GB and RTX 2080 Super 8 GB do not form one 20 GB VRAM pool, and do not have MIG; a fractional share is within an individual compatible physical GPU. - Separate two generations: existing GPU memory fractions with
time-slicingorsm-sharingare distinct from planned guaranteed compute fraction requests/ceilings. The latter is KAI #2191, #2192, gpu-fractioning #157 and #163, Alpha / driver r620+ / CUDA 13.5, not assumed production-ready or usable on r615 alone. - Known live upstream risks to review before any real fractioning canary: #165 documents stale container/metrics mapping after reconnect; PR #166 has a focused unmerged patch. #167 documents
GpuFractioningConfigHelm hook upgrade/uninstall lifecycle pitfalls. Treat "pod Ready" and nonzero metric output as insufficient proof of actual memory enforcement. Verify driver/MPS failures, hard memory cap/OOM, container restart, device placement, metrics cleanup, driver upgrade/drain and uninstall/rollback on an isolated disposable GPU test host before promotion. - Experiment path: (A) read-only version/physical GPU probe and frozen baseline; (B) ordinary exclusive whole-GPU K3s Job on one eligible node, with normal provider throughput/performance and no user-impact regression; (C) on a separate approved isolated GPU test node, two fixed, safe, low-memory test pods on the same GPU with KAI-issued fraction annotations, driver/NRI limits, instrumented OOM negative control and independent metrics; (D) only if worthwhile, compare whole-GPU vs time-slicing vs sm-sharing on verified useful work per GPU-second, prefill/decode p50/p95, model memory, watts, crash/error rate, and tailnet cost. Guard all writes and changes behind human approval. No new CRD or second cluster scheduler.
- Attention gate: actively research prereqs and upstream bug resolution now, but no GPU fractioning PR or live deployment as part of this T3 lane until P0 live storage/route acceptance, P1 one isolated worker, and exclusive GPU baseline each pass. Keep source-specific issues/PRs in their existing repos; consolidate cross-project status in this issue and keep the upstream T3 Ideas discussion high-level.
P4 — optional verified adaptive placement (not in T3 core)
- Reuse the existing Evolution Lab -> z0evals -> z0intelligence -> Kerdoios -> Kueue/KAI/KEDA governance from prior ABAB research. Learn only bounded legal actions (route, warm/cold policy, queue/replicas/placement hints) after deterministic eligibility/security and quota checks. Preserve Kueue admission, KAI GPU placement and KEDA elasticity; never add a second privileged learning scheduler to T3.
- Require offline replay -> shadow -> limited canary -> separately authorized promotion, exact experimental lineage, independent acceptance and fail-safe rollback. Do not train weights or automate new infrastructure during the P0 pilot.
References — consolidated evidence and ownership index (2026-10-28)
This section is the single cross-project reference index for this T3/K3s proposal. The linked upstream idea is for architecture discussion, not a second execution queue. Keep detailed implementation/measurements in their original owning repositories. Source links denote the current published design or reported defect, not confirmation that the related cluster has been tested.
1. T3 execution and discussion
- T3 Code K3s work order, this issue #45 — canonical intent, gates, and dependency/attention ledger; Draft PR #46 — CPU-only, persistent T3 P0 deployment and restart-identity acceptance, LIVE K3s / Tailscale NOT RUN (last inspected head
8da283b04954a02b57c1b0234e27032731fa7fb2; re-read before next worker action). - T3 upstream issue #26535 — converted by maintainer to an Ideas discussion on October 28; original issue is locked, direct discussion URL not supplied by the issue REST view. No upstream PR approval; treat converted thread as a concise design question, not another progress-report destination.
- T3 remote architecture + remote access — stable environment identity, private routes, peer-link OAuth and delegated tasks; #9895 — existing connected-machine load balancing; #16719 — remote delegation; #17051 — record-first launch recovery still needs independently validated durability; #15350 — separate same-host Docker sandbox implementation, avoid overlap.
2. Existing z0 / fleet planning (not a second scheduler)
- z0intelligence #131 — canonical federated capacity plane: typed
CapabilityDemand -> CapacityOffer -> PlacementLease; semantic eligibility in z0, capacity economics in Kerdoios, actual execution/admission in existing runtimes. Fleet migration research comment explicitly rejects fleet-wide GPU/cluster migration without host safety and runtime evidence (its historic fleet report is owner-supplied, not independently re-probed). - z0 #133 / #134 — read-only
z0.capacity.snapshot.v1hardware, Tern and Kerdoios inputs, stable identities and sticky/nonmigratable active sessions; Kerdoios #50 — hardware-aware local SLM resource placement; Kerdoios #51 — quota/offer economics. .files#23 — actual explicit K3s host/cluster onboarding, CPU-first and GPU-fractioning deferred, no automatic host/driver edits; OSS Factory #127 / Draft #128 — existing bounded K8s Job renderer and offline safety tests, live kind/K3s not independently proven. Reuse fixtures, not scheduling authority.
3. GPU fractioning, KAI and release risks
- kai-scheduler/gpu-fractioning — authoritative current README @
dbf2d4efor NRI/MPS/metricsd, memory annotations, runtime prerequisites and CDI caveats; existing downstream fork — research/patch coordination if a concrete, nonduplicated fix emerges, not another T3 scheduler. - KAI Scheduler allocates fractional devices; KAI #2191 / #2192 — compute-fraction feature/design, distinct from memory fraction. GPU Fractioning #157 / #163 — node-side alpha r620+ compute fraction requirements.
- GPU Fractioning #165 / PR #166 — stale-container-mapping bug/unmerged patch; #167 — default CR Helm hook lifecycle issue. Review existing owners and current heads before discussing fixes; do not claim those are resolved or independently measured.
- NVIDIA GPU Operator, NVIDIA container toolkit, NVIDIA device plugin — prereqs: Kubernetes >=1.28, containerd >=2.0 with NRI, GPU Operator >=26.7.0, toolkit >=1.20.1, device plugin >=0.20.1, driver r615+ for memory fractions; compute-fraction prototype targets r620+. No host/runtime upgrade was made or approved.
4. Private-network and inference separation
- Tailscale Kubernetes Operator — first private-cluster ingress/egress candidate; service mirroring for future independent K3s clusters. Headscale and NetBird/WireGuard are alternative transport research, not claimed drop-in Tailscale Operator compatibility.
- NVIDIA Personal AI Router / downstream fork — optional model-serving request routing, not GPU resource scheduler; PAIR #85, #91, #92 and PR #10 track non-multicast VPN node discovery. Fork PR #2 proposed actual serving-node attribution but was closed unmerged, so prove receipts independently.
- Kueue, KAI, KEDA remain admission / GPU placement / elastic scaling owners. Existing z0/Evolution Lab/Tokenomics/z0evals provide only policy research and verified performance evidence; no autonomous routing/driver/platform changes.
5. Attention / next-action contract
ACTIVE NOW: validate current draft #46 on a disposable K3s host: exact-head CI, immutable image, PVC/environment identity across real Pod restart, authenticated private HTTPS/ACL, client pairing and negative controls (NOT RUN until executed). Then isolate/provision one P1 worker and prove existing T3 peer-delegation lifecycle. Next prove exclusive GPU scheduling only when necessary and explicitly approved. WAIT: Tailscale multi-cluster, PAIR engine routing, GPU fractioning, r620 compute fractions, and ABAB autonomous advice are future lanes with independent acceptance; they must not preempt the live P0/P1 blocker or manufacture separate work queues. Do not open duplicate upstream/fork issues or PRs for design-only references. Promotion queue #42 receives verified exact-head outcomes only.
Activity
kvnloo commentedon Oct 28, 2026
Current-head architecture triage — LIMITED (source/issue review; K3s runtime NOT RUN)
New origin design request: pingdotgg/t3code#26535, user-authorized on October 28. This approval covers the one upstream RFC issue only, not recurring origin comments or upstream PRs. Current upstream
mainaudited at5211acfe95a0c6d82a60e2524b5e46b47fe67a3e.Decision: adapter outside T3, provision-then-link
PeerLinks,PeerLinkRequestsandRemoteDelegationnow exist onmain; Linked environments remains beta/off by default and requires explicit authorization on both sides.t3 environment linkuses a peer's pairing code and selectable access.PeerLinksgrant.RemoteDelegationto create Pods.delegate_taskreturning accepted does not prove exactly-once effect or durable recovery. Assign one persisted, scoped provision ID; reconcile by identity and verify receipts on owner.Prior-contribution replay: what improved acceptance
The linked account's recent T3 upstream PR inventory returns 48 PRs, 5 merged and 1 still open as of this triage. Positive examples: #20299, #20736, #14139, #14128, #14140: small fixes, adapted to current main, executable regressions / CI and reviewer feedback. Failure/hold patterns are concrete:
Earlier K3s work: Factory #127 / draft #128 offers a bounded Kubernetes Job manifest renderer, red/green offline contract tests (22 PASS after its repair) and CI evidence. No real kind or K3s run was verified; reuse the security/manifest fixture and avoid creating a competing scheduler.
Next staged downstream implementation
environment-id, private HTTPS route, client pairing and restart survival; pinned image and measured first-connection latency.delegate_task; result recorded/verified by originating environment.Recurring worker: newly created enabled hourly automation
T3 K3s Downstream Build(ID6ae216c385c88191bf2581238548dee2, around minute :10, America/Chicago). It reads the current fork/origin state and contributes only to fork-owned branches/issues/PRs, with exact-head CI and source checks. No local K3s, device, image build, end-to-end agent execution or performance benchmark was run in this triage. Status: LIMITED / architecture established; implementation and runtime proof pending.kvnloo commentedon Oct 28, 2026
P0 code now published (LOCAL_ONLY / LIMITED). Draft fork PR #46, branch
infra/k3s-p0-owner-20261028, exact head0e70b0ddd7221fce5cd494a03ec1200c28b784d7against forkmain. Six new out-of-tree files: checksum-verified T3 v0.1.3 nonroot image recipe; restricted single-Pod namespace/SA/quota/PVC/Deployment/Service; private Tailscale HTTPS Ingress + ingress NetworkPolicy; strict digest renderer; offline checks/CI workflow; human-only deployment, pairing, restart, retention and rollback instructions. No T3 core edits, cluster deployment or upstream writes.PASS (observed, exact-head local): checked-in Python contract suite 5/5, plus 26 independent assertions on resource/security bounds, PVC identity, private ingress and malformed digest input; rendered YAML parse, Python compile and shell checksum-filename fixture PASS. All four exact executed source files matched GitHub blob SHAs:
k3s.yamlb975ecb9b1ca65918381ca7b3b5c9aada9bb091d,render.py79674ec421202de2e47fd4302211fe849f1697b2,test_contract.py9bf71dbf0d2eadb58aedea191abd3e4aa69a17eb,Dockerfile3552af7432aaa8fa74a68488db77a406be087dd4. Python 3, PyYAML 6.0.3. NOT RUN: hosted CI (no exact-head run returned yet); Docker image build;kubectlvalidation; K3s/pod/storage; tailnet ACL/HTTPS; actual T3 pairing, restart persistence, provider/delegation or latency. TCP readiness is not application auth, and actual NetworkPolicy enforcement must be tested.Maintainer scope: origin #26535 was closed and converted to an Ideas discussion Oct 28, with no issue comment/implementation approval; read-only now. Version split remains: upstream main
5211acfe95a0c6d82a60e2524b5e46b47fe67a3evs CLI releasev0.1.3. Origin #17051 remote launch durability and #26475 Tailscale linking remain unproven. Reused safety lessons from independent Factory #127/draft #128, not its job scheduler or unverified cluster proof.Next: get exact-head CI for #46 or fix workflow gate; have a person opt into a disposable K3s P0 run and record digest/K3s version, stable persisted T3 environment ID, private HTTPS client pairing and after-restart data/negative ACL checks. Only then P1 idempotent isolated worker broker and manual OAuth peer link. No promotion authorization or claim of live K3s success.
kvnloo commentedon Oct 28, 2026
P0 identity-restart acceptance improved; LOCAL_ONLY / LIMITED. Continued existing Draft PR #46, branch
infra/k3s-p0-owner-20261028, exact head87b91feaed9db5cf81620eebfbf11335fe592bc4(forkmaintarget). Added read-onlyexamples/k3s-p0/acceptance.pyand 4 fake-kubectl regression tests; README includes private baseline-before/after-human-restart instructions. Probe verifies digest-pinned Ready single server, bound PVC UID, private Tailscale Ingress + ClusterIP, and persistent hashed T3 environment ID/hostname/PVC UID. It neither deploys nor logs raw identity.PASS: 4/4 new offline mocked acceptance tests on source matching fetched GitHub blob SHA. Existing 5/5 contract tests + 26 security assertions passed previously at
0e70b0d, not rerun at this head. NOT RUN: hosted CI (0 exact-head Actions runs/statuses observed), Docker image, actual kubectl/K3s/PVC, Tailscale HTTPS/ACL, pairing/client, threads/provider/restart/delegation. No real K3s execution or performance assertion.Current upstream
main@125b61954dbba5822a8108c8ca47a4846b4b4328(one new unrelated PR-sync change); linked-work blockers remain #17051 and #26475. Upstream design #26535 closed/converted to Ideas; no approval or upstream writes.Next blocker: exact-head CI then human isolated/disposable-cluster image + restart/client/private-route negative acceptance; do not proceed to claiming P1 delegation until this evidence exists. Existing live installation untouched.
kvnloo commentedon Oct 28, 2026
P0 restart proof corrected (LOCAL_ONLY / LIMITED). Continued existing Draft #46, exact fork head
8da283b04954a02b57c1b0234e27032731fa7fb2, base forkmain. Old probe could report success after a second read of the same Pod without a restart. New v2 baseline collects selected Running/Ready Pod UID; the post-restart check requires a different UID, same hashed environment ID/PVC UID/private hostname and same digest-pinned image. Rejects terminating, duplicate or unready Pods; named-Pod read avoids targeting a different replica. Documentation requires a freshly captured v2 baseline. No T3 core/deployment changed.PASS (executed offline at matching blob SHAs): Python 3.13.5, 7/7 focused fake-kubectl tests and compilation.
acceptance.pyblobe216836316a4c7bba28fcbaeea31e848fdc138fc;test_acceptance.pyblobc81f62958f450a60b593623d70caf0b779fabf11. Prior five manifest contract tests were not rerun at this exact head. NOT RUN: hosted Actions (0 exact-head runs/statuses), Docker image, real kubectl/K3s, pod restart/client pairing, PVC/thread persistence, Tailscale HTTPS+negative ACL, provider or remote delegation.Current origin:
main@30eb7e7bb1ef103fdd6f54081b6a4acf66b3ee4cchecked; #15350, #26475 still open. New relevant #26344 is OPEN: remote delegated task approvals/questions are not surfaced on the owner parent today. #17051 record-before-launch remains open. No origin approval from closed #26535.Next gate: exact-head hosted CI (if enabled), then human-opted disposable K3s image/PVC/restart/private-route+client proof. Do not claim live P0 or P1 success; no change to existing infrastructure and no origin writes.
kvnloo commentedon Oct 28, 2026
Roadmap expansion: NVIDIA PAIR + private multi-host mesh — DESIGN ONLY
Human scope update (Oct 28): added an explicit expansion section in the issue body for (P2) independent K3s hosts/clusters over Tailscale or separately validated Headscale/NetBird/WireGuard routes, (P3) NVIDIA PAIR-informed inference routing, and (P4) bounded ABAB shadow/canary policy from existing z0 intelligence/evaluation systems. Existing T3 task ownership, manual peer authorization, K3s pod scheduling and no-cloud-GPU requirements remain unchanged. The existing hourly T3 K3s Downstream Build automation was updated, not duplicated, to see these dependencies after P0/P1.
Verified design blockers/overlap: upstream #26535 was converted to an Ideas discussion (verified GitHub timeline) and grants no PR approval. NVIDIA PAIR's #85/#91/#92 document mDNS/manual-node limitations over VPN; PR #10 is open, not established as shipped. Our prior PAIR draft #2 is closed unmerged: do not assume
X-PAIR-Served-Byattribution exists. Tailscale documents multi-cluster service mirroring. PAIR routes one independent inference request to one eligible node; it neither pools VRAM nor allocates T3 agent tasks.Active implementation unchanged: fork Draft #46 remains open at head
8da283b04954a02b57c1b0234e27032731fa7fb2(last read here); P0 live cluster/storage/Tailscale/client verification still NOT RUN this turn. No code edits, PRs, tests, new tasks, upstream comments, or production infrastructure changes during this roadmap update. Keep P0 proof and P1 one-worker broker first; do not promote design-only roadmap work to queue #42.kvnloo commentedon Oct 28, 2026
T3K3S-GPUFRAC-REFS-20261028 — Design/reference consolidation; implementation NOT RUN.
Added P3b GPU fractioning (HOLD) and a grouped References section to the existing #45 issue body; no second queue or competing worker. Reused prior z0intelligence #131, #133, #134, host plan .files #23, Factory #127/#128, and existing GPU Fractioning README.
Decision: T3 task control → K3s provisioning → optional, later KAI GPU-fraction placement → GPU Fractioning NRI/MPS enforcement → independent evidence; PAIR remains inference-only. Current prerequisites include driver r615+; new compute fractions KAI #2191/GPU #157 call for r620+ and remain Alpha. Upstream #165/PR #166 stale mappings and #167 Helm lifecycle remain review gates. Neither the current two GPUs' VRAM nor remote hosts become one memory device.
Attention: focus only on measured P0 persistent server / P1 single-worker proof first; postpone multi-host/VPN, PAIR and GPU fractioning changes. Existing hourly automation updated in place, no new task. No cluster, GPU, driver, infra, default branch or upstream changes; NO live GPU/K3s acceptance in this update. Upstream #26535 is converted to a Discussion, and the GitHub connector used here has no Discussions write action; do not misreport this downstream reference update as an upstream discussion post.
kvnloo commentedon Oct 28, 2026
P0 acceptance hardened — LOCAL_ONLY / LIMITED. Continued existing Draft #46, branch
infra/k3s-p0-owner-20261028, exact head4cbc64517a1da36d86ff666cc6d53a4b4d44e090. No competing PR, upstream edits, deployments, or new infrastructure.Change: the read-only restart probe now rejects a Ready but wrong-image, wrong-PVC, API-token-mounted or hostPath-mounted Pod, and rejects a ClusterIP Service or Tailscale Ingress misrouted to another workload/port/public host. Two added offline tests exercise 11 fail-closed mutations. Old Pod UID + identity checks remain.
PASS: independently reconstructed source matches the GitHub blob SHA for every executed file (new acceptance/test files + unchanged template, renderer, Dockerfile and contract tests). Python 3.13.5 / PyYAML 6.0.3 local
unittest discover: 14/14 PASS; compilation PASS. Tests execute mockkubectl, not a cluster. NOT RUN: exact-head hosted Actions (0 runs/checks observed), image build, real K3s PVC/Pod restart, private HTTPS/tailnet ACL/NetworkPolicy negative checks, T3 client pairing/thread persistence, OAuth peer link or delegated result.Live blocker: human-approved disposable cluster and private-route/client proof; do not advance P1 proof or treat this as upstream-ready. Relevant origin remains open #15350, #26475, #26344, plus new read-only overlap #26567, #26551 and #26556. GPU Fractioning, PAIR and multi-cluster remain HOLD. Canonical promotion queue #42 not promoted.
kvnloo commentedon Oct 28, 2026
T3K3S-P0-ENDPOINT-SAFETY-005 — P0 false-positive routing proof hardened; LOCAL_ONLY / LIMITED.
infra/k3s-p0-owner-20261028, exact head8514ad87c0addaee297d080f37d91d71645fce99. Three changed files:acceptance.py, fake-kubectl tests and README. No new PR, default-branch/upstream writes, cluster changes or worker scheduler.4cbc645, not silently carried forward.kvnloo commentedon Oct 28, 2026
T3K3S-P0-PRIVATE-ROUTES-006 — P0 offline route guard advanced; LOCAL_ONLY / LIMITED.
Continued existing Draft #46 on
infra/k3s-p0-owner-20261028, exact head8b2d42989918b417b71cfd16338921a82a490499(parent8514ad87), still targeting forkmain. Changed three out-of-tree files only (acceptance.py,test_acceptance.py, README). No K3s/host/provider install, origin write, default merge, or new PR.t3-k3s-p0.<tailnet>.ts.netIngress route (reject arbitrary*.ts.net, extra addresses, IP); list all namespace NetworkPolicies and reject absent, extra/additive or widened owner ingress allowances. This is API-state evidence only, not proof that the CNI enforces NetworkPolicy or that Tailscale ACL/TLS works.8b2d42989918b417b71cfd16338921a82a490499. Locally reproduced new helper logic passed 4 focused unit tests, including 8 ingress + 7 policy negative mutation checks; standalonepy_compilepassed. This is not a full run of the repo's source test suite.delegate_task.kvnloo commentedon Oct 28, 2026
T3K3S-P0-HTTPS-DESCRIPTOR-007 — P0 private-route acceptance now checks the real HTTPS descriptor (LOCAL_ONLY / LIMITED).
6cb505417dff863af900bb0661fdc8aa891f5c05targeting forkmain. Changed 3 out-of-tree paths: newdescriptor_probe.py,test_descriptor_probe.py, and README. No new branch/PR, origin write, default merge, hardware/cluster/network edit or credentials./.well-known/t3/environmentendpoint, comparing the returned identity only as a SHA-256 hash against the privateacceptance.pyv2 Kubernetes baseline. Fail closed on wrong tailnet origin, wrong ID, TLS/connect failure, redirect/non-200, unexpected content type, malformed/oversized JSON, and nonprivate/symlink/stale baseline. No proxy, TLS bypass, OAuth token or scheduler added. Upstream released v0.1.3 contract independently confirms this endpoint.067ae2956b1e0c72ef519dd9c9af86e2a39f222candfe105d2cc0c5e39ae8eb2864c7c1a7471431ad3c. These are fake TLS responses, not a live tailnet test.delegate_task. Current origin main9826dbb; related #17051, #26475, #26344, #26665 remain separate review gates. Origin converted Ideas #26535 is unchanged/read-only.Next human-proof gate: on a deliberately approved disposable K3s cluster, run existing v2 baseline before/after one human Pod restart, and the documented descriptor probe from the approved tailnet client both times. Independently verify negative ACLs, client auth, saved threads/data. P1/PAIR/multi-host/KAI GPU fractioning stay HOLD. Promotion queue #42 remains NOT READY.
kvnloo commentedon Oct 28, 2026
T3K3S-P0-PREFLIGHT-008 — material code progress, LOCAL_ONLY / LIMITED. Continued existing Draft PR #46 (no competing P0 PR), exact head
8a65a319e315726896693c17464c92872b9ca8cainto forkmain. Three out-of-tree files only:examples/k3s-p0/preflight.py,test_preflight.py, README.New operator safety gate: explicit-context, GET-only Kubernetes preflight before human P0 apply; fail closed if an existing
t3-k3s-p0namespace could be overwritten, if not exactly one Ready/untainted node or one named default StorageClass, if Kubernetes <1.28, or if the Tailscale namespace/IngressClass controller is absent or wrong. Query errors suppress stderr; no secrets, live apply, infrastructure or provider change. Reference: Tailscale ingress operator, controllertailscale.com/ts-ingress(source).PASS (real offline Python execution, exact Git blob SHA): 4/4 new mock-kubectl tests + compilation, including 9 invalid/mismatched cluster fixture groups and malformed-query cases; local
git hash-objectmatches fetched upstream fork blobs: preflightf362d80927d9f80646f10500e35fddb581e9a995, tests1efc74e96531da9294c0014cf010757c3b4b3cef. README readback verified. NOT RUN at this head: full combined P0 suite, GitHub Actions (0 runs observed on exact head), Docker image/K3s commands, Tailscale ingress TLS/negative ACL/dataplane, client pairing, thread/PVC identity across real restart, P1 OAuth or delegated result. A passing preflight does not prove the cluster is disposable, owner-approved or actually network-isolated.Origin drift review:
main@e34067edef2eba03c1e40bca7d2b2d933ae12b85; #15350, #26475 still open. New linked-task repair candidates #26665 (correct environment for remote approval navigation) and #26704 (delegated child PR-watch result delivery) are open/unmerged, not T3 P0 proof. #17051 remains a delegated-task durability concern.Next: owner-approved disposable-cluster apply, private client/data/restart/denied-route evidence. Keep P1/P2/PAIR/GPU fractions HOLD, do not mark promotion READY. No upstream/default-branch/cluster writes.
kvnloo commentedon Oct 29, 2026
T3K3S-P1-PLAN-009 — real downstream code, LOCAL_ONLY / P1 HOLD for physical proof. Created stacked Draft #47 from P0 Draft #46 head, not fork main; branch infra/k3s-p1-worker-plan-20261028, exact head 97e42b7ba439e2277aa5d68993e2d0eb372966f1, base P0 8a65a31. Exactly 4 added out-of-tree files; no P0/core/default-branch/other-repo edits or competing P1 PR.
Implemented: inert deterministic hashed-key worker plan for 8 bounded K8s objects (restricted namespace, SA without token, 1-Pod quota, distinct PVC, Recreate server Deployment, ClusterIP, Tailscale ingress + ingress-only policy), private 0600 create-once local receipt, stable repeat output, changed same-key image/expiry conflict rejection. No Kubernetes writes, no real cluster lease, no expiry controller/auto-delete, no pairing/peer consent or task scheduler. P0 live acceptance remains first; original v0.1.3 image peer beta compatibility is unproven.
PASS (executed offline): 5/5 Python unit tests, compilation; CLI process exercised identical-byte YAML retry (8 YAML docs), conflict expiry rejection. Exact GitHub blob SHA matched local executed files: plan 8af66bd933be39c5ceae202639f5ae48e2585295, tests fd407acfd99b4ab9ab13e61908fbd97e0e07c817. PR compare P0→P1 verified exactly 4 added paths. NOT RUN: hosted Actions (0 observed for new head), image build, K3s+PVC, HTTPS/ACL/CNI/remote client, paired owner/restart/data persistence, P1 provision/TTL/peer-delegation/failure injection. No claim of runtime success.
Origin watch: main b287102; sandbox #15350, Tailscale peer #26475, question relay #26344, child result #26704 and workflows #26673 remain open, not P0/P1 proof. #17051 still open. Existing Ideas #26535 unchanged/read-only.
Next: real separately approved disposable K3s P0 identity+data+private-client restart acceptance; only then opt-in P1 lease reconciliation and one actual verified remote delegated result. PAIR/multi-cluster/GPU fractions HOLD. Promotion queue #42 unchanged; nothing READY.
kvnloo commentedon Oct 29, 2026
P1 offline binding slice — LOCAL_ONLY / HOLD. Continued existing stacked Draft PR #47 on
infra/k3s-p1-worker-plan-20261028at exact headb31c7e2b64f89d9ce711eb222eb002ade39b2b80(base: P0 Draft #46 branch). Addedobserve.py, six tests, human-only observation instructions, and README link. Offline Namespace/PVC snapshots now fail closed on lease-hash, Pod Security, retention, bound storage or stable-UID drift. No K8s API access/actuation or worker task scheduler. It does not prove actual Pod, T3 identity, HTTPS, OAuth or delegation.PASS: 6/6 new offline tests + compilation on locally executed files whose
git hash-objectmatches exact published Git blobs (observe.py81c8f3b4f6774259cf7dad04cacbfc393f5e934e;test_observe.pyb7bcb920eece8669a1fc9de9d6e2e627ad6a08e9). Earlier P1 plan 5/5 not rerun here. FAIL: none observed in scope. NOT RUN: hosted CI (zero branch runs), full suite, physical cluster/PVC, Tailscale, paired client, worker image/build, peer linking, task delegation, cancel/restart/expiry.Next blocker: P0 Draft #46 needs explicitly approved disposable-cluster restart, saved-thread/paired-client, private HTTPS and negative ACL proof; then verify latest linked-beta-compatible worker image. P2/PAIR/GPU fractioning still design/HOLD. Origin writes: none. Queue #42 stays not-READY.
kvnloo commentedon Oct 29, 2026
P1 immutable-image binding hardened — LOCAL_ONLY / HOLD. Continued existing stacked Draft #47, exact head
697fefba964600a06ba9dbba60204b0ea39384a2(base: P0 Draft #46 branch). The offline Namespace/PVC lease binding now includes a SHA-256 fingerprint of the full pinned image reference. Changing image registry/repository/digest with the same lease key and PVC UIDs fails comparison; unpinned images are rejected. Bumped private binding receipt to v2; old UID-only baselines fail closed, with migration guidance. No runtime broker or cluster writes.PASS (independently executed against exact GitHub-blob-identical files): Python 3.13.5, 8/8
test_observe.pyoffline tests andpy_compile. Source blobs:observe.py@2628318dba12cadcebd5406ac0abdc7be9cd9c6b,test_observe.py@a6de03d12936aea0190164f4ae4606fb79233cb6; copies verified usinggit hash-object. Includes image swap, invalid reference and legacy baseline regression tests.NOT RUN: full P1 suite, hosted Actions (0 exact-head workflow runs/statuses), actual K3s/Docker/Tailscale, paired client, remote delegated task, lease TTL/cancel/resume. P0 live restart+identity+private access proof remains the deployment gate; P1 is a non-actuating design preview only. Upstream main examined at
861a51a6c4a036a6f14fea2a6f093808de1d96ce; remote launch recovery pingdotgg#17051 and pending linked-peer fix pingdotgg#26475 remain relevant. No origin, default-branch, GPU or home-infrastructure writes.kvnloo commentedon Oct 29, 2026
P1 observed Deployment binding v3 — LOCAL_ONLY / HOLD. Continued existing stacked Draft #47 at exact head
012080ef76a2c2c5537d9696c9ceef156b8f6592, base P0infra/k3s-p0-owner-20261028(Draft #46). No default branch, origin, home infrastructure or cluster writes.Change:
examples/k3s-p1/observe.pynow requires an explicitly supplied read-only Kubernetes Deployment JSON along with Namespace/PVC snapshots and the immutable plan. Rejects an observed desired Pod template with the wrong pinned image, deployment UID/namespace, replica count, selector, ServiceAccount/token mount, or PVC mounts; stores deployment UID in private v3 binding. Legacy v1/v2 baselines fail closed. Documentation includes snapshot and baseline-migration steps. This checks desired Deployment configuration only, never an actually running Pod image, T3 identity, peer permission or lease execution.PASS: independent Python
unittest10/10 focused offline tests pluspy_compileagainst exact-source Git blobs:observe.py@41065010c0f5515cbfc2e69f90de23b6fc3858fc,test_observe.py@91ac775e5193d6aaac9f7a9f8a2aeb219dd054e6, both independently confirmed bygit hash-object. One initial test-call regression was repaired before successful full focused rerun. NOT RUN: full P1 suite; hosted CI (zero exact-head workflow runs/statuses); Docker, real K3s/Tailscale, Pod image or PVC live state, linked OAuth/mobile pairing, TTL/cancel/resume and remote delegation.New upstream watch: pingdotgg/t3code#26757 proposes automatic personal-tailnet sign-in (still OPEN), not a substitute for explicit worker OAuth peer-link consent; do not auto-enroll transient workers. #26776 remote follow-up fix also remains OPEN. P0 Draft #46 remains unproven on a disposable live cluster. Next blocker: real P0 restart + persistent saved-thread/identity + HTTPS/negative ACL and pairing proof, then one P1 worker/delegation. GPU fractioning P3b stays HOLD; no origin publication.
20 remaining items
kvnloo commentedon Oct 30, 2026
P1 snapshot control gate — LOCAL_ONLY / HOLD (2026-10-29). Continued existing stacked Draft #47, branch
infra/k3s-p1-worker-plan-20261028at exact heada5dffeb7, on P0 Draft #465bbb91b3. Read-onlycapture.pynow rejects ResourceQuota scoping/bounds/reconciliation drift (a scoped quota can exclude the actual worker) and ServiceAccount token/secret drift; eight negative fixtures plus README clarification. 3 P1-only files / one commit; no default/origin/cluster write.PASS: GitHub exact-head three-blob readback;
df875c4d..a5dffeb7compare +1/−0 commits, exactly those three paths; 2/2 separate synthetic Linux/Python predicate reproductions, including eight negative cases (not checked-in tests). NOT RUN: 11 checked-intest_capture.pymethods, full suite, CI (zero observed runs/statuses), real disposable/live K3s, quota/SA enforcement, PVC/restart, private Tailscale HTTPS/CNI/ACL, OAuth linking, one delegated result, cancel/TTL. Captured API state is not physical attestation or permission.Next blocker: human-approved disposable P0 private HTTPS, owner identity/PVC restart and paired-client acceptance; then single authorized P1 worker. Upstream pingdotgg#17051 durability gap and private route acceptance remain unproven. P2/PAIR/GPU fractioning HOLD. No upstream readiness.
kvnloo commentedon Oct 30, 2026
P0 private-route exclusivity hardening — LOCAL_ONLY / HOLD (2026-10-29). Continued existing Draft #46 on
infra/k3s-p0-owner-20261028, exact head09e7f4da. Three P0-only paths:acceptance.pynow rejects unapproved Service/Ingress exposure annotations (including ClusterIPtailscale.com/expose), external Service status/fields, and a second/missing/mismatched namespace Service or Ingress. Added ten adversarial fake-kubectl mutations totest_acceptance.py; README documents strict inventory and non-atomic snapshot limits. No T3 core change, default-branch merge, upstream write, infrastructure mutation or P1 deployment.PASS: Independent Python 3.13.5 reproduction of the extracted route/annotation logic, 10/10 positive/negative cases + compilation; exact-head connector readback of all 3 Git blobs. NOT RUN: checked-in P0 test suite (local checkout unavailable), CI (0 observed runs/checks at exact head), image build, live K3s/NetworkPolicy, Tailscale TLS/DNS/ACL, persisted client/thread after pod restart, peer consent and delegation. These are API metadata assertions, not dataplane isolation proof.
Current upstream overlap: #26475 numeric Tailscale peer-link fix and #26757 tailnet-device auto-auth are open/unmerged; neither authorizes auto-enrollment of a transient P1 worker or replaces explicit OAuth peer consent. Next gate: authorized disposable-cluster P0 owner/client/PVC restart + private HTTPS + negative ACL proof, then one consented P1 worker. Stacked Draft #47 remains separate and HOLD at
a5dffeb7.kvnloo commentedon Oct 30, 2026
P0 admitted-Pod security gate — LOCAL_ONLY / HOLD (2026-10-29). Continued existing Draft #46, branch
infra/k3s-p0-owner-20261028, exact head6dfa0e38. A Ready Pod could previously pass the P0 evidence collector despite admission-time host namespace, privilege/seccomp, ServiceAccount, injected credential/command, or PVC/tmp-volume mutations.examples/k3s-p0/acceptance.pynow rejects these effective-Pod drifts; fake-kubectl fixture and 24 negative mutation cases plus README qualification added. Five sequential commits since09e7f4da, exactly three P0 example paths; no T3 core, origin, default-branch, or infrastructure changes.PASS: Independently executed local Python 2/2 source-mirrored isolated test methods (one positive plus 24 adversarial subcases) and
py_compilefor the isolated reproduction; GitHub readback of all three final blobs and exact-head compare (+5/−0 commits; onlyacceptance.py,test_acceptance.py, README). NOT RUN: actual checked-in P0 suite/full tests, hosted Actions (0 exact-head runs/statuses), Docker build, live/disposable K3s, PSA/CNI/ResourceQuota dataplane proof, PVC/owner restart, private Tailscale HTTPS/negative ACL, OAuth client/peer linking, delegation, cancellation/TTL. API assertions alone cannot attest cluster enforcement.Fresh upstream:
pingdotgg/t3code main@8970a321; #26169 proposes HTTPS route preference, #26475 numeric Tailscale peer fix and #27122 sandbox-driver registry remain open; none grants a native K3s seam or automatic worker enrollment. #17051 remote record-before-launch remains open.Next gate: human-approved disposable P0 single-node owner/PVC restart and actual private HTTPS/Tailscale ACL-negative + paired-client proof; then a separately consented single P1 worker. P1 Draft #47 remains HOLD; P2/PAIR/KAI compatibility only. No upstream/cluster write or promotion.
kvnloo commentedon Oct 30, 2026
P0 bounded PVC/PV restart proof — LOCAL_ONLY / HOLD (2026-10-29). Continued existing Draft #46, exact head
6af4fae4. Five out-of-tree files changed (no T3 core): read-only cluster-scoped PV lookup now verifies exact PVC/PV claimRef UID, Bound phase, RWO Filesystem access, storage class, 10Gi owner request and deletion state; rejects option-like PV names. P0 v3 private restart baseline records PV UID/reclaim policy and checks both remain stable; private HTTPS descriptor probe and tests updated for v3. DocumentedDeletereclaim risk: acceptance cannot prove bytes, backups, storage availability or authorize deletion.PASS: 5/5 exact-head GitHub blob readbacks;
6dfa0e38..6af4fae4compare +5/−0, exactly these five files; 16/16 independent isolated Python helper cases (2 valid, 14 negative), not checked-in suite execution. NOT RUN: checked-in Python suites, Docker/image, Actions (0 runs, 0 check-runs/statuses at head), disposable/live K3s/PVC/PV persistence, Tailscale HTTPS/ACL/CNI, paired client and restored thread, OAuth peer linking and remote delegation. Extra cluster-scopedget persistentvolumesread permission is now required for the human P0 acceptance script. No live cluster/default/origin/GPU writes.Gate: human-approved disposable one-node P0 image/persistence/private HTTPS/paired-client and negative policy proof; only then separately consented P1 single-worker trial. P1 Draft #47 remains stacked on an older P0 base, LOCAL_ONLY / HOLD. Upstream pingdotgg#17051 launch durability and pingdotgg#26450/pingdotgg#26475 peer IP route remain unproven; no PROMOTE.
kvnloo commentedon Oct 30, 2026
P0 admitted-host/resource guard — LOCAL_ONLY / HOLD (Oct 29). Advanced Draft #46 to exact head
a647cc74, from6af4fae4. A Ready Pod with an admission-mutated hostname can change T3 usage fingerprint; altered CPU/memory/ephemeral bounds can silently break P0 isolation/capacity assumptions. Read-onlyacceptance.pynow rejects those drifts; added 8 negative fake-Pod cases and updated the human runbook. 3 P0-only files, one commit; no default/origin/cluster changes.PASS: exact-head 3/3 GitHub blob readbacks; compare ahead 1, behind 0, three paths; 2/2 separate synthetic Python mirror methods (baseline and eight adversarial cases), not the checked-in suite. NOT RUN: checked-in tests, CI (0 exact-head runs/statuses), Docker, disposable/live K3s, physical resource admission, private HTTPS/ACL/CNI, PVC/PV restart persistence, pairing and remote delegation. Upstream main
a773dc38read; origin pingdotgg#26535 remains converted/read-only without feedback; pingdotgg#26475, pingdotgg#26757, pingdotgg#27128 still open at inspected heads. P1 #47 still has older P0 base, so no claim of inherited security work.Next: owner-approved disposable P0 test with one Ready pod, stable
hostname=t3-owner, actual resource constraints, private TLS/ACL negative probe, restart-safe state/PVC and paired client; then one explicitly linked P1 worker. P2/PAIR/GPU fractioning HOLD.kvnloo commentedon Oct 30, 2026
P0 malformed Kubernetes response guard — LOCAL_ONLY / HOLD (2026-10-30). Continued existing Draft #46, branch
infra/k3s-p0-owner-20261028, exact9a2e6255froma647cc74. Read-only preflight now raises redactedPreflightErrorinstead of a traceback on malformed Node, taints, conditions, namespace, IngressClass and StorageClass objects. A regression test reproduces nine malformed-object failures on the prior exact source, then passes on the patch. Only two out-of-tree P0 files, one non-default fast-forward commit; no cluster or origin mutation.PASS: exact Git blobs
preflight.py=56c47b078e263bb0f266c4962d865be0eeebc729,test_preflight.py=0f8ef4da299400367365c8c42452f71203f8de04match locally executed copies; 6/6 checked-in Python 3.13.5 preflight tests and compilation; head compare ahead 1/behind 0, two paths; no exact-head Actions runs found. NOT RUN: remaining checked-in P0 suites, Docker/image, disposable/live K3s, PVC/restart, TLS/Tailscale ACL/CNI, client pairing, P1 OAuth/delegation/cancel/TTL.Next: approved disposable single-node P0, persistent owner state/PVC across restart, private HTTPS with negative ACL and paired-client proof; then one consented P1 worker. Stacked P1 Draft #47 still has older P0 parent and is HOLD. PAIR/GPU fractioning remain design/compatibility-only; no promote or undraft.
kvnloo commentedon Oct 30, 2026
P0 admitted-container overrides — LOCAL_ONLY / HOLD (2026-10-30). Advanced existing Draft #46 on
infra/k3s-p0-owner-20261028to exactd56c49e1from9a2e6255.acceptance.pynow rejects container-levelrunAsUser,runAsGroup,runAsNonRootoverrides of the reviewed Pod identity and injected lifecycle hooks; five negative regression fixtures added to the existing fake-kubectl test. Exactly two out-of-tree P0 files and one fast-forward commit. No core, default branch, origin or cluster mutations.PASS: 2/2 committed Git blobs read back at exact head; GitHub compare ahead 1/behind 0 and exactly those two files; isolated Python mirrored-predicate reproduction (one allowed baseline, five rejected mutations). NOT RUN: checked-in P0 suite, GitHub CI (zero observed exact-head runs/statuses), image build, disposable/live K3s, Tailscale private HTTPS/ACL/CNI, physical PVC/restart, client pairing or P1 delegation. The mirrored checks are not proof that Kubernetes admission or network enforcement works.
Upstream: Tailscale device discovery PR #26757 is still open/unmerged; it does not authorize transient worker auto-enrollment or substitute for explicit peer OAuth linking. GPU fractioning remains HOLD (alpha compute fractions; stale-map fix #166 unmerged).
Next blocker: separately owner-approved disposable one-node K3s with real persistent owner/PVC, private HTTPS plus ACL-denial evidence, and a paired client after restart; only then a single consented P1 worker. Stacked Draft #47 remains on an older P0 base and HOLD. No PROMOTE/UNDRAFT_READY. Runnable offline suite:
python3 -m unittest discover -s examples/k3s-p0 -p 'test_*.py' -v.kvnloo commentedon Oct 30, 2026
P0 HTTPS descriptor probe bug fixed — LOCAL_ONLY / HOLD (2026-10-30). Continued existing Draft #46 to exact
6915ce2bfromd56c49e1. Python's realhttp.client.HTTPSConnectiondoes not implement__enter__/__exit__, sowith connect(...)would raiseTypeErrorbefore a single descriptor GET. Replaced withcontextlib.closing; fake TLS connection no longer has an invented context-manager API, and regression checks close on success and non-200. Exactly 2 P0-only files, 1 fast-forward commit, no live K3s/default/origin/core changes.PASS: isolated Python 3.13.5 RED old-context-manager / GREEN closing success + failure paths; GitHub exact-head 2/2 blob readbacks and ahead 1/behind 0, exactly 2 files. NOT RUN: checked-in unit suites, CI (0 observed exact-head runs/statuses), actual HTTPS/Tailscale/ACL, disposable K3s/PVC/restart, paired client, delegated P1 task. Test reproduces an actual stdlib API incompatibility, not live HTTPS validation.
Next: execute checked-in P0 suite in a source-equipped runner, then human-approved disposable single-node P0 private HTTPS descriptor + restart identity/PVC + paired client. P1 Draft #47 remains stacked on older P0 head and HOLD; no automatic rebase/promotion. Origin main observed at
97e35bcf, peer-link authorization still distinct from client-saved environment.kvnloo commentedon Oct 30, 2026
P0 restart baseline input validation — LOCAL_ONLY / HOLD (2026-10-30). Continued existing Draft #46 to exact
5158a707, branchinfra/k3s-p0-owner-20261028. The read-only acceptance comparator previously loaded unbounded baseline JSON and assumed a valid object. It now requires a private regular file <=4096 bytes, opens FIFOs without blocking, preserves symlink refusal, and validates exact v3 identity fields before comparing restart evidence. Added a checked-in adversarial test method (mode, symlink, oversized/corrupt/partial JSON, FIFO, valid compare); changed 2 P0-only files in 2 fast-forward commits. No new PR, T3 core change, default-branch/upstream/cluster write or deployment.PASS: isolated Python 3.13.5 helper checks 8/8; exact-head 2/2 GitHub blob reads; compare from
6915ce2bahead 2 / behind 0 (2 files). NOT RUN: actual checked-in Python suite, exact-head CI (0 Actions/check runs), real/disposable K3s, Tailscale HTTPS/ACL, PVC/thread restart, client pairing, provider/delegation. The isolated tests do not validate the full acceptance module.Next blocker: checked-in offline tests and human-authorized disposable one-node P0 private HTTPS + stable PVC/identity + real paired client, then consented single P1 worker. P1 Draft #47 remains stacked/HOLD; no PROMOTE / UNDRAFT_READY. Upstream Ideas pingdotgg#26535 remains read-only.
kvnloo commentedon Oct 30, 2026
P0 descriptor baseline FIFO deadlock fixed — LOCAL_ONLY / HOLD (2026-10-30). Advanced existing Draft #46 to exact
e11d6de0, a fast-forward from5158a707changing only two P0 files. Unlikeacceptance.py,descriptor_probe.read_baselineopened untrusted local baseline paths with a blockingO_RDONLY; a private FIFO would hang beforefstat. AddO_NONBLOCKand a real FIFO subprocess timeout regression; no live Kubernetes or T3 access.PASS: source copies hash to exact original GitHub blobs (
0e1dace3,20a083b5); on Python 3.13.5 the new FIFO test RED timed out on the old code, then GREEN with 9/9 actual checked-intest_descriptor_probe.pymethods passing and compilation. New GitHub blobsb7d01491/0033d6eematch the tested files and exact-head readback. GitHub compare: +1 commit/−0, exactly those two paths.NOT RUN: other checked-in P0 suites, CI (0 exact-head Actions/checks), image build, disposable/live K3s, Tailscale HTTPS/ACL/CNI, PVC/thread restart, client pairing, P1 OAuth peer-link/delegation. Upstream main now closes host-fingerprint issue #11849 via
9e60e945, but the P0 fixture packages older CLIv0.1.3; no runtime compatibility inference. Peer-link Tailscale #26475 remains open.Next: run full offline P0 suite, then owner-approved disposable one-node K3s P0 private HTTPS/identity/PVC restart and actual paired client; P1 #47 stays HOLD/stacked. No default-branch, origin, or infrastructure writes.
kvnloo commentedon Oct 30, 2026
P0 bound PVC/PV capacity — LOCAL_ONLY / HOLD (2026-10-30). Continued existing Draft #46 at exact
0bcc804a(parente11d6de0). The prior read-only acceptance check could pass with a PVC requesting 10Gi but a missing or undersized API-reported PV/PVC capacity. RequirePVC.status.capacity.storageandPV.spec.capacity.storageboth equal the fixed 10Gi P0 fixture; add four negative cases and document the limitation. Exactly one fast-forward commit / three P0-only paths; no new PR, core T3 code, default/origin or cluster writes.PASS: isolated local Python capacity-predicate reproduction 1 clean + 4 negatives, including verifying the old predicate accepted all four bad cases; exact-head GitHub blob readback 3/3; compare parent→head ahead 1, behind 0, three intended files. NOT RUN: checked-in
python3 -m unittest discover -s examples/k3s-p0 -p 'test_*.py' -v, hosted Actions (0 head runs/statuses), image build, disposable/live K3s, Tailscale HTTPS/CNI/ACL, PVC restart/data, paired client or remote delegation. The isolated reproduction is not a checked-in suite run, real storage-capacity observation, or durability attestation.Upstream change: container host fingerprint issue #11849 closed Oct 30 as fixed by
9e60e945d2, verified ancestor of upstreammain@32ecd717; the P0 image still packages CLI v0.1.3, so do not infer the fix is released or change its pinned host guard. Numeric Tailscale peer-link #26475 remains open, and remote record-before-launch #17051 remains unproven.Next blocker: run the full checked-in P0 offline suite, then obtain explicit authorization for a disposable one-node private HTTPS/PVC/restart/paired-client test; only then one consented P1 worker (#47). No PROMOTE or upstream approval; P2/PAIR/GPU fractioning remain held.
kvnloo commentedon Oct 30, 2026
P0 malformed named API response guard — LOCAL_ONLY / HOLD (2026-10-30). Continued existing Draft #46 to exact
497e43e8from0bcc804a. The read-only restart acceptance collector previously JSON-decoded Kubernetes Deployment/PVC/Service/Ingress GET responses without checking their top-level type: syntactically valid[],null, scalar JSON could escape asAttributeErrorinstead of a fail-closed evidence error. It now rejects non-object responses. Added a checked-in regression method with 4 named resources × 4 malformed response types. Exactly two out-of-tree P0 paths, two fast-forward commits, no new PR/core/default/origin/cluster change.PASS: independent Python 3 isolated boundary tests 3/3, including 16 negative combinations, valid object and bad JSON; the old predicate accepted those 16 malformed values, new guard rejects them; exact-head GitHub blob readbacks for both files and compare
0bcc804a..497e43e8ahead 2 / behind 0, two intended paths. NOT RUN: checked-intest_acceptance.py/full P0 suite, GitHub Actions (no exact-head runs/statuses observed), image build, disposable/live K3s, PVC/restart, private HTTPS/ACL, client pairing or P1 delegation. Isolated boundary reproduction is not checked-in or live K3s proof.Dependency refresh: upstream container hostname issue #11849 closed as fixed on main at
9e60e945d2, but this does not satisfy persistence tests. Tailscale selection #26475, follow-up delegation #26776, and remote approvals #26344 remain open. Stacked P1 Draft #47 is HOLD; do not assume its old P0 parent matches this head.Next gate: execute checked-in offline tests on a source-equipped runner; then independently authorized disposable one-node K3s P0 image/PVC/restart + private HTTPS/negative ACL + paired-client proof before P1. No promotion or undraft.
kvnloo commentedon Oct 30, 2026
P0 namespace controls — LOCAL_ONLY / HOLD (2026-10-30). Continued existing Draft #46, exact head
05656020from497e43e8. Read-only acceptance now checks the actual named ResourceQuota (exact, unscoped hard bounds plus reconciled status) and ServiceAccount (no token automount, referenced secrets, deletion, or identity drift) before reading the private persisted T3 environment ID. Added checked-in fixtures/15 negative regressions and documented the extra GET permissions and API-vs-enforcement boundary. Three fast-forward commits, exactly three P0-only files; no core/default/upstream/cluster changes.PASS: exact-head GitHub 3/3 blob readbacks; compare +3/−0 commits, 3 paths; Python 3.13.5 independent isolated reproduction 2/2 methods (one positive + 15 adversarial cases), bytecode compilation. NOT RUN: checked-in P0 suite (new tests present but not executed), CI (0 exact-head workflow runs/statuses), Docker, disposable/live K3s, quota admission/CNI/ACL, tailnet TLS, PVC/thread restart, consented pairing and P1 delegation. The isolated reproduction does not prove the checked-in suite or runtime policy enforcement.
Dependency check: origin
main32ecd717; container-host identity issue #11849 closed with maintainer's fix reference9e60e945d2, but this P0 example still pins released CLI 0.1.3; tailnet-link PR #26475 remains unmerged with review concerns. Do not assume either change shipped in the P0 image.Next: run
python3 -m unittest discover -s examples/k3s-p0 -p 'test_*.py' -vat this SHA, then separately obtain human approval for a disposable one-node K3s P0 storage/identity/HTTPS/pairing/restart proof. P1 Draft #47 remains stacked/HOLD. No PROMOTE / UNDRAFT_READY.kvnloo commentedon Oct 30, 2026
P0 renderer no-clobber fix — LOCAL_ONLY / HOLD (2026-10-30). Continued existing Draft #46 at exact
838cda838fa8d3f2902e27cbbe470d7fd9f5d678from05656020: 1 fast-forward commit, only 3 P0 example files (render.py, newtest_render_output.py, README). Before:--outputusedPath.write_text, overwriting an existing file or following an output symlink. After: mode-0600 same-directory staged file, fsync, exclusive hard-link publish and temporary cleanup; existing output/symlink never overwritten. Rerun with a fresh output path. No cluster, default, upstream or T3 core change.PASS: previous sink's symlink overwrite reproduced locally (RED); exact post-fix
render.pyand checked-intest_render_output.pyGit blobs matched locally executed source (302c293e,72e7a48c);python3 -m unittest -v test_render_output3/3 GREEN (private/exclusive output, symlink non-clobber, injected publication failure cleanup); Python compilation; 3/3 GitHub blob readbacks and compare ahead 1 / behind 0. NOT RUN: other checked-in P0 suites, Actions (0 exact-head runs/statuses observed), image build, disposable/live K3s, Tailscale TLS/CNI/ACL, PVC/restart, actual web/mobile pairing or P1 delegation.Next: source-equipped full offline P0 suite, then separately human-approved disposable one-node runtime P0 proof (HTTPS/ACL denial, stable persisted identity/data, paired client). P1 #47 stays stacked on older P0 head and HOLD; do not auto-rebase or promote. Independent factory kind→K3s CPU Job pilot #127 / draft #128 is not runtime proof for T3. Peer route origin #26475 remains open; upstream Ideas #26535 is read-only. P2/PAIR/GPU-fractioning remain design-only/HOLD. No PROMOTE / UNDRAFT_READY.
kvnloo commentedon Oct 30, 2026
P0 digest image repository validation — LOCAL_ONLY / HOLD (2026-10-30). Advanced existing Draft #46
infra/k3s-p0-owner-20261028to exact162bc62b7, 2 fast-forward commits from838cda83, exactly two out-of-tree P0 paths:render.pyand extended existingtest_contract.py. The renderer previously accepted impossible OCI image repositories despite an apparently valid SHA256 digest (tag/colon inside repository path, nonnumeric/overflow registry port, invalid segment separators). A six-case RED reproduction now fails closed; four valid registry/path cases retained, including numeric registry ports. No core, origin, default-branch, cluster or existing deployment modification.PASS: Python 3.13.5 and PyYAML 6.0.3 local: 8/8 actual checked-in focused contract+renderer tests (Git blob hashes match committed SHA: renderer
693e2837, contract testsb206c8ac, fixturesb975ecb9/3552af74, output tests72e7a48c); separate 2/2 temporary adversarial methods; old source accepted all 6 malformed refs (RED), new source rejected them (GREEN); actual render CLI produced eight-resource YAML mode 0600 and rejected malformed image plus overwrite; compilation PASS. Verified exact GitHub head, +2/−0 comparison, only expected paths. No local credentials or live infrastructure involved.NOT RUN: full checked-in P0 suite, pinned PyYAML 6.0.2 environment, GitHub Actions (0 runs/statuses observed at this head despite existing
k3s-p0-contract.yml), Docker/image build, disposable/live K3s, real Tailscale TLS/ACL/CNI, persisted PVC/thread/client pairing, and P1 remotedelegate_task. This is offline source proof only.Next: run remaining checked-in P0 suites and approved disposable single-node owner proof; then one isolated consented P1 worker. P1 Draft #47 stays stacked/HOLD. No PROMOTE/UNDRAFT_READY; no upstream writes. Reference independent Factory kind→K3s CPU Job pilot #127 / Draft #128 only for Kubernetes job guardrails, not T3 identity/delegation.