[rohitrsh]

Summary

Replaces the KubernetesProvider placeholder with a real ContainerProvider. A trainer running in the same cluster gets one Pod and one Service per environment, and an in-cluster HTTP URL, matching the decisions on #1153.

• One Pod and one Service per provider instance. No Deployment, Ingress, or port-forward.
• base_url is http://{name}.{namespace}.svc.cluster.local:{port}. This is the in-cluster HTTP exception. ModalProvider and ACASandboxProvider still require https://.
• Optional extra: pip install openenv[kubernetes]. The client is imported only when the provider is constructed, and it is not re-exported from openenv.core.containers.runtime.
• Constructor takes namespace and kube config. Resources, runtimeClassName, imagePullSecrets, and labels are start_container kwargs.
• Readiness polls GET /health once the Pod is Running, as LocalDockerProvider does. A Failed or Succeeded Pod raises immediately.
• A failed start deletes only the objects this instance created. stop_container() and close() are safe to call twice. The Service URL and raw API bodies are left out of errors.
• Names are DNS-1123 labels. Ownership labels (app.kubernetes.io/managed-by=openenv and the instance label the Service selects on) cannot be replaced by caller labels.

Closes nothing by itself. Follows #1153 and #441.

Test plan

• pytest tests/test_core/test_kubernetes_provider.py (46 passed, no cluster)
• ruff check, ruff format, and usort on the changed files
• CI-equivalent pytest tests/ excluding integration, network, and docker: 3075 passed, 103 skipped. The only failure was the existing test_start_refuses_a_port_held_by_a_non_capture_listener timeout, which does not import this provider.
• Live on kind context kind-acroflow, namespace openenv: one Pod and one Service, in-cluster GET /health returned ok, stop_container() removed both.
• @k21993 review, particularly failure cleanup, idempotent shutdown, naming and ownership labels, readiness timeouts, and the in-cluster URL.

---

Note

Medium Risk
Adds optional cluster lifecycle code (create/delete Pods and Services) with RBAC and partial-failure cleanup implications, though scope is limited to labeled resources in a caller-chosen namespace and is covered by extensive unit tests.

Overview
Implements in-cluster Kubernetes runtime support by replacing the KubernetesProvider stub with a real ContainerProvider that provisions one Pod and one Service per instance in an existing namespace (no Deployment, Ingress, or port-forward).

Install via pip install openenv[kubernetes]; the client loads only when the provider is constructed and KubernetesProvider is not re-exported from openenv.core.containers.runtime (import from kubernetes_provider). start_container returns a plaintext in-cluster URL http://{name}.{namespace}:{port} for trainers running inside the cluster; readiness follows LocalDockerProvider (Running + GET /health). Failed starts and shutdown use ownership labels and conservative cleanup so only OpenEnv-managed objects are deleted; errors avoid leaking URLs and API bodies.

Docs and pyproject.toml mark Kubernetes as a supported provider; a large test_kubernetes_provider.py suite covers lifecycle, cleanup edge cases, and the SDK adapter without requiring a live cluster.

^{Reviewed by Cursor Bugbot for commit dacf113. Bugbot is set up for automated code reviews on this repo. Configure here.}

[k21993]

Thanks for putting this together. The overall design matches the discussion on the issue. A few things I think should be fixed before this leaves draft:

• ensure_namespace() calls read_namespace, which requires cluster-level permission. A trainer with only namespace-level Pod and Service permissions will fail before creating anything. Please remove this preflight. Also handle a 404 from Pod or Service creation as "namespace does not exist" so that error does not become the generic "Failed to create Pod" message.

• A Pod or Service can be created even if the client loses the API response. In that case, the current flags stay false, cleanup skips the resource, and _name is cleared. For a non-409 create error, please check the generated name, verify the OpenEnv ownership labels, and try to delete it. If cleanup cannot be confirmed, keep the name and state so stop_container() can retry.

• The returned URL hardcodes .svc.cluster.local. Kubernetes clusters can use a different cluster domain. Since this provider is only for in-cluster use, service.namespace is the simplest portable address. A configurable cluster domain would also work.

• Separate non-blocking note: proxies={"http": None, "https": None} still allows ALL_PROXY. The same pattern already exists in LocalDockerProvider and DockerSwarmProvider, so I do not think this PR should own that fix. It should be tracked separately for all three providers.

[rohitrsh]

Thanks for the review. The three points are in 74d80ba:

• The namespace preflight is gone. A 404 from Pod or Service create reports that the namespace does not exist, and the API body is not included.
• If a create response is lost, cleanup reads the generated name and deletes it only when the OpenEnv ownership labels match. A 409 is left alone. If that read or delete cannot be confirmed, the name and flags stay so stop_container() can retry.
• The Service URL is now http://{name}.{namespace}:{port}. Pod DNS search fills in svc.<cluster-domain>.

proxies={"http": None, "https": None} still lets ALL_PROXY through. That is the same gap in LocalDockerProvider and DockerSwarmProvider, so I left it for a separate change.

The PR is still a draft.

[rohitrsh]

Hey @k21993, the three points from your review are in 74d80ba. Ready for another look when you have a chance.

[k21993]

Thanks for the update @rohitrsh . I reviewed 74d80ba.

The original three concerns are addressed, but I found one remaining cleanup issue.

_lookup_owned() sets the created flag to True when the ownership read fails. stop_container() later treats that flag as confirmed ownership and deletes without re-checking labels. This can delete a foreign Pod or Service after an ambiguous create failure.

There is also a sequencing issue: if Pod creation fails before Service creation is attempted, cleanup still probes the Service with uncertain=True. If that read fails, the Service is marked as created even though create_service() was never called.

I think the fix should keep ownership-unknown separate from confirmed ownership, only mark uncertainty for the resource whose create was actually attempted, and re-check labels before deleting an unconfirmed resource. If that read fails again, retain the state for a later retry.

[rohitrsh]

Thanks @k21993. That cleanup hole is fixed in 7eed439.

A failed ownership read is now unconfirmed, not a confirmed create. stop_container() reads the labels again and deletes only when they match. If that read fails, the unconfirmed state stays for another retry.

Only the resource whose create was attempted can become unconfirmed. A Pod create failure does not mark the Service.

[k21993]

Thanks, reviewed 7eed439. This addresses the cleanup issue I raised.

The unconfirmed ownership state is now kept separate, stop_container() re-checks labels before deleting, and uncertainty is only tracked for the resource whose create was actually attempted.

LGTM!

[rohitrsh]

Hey @k21993, this is ready for review. The workflows are waiting for approval so CI can run.

[cursor]

Stale Bugbot comment from a previous run.

[rohitrsh]

Hey @k21993, thanks again for the LGTM on 7eed439. I merged main to clear two doc conflicts. The provider code is unchanged.

The workflows are still waiting on a maintainer to approve them so CI can run. Could you approve the workflows?

[cursor]

Stale Bugbot comment from a previous run.

[rohitrsh]

Hey @sergiopaniego, @k21993 LGTM'd the Kubernetes provider. CI is waiting on a maintainer to approve the workflows so they can run. Could you approve them?

[rohitrsh]

Hey @burtenshaw, this is the Kubernetes provider from the decisions you made on #1153 (#1153 (comment)). @k21993 LGTM'd it. CI is waiting on a maintainer to approve the workflows so they can run. Could you approve them, or point me at who can?

[rohitrsh]

Hey @burtenshaw, bumping this. CI is still waiting on a maintainer to approve the workflows. Could you approve them, or point me at who can?

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[cursor]

Stale Bugbot comment from a previous run.

[burtenshaw]

The independent readiness review is addressed in 14fbf7e: in-cluster health checks now exclude ALL_PROXY as well as HTTP/HTTPS proxies, and Kubernetes status reads use the remaining monotonic deadline with SDK retries disabled on a private configuration. HTTP polling and sleeps also respect the remaining budget; transport failures do not expose the API endpoint. The provider suite passes 65 tests, including transport-level proxy selection and bounded SDK request regressions.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 14fbf7e. Configure here.}

[burtenshaw]

Validated exact head dacf113 against a disposable Kind cluster using Kubernetes SDK 35.0.0. The provider created its Pod and Service, reached health through in-cluster DNS with a deliberately broken ALL_PROXY, and served health through a host port-forward. The child Pod had service-account token mounting disabled. Two stop calls completed successfully; readback confirmed both provider resources were deleted. The temporary cluster was also removed. Provider source SHA-256: 23ead80db8c78ee380f6051a5f75057b814d390209186b07729bf9d23e2cc4ec.

[burtenshaw]

Reviewed the one-Pod/one-Service provider and its ownership and cleanup paths, configuration validation, lazy optional dependency, in-cluster HTTP scope, and readiness handling against the agreed #1153 design. Startup interrupts retain ownership-checked cleanup; context exit preserves the original failure; proxy inheritance, transient API retries, and deadline propagation are covered.

All 72 focused tests pass with Kubernetes SDK 29.0.0 and 35.0.0, including timeout construction through the real SDK transport. The exact-head Kind smoke passed Pod/Service creation, readiness with broken ALL_PROXY, in-cluster and forwarded health, repeat-safe resource cleanup, and cluster removal. All 13 checks pass and all review threads are resolved at dacf113. The serialized merger will refresh against main and verify the resulting head again before merging.