sync to refactored break - #2
Open
brownzebra wants to merge 2451 commits into
Open
Conversation
…ath-storage-class small-openbao-init-needs-local-path-storage-class
…ues-small-yaml small-argocd-challenges-with-values-small-yaml
…ues-small-yaml small to contain github link directly
* update migration manual * update httproute for otel * add httproute for lognhorn in migration doc
fix: remove duplicate aim
Leftovers from kgateway -> envoy migration
…n-amd-gpu-operator-config EAI-6555 Use Helm .Release.Namespace instead of extra valuesObject
fix(tls): Reference correct namespace
Lines 98, 106, 113, 124 had hardcoded 9100 while the Service (lines 37-38) already used .Values.services.nodeExporter.metrics. This caused CrashLoopBackOff on clusters where AMD fleet-observability or a host systemd node_exporter pre-occupies port 9100 (rck-g03, workload-dev tw016). The cluster-values port override had no effect on the DaemonSet. Default in values.yaml remains 9100 — no change to other clusters. Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
Convert sources/aim-cluster-model-source from an ArgoCD directory app into a Helm chart that renders either the legacy generic model sources (default, when hardwareFamilies is empty) or per-hardware-family AIMClusterModelSource resources (cpu, epyc, instinct, radeon). The legacy branch reproduces the existing amd-aim-release-* resources unchanged so ArgoCD does not prune or recreate existing installs. The app's hardwareFamilies value is supplied as a structured YAML list via valuesObject (cluster-bloom injects the selected families at deploy time), so no comma parsing is involved on any hop. The base default is an empty list, preserving legacy behavior. Part of EAI-6030.
…-workflow-logic-in-cluster-forge-to-reduce-confusion EAI-1500 update Github Workflows documentation
For medium/large clusters ArgoCD reads cluster-values/values.yaml, which the gitea-init-job rebuilds from a template rather than copying the seeded complete_values.yaml wholesale. Add an aimHardwareFamily value and emit the apps.aim-cluster-model-source.valuesObject.hardwareFamilies block into cluster-values when set, mirroring the existing airmImageRepository handling. Without it the chart fell back to the legacy install-all branch on medium/large. Part of EAI-6030.
The upstream Helm chart uses replicas: ~ (YAML null) as the default for all four Kyverno controllers. The chart's null-replica guard in _deployment.tpl only checks kindIs "invalid", which null passes — causing Kubernetes to coerce the empty rendered value to 0 replicas. This silently breaks the admission controller webhook, meaning generate policies (e.g. dynamic-pvc-creation) never fire, leaving workspace pods stuck in Pending due to missing PVCs. Set replicas: 1 explicitly for admissionController, backgroundController, cleanupController, and reportsController so all cluster-forge installs get a working Kyverno from day one. Clusters needing HA can override via their cluster-values. Upstream bug: kyverno/kyverno#8941, #6182 Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
Parameterizes the amd-gpu-operator-config DeviceConfig out-of-tree ROCm driver version by GPU family (radeon | instinct) via a gpuStackFamily / driverVersion value and a gpuStack.driverVersion helper. Empty resolves to instinct (7.0), so existing installs are unchanged; radeon resolves to the ROCm 7.13 tech-preview pin. The GPU Operator chart version is selected separately through the app-level apps.amd-gpu-operator.path field. cluster-bloom injects both the path and the config valuesObject: small clusters via --set on the helm render, medium/large via the gitea-init-job, which now emits the amd-gpu-operator path and amd-gpu-operator-config valuesObject into cluster-values. Radeon driver version is a placeholder pending EAI-5906. Part of EAI-6030.
Split the previously unversioned amd-gpu-operator-config chart into v1.4.1 (the existing DeviceConfig example, default) and v1.5.1-beta.0 (the new-schema DeviceConfig for the radeon tech-preview stack). The beta DeviceConfig resolves its ROCm driver version through the existing gpuStack.driverVersion helper, so radeon gets 7.13 and the default stays 7.0. root/values.yaml defaults the config app path to amd-gpu-operator-config/v1.4.1; the beta chart is selected only when cluster-bloom injects apps.amd-gpu-operator-config.path for GPU_STACK_FAMILY=radeon. The gitea-init-job now emits that path for medium/large clusters. Part of EAI-6030.
Read the imageRegistrySecret name from values.yaml instead of hardcoding dockerhub-amdpsdo-regcred across the four DeviceConfig component sections. Default keeps the existing secret name. Part of EAI-6030.
Guard each imageRegistrySecret block with the resolved name so an empty or absent imageRegistrySecret.name renders no key at all, instead of emitting an empty name that produces invalid DeviceConfig YAML. Part of EAI-6030.
EAI-8232 Stop using PreSync hooks for seaweedfs ExternalSecrets
The ai-gateway mutating webhook took its serving cert from the chart's built-in
Helm genCA path, which upstream's own values.yaml describes as "not recommended
for production use". On int-test the CA advertised in the
MutatingWebhookConfiguration stopped matching the cert the controller served, and
pod admission began failing with:
x509: certificate signed by unknown authority ("ai-gateway-controller-ca")
Because the webhook is failurePolicy: Fail and selects envoy-gateway managed
pods, the ai-gateway proxy Deployment could no longer create pods. It sat at
UP-TO-DATE 0 for 14h, still serving from a ReplicaSet created during cluster
bring-up whose Envoy configuration was stale, and every inference request
returned an empty 500 regardless of whether a valid key, an invalid key, or no
key was supplied. Recovery required deleting the webhook by hand so the
Deployment could roll.
Exactly how the CA and the served cert diverged is not established; the failure
mode is what matters here. The genCA path leaves two copies of the trust
material that must agree - the caBundle written into the webhook object, and the
cert in the Secret the controller mounts - with nothing keeping them consistent
once either is rewritten.
Enable the chart's cert-manager path instead. The Certificate and Issuer are
declarative, and ca-injector maintains the caBundle from the issued cert, so the
two cannot drift apart. Both paths populate the same Secret the controller
mounts (self-signed-cert-for-mutating-webhook), and cert-manager is already
installed at syncWave -40, ahead of this app at -5.
Renew AIWB OpenBao token instead of rotating it
cluster-auth no longer authenticates anything. The AIM serving routes are covered by the route-scoped workloads-extauth policy and ai-gateway by its gateway-scoped default-deny, and neither the airm nor the aiwb chart has a live reference to it. On app-dev its own counters show 230k checks over 15 days, all fast_path with authenticator=none and zero denies. - drop the cluster-auth and cluster-auth-config apps from root/values.yaml and from enabledApps in all three sizings, plus the medium resources override - delete the cluster-auth (0.5.0, 0.5.9) and cluster-auth-config chart sources - delete the cluster-auth ext_authz SecurityPolicy and its ReferenceGrant - stop writing the OpenBao root token to secrets/cluster-auth-openbao-token, and drop the now-inert INIT_MODE export that only drove that write - drop the security-policy-extauth patch block from the OpenShift installer, which would have aborted the install under set -e once the template is gone - refresh the SBOM and the stale comments that referenced the ext_authz hop
[EAI-7305]: Remove cluster-auth from the stack
Openshift custom values feature
Issue the ai-gateway webhook cert via cert-manager
Rebase onto main after #824 (cert-manager webhook TLS). Drop chart template caBundle prevention; keep operator heal path, post-handoff platform gates, CI install-script checks, and OpenShift install wiring. Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(extra-apps): Add blueprint for vllm-sr Example blueprint for adding vllm-sr as an extra app. The config for it is just a placeholder for demonstrating the idea. The user is required to just use these as reference and implement these in the per cluster cluster-values repo. * redo docs * fix(dashboard): update dashboard docs and router access * feat: retries and timeouts * more retry docs
feat(EAI-8292): webhook heal script and platform gates (complements #824)
Explain the Gitea and Argo CD workflow for adding, validating, replacing, and removing additional model sources, and index repository documentation. Co-authored-by: Cursor <cursoragent@cursor.com>
Align filename with underscore convention used by other docs/. Co-authored-by: Cursor <cursoragent@cursor.com>
Expand aim_model_management.md with packaged baseline, base catalog, and version policy. Add adding_aim_catalog_models.md as a streamlined admin procedure. Update README links. Co-authored-by: Cursor <cursoragent@cursor.com>
Clarify aim_model_management.md is customer-facing and not tied to release cadence; rename inter-release wording to cluster-managed additions. Co-authored-by: Cursor <cursoragent@cursor.com>
Replace private repo references in customer-facing docs with the public amd-enterprise-ai/aim-build repository and remove private-repo path details. Co-authored-by: Cursor <cursoragent@cursor.com>
…rdwareFamilies helm value
…urce docs: document inter-release AIM catalog management
Cluster-auth has been removed from the stack. Replaced with envoy SecurityPolicy in the blueprint.
sanity check to get script if missing from release
fix(blueprint): vllm-sr remove cluster-auth references
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
15 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.