Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 58 additions & 0 deletions clusters/unraid-lab/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# clusters/unraid-lab

Apps scoped to the Omni-managed **`unraid-lab`** Talos cluster. The repo's shared
`bootstrap/argocd-app-of-apps.yaml` only syncs `infrastructure/` (shared across
clusters); it does **not** sync `clusters/`. So apps placed here run on
`unraid-lab` only and never land on OCI.

First apps:

- **Harbor** — backing registry for the Image Factory
(`syscode-ai-internal-plans/projects/image-factory-registry`).
- **ARC runner scale sets** — self-hosted GitHub Actions runners for the
Harbor↔GHCR mirror and later homelab jobs.

## Wiring (how these apps get synced)

The `unraid-lab` cluster's inline root Argo CD Application (defined in its Omni
cluster template — see `omni-on-unraid` / the Omni machine config) must sync
**both**:

- `bootstrap/` — shared infra (Argo CD app-of-apps → `infrastructure/*`).
- `clusters/unraid-lab/app-of-apps.yaml` — this cluster's app-of-apps
(→ `clusters/unraid-lab/apps/*`).

## Prerequisites / action items

- [ ] **Argo CD + Tailscale operator on `unraid-lab`.** Bootstrap Argo CD via
Talos `cluster.inlineManifests` (as OCI does); install the Tailscale operator
(`bootstrap/tailscale-operator.yaml`) for the `tailscale` IngressClass + auto
TLS.
- [ ] Point the inline root Application at `clusters/unraid-lab/app-of-apps.yaml`
(plus `bootstrap/`).
- [ ] Set the real MagicDNS name in `apps/harbor/values.yaml`
(`externalURL` + `expose.ingress.hosts.core` → `harbor.<tailnet>.ts.net`).
- [ ] Create the `harbor-admin-password` Secret (`HARBOR_ADMIN_PASSWORD`) in the
`harbor` namespace before first sync.
- [ ] Create the `arc-gha-secret` GitHub App Secret in the `arc-runners`
namespace before syncing `arc-runners`.
- [ ] Create the `harbor-mirror-robot` Secret in the `arc-runners` namespace
with `username` and `password` keys before runner pods start.

## Harbor bootstrap (post-deploy, plan decision (b): idempotent API script)

After Harbor is up, an idempotent script (a `local-exec` or k8s Job against the
Harbor API) provisions:

- Project **`image-factory`** (private).
- **Push robot** — for the Image Factory service (`omni-on-unraid`) to push
schematics / installer / cache.
- **Read-only robot** — for the GHCR mirror job; its credential is delivered to
the ARC runner pod as the `harbor-mirror-robot` k8s Secret (see the
`talos-arc-kvm-unraid` handoff), NOT to GitHub secrets.
- **PocketID OIDC** for human/UI + `docker login` (`auth_mode = oidc_auth`).
- A retention policy on `installer` / `cache`.

> v1.1 (cross-project): mint the robot credentials on demand from
> [tessera](https://github.com/syscode-labs/tessera) (a Harbor create-then-delete
> Source) instead of static robots.
28 changes: 28 additions & 0 deletions clusters/unraid-lab/app-of-apps.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: unraid-lab-apps
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "-1"
spec:
# Cluster-scoped app-of-apps. The unraid-lab cluster's inline root Application
# syncs THIS in addition to bootstrap/ (shared infra). OCI does NOT sync this
# path, so unraid-lab-only apps (Harbor) never land on OCI.
project: default
source:
repoURL: https://github.com/syscode-labs/syscode-homelab-gitops-apps.git
targetRevision: HEAD
path: clusters/unraid-lab/apps
directory:
recurse: true
include: "*/application.yaml"
destination:
server: https://kubernetes.default.svc
namespace: argocd
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
30 changes: 30 additions & 0 deletions clusters/unraid-lab/apps/arc-controller/application.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: arc-controller
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "1"
spec:
project: default
sources:
- repoURL: ghcr.io/actions/actions-runner-controller-charts
chart: gha-runner-scale-set-controller
targetRevision: "0.14.2"
helm:
releaseName: arc
valueFiles:
- $values/clusters/unraid-lab/apps/arc-controller/values.yaml
- repoURL: https://github.com/syscode-labs/syscode-homelab-gitops-apps.git
targetRevision: HEAD
ref: values
destination:
server: https://kubernetes.default.svc
namespace: arc-systems
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
- ServerSideApply=true
8 changes: 8 additions & 0 deletions clusters/unraid-lab/apps/arc-controller/values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# ARC controller for GitHub Actions runner scale sets.
# The Helm release name is pinned to "arc" so runner scale-set values can bind
# to the stable service account: arc-gha-runner-scale-set-controller.

metrics:
controllerManagerAddr: ":8080"
listenerAddr: ":8080"
listenerEndpoint: "/metrics"
55 changes: 55 additions & 0 deletions clusters/unraid-lab/apps/arc-runners/application.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: arc-runner-scale-sets
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "3"
spec:
goTemplate: true
goTemplateOptions:
- missingkey=error
generators:
- list:
elements:
- cluster: unraid-lab
server: https://kubernetes.default.svc
namespace: arc-runners
scaleSetName: arc-runners-container
valuesPath: clusters/unraid-lab/apps/arc-runners/values.yaml
# Add OCI here once the OCI cluster is registered in this Argo CD:
# - cluster: oci-lab
# server: https://kubernetes.default.svc
# namespace: arc-runners
# scaleSetName: arc-runners-oci
# valuesPath: clusters/oci/apps/arc-runners/values.yaml
template:
metadata:
name: "{{ .cluster }}-{{ .scaleSetName }}"
namespace: argocd
labels:
syscode.dev/cluster: "{{ .cluster }}"
syscode.dev/app: arc-runners
spec:
project: default
sources:
- repoURL: ghcr.io/actions/actions-runner-controller-charts
chart: gha-runner-scale-set
targetRevision: "0.14.2"
helm:
releaseName: "{{ .scaleSetName }}"
valueFiles:
- "$values/{{ .valuesPath }}"
- repoURL: https://github.com/syscode-labs/syscode-homelab-gitops-apps.git
targetRevision: HEAD
ref: values
destination:
server: "{{ .server }}"
namespace: "{{ .namespace }}"
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
- ServerSideApply=true
49 changes: 49 additions & 0 deletions clusters/unraid-lab/apps/arc-runners/values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
githubConfigUrl: https://github.com/syscode-labs
githubConfigSecret: arc-gha-secret

runnerGroup: default
runnerScaleSetName: arc-runners-container
minRunners: 0
maxRunners: 3

scaleSetLabels:
- self-hosted
- linux
- x64
- unraid-lab
- arc-runners-container
- harbor-mirror

controllerServiceAccount:
namespace: arc-systems
name: arc-gha-runner-scale-set-controller

template:
metadata:
labels:
syscode.dev/cluster: unraid-lab
syscode.dev/runner-purpose: harbor-mirror
spec:
containers:
- name: runner
image: ghcr.io/actions/actions-runner:2.334.0
command:
- /home/runner/run.sh
env:
- name: HARBOR_ROBOT_USERNAME
valueFrom:
secretKeyRef:
name: harbor-mirror-robot
key: username
- name: HARBOR_ROBOT_PASSWORD
valueFrom:
secretKeyRef:
name: harbor-mirror-robot
key: password
resources:
requests:
cpu: 500m
memory: 1Gi
limits:
cpu: "2"
memory: 4Gi
29 changes: 29 additions & 0 deletions clusters/unraid-lab/apps/harbor/application.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: harbor
namespace: argocd
annotations:
argocd.argoproj.io/sync-wave: "2"
spec:
project: default
sources:
- repoURL: https://helm.goharbor.io
chart: harbor
targetRevision: "1.15.1" # verify against a current Harbor 2.12.x chart
helm:
valueFiles:
- $values/clusters/unraid-lab/apps/harbor/values.yaml
- repoURL: https://github.com/syscode-labs/syscode-homelab-gitops-apps.git
targetRevision: HEAD
ref: values
destination:
server: https://kubernetes.default.svc # the unraid-lab cluster's own Argo CD
namespace: harbor
syncPolicy:
automated:
prune: false # registry data is precious; no auto-prune
selfHeal: true
syncOptions:
- CreateNamespace=true
- ServerSideApply=true
52 changes: 52 additions & 0 deletions clusters/unraid-lab/apps/harbor/values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Harbor Helm values — unraid-lab. Backing registry for the Image Factory.
# See syscode-ai-internal-plans/projects/image-factory-registry.
#
# TLS/exposure decision: served on the tailnet via the Tailscale Kubernetes
# operator Ingress; Tailscale provisions the (Let's Encrypt) cert, so no private
# CA and no TrustedRootsConfig on nodes. externalURL is the MagicDNS name, stable
# across an Unraid→OCI move.

externalURL: https://harbor.<tailnet>.ts.net # set to the real ts.net MagicDNS name

expose:
type: ingress
tls:
enabled: true
certSource: none # Tailscale terminates TLS at the ingress
ingress:
hosts:
core: harbor.<tailnet>.ts.net
className: tailscale # Tailscale operator IngressClass
# annotations: {} # verify: Tailscale operator hostname annotations if needed

# Admin password comes from a pre-created Secret, not committed here.
existingSecretAdminPassword: harbor-admin-password # key: HARBOR_ADMIN_PASSWORD
existingSecretAdminPasswordKey: HARBOR_ADMIN_PASSWORD

# Chart-internal Postgres + Redis to start (single-node). NOTE: this is the
# migration-time data liability flagged in the plan — revisit external DB before
# an Unraid→OCI move.
database:
type: internal
redis:
type: internal

persistence:
enabled: true
persistentVolumeClaim:
registry:
size: 100Gi # installer/schematic/cache blobs; size for real churn
database:
size: 8Gi
redis:
size: 2Gi
jobservice:
jobLog:
size: 2Gi

# Human auth = PocketID OIDC; robots for machines. OIDC is configured POST-deploy
# (Harbor API / UI), not via chart values — see the bootstrap script in README.

# Trim to the components actually needed (no Trivy/Notary to start; add later).
trivy:
enabled: false
Loading