Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ test/e2e/ Kind E2E suite (//go:build e2e)
docs/ architecture, security, modules; layout.md (target folder shape per module); archive/ (read-only old designs)
design.pen canonical Pencil dashboard design
signing/ cosign public keys, dated legacy key + cross-signature (image + module)
.devcontainer/ dev container profiles, generated by hack/gen-devcontainers.py from versions.toml, profiles.toml, devcontainer-lock.json (full profile's feature lock), setup.sh (shared setup script, copied into the submodule copies) and each folder's system-deps.toml
go.work links all 17 Go modules (incl. test/e2e)
Makefile canonical task runner
```
Expand All @@ -72,6 +73,7 @@ After cloning: `git submodule update --init modules` (required for `make dev-up`
- **Dev:** `make dev-up` (Kind + OCI registry :5001 + Helm), `make web-dev` (Vite, proxies in-cluster API), `make dev-load` (load built images; run `make images` first), `make dev-install` (Helm upgrade), `make dev-down`.
- **Build:** `make build` (all Go + web), `make build-go` (15 modules in GO_MODULES, all but test/e2e), `make build-web` (`npm ci && npm run build`), `make images`.
- **Codegen:** `make generate` (deepcopy), `make manifests` (CRD/RBAC YAML, synced to `charts/gameplane/charts/gameplane-crds/crds/` and `charts/gameplane/crd-manifests/`), `make modules-push`, `make tidy`.
- **Dev containers:** `make devcontainers` (regenerate every profile and the `website/` and `modules/` copies after editing `.devcontainer/versions.toml`, `.devcontainer/profiles.toml`, `.devcontainer/devcontainer-lock.json`, `.devcontainer/setup.sh` or a `system-deps.toml`), `make check-devcontainers` (drift check). Profile table: `docs/contributing.md`.
- **Tests (CI only — never run locally, see Rule 8):** `make test`, `make test-integration` (envtest operator+api), `make test-e2e` (~10–20 min), `make test-e2e-bucket BUCKET=(operator|api-auth|api-roles|api-rbac|api-agent|api-mods|ratelimit|bot-fast|bot-heavy|multicluster|upgrade)`.
- **E2E conventions:** register new tests in `test/e2e/buckets.sh`; `t.Parallel()` + unique names; guard shared resources (`ociPushMu` for module pushes, `ensureResticRepo(t)` for shared backup repos). Rate limits per cluster: IP burst 10 (5/min), user burst 6 (3/min) — cap an API bucket at ~7 admin logins.
- **Lint:** `make lint` = gofmt, go vet, golangci-lint, ESLint, `check-specs`, `check-doc-versions`, `check-links`.
Expand Down
43 changes: 43 additions & 0 deletions docs/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,49 @@ make dev-up # creates kind cluster, builds images, installs Helm chart
make web-dev # starts the Vite dev server with proxy to in-cluster API
```

## Dev container profiles

The repo ships one dev container per area, so you only install what your change needs. Every profile is generated by `hack/gen-devcontainers.py` from these inputs: `.devcontainer/versions.toml` (every tool version, written once), `.devcontainer/profiles.toml` (the profile catalogue), `.devcontainer/devcontainer-lock.json` (the full profile's feature lock), `.devcontainer/setup.sh` (the shared setup script, copied into the `website/` and `modules/` copies) and the `system-deps.toml` file in each component folder (what that folder needs). Never edit the generated `devcontainer.json`, `setup.env`, `ready.txt` or per-profile lock files by hand.

| Profile | Size | Scope | What it installs | Setup time in CI (amd64 / arm64) | Use it for |
| --- | --- | --- | --- | --- | --- |
| `web` | component | `web/` | Node, `npm ci` in `web/` | 3.1 / 2.7 min | Dashboard code with mocked data |
| `api` | component | `api/`, `svcutil/` and their Go libraries | Go, golangci-lint, envtest | 5.1 / 4.7 min | API handlers, auth, DB |
| `operator` | component | `operator/` and its Go libraries | Go, golangci-lint, envtest | 6.9 / 4.4 min | Reconcilers and CRD types |
| `agent` | component | `agent/` and its Go libraries | Go, golangci-lint | 5.4 / 3.8 min | The in-pod sidecar |
| `go-services` | component | sentinel, tunnel, gp-module, mcp-server, telemetry-receiver, audit-syslog-bridge, capture-sidecar, gameproto and their Go libraries | Go, golangci-lint | 7.3 / 5.0 min | The smaller Go services |
| `website` | component | `website/` submodule | Node, `npm ci` in `website/` | 2.4 / 2.3 min | Docs and marketing site |
| `modules` | component | `modules/` submodule | Go, Python with PyYAML, oras | 3.0 / 2.7 min | Game module templates |
| `dashboard-api` | part | web + api + operator + agent, kind, chart, `modules` submodule | Docker, kind, kubectl, Helm, Go, Node, Python with PyYAML, oras | 9.1 / 7.4 min | Dashboard work against a live local cluster |
| `control-plane` | part | operator + api + agent, kind, chart, e2e, `modules` submodule | Docker, kind, kubectl, Helm, Go, Python with PyYAML, oras | 7.1 / 5.9 min | Backend work with a live cluster, e2e code |
| `modules-gp` | part | modules + `gp-module/` | Go, Python with PyYAML, oras | 5.0 / 4.2 min | Module work with the `gp-module` CLI |
| `full` | full | everything | everything above | 11.1 / 8.4 min | Cross-cutting changes; the default. Leaves out the website's Astro port 4321 and extension (open `website` for those) |

Setup times are the cold "build profile and run its checks" step of the `dev containers` workflow on the PR that added the profiles. They include the profile's own compile checks. Every profile also gets git, GitHub CLI, uv, Spec Kit and the AI coding CLIs (not logged in).

Pick a profile:

- **VS Code:** run "Dev Containers: Reopen in Container" and choose the profile by its title, for example `Gameplane · Web dashboard`, `Gameplane · Dashboard and API` or `Gameplane · Full`.
- **Codespaces:** "New with options", then pick the profile under "Dev container configuration". The default is the full profile.
- **CLI:** `devcontainer up --workspace-folder . --config .devcontainer/<name>/devcontainer.json` (the full profile is `.devcontainer/devcontainer.json`).

When setup finishes, the terminal prints the profile's `ready.txt`: the commands that work in it and its forwarded ports.

Add a component or change a version:

1. Edit the inputs: a version in `.devcontainer/versions.toml`, a profile in `.devcontainer/profiles.toml`, a folder's `system-deps.toml` (a new component folder needs one), the full profile's feature lock in `.devcontainer/devcontainer-lock.json`, or the shared `.devcontainer/setup.sh`.
2. Run `make devcontainers`. It rewrites the generated files here and the copies in `website/` and `modules/`.
3. Commit the inputs and the generated files together. CI's `dev containers / drift` job fails if they don't match (`make check-devcontainers` runs the same check). A changed submodule copy goes to that repo's own PR first, then a pointer bump here.
Comment thread
ValgulNecron marked this conversation as resolved.
Resolved

**`need:` messages.** Make targets that need a tool the current profile doesn't install stop early, for example:

```text
need: kind is not installed in this dev container.
Open the "dashboard-api" or "control-plane" profile (declared in deploy/kind/system-deps.toml).
```

Reopen the container with one of the named profiles (or `full`) and run the target again.

## Design first for UI changes

**Any change to the web dashboard's visual surface starts in
Expand Down
16 changes: 15 additions & 1 deletion specs/026-modular-dev-containers/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Pick `Gameplane · dashboard + API`, run `make dev-up` and then `make web-dev`.

## 4. Full profile parity (US3)

Compare the generated `.devcontainer/devcontainer.json` against `git show 8a67f2e:.devcontainer/devcontainer.json`. Expected: same features, ports, extensions, settings and env. The only differences are the `postCreateCommand` path and the three OD-5 alignments (kubectl 1.36, setup-envtest `release-0.24`, pinned helm). Open Codespaces with default options. Expected: the full profile is used.
Compare the generated `.devcontainer/devcontainer.json` against `git show 8a67f2e:.devcontainer/devcontainer.json`. Expected: same features, ports, extensions, settings and env. The only differences are the generated `name` (`Gameplane · <title>`, T006), the `postCreateCommand` path and the three OD-5 alignments (kubectl 1.36, setup-envtest `release-0.24`, pinned helm). Open Codespaces with default options. Expected: the full profile is used.

## 5. CI build matrix (FR-014, SC-001, SC-002, SC-004)

Expand All @@ -37,3 +37,17 @@ Clone `GameplanePanel/website` on its own and open it in a dev container. Expect
## 7. Docs (US4)

Give `docs/contributing.md` to a new reader with three example changes: a dashboard button, a new Valheim setting, a website typo. Expected picks: `dashboard-api` (or `web` if no live data is needed), `modules`, `website`.

## Results

Recorded on 2026-10-29 against master `b07e4d4f` (PR #809, which also carries PR #810). Evidence is from CI runs; steps that need an interactive editor or a person are marked as not run.

| Section | Result | Evidence |
| --- | --- | --- |
| §1 Drift and format | Pass (CI) | The `drift` job runs `--check` and `make test-gen-devcontainers` (fixture tests for the `system-deps.toml` rules, including literal versions). It caught real drift on PR #809 run 40607607262: after master moved to Go 1.27.3, it failed naming `modules/.devcontainer/devcontainer.json` until module PR #55 merged. The manual edit-and-revert in §1 was not run. |
| §2 Component profile | Pass (CI build); interactive steps not run | Every component profile built on amd64 and arm64 and ran its checks (`devcontainers` run 40609123253). The VS Code walk-through (`npx tsc`, `command -v`, `make dev-up` printing `need:`) was not run in an editor. `hack/need.sh kind` outside a dev container prints the `need:` message naming `dashboard-api` or `control-plane` and exits 127. |
| §3 Part profile | Pass (CI build); live dashboard login not run | `dashboard-api`, `control-plane` and `modules-gp` built and ran their checks on both architectures. `make dev-up` + `make web-dev` + login was not run. |
| §4 Full profile parity | Pass | The generated `.devcontainer/devcontainer.json` matches the pre-026 file except the name, `postCreateCommand`, and the OD-5 alignments (kubectl 1.36, helm 4.3.1, setup-envtest `release-0.24`). Setup also installs `python3` (the base image has none), which the old script assumed. Go 1.27.3 and golangci-lint v2.14.0 come from master (#765) through `versions.toml`. |
Comment thread
ValgulNecron marked this conversation as resolved.
Resolved
| §5 CI build matrix | Builds pass; size target partly met | All 22 profile × architecture builds green. Setup time (build plus checks, minutes, amd64 / arm64): web 3.1 / 2.7, website 2.4 / 2.3, modules 3.0 / 2.7, agent 5.4 / 3.8, api 5.1 / 4.7, operator 6.9 / 4.4, go-services 7.3 / 5.0, modules-gp 5.0 / 4.2, control-plane 7.1 / 5.9, dashboard-api 9.1 / 7.4, full 11.1 / 8.4. Against "component at or under half of full" (5.6 min amd64, 4.2 min arm64): web, website, modules and agent meet it on both; api meets it on amd64 only; operator and go-services do not, because their checks compile every Go module in scope. Disk use is in each job's step summary, which the API does not expose; it was not copied here. |
| §6 Submodule repos | Pass (CI) | The website (PR #26, #27) and module (PR #53, #54, #55) repos each build their own dev container in CI and run `npm run check` / the validate syntax check. The main-repo `--check` fails on a stale submodule copy (see §1). |
| §7 Docs | Not run | Needs a new reader. The table in `docs/contributing.md` maps the three examples to `dashboard-api` or `web`, `modules`, and `website`. |
2 changes: 1 addition & 1 deletion specs/026-modular-dev-containers/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ A contributor (or a maintainer setting up a machine without dev containers) open
### Functional Requirements

- **FR-001**: The repository MUST offer several named dev container profiles that a dev-container-capable editor lists for the contributor to pick from when opening the repository.
- **FR-002**: Profiles MUST come in three sizes: component profiles (one component each), part profiles (a named group of components worked on together) and one full profile. The initial catalogue (OD-1) is: component profiles for web, api, operator, agent, website and modules, plus one "Go services and libraries" profile covering every other Go module in the workspace; part profiles "dashboard + API" (web, api, the shared libraries they use, and a local Kubernetes cluster per OD-2), "control plane" (operator, api, agent, the Helm chart, a local Kubernetes cluster and the end-to-end test suite) and "modules + gp-module" (game modules and the module authoring tool); and the full profile.
- **FR-002**: Profiles MUST come in three sizes: component profiles (one component each), part profiles (a named group of components worked on together) and one full profile. The initial catalogue (OD-1) is: component profiles for web, api, operator, agent, website and modules, plus one "Go services and libraries" profile covering every other Go module in the workspace; part profiles "dashboard + API" (web, api, the shared libraries they use, and a local Kubernetes cluster per OD-2, plus the operator and agent that cluster runs so the dashboard has servers to show), "control plane" (operator, api, agent, the Helm chart, a local Kubernetes cluster and the end-to-end test suite) and "modules + gp-module" (game modules and the module authoring tool); and the full profile.
- **FR-003**: The full profile MUST include everything today's dev container includes: every tool and version, forwarded port, editor extension, editor setting, environment variable and setup step. The only allowed version changes are the three drift fixes from OD-5: kubectl 1.31 → 1.36, `setup-envtest@latest` → `@release-0.24`, and helm `latest` → a pinned version. It MUST be the profile used when no profile is picked.
- **FR-004**: Each component and part profile MUST install only the tools, dependencies, submodules, editor extensions and forwarded ports its scope needs, as given by its components' dependency lists and the profile catalogue.
- **FR-004a**: Every component in the profile catalogue MUST list its required system dependencies in its own folder, in one documented location and format used by all components. Each entry names the dependency, its kind (toolchain, command-line tool, OS package, service or submodule), the version it needs (taken from the version source) and a one-line reason. A component with no needs beyond the shared baseline MUST still have a list that says so.
Expand Down
14 changes: 7 additions & 7 deletions specs/026-modular-dev-containers/tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,8 +124,8 @@ description: "Task list for spec 026 modular dev containers"

**Independent Test**: quickstart §6.

- [ ] T024 [P] [US1] In `GameplanePanel/website` (branch `claude/project-thread-427u3v`), add `system-deps.toml` from the T001 brief.
- [ ] T025 [P] [US1] In `GameplanePanel/module` (branch `claude/project-thread-427u3v`), add `system-deps.toml` from the T001 brief.
- [X] T024 [P] [US1] In `GameplanePanel/website` (branch `claude/project-thread-427u3v`), add `system-deps.toml` from the T001 brief. Done in website PR #26.
- [X] T025 [P] [US1] In `GameplanePanel/module` (branch `claude/project-thread-427u3v`), add `system-deps.toml` from the T001 brief. Done in module PR #53.
- [X] T026 [US1] From the main repo, run `make devcontainers` (with `--submodules`) to write `website/.devcontainer/*` and `modules/.devcontainer/*` (devcontainer.json, lock, a copied `setup.sh`, setup.env, ready.txt), and commit them in each submodule repo.
- [X] T027 [P] [US5] Add a dev container build job to `website/.github/workflows/ci.yaml` (amd64 and arm64, `devcontainers/ci` SHA-pinned, runs `npm run check`).
- [X] T028 [P] [US5] Add a dev container build job to the module repo's CI workflow (amd64 and arm64, runs the validate command from T001).
Expand All @@ -139,17 +139,17 @@ description: "Task list for spec 026 modular dev containers"

**Independent Test**: quickstart §7.

- [ ] T030 [P] [US4] Add a "Dev container profiles" section to `docs/contributing.md`: a table (name, size, scope, what it installs, typical setup time from the PR-B job summary, when to use it), how to pick one in VS Code, Codespaces and the CLI, how to add a component or change a version (edit inputs, then `make devcontainers`), and what a `need:` message means.
- [ ] T031 [P] [US4] Update `docs/dependencies.md` so the devcontainer pin lines point to `.devcontainer/versions.toml`, and drop the stale "built from source" note if T001 shows it is out of date.
- [ ] T032 [P] [US4] Update `CLAUDE.md` "Repository map" (`.devcontainer/` line, `system-deps.toml` mention) and "Commands" (`make devcontainers`, `make check-devcontainers`).
- [X] T030 [P] [US4] Add a "Dev container profiles" section to `docs/contributing.md`: a table (name, size, scope, what it installs, typical setup time from the PR-B job summary, when to use it), how to pick one in VS Code, Codespaces and the CLI, how to add a component or change a version (edit inputs, then `make devcontainers`), and what a `need:` message means.
- [X] T031 [P] [US4] Update `docs/dependencies.md` so the devcontainer pin lines point to `.devcontainer/versions.toml`, and drop the stale "built from source" note if T001 shows it is out of date. Done with T012 in PR #809: every pin line names `versions.toml`; the "built from source" note is still true (`setup.sh` runs `go install` for golangci-lint).
- [X] T032 [P] [US4] Update `CLAUDE.md` "Repository map" (`.devcontainer/` line, `system-deps.toml` mention) and "Commands" (`make devcontainers`, `make check-devcontainers`).
- [ ] T033 [US4] In `GameplanePanel/website`, edit `src/content/docs/local-development-setup.mdx`: replace the `.devcontainer/post-create.sh` reference with the profile list and a link to the `docs/contributing.md` table. Text only, no new visual component (Principle II exempt, plan). Open a website PR, then bump the pointer after it merges.

---

## Phase 10: Polish & cross-cutting (PR-D)

- [ ] T034 Remove any leftover reference to `post-create.sh` repo-wide (`grep -rn post-create`), excluding `specs/done_*`.
- [ ] T035 Run quickstart §1–§7 against the merged state and record the results (times, disk, pass/fail) in `specs/026-modular-dev-containers/quickstart.md` under a "Results" heading.
- [X] T034 Remove any leftover reference to `post-create.sh` repo-wide (`grep -rn post-create`), excluding `specs/done_*`. Checked 2026-10-29: no `post-create.sh` reference remains outside this spec folder (the website page never named it after website PR #25).
- [X] T035 Run quickstart §1–§7 against the merged state and record the results (times, disk, pass/fail) in `specs/026-modular-dev-containers/quickstart.md` under a "Results" heading. CI results recorded 2026-10-29. The interactive steps in §1–§3 and §7 and the disk numbers were not run; valgul accepted the CI evidence on 2026-10-29 and chose to archive without them.
- [ ] T036 Archive once every task is done and the last PR is merged: `git mv specs/026-modular-dev-containers specs/done_026-modular-dev-containers` and update in-repo references in the same commit (CLAUDE.md rule 16).

---
Expand Down
Loading