diff --git a/.backlog/FIX-CONFIG-NEW-LISTS/PRD.md b/.backlog/FIX-CONFIG-NEW-LISTS/PRD.md new file mode 100644 index 00000000..6163839a --- /dev/null +++ b/.backlog/FIX-CONFIG-NEW-LISTS/PRD.md @@ -0,0 +1,28 @@ +# FIX-CONFIG-NEW-LISTS — a list newer than the mission configuration is not a removal + +Status: done — 2026-10-27 (PR #275) + +## Problem + +A mission configuration (`ctld.configUser`) omits a list on purpose to remove it (ADR 0011). +But a configuration written before a catalogue version never had the lists that version added. +The Fat Cow (catalogue `2.3.0`) added `fatCowTypes`, `fatCowCrates`, `fatCowLoadouts` and `fatCowAICows`: in a `2.2.0` configuration CTLD read them as removed, and the Fat Cow was on with no aircraft, no crate and no loadout. + +Found on 2026-10-27 vendoring rc14 into VMCT: the VEAF Open Trainings' `ctld-config.yaml` date from `2.2.0`, and the `.miz` published that morning carried a `CTLD_userConfig.lua` without a single Fat Cow list. +`ctld-tools` had the same gap: its completion (ADR 0011 Addendum 2) adds scalars and entry fields, never a whole list, and saving stamps the new version, which makes the omission a removal for good. + +## Decision + +ADR 0011 Addendum 3. +The build dates every list from the pinned shapes (`ctld-tools since` → `ctld.configListSince`); the engine and `ctld-tools` give a list its default when the configuration's `configVersion` is older than the list. + +## Definition of done + +- `ctld.configListSince` generated by the build and merged into `CTLD.lua`. +- `CTLDConfig:load()` gives a newer list its default, localised, and names it in the startup notice; an omission at or after the list's version stays a removal. +- `complete()` adds such a list when `ctld-tools` opens the configuration. +- busted and pytest cover both sides; docs (mission-maker configuration EN + FR), ADR, CHANGELOG. + +## Not claimed + +Not flown: the Open Trainings were completed by hand in their own `ctld-config.yaml` on 2026-10-27 and run with that, not with this fix. diff --git a/.backlog/FIX-CONFIG-NEW-LISTS/tickets/01-date-the-lists.md b/.backlog/FIX-CONFIG-NEW-LISTS/tickets/01-date-the-lists.md new file mode 100644 index 00000000..3caf85bf --- /dev/null +++ b/.backlog/FIX-CONFIG-NEW-LISTS/tickets/01-date-the-lists.md @@ -0,0 +1,11 @@ +# 01 — date the lists, default the newer ones + +Status: done + +See the [PRD](../PRD.md). + +## Acceptance + +- A `2.2.0` snapshot without `fatCowCrates` reads the catalogue's crates, reported by `getDefaultedParameters()`. +- A `2.3.0` snapshot without it reads nothing; `aiZones` (not dated) stays removed at any version. +- `complete()` adds `fatCowCrates` to a `2.2.0` configuration, not to a `2.3.0` one. diff --git a/.backlog/README.md b/.backlog/README.md index 1bca3044..8933b1f9 100644 --- a/.backlog/README.md +++ b/.backlog/README.md @@ -14,6 +14,7 @@ authored **per lot, when the lot is started** (not in batch). | Lot | Status | Description | Branch | |-----|--------|-------------|--------| +| [`FIX-CONFIG-NEW-LISTS`](FIX-CONFIG-NEW-LISTS/PRD.md) | merged (PR #275) — 1 ticket ([date the lists](FIX-CONFIG-NEW-LISTS/tickets/01-date-the-lists.md)) | A list the catalogue gained after a mission configuration's version takes its default instead of reading as removed: the Fat Cow lists were empty on every `2.2.0` configuration (ADR 0011 Addendum 3). | `fix/config-new-lists-default` | | [`FEAT-FAT-COW`](FEAT-FAT-COW/PRD.md) | merged (PR #273) — 06 waiting-human (live check with the squadron, Zip) — 01 spike ✅ (a runtime Invisible FARP serves, enforces a finite stock, and needs only a FARP Fuel Depot as supplier), 02-05 ✅ | A CH-47 that lands in the field and becomes a forward refuel/rearm point (AI escort of a flight, or player-flown with cow crates), stocked with what it carries. | `feature/fat-cow` | | [`FIX-I18N-TRANSLATE-ALIAS`](FIX-I18N-TRANSLATE-ALIAS/PRD.md) | merged (PR #269) — 1 ticket ([JTAC through tr, guard sees the alias](FIX-I18N-TRANSLATE-ALIAS/tickets/01-jtac-through-tr-and-guard-sees-the-alias.md)) | The 16 JTAC radio keys reach the four dictionaries; the JTAC module calls ctld.tr and the generator also recognises the alias. | | [`FIX-FARP-PACK-FALSE-SUCCESS`](FIX-FARP-PACK-FALSE-SUCCESS/PRD.md) | merged (PR #268) — 1 ticket ([report the crates created](FIX-FARP-PACK-FALSE-SUCCESS/tickets/01-report-the-crates-created.md)) | Packing a FARP now reports a pack that created no crate, or only some, instead of always announcing a success. | diff --git a/.gitignore b/.gitignore index 1cc0a620..4d0152b6 100644 --- a/.gitignore +++ b/.gitignore @@ -43,6 +43,7 @@ __pycache__/ # ── Generated verbatim YAML embed (ctld.configDefault; regenerated by the build) ── /src/CTLD_config_default_yaml.lua +/src/CTLD_config_list_since.lua # ── ctld-tools web app: built frontend assets (bundled at CI, CTLD-TOOLS-WEBAPP t07) ── /tools/ctld-tools/ctld_tools/web/static/ diff --git a/CHANGELOG.md b/CHANGELOG.md index 2ee4c9b2..314d7aad 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,16 @@ Versioning follows [Semantic Versioning](https://semver.org/). ## [Unreleased] +### Fixed — a mission configuration written before the Fat Cow gets its crates and loadouts (FIX-CONFIG-NEW-LISTS) + +- A mission configuration omits a list on purpose to remove it (ADR 0011), but a configuration written before + catalogue `2.3.0` never had the Fat Cow lists: CTLD read `fatCowTypes`, `fatCowCrates`, `fatCowLoadouts` and + `fatCowAICows` as removed, and the Fat Cow was on with no aircraft, crate or loadout (seen on the VEAF Open + Trainings, whose configurations date from `2.2.0`). The build now dates every list from the pinned catalogue + shapes (`ctld.configListSince`, `ctld-tools since`): a list newer than the configuration's `configVersion` takes + its default and is named in the startup notice, and `ctld-tools` adds it when it opens the configuration. A list + omitted at or after its own version is still a removal (ADR 0011 Addendum 3). + ### Added — Fat Cow: a CH-47 that lands in the field and becomes a forward refuel and rearm point (FEAT-FAT-COW) ED announced a native Fat Cow on 2024-04-19, and again on 2024-08-23; DCS 2.9.30.28738 still has none, so CTLD brings its own while the Chinooks wait. diff --git a/dev/adr/0011-complete-yaml-config-and-webapp-tooling.md b/dev/adr/0011-complete-yaml-config-and-webapp-tooling.md index 69cb7613..8bbc774f 100644 --- a/dev/adr/0011-complete-yaml-config-and-webapp-tooling.md +++ b/dev/adr/0011-complete-yaml-config-and-webapp-tooling.md @@ -241,3 +241,39 @@ version at `2.0.0` on both sides the version-gap detection reported nothing. The - A completion only helps once an `ctld-tools.exe` that contains it is built and used: an older exe knows neither the new settings nor the completion, and its own validation cannot flag what it does not know. - Every change to the catalogue's keys or list-entry fields now costs a version increment and a snapshot file. + +## Addendum 3 — a list newer than the configuration is not a removal (2026-10-27) + +Point 1 reads an omitted list as a deliberate removal. That holds only for a list the configuration's own catalogue +already had. The Fat Cow (catalogue `2.3.0`) added `fatCowTypes`, `fatCowCrates`, `fatCowLoadouts` and +`fatCowAICows`; every mission configuration written against `2.2.0` lacks them, so the engine read them as removed +and the Fat Cow was on with no aircraft, no crate and no loadout. +Measured on the VEAF Open Trainings the day rc14 shipped: their `ctld-config.yaml` (VMCT, `2.2.0`) became +`ctld.configUser` without a single Fat Cow list. +Addendum 2's completion did not help either: it adds scalars and entry fields, never a whole list, and saving +stamps the new version, which turns the missing lists into removals for good. + +### Decision + +- **The build dates every list** from the pinned shapes (`tests/ci/data/catalogue_shapes`): `ctld-tools since` + writes `ctld.configListSince = { [list] = "" }` next to `ctld.configDefault`, for the + lists that appeared after the oldest pinned shape. +- **The engine** gives such a list its default when the snapshot omits it and its `configVersion` is older than the + list (a snapshot with no version is older than any); the list is named in the startup notice with the omitted + parameters. A list the snapshot omits at or after its version stays a removal, as does any list the table does not + name. +- **`ctld-tools`** applies the same rule when it opens a configuration (`complete()`, reading the table from the + engine it installs), so saving keeps the list instead of stamping its absence. + +### Considered options + +- **Merge every list default at runtime.** Still rejected: it resurrects deliberate removals. +- **Fix it downstream, in each tool that writes configurations (VMCT's `ctld-upgrade`).** Rejected: a hand-written or + older configuration would keep the trap, and VMCT has no shapes to tell a new list from a removed one. + +### Consequences + +- The shapes become a build input, not only a test guard: a catalogue change without its snapshot now also leaves + the new lists undated, on top of failing the guard. +- Lists already in the oldest pinned shape (`2.1.0`) cannot be dated: an older configuration that lacks one still + reads it as removed. diff --git a/docs/mission-maker/configuration.fr.md b/docs/mission-maker/configuration.fr.md index b63f2067..f656339b 100644 --- a/docs/mission-maker/configuration.fr.md +++ b/docs/mission-maker/configuration.fr.md @@ -96,6 +96,10 @@ réglages devenus inutiles, valeurs par défaut déplacées — sans jamais rien - une **liste** (une section de caisses, un groupe de troupes, une entrée de zone, un nom de pilote) que vous omettez est réellement absente. C'est ainsi qu'on en retire une — et c'est pourquoi rien n'est fusionné. + Seule exception : une liste que CTLD a ajoutée après votre `configVersion` (les caisses et + chargements de la Fat Cow sont arrivés en `2.3.0`). Vous ne pouvez pas avoir retiré ce qui + n'existait pas encore : CTLD utilise sa liste par défaut et la nomme dans le même rapport de + démarrage. Ouvrir l'instantané dans `ctld-tools` l'ajoute pour de bon. La validation de `ctld-tools` signale un réglage manquant comme une erreur et refuse d'injecter : un instantané produit par l'outil est donc toujours complet. diff --git a/docs/mission-maker/configuration.md b/docs/mission-maker/configuration.md index a4053db0..2c47bc67 100644 --- a/docs/mission-maker/configuration.md +++ b/docs/mission-maker/configuration.md @@ -90,6 +90,10 @@ used, defaults that moved — without ever merging anything behind your back. still runs and still tells you. - a **list** (a crate section, a troop group, a zone entry, a pilot name) you leave out is genuinely absent. That is how you remove one — and it is why nothing is merged. + The one exception is a list CTLD added after your `configVersion` (the Fat Cow crates and + loadouts arrived in `2.3.0`): you cannot have removed what did not exist yet, so CTLD uses its + default list and names it in the same startup report. Opening the snapshot in `ctld-tools` adds it + for good. `ctld-tools`' validation reports a missing setting as an error and refuses to inject, so a snapshot produced by the tool is always complete. diff --git a/src/CTLD_config.lua b/src/CTLD_config.lua index 94507b0a..8ec5a091 100644 --- a/src/CTLD_config.lua +++ b/src/CTLD_config.lua @@ -83,14 +83,26 @@ function CTLDConfig:load() self._defaultVersion = nil if usingUser and ctld.configDefault then self._defaults = CTLDConfig.flatten(CTLDConfig.parseYAML(ctld.configDefault)) + self._userVersion = flat.configVersion + self._defaultVersion = self._defaults.configVersion for k, v in pairs(self._defaults) do if type(v) ~= "table" and flat[k] == nil then self._defaultedParams[#self._defaultedParams + 1] = k end end + -- ADR 0011 Addendum 3 — a list the catalogue gained after the snapshot's version was not removed + -- by its author, who never saw it: it takes its default, reported like an omitted parameter. + -- A snapshot with no version predates every list in ctld.configListSince. + for k, since in pairs(ctld.configListSince or {}) do + local d = self._defaults[k] + if type(d) == "table" and flat[k] == nil + and (self._userVersion == nil or ctld.utils.compareVersions(self._userVersion, since) < 0) then + CTLDConfig.localiseI18n(d) + self.settings[k] = d + self._defaultedParams[#self._defaultedParams + 1] = k + end + end table.sort(self._defaultedParams) - self._userVersion = flat.configVersion - self._defaultVersion = self._defaults.configVersion end return true, "CTLDConfig: loaded (" .. (usingUser and "user" or "default") .. " config)." @@ -150,7 +162,7 @@ function CTLDConfig:reportRetiredSettings() end end ---- Parameters the loaded snapshot omitted, resolved from the default catalogue. +--- Parameters the loaded snapshot omitted, and lists newer than its version, resolved from the default catalogue. -- Sorted; empty when the snapshot is complete, or when the default catalogue is the winner. -- Reported once on screen at startup by CTLD_bootstrap (ADR 0011 Addendum 1). -- @return table array of setting names diff --git a/tests/ci/unit/config_param_tier_spec.lua b/tests/ci/unit/config_param_tier_spec.lua index 3159d7aa..ae8d8f90 100644 --- a/tests/ci/unit/config_param_tier_spec.lua +++ b/tests/ci/unit/config_param_tier_spec.lua @@ -79,6 +79,76 @@ describe("CTLDConfig parameter/list tiers (ADR 0011 addendum 1)", function() end) end) + -- ── ADR 0011 Addendum 3: a list newer than the snapshot was never removed by its author ── + describe("a missing list newer than the snapshot's version takes its default", function() + + local SINCE = { fatCowCrates = "2.3.0", fatCowLoadouts = "2.3.0" } + local savedSince + + before_each(function() + savedSince = ctld.configListSince + ctld.configListSince = SINCE + end) + after_each(function() + ctld.configListSince = savedSince + end) + + -- Drop `key` and its block (the `- ` items at its indent, and anything deeper). + local function withoutList(yaml, key) + local out, indent = {}, nil + for line in yaml:gmatch("[^\r\n]+") do + local lead = #line:match("^(%s*)") + if indent and (lead > indent or (lead == indent and line:match("^%s*%- "))) then + -- inside the dropped block + else + indent = nil + if line:match("^%s*" .. key .. "%s*:") then indent = lead else out[#out + 1] = line end + end + end + return table.concat(out, "\n") + end + + local function atVersion(yaml, version) + return (yaml:gsub('configVersion: "[^"]*"', 'configVersion: "' .. version .. '"', 1)) + end + + it("a 2.2.0 snapshot without fatCowCrates gets the catalogue's crates", function() + local cfg = loadUser(atVersion(withoutList(DEFAULT, "fatCowCrates"), "2.2.0")) + local crates = cfg:getSetting("fatCowCrates") + assert.is_table(crates) + assert.is_true(#crates > 0) + assert.equals(2400, crates[1].fuelKg) + end) + + it("reports the list among the settings taken from the default", function() + local cfg = loadUser(atVersion(withoutList(withoutList(DEFAULT, "fatCowCrates"), "fatCowLoadouts"), "2.2.0")) + assert.same({ "fatCowCrates", "fatCowLoadouts" }, cfg:getDefaultedParameters()) + end) + + it("a snapshot with no configVersion predates the list too", function() + local yaml = withoutList(DEFAULT, "fatCowCrates"):gsub('configVersion: "[^"]*"\n?', "", 1) + local cfg = loadUser(yaml) + assert.is_table(cfg:getSetting("fatCowCrates")) + end) + + it("a 2.3.0 snapshot without fatCowCrates removed it: it stays absent", function() + local cfg = loadUser(withoutList(DEFAULT, "fatCowCrates")) + assert.is_nil(cfg:getSetting("fatCowCrates")) + assert.same({}, cfg:getDefaultedParameters()) + end) + + it("a list kept by the snapshot is the snapshot's, even when older", function() + local yaml = atVersion(DEFAULT, "2.2.0"):gsub("desc: Fat Cow fuel tank %(800 gal%)", "desc: My tank", 1) + local cfg = loadUser(yaml) + assert.equals("My tank", cfg:getSetting("fatCowCrates")[1].desc) + end) + + it("a list the table does not name stays a removal at any version", function() + local cfg = loadUser(atVersion(without(DEFAULT, "aiZones"), "2.0.0")) + assert.is_nil(cfg:getSetting("aiZones")) + end) + end) + -- ── Ticket 03: a parameter must have exactly one default, in the catalogue ── describe("no parameter carries a duplicate default in code", function() diff --git a/tools/build/listToMerge.txt b/tools/build/listToMerge.txt index e6fa8931..b58c263f 100644 --- a/tools/build/listToMerge.txt +++ b/tools/build/listToMerge.txt @@ -8,6 +8,7 @@ CTLD_i18n_es.lua CTLD_i18n_ko.lua -- Verbatim config YAML embedded as ctld.configDefault string; parsed by CTLDConfig:load(). CTLD_config_default_yaml.lua +CTLD_config_list_since.lua CTLD_utils.lua CTLD_menu.lua core/CTLD_objectRegistry.lua diff --git a/tools/build/merge_CTLD.ps1 b/tools/build/merge_CTLD.ps1 index 5daa0e32..162e6c4d 100644 --- a/tools/build/merge_CTLD.ps1 +++ b/tools/build/merge_CTLD.ps1 @@ -85,6 +85,12 @@ Push-Location $ctldToolsDir try { & poetry run ctld-tools embed --yaml $configYaml --out $configDefaultLua --var configDefault if ($LASTEXITCODE -ne 0) { throw "embed returned $LASTEXITCODE" } + # The version each list appeared in (ctld.configListSince, ADR 0011 Addendum 3): a mission config + # older than a list gets its default instead of reading its absence as a removal. + $configListSinceLua = Join-Path $srcDir "CTLD_config_list_since.lua" + $catalogueShapes = Join-Path (Split-Path $srcDir -Parent) "tests\ci\data\catalogue_shapes" + & poetry run ctld-tools since --yaml $configYaml --shapes $catalogueShapes --out $configListSinceLua + if ($LASTEXITCODE -ne 0) { throw "since returned $LASTEXITCODE" } } catch { Write-Host "[ERROR] Could not embed CTLD_config.yaml via ctld-tools." diff --git a/tools/ctld-tools/ctld_tools/cli.py b/tools/ctld-tools/ctld_tools/cli.py index f8f7e826..27b361c2 100644 --- a/tools/ctld-tools/ctld_tools/cli.py +++ b/tools/ctld-tools/ctld_tools/cli.py @@ -89,6 +89,20 @@ def shape_cmd( typer.echo(f"shape: wrote {out}") +@app.command("since") +def since_cmd( + yaml_path: Path = typer.Option(..., "--yaml", help="path to the catalogue YAML (src/CTLD_config.yaml)"), + shapes_dir: Path = typer.Option(..., "--shapes", help="the pinned shapes (tests/ci/data/catalogue_shapes)"), + out: Path = typer.Option(..., "--out", help="path to the Lua module to write (ctld.configListSince)"), +) -> None: + """Write the catalogue version each list appeared in, as a Lua module (build step).""" + from ctld_tools.catalog import Catalog + from ctld_tools.since import write_since + + write_since(Catalog.load(yaml_path), shapes_dir, out) + typer.echo(f"since: wrote {out} (ctld.configListSince)") + + @app.command("validate") def validate_cmd( yaml_path: Path = typer.Option(..., "--yaml", help="path to a complete config YAML to validate"), diff --git a/tools/ctld-tools/ctld_tools/completion.py b/tools/ctld-tools/ctld_tools/completion.py index 695e08ff..db11eb61 100644 --- a/tools/ctld-tools/ctld_tools/completion.py +++ b/tools/ctld-tools/ctld_tools/completion.py @@ -11,13 +11,18 @@ only for entries the catalogue knows. Saving stamps the current version, so a later removal is respected. -Lists and entries that are absent altogether are never re-created. +- A **whole list** absent is a removal (ADR 0011), except a list the catalogue gained after the + configuration's version (`since`, from `ctld.configListSince`, ADR 0011 Addendum 3): nobody could + have removed it, so it is added with its default, as the engine would read it. + +Entries absent from a list are never re-created. A value the configuration already carries is never changed. """ from __future__ import annotations +import copy from dataclasses import dataclass from typing import Any @@ -46,25 +51,29 @@ def _is_scalar(value: Any) -> bool: return not isinstance(value, (dict, list, tuple)) -def complete(user: Catalog, default: Catalog) -> list[Addition]: +def complete(user: Catalog, default: Catalog, since: dict[str, str] | None = None) -> list[Addition]: """Add to `user` what `default` has and it lacks; return what was added. Scalar parameters are always added. The tier is derived from the shape of the default value (a scalar is a parameter), the same rule the engine applies, so tool and engine cannot disagree about - what an absent key means. Fields of list entries are added only when `user` predates `default`. + what an absent key means. Fields of list entries are added only when `user` predates `default`, and + a whole list only when `user` predates the version `since` gives it. """ added: list[Addition] = [] + since = since or {} + user_version = parse_version(user.get(_VERSION_KEY)) for key in default.keys(): - if key == _VERSION_KEY: + if key == _VERSION_KEY or user.has(key): continue value = default.get(key) - if not _is_scalar(value) or user.has(key): + if not _is_scalar(value) and not (key in since and user_version < parse_version(since[key])): continue section = default.section_of(key) + value = copy.deepcopy(value) user.add_setting(key, value, section=section) added.append(Addition(key, value, section)) - if parse_version(user.get(_VERSION_KEY)) < parse_version(default.get(_VERSION_KEY)): + if user_version < parse_version(default.get(_VERSION_KEY)): for f in entry_field_gap(user, default)[0]: user.get(f.container)[f.entry][f.field] = f.value added.append(Addition(f.field, f.value, user.section_of(f.container), f.container, f.entry)) diff --git a/tools/ctld-tools/ctld_tools/resources.py b/tools/ctld-tools/ctld_tools/resources.py index 981c3d03..2bea56f8 100644 --- a/tools/ctld-tools/ctld_tools/resources.py +++ b/tools/ctld-tools/ctld_tools/resources.py @@ -122,6 +122,30 @@ def read_engine() -> bytes: return path.read_bytes() +def list_since() -> dict[str, str]: + """The catalogue version each list appeared in (`ctld.configListSince`, ADR 0011 Addendum 3). + + Read from the engine the tool installs, so the tool and the engine agree; in a checkout with no built + engine, computed from the pinned shapes the build reads. + """ + import re + + path = engine_path() + if path.is_file(): + text = path.read_text(encoding="utf-8", errors="replace") + block = re.search(r"ctld\.configListSince\s*=\s*\{(.*?)\n\}", text, re.S) + if block: + return dict(re.findall(r'\["([^"]+)"\]\s*=\s*"([^"]+)"', block.group(1))) + shapes = _repo_root() / "tests" / "ci" / "data" / "catalogue_shapes" + if _frozen_dir() is None and shapes.is_dir(): + from ctld_tools.catalog import Catalog + from ctld_tools.since import list_since as compute + from ctld_tools.since import load_shapes + + return compute(Catalog.load(default_catalog_path()), load_shapes(shapes)) + return {} + + def read_sounds() -> dict[str, bytes]: """The beacon sounds, keyed by file name.""" out: dict[str, bytes] = {} diff --git a/tools/ctld-tools/ctld_tools/since.py b/tools/ctld-tools/ctld_tools/since.py new file mode 100644 index 00000000..5510de22 --- /dev/null +++ b/tools/ctld-tools/ctld_tools/since.py @@ -0,0 +1,53 @@ +"""The catalogue version each list first appeared in, read from the pinned shapes (build step). + +A list a configuration omits is an intentional removal (ADR 0011), but only when the list existed in the +catalogue that configuration was written against: nobody can have removed a list that came later. The engine +tells the two apart with this table, `ctld.configListSince`, embedded by the build next to `ctld.configDefault` +(ADR 0011 Addendum 3). + +Only the lists that appeared after the oldest pinned shape are listed: for the others nothing earlier is known, +and an omission stays a removal, as before. +""" + +from __future__ import annotations + +import json +from pathlib import Path +from typing import Any + +from ctld_tools.catalog import Catalog +from ctld_tools.versiongap import parse_version + +_VERSION_KEY = "configVersion" +_HEADER = ( + "-- GENERATED by ctld-tools (since) from tests/ci/data/catalogue_shapes. DO NOT EDIT.\n" + "-- The catalogue version each list appeared in (ADR 0011 Addendum 3).\nctld = ctld or {}\n" +) + + +def list_since(catalog: Catalog, shapes: list[dict[str, Any]]) -> dict[str, str]: + """`{list key: version}` for the lists of `catalog` that a later shape than the oldest one introduced.""" + ordered = sorted(shapes, key=lambda s: parse_version(s["version"])) + if not ordered: + return {} + since: dict[str, str] = {} + for key in catalog.keys(): + if key == _VERSION_KEY or not isinstance(catalog.get(key), (dict, list, tuple)): + continue + first = next((s["version"] for s in ordered if key in s["keys"]), None) + if first is not None and first != ordered[0]["version"]: + since[key] = first + return since + + +def load_shapes(shapes_dir: str | Path) -> list[dict[str, Any]]: + return [json.loads(p.read_text(encoding="utf-8")) for p in sorted(Path(shapes_dir).glob("*.json"))] + + +def to_lua(since: dict[str, str]) -> str: + lines = [f' ["{key}"] = "{since[key]}",' for key in sorted(since)] + return _HEADER + "ctld.configListSince = {\n" + "".join(line + "\n" for line in lines) + "}\n" + + +def write_since(catalog: Catalog, shapes_dir: str | Path, out: str | Path) -> None: + Path(out).write_text(to_lua(list_since(catalog, load_shapes(shapes_dir))), encoding="utf-8", newline="\n") diff --git a/tools/ctld-tools/ctld_tools/web/state.py b/tools/ctld-tools/ctld_tools/web/state.py index 1f6f7144..8e06c4d4 100644 --- a/tools/ctld-tools/ctld_tools/web/state.py +++ b/tools/ctld-tools/ctld_tools/web/state.py @@ -82,14 +82,14 @@ def load_path(self, path: str | Path) -> None: if found is None: raise ValueError(f"no CTLD configuration in {path.name}") self._catalog = Catalog.loads(found.yaml) - self._completion = complete(self._catalog, self.default_catalog()) + self._completion = complete(self._catalog, self.default_catalog(), resources.list_since()) self._path = None self._mission_path = path self._config_shape = found.shape self._sounds = read_sounds_from_miz(path, self._catalog) return self._catalog = Catalog.load(path) - self._completion = complete(self._catalog, self.default_catalog()) + self._completion = complete(self._catalog, self.default_catalog(), resources.list_since()) self._path = path self._mission_path = None self._config_shape = "file" diff --git a/tools/ctld-tools/tests/test_list_since.py b/tools/ctld-tools/tests/test_list_since.py new file mode 100644 index 00000000..8e13bac0 --- /dev/null +++ b/tools/ctld-tools/tests/test_list_since.py @@ -0,0 +1,91 @@ +"""A list the catalogue gained after a configuration's version is not a removal (ADR 0011 Addendum 3). + +The build writes `ctld.configListSince` from the pinned shapes; the engine and `complete()` read it, so a +mission configuration written before the Fat Cow gets its crates and loadouts instead of empty lists. +""" + +from pathlib import Path + +from typer.testing import CliRunner + +from ctld_tools import resources +from ctld_tools.catalog import Catalog +from ctld_tools.cli import app as cli_app +from ctld_tools.completion import complete +from ctld_tools.since import list_since, load_shapes, to_lua + +REPO = Path(__file__).resolve().parents[3] +SHAPES = REPO / "tests" / "ci" / "data" / "catalogue_shapes" + +DEFAULT = """\ +configVersion: "2.3.0" +mm_facing: + numberOfTroops: 10 + aiZones: [] + fatCowCrates: + - desc: tank + fuelKg: 2400 +""" + +SHAPES_FIXTURE = [ + {"version": "2.1.0", "keys": ["numberOfTroops", "aiZones"], "entryFields": []}, + {"version": "2.3.0", "keys": ["numberOfTroops", "aiZones", "fatCowCrates"], "entryFields": []}, + {"version": "2.2.0", "keys": ["numberOfTroops", "aiZones"], "entryFields": []}, +] + + +def cat(text: str) -> Catalog: + return Catalog.loads(text) + + +def test_a_list_later_than_the_oldest_shape_is_dated(): + assert list_since(cat(DEFAULT), SHAPES_FIXTURE) == {"fatCowCrates": "2.3.0"} + + +def test_the_shipped_catalogue_dates_the_fat_cow_lists(): + since = list_since(Catalog.load(REPO / "src" / "CTLD_config.yaml"), load_shapes(SHAPES)) + for key in ("fatCowCrates", "fatCowLoadouts", "fatCowTypes"): + assert since[key] == "2.3.0" + assert "aiZones" not in since + assert "fatCowEnabled" not in since # a parameter, not a list + + +def test_the_lua_module_round_trips_through_the_engine_reader(tmp_path, monkeypatch): + engine = tmp_path / "CTLD.lua" + engine.write_text("ctld = {}\n" + to_lua({"fatCowCrates": "2.3.0", "fatCowTypes": "2.3.0"}), encoding="utf-8") + monkeypatch.setattr(resources, "engine_path", lambda: engine) + assert resources.list_since() == {"fatCowCrates": "2.3.0", "fatCowTypes": "2.3.0"} + + +def test_the_cli_writes_the_module(tmp_path): + out = tmp_path / "since.lua" + result = CliRunner().invoke( + cli_app, ["since", "--yaml", str(REPO / "src" / "CTLD_config.yaml"), "--shapes", str(SHAPES), "--out", str(out)] + ) + assert result.exit_code == 0, result.output + assert '["fatCowLoadouts"] = "2.3.0",' in out.read_text(encoding="utf-8") + + +def test_complete_adds_a_list_newer_than_the_configuration(): + user = cat('configVersion: "2.2.0"\nmm_facing:\n numberOfTroops: 10\n aiZones: []\n') + added = complete(user, cat(DEFAULT), {"fatCowCrates": "2.3.0"}) + assert [a.key for a in added] == ["fatCowCrates"] + assert user.get("fatCowCrates")[0]["fuelKg"] == 2400 + + +def test_complete_leaves_a_list_removed_at_its_own_version(): + user = cat('configVersion: "2.3.0"\nmm_facing:\n numberOfTroops: 10\n aiZones: []\n') + assert complete(user, cat(DEFAULT), {"fatCowCrates": "2.3.0"}) == [] + assert not user.has("fatCowCrates") + + +def test_complete_leaves_an_undated_list_removed_at_any_version(): + user = cat('configVersion: "2.0.0"\nmm_facing:\n numberOfTroops: 10\n') + added = complete(user, cat(DEFAULT), {"fatCowCrates": "2.3.0"}) + assert [a.key for a in added] == ["fatCowCrates"] + assert not user.has("aiZones") + + +def test_complete_without_the_table_adds_no_list(): + user = cat('configVersion: "2.2.0"\nmm_facing:\n numberOfTroops: 10\n') + assert complete(user, cat(DEFAULT)) == []