From 2b62af4f300af6cc22ea0bf92adc688a181f6027 Mon Sep 17 00:00:00 2001 From: David Pierron Date: Tue, 27 Oct 2026 19:21:41 +0100 Subject: [PATCH 1/2] fix(config): a list newer than the mission configuration takes its default (FIX-CONFIG-NEW-LISTS) A configuration omits a list to remove it (ADR 0011), but one written before catalogue 2.3.0 never had the Fat Cow lists: CTLD read them as removed and the Fat Cow had no aircraft, crate or loadout (VEAF Open Trainings, configs at 2.2.0). The build dates every list from the pinned shapes (ctld-tools since -> ctld.configListSince); the engine and ctld-tools complete() give a list newer than the configuration's version its default. ADR 0011 Addendum 3. Co-Authored-By: Claude Opus 5.5 --- .backlog/FIX-CONFIG-NEW-LISTS/PRD.md | 28 ++++++ .../tickets/01-date-the-lists.md | 11 +++ .backlog/README.md | 1 + .gitignore | 1 + CHANGELOG.md | 10 ++ ...complete-yaml-config-and-webapp-tooling.md | 36 ++++++++ docs/mission-maker/configuration.fr.md | 4 + docs/mission-maker/configuration.md | 4 + src/CTLD_config.lua | 18 +++- tests/ci/unit/config_param_tier_spec.lua | 70 ++++++++++++++ tools/build/listToMerge.txt | 1 + tools/build/merge_CTLD.ps1 | 6 ++ tools/ctld-tools/ctld_tools/cli.py | 14 +++ tools/ctld-tools/ctld_tools/completion.py | 21 +++-- tools/ctld-tools/ctld_tools/resources.py | 24 +++++ tools/ctld-tools/ctld_tools/since.py | 53 +++++++++++ tools/ctld-tools/ctld_tools/web/state.py | 4 +- tools/ctld-tools/tests/test_list_since.py | 91 +++++++++++++++++++ 18 files changed, 386 insertions(+), 11 deletions(-) create mode 100644 .backlog/FIX-CONFIG-NEW-LISTS/PRD.md create mode 100644 .backlog/FIX-CONFIG-NEW-LISTS/tickets/01-date-the-lists.md create mode 100644 tools/ctld-tools/ctld_tools/since.py create mode 100644 tools/ctld-tools/tests/test_list_since.py diff --git a/.backlog/FIX-CONFIG-NEW-LISTS/PRD.md b/.backlog/FIX-CONFIG-NEW-LISTS/PRD.md new file mode 100644 index 00000000..e4024ad6 --- /dev/null +++ b/.backlog/FIX-CONFIG-NEW-LISTS/PRD.md @@ -0,0 +1,28 @@ +# FIX-CONFIG-NEW-LISTS — a list newer than the mission configuration is not a removal + +Status: done + +## Problem + +A mission configuration (`ctld.configUser`) omits a list on purpose to remove it (ADR 0011). +But a configuration written before a catalogue version never had the lists that version added. +The Fat Cow (catalogue `2.3.0`) added `fatCowTypes`, `fatCowCrates`, `fatCowLoadouts` and `fatCowAICows`: in a `2.2.0` configuration CTLD read them as removed, and the Fat Cow was on with no aircraft, no crate and no loadout. + +Found on 2026-10-27 vendoring rc14 into VMCT: the VEAF Open Trainings' `ctld-config.yaml` date from `2.2.0`, and the `.miz` published that morning carried a `CTLD_userConfig.lua` without a single Fat Cow list. +`ctld-tools` had the same gap: its completion (ADR 0011 Addendum 2) adds scalars and entry fields, never a whole list, and saving stamps the new version, which makes the omission a removal for good. + +## Decision + +ADR 0011 Addendum 3. +The build dates every list from the pinned shapes (`ctld-tools since` → `ctld.configListSince`); the engine and `ctld-tools` give a list its default when the configuration's `configVersion` is older than the list. + +## Definition of done + +- `ctld.configListSince` generated by the build and merged into `CTLD.lua`. +- `CTLDConfig:load()` gives a newer list its default, localised, and names it in the startup notice; an omission at or after the list's version stays a removal. +- `complete()` adds such a list when `ctld-tools` opens the configuration. +- busted and pytest cover both sides; docs (mission-maker configuration EN + FR), ADR, CHANGELOG. + +## Not claimed + +Not flown: the Open Trainings were completed by hand in their own `ctld-config.yaml` on 2026-10-27 and run with that, not with this fix. diff --git a/.backlog/FIX-CONFIG-NEW-LISTS/tickets/01-date-the-lists.md b/.backlog/FIX-CONFIG-NEW-LISTS/tickets/01-date-the-lists.md new file mode 100644 index 00000000..3caf85bf --- /dev/null +++ b/.backlog/FIX-CONFIG-NEW-LISTS/tickets/01-date-the-lists.md @@ -0,0 +1,11 @@ +# 01 — date the lists, default the newer ones + +Status: done + +See the [PRD](../PRD.md). + +## Acceptance + +- A `2.2.0` snapshot without `fatCowCrates` reads the catalogue's crates, reported by `getDefaultedParameters()`. +- A `2.3.0` snapshot without it reads nothing; `aiZones` (not dated) stays removed at any version. +- `complete()` adds `fatCowCrates` to a `2.2.0` configuration, not to a `2.3.0` one. diff --git a/.backlog/README.md b/.backlog/README.md index 1bca3044..2a4d28ae 100644 --- a/.backlog/README.md +++ b/.backlog/README.md @@ -14,6 +14,7 @@ authored **per lot, when the lot is started** (not in batch). | Lot | Status | Description | Branch | |-----|--------|-------------|--------| +| [`FIX-CONFIG-NEW-LISTS`](FIX-CONFIG-NEW-LISTS/PRD.md) | merged (PR #NN) — 1 ticket ([date the lists](FIX-CONFIG-NEW-LISTS/tickets/01-date-the-lists.md)) | A list the catalogue gained after a mission configuration's version takes its default instead of reading as removed: the Fat Cow lists were empty on every `2.2.0` configuration (ADR 0011 Addendum 3). | `fix/config-new-lists-default` | | [`FEAT-FAT-COW`](FEAT-FAT-COW/PRD.md) | merged (PR #273) — 06 waiting-human (live check with the squadron, Zip) — 01 spike ✅ (a runtime Invisible FARP serves, enforces a finite stock, and needs only a FARP Fuel Depot as supplier), 02-05 ✅ | A CH-47 that lands in the field and becomes a forward refuel/rearm point (AI escort of a flight, or player-flown with cow crates), stocked with what it carries. | `feature/fat-cow` | | [`FIX-I18N-TRANSLATE-ALIAS`](FIX-I18N-TRANSLATE-ALIAS/PRD.md) | merged (PR #269) — 1 ticket ([JTAC through tr, guard sees the alias](FIX-I18N-TRANSLATE-ALIAS/tickets/01-jtac-through-tr-and-guard-sees-the-alias.md)) | The 16 JTAC radio keys reach the four dictionaries; the JTAC module calls ctld.tr and the generator also recognises the alias. | | [`FIX-FARP-PACK-FALSE-SUCCESS`](FIX-FARP-PACK-FALSE-SUCCESS/PRD.md) | merged (PR #268) — 1 ticket ([report the crates created](FIX-FARP-PACK-FALSE-SUCCESS/tickets/01-report-the-crates-created.md)) | Packing a FARP now reports a pack that created no crate, or only some, instead of always announcing a success. | diff --git a/.gitignore b/.gitignore index 1cc0a620..4d0152b6 100644 --- a/.gitignore +++ b/.gitignore @@ -43,6 +43,7 @@ __pycache__/ # ── Generated verbatim YAML embed (ctld.configDefault; regenerated by the build) ── /src/CTLD_config_default_yaml.lua +/src/CTLD_config_list_since.lua # ── ctld-tools web app: built frontend assets (bundled at CI, CTLD-TOOLS-WEBAPP t07) ── /tools/ctld-tools/ctld_tools/web/static/ diff --git a/CHANGELOG.md b/CHANGELOG.md index 2ee4c9b2..314d7aad 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,16 @@ Versioning follows [Semantic Versioning](https://semver.org/). ## [Unreleased] +### Fixed — a mission configuration written before the Fat Cow gets its crates and loadouts (FIX-CONFIG-NEW-LISTS) + +- A mission configuration omits a list on purpose to remove it (ADR 0011), but a configuration written before + catalogue `2.3.0` never had the Fat Cow lists: CTLD read `fatCowTypes`, `fatCowCrates`, `fatCowLoadouts` and + `fatCowAICows` as removed, and the Fat Cow was on with no aircraft, crate or loadout (seen on the VEAF Open + Trainings, whose configurations date from `2.2.0`). The build now dates every list from the pinned catalogue + shapes (`ctld.configListSince`, `ctld-tools since`): a list newer than the configuration's `configVersion` takes + its default and is named in the startup notice, and `ctld-tools` adds it when it opens the configuration. A list + omitted at or after its own version is still a removal (ADR 0011 Addendum 3). + ### Added — Fat Cow: a CH-47 that lands in the field and becomes a forward refuel and rearm point (FEAT-FAT-COW) ED announced a native Fat Cow on 2024-04-19, and again on 2024-08-23; DCS 2.9.30.28738 still has none, so CTLD brings its own while the Chinooks wait. diff --git a/dev/adr/0011-complete-yaml-config-and-webapp-tooling.md b/dev/adr/0011-complete-yaml-config-and-webapp-tooling.md index 69cb7613..8bbc774f 100644 --- a/dev/adr/0011-complete-yaml-config-and-webapp-tooling.md +++ b/dev/adr/0011-complete-yaml-config-and-webapp-tooling.md @@ -241,3 +241,39 @@ version at `2.0.0` on both sides the version-gap detection reported nothing. The - A completion only helps once an `ctld-tools.exe` that contains it is built and used: an older exe knows neither the new settings nor the completion, and its own validation cannot flag what it does not know. - Every change to the catalogue's keys or list-entry fields now costs a version increment and a snapshot file. + +## Addendum 3 — a list newer than the configuration is not a removal (2026-10-27) + +Point 1 reads an omitted list as a deliberate removal. That holds only for a list the configuration's own catalogue +already had. The Fat Cow (catalogue `2.3.0`) added `fatCowTypes`, `fatCowCrates`, `fatCowLoadouts` and +`fatCowAICows`; every mission configuration written against `2.2.0` lacks them, so the engine read them as removed +and the Fat Cow was on with no aircraft, no crate and no loadout. +Measured on the VEAF Open Trainings the day rc14 shipped: their `ctld-config.yaml` (VMCT, `2.2.0`) became +`ctld.configUser` without a single Fat Cow list. +Addendum 2's completion did not help either: it adds scalars and entry fields, never a whole list, and saving +stamps the new version, which turns the missing lists into removals for good. + +### Decision + +- **The build dates every list** from the pinned shapes (`tests/ci/data/catalogue_shapes`): `ctld-tools since` + writes `ctld.configListSince = { [list] = "" }` next to `ctld.configDefault`, for the + lists that appeared after the oldest pinned shape. +- **The engine** gives such a list its default when the snapshot omits it and its `configVersion` is older than the + list (a snapshot with no version is older than any); the list is named in the startup notice with the omitted + parameters. A list the snapshot omits at or after its version stays a removal, as does any list the table does not + name. +- **`ctld-tools`** applies the same rule when it opens a configuration (`complete()`, reading the table from the + engine it installs), so saving keeps the list instead of stamping its absence. + +### Considered options + +- **Merge every list default at runtime.** Still rejected: it resurrects deliberate removals. +- **Fix it downstream, in each tool that writes configurations (VMCT's `ctld-upgrade`).** Rejected: a hand-written or + older configuration would keep the trap, and VMCT has no shapes to tell a new list from a removed one. + +### Consequences + +- The shapes become a build input, not only a test guard: a catalogue change without its snapshot now also leaves + the new lists undated, on top of failing the guard. +- Lists already in the oldest pinned shape (`2.1.0`) cannot be dated: an older configuration that lacks one still + reads it as removed. diff --git a/docs/mission-maker/configuration.fr.md b/docs/mission-maker/configuration.fr.md index b63f2067..f656339b 100644 --- a/docs/mission-maker/configuration.fr.md +++ b/docs/mission-maker/configuration.fr.md @@ -96,6 +96,10 @@ réglages devenus inutiles, valeurs par défaut déplacées — sans jamais rien - une **liste** (une section de caisses, un groupe de troupes, une entrée de zone, un nom de pilote) que vous omettez est réellement absente. C'est ainsi qu'on en retire une — et c'est pourquoi rien n'est fusionné. + Seule exception : une liste que CTLD a ajoutée après votre `configVersion` (les caisses et + chargements de la Fat Cow sont arrivés en `2.3.0`). Vous ne pouvez pas avoir retiré ce qui + n'existait pas encore : CTLD utilise sa liste par défaut et la nomme dans le même rapport de + démarrage. Ouvrir l'instantané dans `ctld-tools` l'ajoute pour de bon. La validation de `ctld-tools` signale un réglage manquant comme une erreur et refuse d'injecter : un instantané produit par l'outil est donc toujours complet. diff --git a/docs/mission-maker/configuration.md b/docs/mission-maker/configuration.md index a4053db0..2c47bc67 100644 --- a/docs/mission-maker/configuration.md +++ b/docs/mission-maker/configuration.md @@ -90,6 +90,10 @@ used, defaults that moved — without ever merging anything behind your back. still runs and still tells you. - a **list** (a crate section, a troop group, a zone entry, a pilot name) you leave out is genuinely absent. That is how you remove one — and it is why nothing is merged. + The one exception is a list CTLD added after your `configVersion` (the Fat Cow crates and + loadouts arrived in `2.3.0`): you cannot have removed what did not exist yet, so CTLD uses its + default list and names it in the same startup report. Opening the snapshot in `ctld-tools` adds it + for good. `ctld-tools`' validation reports a missing setting as an error and refuses to inject, so a snapshot produced by the tool is always complete. diff --git a/src/CTLD_config.lua b/src/CTLD_config.lua index 94507b0a..8ec5a091 100644 --- a/src/CTLD_config.lua +++ b/src/CTLD_config.lua @@ -83,14 +83,26 @@ function CTLDConfig:load() self._defaultVersion = nil if usingUser and ctld.configDefault then self._defaults = CTLDConfig.flatten(CTLDConfig.parseYAML(ctld.configDefault)) + self._userVersion = flat.configVersion + self._defaultVersion = self._defaults.configVersion for k, v in pairs(self._defaults) do if type(v) ~= "table" and flat[k] == nil then self._defaultedParams[#self._defaultedParams + 1] = k end end + -- ADR 0011 Addendum 3 — a list the catalogue gained after the snapshot's version was not removed + -- by its author, who never saw it: it takes its default, reported like an omitted parameter. + -- A snapshot with no version predates every list in ctld.configListSince. + for k, since in pairs(ctld.configListSince or {}) do + local d = self._defaults[k] + if type(d) == "table" and flat[k] == nil + and (self._userVersion == nil or ctld.utils.compareVersions(self._userVersion, since) < 0) then + CTLDConfig.localiseI18n(d) + self.settings[k] = d + self._defaultedParams[#self._defaultedParams + 1] = k + end + end table.sort(self._defaultedParams) - self._userVersion = flat.configVersion - self._defaultVersion = self._defaults.configVersion end return true, "CTLDConfig: loaded (" .. (usingUser and "user" or "default") .. " config)." @@ -150,7 +162,7 @@ function CTLDConfig:reportRetiredSettings() end end ---- Parameters the loaded snapshot omitted, resolved from the default catalogue. +--- Parameters the loaded snapshot omitted, and lists newer than its version, resolved from the default catalogue. -- Sorted; empty when the snapshot is complete, or when the default catalogue is the winner. -- Reported once on screen at startup by CTLD_bootstrap (ADR 0011 Addendum 1). -- @return table array of setting names diff --git a/tests/ci/unit/config_param_tier_spec.lua b/tests/ci/unit/config_param_tier_spec.lua index 3159d7aa..ae8d8f90 100644 --- a/tests/ci/unit/config_param_tier_spec.lua +++ b/tests/ci/unit/config_param_tier_spec.lua @@ -79,6 +79,76 @@ describe("CTLDConfig parameter/list tiers (ADR 0011 addendum 1)", function() end) end) + -- ── ADR 0011 Addendum 3: a list newer than the snapshot was never removed by its author ── + describe("a missing list newer than the snapshot's version takes its default", function() + + local SINCE = { fatCowCrates = "2.3.0", fatCowLoadouts = "2.3.0" } + local savedSince + + before_each(function() + savedSince = ctld.configListSince + ctld.configListSince = SINCE + end) + after_each(function() + ctld.configListSince = savedSince + end) + + -- Drop `key` and its block (the `- ` items at its indent, and anything deeper). + local function withoutList(yaml, key) + local out, indent = {}, nil + for line in yaml:gmatch("[^\r\n]+") do + local lead = #line:match("^(%s*)") + if indent and (lead > indent or (lead == indent and line:match("^%s*%- "))) then + -- inside the dropped block + else + indent = nil + if line:match("^%s*" .. key .. "%s*:") then indent = lead else out[#out + 1] = line end + end + end + return table.concat(out, "\n") + end + + local function atVersion(yaml, version) + return (yaml:gsub('configVersion: "[^"]*"', 'configVersion: "' .. version .. '"', 1)) + end + + it("a 2.2.0 snapshot without fatCowCrates gets the catalogue's crates", function() + local cfg = loadUser(atVersion(withoutList(DEFAULT, "fatCowCrates"), "2.2.0")) + local crates = cfg:getSetting("fatCowCrates") + assert.is_table(crates) + assert.is_true(#crates > 0) + assert.equals(2400, crates[1].fuelKg) + end) + + it("reports the list among the settings taken from the default", function() + local cfg = loadUser(atVersion(withoutList(withoutList(DEFAULT, "fatCowCrates"), "fatCowLoadouts"), "2.2.0")) + assert.same({ "fatCowCrates", "fatCowLoadouts" }, cfg:getDefaultedParameters()) + end) + + it("a snapshot with no configVersion predates the list too", function() + local yaml = withoutList(DEFAULT, "fatCowCrates"):gsub('configVersion: "[^"]*"\n?', "", 1) + local cfg = loadUser(yaml) + assert.is_table(cfg:getSetting("fatCowCrates")) + end) + + it("a 2.3.0 snapshot without fatCowCrates removed it: it stays absent", function() + local cfg = loadUser(withoutList(DEFAULT, "fatCowCrates")) + assert.is_nil(cfg:getSetting("fatCowCrates")) + assert.same({}, cfg:getDefaultedParameters()) + end) + + it("a list kept by the snapshot is the snapshot's, even when older", function() + local yaml = atVersion(DEFAULT, "2.2.0"):gsub("desc: Fat Cow fuel tank %(800 gal%)", "desc: My tank", 1) + local cfg = loadUser(yaml) + assert.equals("My tank", cfg:getSetting("fatCowCrates")[1].desc) + end) + + it("a list the table does not name stays a removal at any version", function() + local cfg = loadUser(atVersion(without(DEFAULT, "aiZones"), "2.0.0")) + assert.is_nil(cfg:getSetting("aiZones")) + end) + end) + -- ── Ticket 03: a parameter must have exactly one default, in the catalogue ── describe("no parameter carries a duplicate default in code", function() diff --git a/tools/build/listToMerge.txt b/tools/build/listToMerge.txt index e6fa8931..b58c263f 100644 --- a/tools/build/listToMerge.txt +++ b/tools/build/listToMerge.txt @@ -8,6 +8,7 @@ CTLD_i18n_es.lua CTLD_i18n_ko.lua -- Verbatim config YAML embedded as ctld.configDefault string; parsed by CTLDConfig:load(). CTLD_config_default_yaml.lua +CTLD_config_list_since.lua CTLD_utils.lua CTLD_menu.lua core/CTLD_objectRegistry.lua diff --git a/tools/build/merge_CTLD.ps1 b/tools/build/merge_CTLD.ps1 index 5daa0e32..162e6c4d 100644 --- a/tools/build/merge_CTLD.ps1 +++ b/tools/build/merge_CTLD.ps1 @@ -85,6 +85,12 @@ Push-Location $ctldToolsDir try { & poetry run ctld-tools embed --yaml $configYaml --out $configDefaultLua --var configDefault if ($LASTEXITCODE -ne 0) { throw "embed returned $LASTEXITCODE" } + # The version each list appeared in (ctld.configListSince, ADR 0011 Addendum 3): a mission config + # older than a list gets its default instead of reading its absence as a removal. + $configListSinceLua = Join-Path $srcDir "CTLD_config_list_since.lua" + $catalogueShapes = Join-Path (Split-Path $srcDir -Parent) "tests\ci\data\catalogue_shapes" + & poetry run ctld-tools since --yaml $configYaml --shapes $catalogueShapes --out $configListSinceLua + if ($LASTEXITCODE -ne 0) { throw "since returned $LASTEXITCODE" } } catch { Write-Host "[ERROR] Could not embed CTLD_config.yaml via ctld-tools." diff --git a/tools/ctld-tools/ctld_tools/cli.py b/tools/ctld-tools/ctld_tools/cli.py index f8f7e826..27b361c2 100644 --- a/tools/ctld-tools/ctld_tools/cli.py +++ b/tools/ctld-tools/ctld_tools/cli.py @@ -89,6 +89,20 @@ def shape_cmd( typer.echo(f"shape: wrote {out}") +@app.command("since") +def since_cmd( + yaml_path: Path = typer.Option(..., "--yaml", help="path to the catalogue YAML (src/CTLD_config.yaml)"), + shapes_dir: Path = typer.Option(..., "--shapes", help="the pinned shapes (tests/ci/data/catalogue_shapes)"), + out: Path = typer.Option(..., "--out", help="path to the Lua module to write (ctld.configListSince)"), +) -> None: + """Write the catalogue version each list appeared in, as a Lua module (build step).""" + from ctld_tools.catalog import Catalog + from ctld_tools.since import write_since + + write_since(Catalog.load(yaml_path), shapes_dir, out) + typer.echo(f"since: wrote {out} (ctld.configListSince)") + + @app.command("validate") def validate_cmd( yaml_path: Path = typer.Option(..., "--yaml", help="path to a complete config YAML to validate"), diff --git a/tools/ctld-tools/ctld_tools/completion.py b/tools/ctld-tools/ctld_tools/completion.py index 695e08ff..db11eb61 100644 --- a/tools/ctld-tools/ctld_tools/completion.py +++ b/tools/ctld-tools/ctld_tools/completion.py @@ -11,13 +11,18 @@ only for entries the catalogue knows. Saving stamps the current version, so a later removal is respected. -Lists and entries that are absent altogether are never re-created. +- A **whole list** absent is a removal (ADR 0011), except a list the catalogue gained after the + configuration's version (`since`, from `ctld.configListSince`, ADR 0011 Addendum 3): nobody could + have removed it, so it is added with its default, as the engine would read it. + +Entries absent from a list are never re-created. A value the configuration already carries is never changed. """ from __future__ import annotations +import copy from dataclasses import dataclass from typing import Any @@ -46,25 +51,29 @@ def _is_scalar(value: Any) -> bool: return not isinstance(value, (dict, list, tuple)) -def complete(user: Catalog, default: Catalog) -> list[Addition]: +def complete(user: Catalog, default: Catalog, since: dict[str, str] | None = None) -> list[Addition]: """Add to `user` what `default` has and it lacks; return what was added. Scalar parameters are always added. The tier is derived from the shape of the default value (a scalar is a parameter), the same rule the engine applies, so tool and engine cannot disagree about - what an absent key means. Fields of list entries are added only when `user` predates `default`. + what an absent key means. Fields of list entries are added only when `user` predates `default`, and + a whole list only when `user` predates the version `since` gives it. """ added: list[Addition] = [] + since = since or {} + user_version = parse_version(user.get(_VERSION_KEY)) for key in default.keys(): - if key == _VERSION_KEY: + if key == _VERSION_KEY or user.has(key): continue value = default.get(key) - if not _is_scalar(value) or user.has(key): + if not _is_scalar(value) and not (key in since and user_version < parse_version(since[key])): continue section = default.section_of(key) + value = copy.deepcopy(value) user.add_setting(key, value, section=section) added.append(Addition(key, value, section)) - if parse_version(user.get(_VERSION_KEY)) < parse_version(default.get(_VERSION_KEY)): + if user_version < parse_version(default.get(_VERSION_KEY)): for f in entry_field_gap(user, default)[0]: user.get(f.container)[f.entry][f.field] = f.value added.append(Addition(f.field, f.value, user.section_of(f.container), f.container, f.entry)) diff --git a/tools/ctld-tools/ctld_tools/resources.py b/tools/ctld-tools/ctld_tools/resources.py index 981c3d03..2bea56f8 100644 --- a/tools/ctld-tools/ctld_tools/resources.py +++ b/tools/ctld-tools/ctld_tools/resources.py @@ -122,6 +122,30 @@ def read_engine() -> bytes: return path.read_bytes() +def list_since() -> dict[str, str]: + """The catalogue version each list appeared in (`ctld.configListSince`, ADR 0011 Addendum 3). + + Read from the engine the tool installs, so the tool and the engine agree; in a checkout with no built + engine, computed from the pinned shapes the build reads. + """ + import re + + path = engine_path() + if path.is_file(): + text = path.read_text(encoding="utf-8", errors="replace") + block = re.search(r"ctld\.configListSince\s*=\s*\{(.*?)\n\}", text, re.S) + if block: + return dict(re.findall(r'\["([^"]+)"\]\s*=\s*"([^"]+)"', block.group(1))) + shapes = _repo_root() / "tests" / "ci" / "data" / "catalogue_shapes" + if _frozen_dir() is None and shapes.is_dir(): + from ctld_tools.catalog import Catalog + from ctld_tools.since import list_since as compute + from ctld_tools.since import load_shapes + + return compute(Catalog.load(default_catalog_path()), load_shapes(shapes)) + return {} + + def read_sounds() -> dict[str, bytes]: """The beacon sounds, keyed by file name.""" out: dict[str, bytes] = {} diff --git a/tools/ctld-tools/ctld_tools/since.py b/tools/ctld-tools/ctld_tools/since.py new file mode 100644 index 00000000..5510de22 --- /dev/null +++ b/tools/ctld-tools/ctld_tools/since.py @@ -0,0 +1,53 @@ +"""The catalogue version each list first appeared in, read from the pinned shapes (build step). + +A list a configuration omits is an intentional removal (ADR 0011), but only when the list existed in the +catalogue that configuration was written against: nobody can have removed a list that came later. The engine +tells the two apart with this table, `ctld.configListSince`, embedded by the build next to `ctld.configDefault` +(ADR 0011 Addendum 3). + +Only the lists that appeared after the oldest pinned shape are listed: for the others nothing earlier is known, +and an omission stays a removal, as before. +""" + +from __future__ import annotations + +import json +from pathlib import Path +from typing import Any + +from ctld_tools.catalog import Catalog +from ctld_tools.versiongap import parse_version + +_VERSION_KEY = "configVersion" +_HEADER = ( + "-- GENERATED by ctld-tools (since) from tests/ci/data/catalogue_shapes. DO NOT EDIT.\n" + "-- The catalogue version each list appeared in (ADR 0011 Addendum 3).\nctld = ctld or {}\n" +) + + +def list_since(catalog: Catalog, shapes: list[dict[str, Any]]) -> dict[str, str]: + """`{list key: version}` for the lists of `catalog` that a later shape than the oldest one introduced.""" + ordered = sorted(shapes, key=lambda s: parse_version(s["version"])) + if not ordered: + return {} + since: dict[str, str] = {} + for key in catalog.keys(): + if key == _VERSION_KEY or not isinstance(catalog.get(key), (dict, list, tuple)): + continue + first = next((s["version"] for s in ordered if key in s["keys"]), None) + if first is not None and first != ordered[0]["version"]: + since[key] = first + return since + + +def load_shapes(shapes_dir: str | Path) -> list[dict[str, Any]]: + return [json.loads(p.read_text(encoding="utf-8")) for p in sorted(Path(shapes_dir).glob("*.json"))] + + +def to_lua(since: dict[str, str]) -> str: + lines = [f' ["{key}"] = "{since[key]}",' for key in sorted(since)] + return _HEADER + "ctld.configListSince = {\n" + "".join(line + "\n" for line in lines) + "}\n" + + +def write_since(catalog: Catalog, shapes_dir: str | Path, out: str | Path) -> None: + Path(out).write_text(to_lua(list_since(catalog, load_shapes(shapes_dir))), encoding="utf-8", newline="\n") diff --git a/tools/ctld-tools/ctld_tools/web/state.py b/tools/ctld-tools/ctld_tools/web/state.py index 1f6f7144..8e06c4d4 100644 --- a/tools/ctld-tools/ctld_tools/web/state.py +++ b/tools/ctld-tools/ctld_tools/web/state.py @@ -82,14 +82,14 @@ def load_path(self, path: str | Path) -> None: if found is None: raise ValueError(f"no CTLD configuration in {path.name}") self._catalog = Catalog.loads(found.yaml) - self._completion = complete(self._catalog, self.default_catalog()) + self._completion = complete(self._catalog, self.default_catalog(), resources.list_since()) self._path = None self._mission_path = path self._config_shape = found.shape self._sounds = read_sounds_from_miz(path, self._catalog) return self._catalog = Catalog.load(path) - self._completion = complete(self._catalog, self.default_catalog()) + self._completion = complete(self._catalog, self.default_catalog(), resources.list_since()) self._path = path self._mission_path = None self._config_shape = "file" diff --git a/tools/ctld-tools/tests/test_list_since.py b/tools/ctld-tools/tests/test_list_since.py new file mode 100644 index 00000000..8e13bac0 --- /dev/null +++ b/tools/ctld-tools/tests/test_list_since.py @@ -0,0 +1,91 @@ +"""A list the catalogue gained after a configuration's version is not a removal (ADR 0011 Addendum 3). + +The build writes `ctld.configListSince` from the pinned shapes; the engine and `complete()` read it, so a +mission configuration written before the Fat Cow gets its crates and loadouts instead of empty lists. +""" + +from pathlib import Path + +from typer.testing import CliRunner + +from ctld_tools import resources +from ctld_tools.catalog import Catalog +from ctld_tools.cli import app as cli_app +from ctld_tools.completion import complete +from ctld_tools.since import list_since, load_shapes, to_lua + +REPO = Path(__file__).resolve().parents[3] +SHAPES = REPO / "tests" / "ci" / "data" / "catalogue_shapes" + +DEFAULT = """\ +configVersion: "2.3.0" +mm_facing: + numberOfTroops: 10 + aiZones: [] + fatCowCrates: + - desc: tank + fuelKg: 2400 +""" + +SHAPES_FIXTURE = [ + {"version": "2.1.0", "keys": ["numberOfTroops", "aiZones"], "entryFields": []}, + {"version": "2.3.0", "keys": ["numberOfTroops", "aiZones", "fatCowCrates"], "entryFields": []}, + {"version": "2.2.0", "keys": ["numberOfTroops", "aiZones"], "entryFields": []}, +] + + +def cat(text: str) -> Catalog: + return Catalog.loads(text) + + +def test_a_list_later_than_the_oldest_shape_is_dated(): + assert list_since(cat(DEFAULT), SHAPES_FIXTURE) == {"fatCowCrates": "2.3.0"} + + +def test_the_shipped_catalogue_dates_the_fat_cow_lists(): + since = list_since(Catalog.load(REPO / "src" / "CTLD_config.yaml"), load_shapes(SHAPES)) + for key in ("fatCowCrates", "fatCowLoadouts", "fatCowTypes"): + assert since[key] == "2.3.0" + assert "aiZones" not in since + assert "fatCowEnabled" not in since # a parameter, not a list + + +def test_the_lua_module_round_trips_through_the_engine_reader(tmp_path, monkeypatch): + engine = tmp_path / "CTLD.lua" + engine.write_text("ctld = {}\n" + to_lua({"fatCowCrates": "2.3.0", "fatCowTypes": "2.3.0"}), encoding="utf-8") + monkeypatch.setattr(resources, "engine_path", lambda: engine) + assert resources.list_since() == {"fatCowCrates": "2.3.0", "fatCowTypes": "2.3.0"} + + +def test_the_cli_writes_the_module(tmp_path): + out = tmp_path / "since.lua" + result = CliRunner().invoke( + cli_app, ["since", "--yaml", str(REPO / "src" / "CTLD_config.yaml"), "--shapes", str(SHAPES), "--out", str(out)] + ) + assert result.exit_code == 0, result.output + assert '["fatCowLoadouts"] = "2.3.0",' in out.read_text(encoding="utf-8") + + +def test_complete_adds_a_list_newer_than_the_configuration(): + user = cat('configVersion: "2.2.0"\nmm_facing:\n numberOfTroops: 10\n aiZones: []\n') + added = complete(user, cat(DEFAULT), {"fatCowCrates": "2.3.0"}) + assert [a.key for a in added] == ["fatCowCrates"] + assert user.get("fatCowCrates")[0]["fuelKg"] == 2400 + + +def test_complete_leaves_a_list_removed_at_its_own_version(): + user = cat('configVersion: "2.3.0"\nmm_facing:\n numberOfTroops: 10\n aiZones: []\n') + assert complete(user, cat(DEFAULT), {"fatCowCrates": "2.3.0"}) == [] + assert not user.has("fatCowCrates") + + +def test_complete_leaves_an_undated_list_removed_at_any_version(): + user = cat('configVersion: "2.0.0"\nmm_facing:\n numberOfTroops: 10\n') + added = complete(user, cat(DEFAULT), {"fatCowCrates": "2.3.0"}) + assert [a.key for a in added] == ["fatCowCrates"] + assert not user.has("aiZones") + + +def test_complete_without_the_table_adds_no_list(): + user = cat('configVersion: "2.2.0"\nmm_facing:\n numberOfTroops: 10\n') + assert complete(user, cat(DEFAULT)) == [] From 87610822a85c6bc64040cd372c532ccaa472703f Mon Sep 17 00:00:00 2001 From: David Pierron Date: Tue, 27 Oct 2026 19:21:57 +0100 Subject: [PATCH 2/2] docs(backlog): FIX-CONFIG-NEW-LISTS index line, PR #275 Co-Authored-By: Claude Opus 5.5 --- .backlog/FIX-CONFIG-NEW-LISTS/PRD.md | 2 +- .backlog/README.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.backlog/FIX-CONFIG-NEW-LISTS/PRD.md b/.backlog/FIX-CONFIG-NEW-LISTS/PRD.md index e4024ad6..6163839a 100644 --- a/.backlog/FIX-CONFIG-NEW-LISTS/PRD.md +++ b/.backlog/FIX-CONFIG-NEW-LISTS/PRD.md @@ -1,6 +1,6 @@ # FIX-CONFIG-NEW-LISTS — a list newer than the mission configuration is not a removal -Status: done +Status: done — 2026-10-27 (PR #275) ## Problem diff --git a/.backlog/README.md b/.backlog/README.md index 2a4d28ae..8933b1f9 100644 --- a/.backlog/README.md +++ b/.backlog/README.md @@ -14,7 +14,7 @@ authored **per lot, when the lot is started** (not in batch). | Lot | Status | Description | Branch | |-----|--------|-------------|--------| -| [`FIX-CONFIG-NEW-LISTS`](FIX-CONFIG-NEW-LISTS/PRD.md) | merged (PR #NN) — 1 ticket ([date the lists](FIX-CONFIG-NEW-LISTS/tickets/01-date-the-lists.md)) | A list the catalogue gained after a mission configuration's version takes its default instead of reading as removed: the Fat Cow lists were empty on every `2.2.0` configuration (ADR 0011 Addendum 3). | `fix/config-new-lists-default` | +| [`FIX-CONFIG-NEW-LISTS`](FIX-CONFIG-NEW-LISTS/PRD.md) | merged (PR #275) — 1 ticket ([date the lists](FIX-CONFIG-NEW-LISTS/tickets/01-date-the-lists.md)) | A list the catalogue gained after a mission configuration's version takes its default instead of reading as removed: the Fat Cow lists were empty on every `2.2.0` configuration (ADR 0011 Addendum 3). | `fix/config-new-lists-default` | | [`FEAT-FAT-COW`](FEAT-FAT-COW/PRD.md) | merged (PR #273) — 06 waiting-human (live check with the squadron, Zip) — 01 spike ✅ (a runtime Invisible FARP serves, enforces a finite stock, and needs only a FARP Fuel Depot as supplier), 02-05 ✅ | A CH-47 that lands in the field and becomes a forward refuel/rearm point (AI escort of a flight, or player-flown with cow crates), stocked with what it carries. | `feature/fat-cow` | | [`FIX-I18N-TRANSLATE-ALIAS`](FIX-I18N-TRANSLATE-ALIAS/PRD.md) | merged (PR #269) — 1 ticket ([JTAC through tr, guard sees the alias](FIX-I18N-TRANSLATE-ALIAS/tickets/01-jtac-through-tr-and-guard-sees-the-alias.md)) | The 16 JTAC radio keys reach the four dictionaries; the JTAC module calls ctld.tr and the generator also recognises the alias. | | [`FIX-FARP-PACK-FALSE-SUCCESS`](FIX-FARP-PACK-FALSE-SUCCESS/PRD.md) | merged (PR #268) — 1 ticket ([report the crates created](FIX-FARP-PACK-FALSE-SUCCESS/tickets/01-report-the-crates-created.md)) | Packing a FARP now reports a pack that created no crate, or only some, instead of always announcing a success. |