Skip to content

fix(config): refuse unknown keys in [tool.fastskill] and metrics files - #363

Merged
aroff merged 1 commit into
mainfrom
fix/reject-unknown-config-keys
Sep 23, 2026
Merged

aroff merged 1 commit into
mainfrom
fix/reject-unknown-config-keys

Conversation

@aroff

@aroff aroff commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Summary

FastSkill ignored misspelt settings in the two places where a typo changes what it does:

  • [tool.fastskill] in skill-project.toml: skip_transitiv = true or a mistyped repository field was dropped without a word.
  • Eval metrics files: a misspelt criterion made a metric score nothing, and the run still looked successful.

Both are now refused, and the error names every unknown key with its path:

skill-project.toml has settings that FastSkill does not recognise: tool.fastskill.skip_transitiv, tool.fastskill.repositories.0.brnach. Check the spelling, or remove them if they are meant for a newer FastSkill.

The rest of the manifest stays permissive. Top-level keys and other tools' tables such as [tool.other] are still accepted and kept on save, as lifecycle PRD S-01 requires ("preserved or rejected before writing, never dropped").

How unknown keys are found

core/unknown_keys.rs parses the file, serializes the typed value back to TOML, and reports every raw key that is missing from the result.

We didn't use serde(deny_unknown_fields) or serde_ignored. Neither can see keys inside #[serde(flatten)] or untagged enums, because serde buffers that content before choosing a variant, and RepositoryDefinition uses both. The round-trip diff also catches keys that an untagged enum silently discarded when it picked a variant without them.

Compatibility

  • An older CLI will refuse a manifest that uses a newer [tool.fastskill] key. The error message says so.
  • Test fixtures that used [tool.fastskill.extension] as an "unknown table" example now use [tool.other], which is the permissive case the PRD describes.

Tests

  • manifest/tests.rs: typos under [tool.fastskill] are refused, a typo in a flattened repository entry is refused, and tables outside [tool.fastskill] stay permissive.
  • eval/scorecard/metrics.rs: a misspelt metrics key is refused, and new tests cover the read, parse and missing-checks.toml errors, which keep the changed file above the 90% coverage gate.
  • core/unknown_keys.rs: unit tests for the diff.
  • scripts/run-tests.sh passes locally.

🤖 Generated with Claude Code

A misspelt key under [tool.fastskill] (say `skip_transitiv`), inside a
[[tool.fastskill.repositories]] entry (`brnach`), or in an eval metrics
file (`critera`, `suite`) parsed cleanly and was then ignored.

serde's deny_unknown_fields cannot help here, and neither can
serde_ignored: repository entries flatten an untagged enum and metrics
flatten an internally tagged one, and serde buffers those keys where
neither sees them. core::unknown_keys instead serializes the parsed value
back and reports every input key the round trip lost. That also catches a
key dropped because an untagged enum chose a variant without it.

The rest of the manifest stays permissive, per the lifecycle PRD
("preserved or rejected before writing, never dropped"). Two
preservation tests used [tool.fastskill.extension] as their example
unknown table; they now use [tool.other].

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@aroff
aroff enabled auto-merge (squash) September 23, 2026 13:43
@aroff
aroff merged commit b8b9d82 into main Sep 23, 2026
14 checks passed
@aroff
aroff deleted the fix/reject-unknown-config-keys branch September 23, 2026 13:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants