Validate AI coding assistant skills against the Agent Skills specification.
A skill that fails to load fails silently. A skill that loads but never triggers is invisible. A skill that triggers but teaches the agent nothing costs context in every session and gives nothing back. This plugin catches all three — the first mechanically, the other two with a structured review.
It works on skills for any client. SKILL.md is a portable format, and nothing
here is Claude-specific except the plugin packaging.
In Claude Code:
/plugin marketplace add blakebauman/skillist-validator
/plugin install skillist-validator@skillist-validator
Using a different agent? The three skills are portable SKILL.md — copy
skills/* into that tool's skills directory. The validator itself is a
dependency-free Python script and runs anywhere:
python3 skills/skill-validation/scripts/validate_skill.py path/to/skills| Command | What it does |
|---|---|
/skillist-validator:validate [path] |
Spec conformance check. Reports findings with fixes; changes nothing. |
/skillist-validator:audit [path] |
Full review — conformance, instruction quality, and triggering. |
/skillist-validator:fix [path] |
Validates, fixes the violations, re-validates, reports before/after. |
| Skill | What it does |
|---|---|
skill-validation |
Spec conformance: frontmatter, name and description rules, limits, layout, bundled references, progressive disclosure. Wraps the validator script. |
skill-quality-audit |
Past conformance: is the skill grounded in real expertise, calibrated between prescriptive and open, economical with context, and scoped as a coherent unit? |
skill-description-tuning |
Why a skill triggers on the wrong prompts or not at all — rewriting the description and measuring the change against labelled eval queries. |
skill-auditor — runs all three in the order that costs least: conformance
first (fast, scriptable), then quality (judgment), then triggering (expensive
to measure). Delegate to it for anything spanning more than one.
skills/skill-validation/scripts/validate_skill.py — 41 rules, every one
carrying a fix hint.
# One skill, or a whole tree — it discovers every SKILL.md beneath the path
python3 validate_skill.py .claude/skills
# Machine-readable, for CI or a fix loop
python3 validate_skill.py --format json skills/
# Warnings fail too — the pre-publish gate
python3 validate_skill.py --strict skills/
# What does it check?
python3 validate_skill.py --list-rulesExit codes: 0 clean, 1 violations, 2 bad path or no skill there.
| Group | Covers |
|---|---|
| AS0xx structure | SKILL.md present, frontmatter parses, only spec-defined fields |
| AS1xx name | 64-char limit, character set, hyphen rules, directory match |
| AS2xx description | 1024-char limit, trigger language, imperative phrasing, startup cost |
| AS3xx optional fields | license, compatibility, metadata string values, allowed-tools format |
| AS4xx body | 500-line and 5,000-token recommendations, bundled files with no load condition |
| AS5xx references | Broken paths, escapes, nesting depth, dead weight, empty directories |
| AS6xx scripts | Shebang and executable-bit agreement |
Three levels, reported differently: ERROR is a spec violation, WARN is a spec recommendation or agentskills.io authoring guidance, NOTE is portability information rather than a defect.
Full catalog with the reasoning and a fix for each rule:
skills/skill-validation/references/rule-catalog.md.
No dependencies. It uses PyYAML when installed and falls back to a built-in parser for the YAML subset frontmatter actually uses. The fallback is deliberately strict — it raises rather than guessing, so a skill is never passed on a misparse. Both paths are covered by the test suite.
No false positives on project paths. Skills legitimately name paths in the
user's project (.workspace/INDEX.md, ~/Downloads/data.csv). Only paths that
claim to be bundled — under scripts/, references/, assets/, or whose
first segment exists in the skill — are checked for existence. Where both
readings are plausible (a scripts/deploy.sh in a skill that bundles no
scripts/) the finding is a warning naming both, not a missing-file error.
Validated against 78 real-world skills across four projects while being built;
every check that cried wolf was split or dropped rather than shipped.
Warnings are judgment calls, and say so. ${CLAUDE_PLUGIN_ROOT} is correct
in a Claude Code plugin and a portability problem elsewhere, so it is a NOTE,
not a warning. The rule catalog says when ignoring each warning is defensible.
- name: Validate skills
run: |
python3 skills/skill-validation/scripts/validate_skill.py --strict .claude/skillsOr with the JSON output, to gate on errors only while surfacing warnings:
python3 validate_skill.py --format json skills/ \
| jq -r '.skills[] | select(.conforms | not) | .name'python3 tests/run_tests.py # validator against fixtures with known defects
python3 scripts/validate_plugin.py # plugin structure + the plugin's own skills, strictlyvalidate_plugin.py runs the validator against this plugin's own skills with
--strict and runs the test suite, so the plugin cannot ship violating the
specification it enforces.
MIT. See LICENSE.