Skip to content

Add opt-in instruction-context static analysis backed by shared host semantics #279

Description

@Jamie-BitFlight

Outcome

Add an opt-in instruction-context static-analysis profile to skilllint that consumes a shared/versioned deterministic context model from Skill Lapidary rather than independently reimplementing host loading semantics.

Upstream provider issue: https://github.com/Jamie-BitFlight/skill-lapidary/issues/202

Why

skilllint already owns fast deterministic validation for agent plugins, skills and agents, with explicit rule provenance, severity, adapters, schemas and suppression. Skill Lapidary is developing a richer instruction-context model for repository-wide auditing.

There is a useful overlap, but only for facts that have deterministic falsifiers. skilllint should gain the structural checks while preserving the boundary:

skilllint proves structural/static conditions; Skill Lapidary interprets semantic meaning, authority, conflict and conservation.

The two tools should not independently implement Claude/Codex/AGENTS loading semantics.

Architectural boundary

In scope for skilllint

Mechanically provable facts derived from a versioned effective-context graph:

  • broken/dangling instruction references;
  • repository-boundary/path escapes;
  • malformed or unsupported selectors;
  • mechanically shadowed carriers;
  • byte/normalized-exact duplicate carriers/directives;
  • dead selectors;
  • unreachable carriers;
  • ambiguous recognized carriers whose host loading is not established;
  • import cycles/depth;
  • effective-context token count;
  • effective-context nesting/import depth;
  • context-count/explosion measurements;
  • narrowly structured literal opposition where a deterministic parser can prove both obligations and overlapping applicability.

Out of scope

Do not add rules claiming to determine:

  • semantic paraphrase equivalence;
  • general prose contradiction;
  • semantic refinement/specialization;
  • justified exceptions;
  • semantic supersession;
  • inferred/approved goals;
  • semantic conservation;
  • behavioral equivalence;
  • whether the instruction system achieves its purpose.

Those remain Skill Lapidary concerns.

Proposed opt-in rule family

Use a dedicated family such as ICxxx (Instruction Context). Final IDs/severities require normal rule-provenance review.

Initial candidate set:

Candidate Static condition Suggested initial severity
IC001 repository-relative instruction reference cannot resolve warning
IC002 instruction import/reference escapes authorized boundary or traverses unsafe symlink error
IC003 selector/import/loading syntax is unsupported or unresolved; must not broaden to unconditional applicability warning
IC004 carrier is mechanically shadowed by established host precedence info
IC005 byte-identical/normalized-identical carriers load into the same effective context info
IC006 exact normalized directive occurs multiple times in one effective context info
IC007 narrowly parseable literal opposition over demonstrably overlapping scope opt-in warning
IC008 scoped selector matches no concrete target info
IC009 nominally scoped selector resolves to effectively whole-tree applicability info
IC010 effective-context count/ratio measurement exceeds configured threshold info
IC011 carrier/import exists but no supported loading route reaches it info
IC012 recognized instruction/memory filename exists but selected host profile cannot establish loading info
IC013 identical directive appears under different established structural authority/scope; inspection signal only info
IC014 effective-context token budget measurement/threshold info/warning
IC015 effective-context nesting/import depth measurement/threshold info
IC016 import cycle or configured maximum import depth exceeded warning

IC007 and IC013 should remain experimental until corpus evaluation demonstrates precision. A rule must not emit a semantic conclusion merely because text looks similar.

UX / configuration

These checks should not silently join the default fast path initially.

Provide an opt-in profile, for example:

{
  "analysis": {
    "instruction-context": {
      "enabled": true,
      "hosts": ["claude-code", "codex"],
      "checks": ["references", "scope", "shadowing", "duplicates", "token-budget"]
    }
  }
}

and/or a CLI surface equivalent to:

skilllint check . --analysis instruction-context

The exact CLI/config shape should follow current skilllint configuration architecture rather than this illustrative syntax.

Rules must still participate in the existing registry/provenance/severity/suppression/reporting architecture.

Planned slices

  • S1 — Consumer contract and profile plumbing.

  • S2 — Graph integrity checks.

    • IC001/IC002/IC003/IC008/IC011/IC012/IC016.
    • These have the strongest deterministic falsifiers.
  • S3 — Structural optimization/measurement checks.

    • IC004/IC005/IC014/IC015.
    • Add effective-context token/depth measurements without presenting measurements as vendor requirements.
  • S4 — Experimental architectural-smell checks.

    • Evaluate IC006/IC007/IC009/IC010/IC013 on representative repositories.
    • Ship only checks whose precision and provenance justify them; otherwise retain as measurements/research.
  • S5 — Cross-tool conformance suite.

    • Run shared fixtures/corpora through both tools.
    • Assert identical structural source/context graph facts for the same host profile/version.
    • Demonstrate that skilllint does not emit semantic Lapidary relationships.

Acceptance criteria

  • Instruction-context analysis is opt-in and does not materially slow ordinary skilllint check when disabled.
  • skilllint consumes shared/versioned host semantics or a generated compatibility artifact; it does not fork a private interpretation of host loading.
  • Unsupported host mechanics become unresolved findings/coverage, never unconditional applicability.
  • Every new rule has normal skilllint provenance/authority classification and discriminating fixtures.
  • Measurement rules clearly distinguish measurement from normative/vendor constraints, consistent with TC001's architecture.
  • Context-token checks measure the complete effective composition, not merely individual files.
  • Shadowing/unreachability checks are structural and never report semantic supersession.
  • Exact duplication checks do not claim paraphrase equivalence.
  • Any literal-opposition rule is constrained to mechanically parseable cases and evaluated for false positives before promotion.
  • Cross-tool fixtures prove structural parity with the upstream provider.
  • Documentation explains when to use skilllint static analysis versus a Skill Lapidary semantic audit.

Performance requirements

This feature must preserve skilllint's fast-linter role:

  • disabled profile should have effectively zero graph-analysis cost;
  • discovery/context construction should be one repository pass where practical;
  • parse each relevant file once and reuse parsed state across IC rules;
  • rules consume the shared graph rather than independently rescanning;
  • expensive/experimental checks remain individually configurable;
  • benchmark representative small, medium and large repositories before enabling any check by default.

Dependency strategy

Do not immediately couple skilllint's release lifecycle to an unpublished Skill Lapidary Python package. Acceptable first integrations include:

  1. a small stable shared package only if fix(scan): discover skills under known provider dirs (.claude/skills etc.) #202 L4 establishes one as part of the consumer contract;
  2. preferably, a generated/versioned L4 schema + host-profile artifact and L5 shared fixture corpus;
  3. vendored generated data whose provenance/version is mechanically checked.

#202 L1's Source/Context Python records are an ownership seam inside Lapidary, not yet the cross-repository integration contract. skilllint must not import or vendor Lapidary's internal Tree, Composition, filesystem traversal, or current unversioned CLI JSON shape.

Copy/pasting the discovery implementation is not an acceptable long-term integration because it permits semantic drift between tools.

Non-goals

  • LLM calls from skilllint.
  • SQLite audit-store dependency.
  • HTML semantic audit reports.
  • Automatic rewriting/consolidation.
  • Turning recommendations or measurements into hard failures without documented authority/evidence.

Upstream boundary clarification — 2026-09-30

Skill Lapidary #202 was walked through with this consumer before freezing L1. The provider deliberately narrowed L1 to immutable Source/Context records plus source identity. Discovery workspace (Tree), mutable composition construction, filesystem/path policy and serialization remain private/later-slice concerns.

Consequences for this issue:

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions