Skip to content

Add shared emitter conformance fixtures and a cross-emitter check - #17

Open
magic-peach wants to merge 2 commits into
runtimeconditions:mainfrom
magic-peach:conformance-fixtures
Open

magic-peach wants to merge 2 commits into
runtimeconditions:mainfrom
magic-peach:conformance-fixtures

Conversation

@magic-peach

Copy link
Copy Markdown

Each emitter tests itself with its own harness, so nothing checks that they agree. Review of the Java emitter (#10) found bugs that other emitters had already fixed or avoided, which a shared fixture set would have caught once.

  • Two new cases in model/conformance: 14-scalar-definition-references (properties that reference scalar, unconstrained, enum, and chained $defs) and 15-keyword-property-names (reserved words in common languages). Their expected models are generated by the normalizer conformance test, which now lists them.
  • expected/<case>/manifest-neutral.yaml for every positive case: the part of the binding manifest that comes from the model, not the language (declarations, imported marker contracts, and root bindings).
  • ci/emitter_conformance.py generates every positive case with each emitter and requires the neutral part of its manifest to equal the fixture. It has adapters for the Python, TypeScript and Rust emitters, an --update mode, and unit tests that fail if a case has no fixture. A new verify-emitter-conformance job runs it for Python and TypeScript.
  • model/conformance/README.md explains the fixtures, how to check an emitter, and how to add a case.

I ran the Python, TypeScript and Go emitter test suites against the two new cases (all pass), and the checker for Python and TypeScript: all 12 positive cases agree.

Two things it showed about the Go emitter, which is why it is not in the checker:

  • Its manifest uses vocabulary coordinates for model references and leaves out the schema and vocabulary coordinates, so it does not match the shape the other emitters share.
  • It rejects an unconstrained definition (payload: {} in case 14) with RCG1024, where the other emitters produce an unconstrained type.

I did not change the Go emitter, since that would change published manifests. The Rust emitter in #11 has an adapter here and agrees on all twelve cases when run locally. The CI job runs Python and TypeScript for now; adding Rust is one flag once #11 lands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant