Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ from `cargo run --release --example coverage_report -- <path>`.
<!--
- [ ] `cargo fmt --all -- --check` passes
- [ ] `cargo clippy --all-targets --all-features -- -D warnings` passes
- [ ] `cargo test --release` passes
- [ ] `tools/ci/run_test_targets.sh --release --all-features` passes
- [ ] `RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features` passes
- [ ] Shown working on real files: the corpus records it was measured on, with the command that reproduces it (no new unit tests)
- [ ] Shown working on real files: the corpus records it was measured on, with the command that reproduces it (unit tests are banned: no new ones, and no regression test for a fix)
- [ ] For decoder changes: coverage-report numbers attached
-->

Expand Down
23 changes: 21 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -73,10 +73,23 @@ jobs:
- uses: Swatinem/rust-cache@42dc69e1aa15d09112580998cf2ef0119e2e91ae # v2
with:
key: ${{ matrix.os }}-${{ matrix.toolchain }}
- name: Every tests/*.rs target is classified
shell: bash
run: tools/ci/run_test_targets.sh --check-only
- name: cargo build --release
run: cargo +${{ matrix.toolchain }} build --release --all-features --all-targets
- name: cargo test --release
run: cargo +${{ matrix.toolchain }} test --release --all-features
# Unit tests are banned: tools/ci/test-targets.txt classes every
# tests/*.rs target as real, contract, hardening, cli or unit, and the
# script runs all but unit, then the documentation examples. It fails
# on a target the manifest does not class, so a new one cannot run or
# skip unnoticed. The inline #[cfg(test)] suites in src/ are not run.
# Corpus-reading targets skip here (no ../../samples) and run in the
# Corpus (public DWG) job.
- name: Real-file, contract and hardening targets, and the documentation examples
shell: bash
env:
RUSTUP_TOOLCHAIN: ${{ matrix.toolchain }}
run: tools/ci/run_test_targets.sh --release --all-features
- name: The specification's worked examples (#107)
run: cargo +${{ matrix.toolchain }} run --release --all-features --example spec_examples

Expand Down Expand Up @@ -603,6 +616,12 @@ jobs:
# up-to-date off, so two PRs can each add the same `## 7x.`.
- name: ARCHITECTURE.md's numbered sections run in order
run: python3 tools/maintainer/train/arch.py check
# The inline test that pinned fixed_entity_label to
# decode_fixed_entity_body was a unit test and no longer runs; this
# reads both lists from src/entities/dispatch.rs, so a new decoder that
# adds its arm to one and not the other fails here.
- name: The fixed-code dispatcher's label list and body arms agree
run: python3 tools/ci/dispatch_arms.py --check

pii-guard:
name: PII guard
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@ The repository's guidance for every coding agent is in `CLAUDE.md`; read it firs
## Review guidelines

- dwg-rs is a clean-room implementation (`CLEANROOM.md`). Never cite, quote, link or paraphrase LibreDWG, the ODA/Teigha SDK, an Autodesk SDK or any other GPL or proprietary DWG implementation in a review comment, and never suggest consulting one. Ground a finding in the ODA *Open Design Specification for .dwg files* v5.4.1 (cite its section), Autodesk's public *DXF Reference*, AutoCAD's own DWG and DXF output in the pinned corpus (`tools/corpus/pins.tsv`), or this repository's code.
- Do not ask for unit tests. The project adds none; it proves behaviour on the pinned public corpus, the DXF oracle and the end-to-end verifiers, so ask for that evidence instead.
- Do not ask for unit tests: they are banned here (`CLAUDE.md`). The project adds none; it proves behaviour on the pinned public corpus, the DXF oracle and the end-to-end verifiers, so ask for that evidence instead.
34 changes: 23 additions & 11 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,9 +176,11 @@ bit granularity. Key methods:
| `read_ms` | Two-byte modular stream | `u64` (unsigned) |
| `read_handle` | 4-bit code + 4-bit counter + payload | `Handle { code, value }` |

The `BitWriter` methods mirror each `read_*` method exactly. Property
tests in `tests/proptest_roundtrip.rs` lock the invariant that
every primitive round-trips bit-exactly.
The `BitWriter` methods mirror each `read_*` method exactly. Every
primitive round-trips bit-exactly; CI exercises that end to end, because
the committed canonical fixtures are written with `BitWriter` and read back
by the reader, and `tests/proptest_roundtrip.rs` (class `unit`, not run)
states it as a property.

## 5. LZ77 decompression (spec §4.7)

Expand Down Expand Up @@ -5232,9 +5234,13 @@ input to a full `DwgFile::to_bytes()` once Stages 2-5 are completed.

## 10. Test strategy

Evidence comes from real executables run on real files. New work is
shown on the corpus, not with new unit tests; the suites already in
the tree stay and must keep passing.
Evidence comes from real executables run on real files. Unit tests
are banned: none are added, and none run in CI. `tools/ci/test-targets.txt`
classes every file under `tests/` as `real`, `cli`, `contract`,
`hardening` or `unit`; the Test job runs all but `unit`
(`tools/ci/run_test_targets.sh`), and fails on a file the manifest
does not class. The inline `#[cfg(test)]` suites in `src/` are being
deleted area by area, and git history is the archive.

1. **Self-validating decoders.** Every entity and object decoder must
end exactly on its record's data-stream boundary (§7i), so a wrong
Expand All @@ -5252,15 +5258,21 @@ the tree stay and must keep passing.
generator, the coverage report must stay above its floor, and the
class-census and reference-closure probes must pass. CI does not
read the real corpus yet (#95).
4. **Property, golden and fuzz checks.** `tests/proptest_roundtrip.rs`
round-trips the bit-level primitives, `tests/integration_svg_goldens.rs`
pins SVG output, and `fuzz/` runs nightly against every byte-level
attack surface.
4. **Fuzz and contract checks.** `tests/integration_fuzz_corpus_regression.rs`
replays every fuzz seed through its entry point (class `hardening`),
`tests/code_table_agreement.rs` checks the object-type tables against
the specification's Table 4 (class `contract`),
`tools/ci/dispatch_arms.py` keeps the fixed-code dispatcher's label list
and body arms in step, and `fuzz/` runs nightly against every byte-level
attack surface. `tests/proptest_roundtrip.rs` and
`tests/integration_svg_goldens.rs` are classed `unit`: kept as source,
not run.
5. **Streaming E2E.** `tools/streaming/verify.py` drives the built
`stream_header` example in separate processes under generated chunk
schedules (docs/streaming/README.md).

`cargo test --release --all-features` must pass before a PR merges.
`tools/ci/run_test_targets.sh --release --all-features` must pass before
a PR merges.

## 11. Legal posture

Expand Down
16 changes: 8 additions & 8 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,22 +11,22 @@ dwg-rs is an Apache-2.0 Rust reader (and early writer) for Autodesk DWG, R13 to
```sh
cargo fmt --all -- --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --release --all-features
tools/ci/run_test_targets.sh --release --all-features # every non-unit target of tools/ci/test-targets.txt, then the doc tests
RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features
cargo deny check advisories bans licenses sources # CI pins cargo-deny 0.19.4
CARGO_TARGET_DIR=target-msrv cargo +1.85 check --all-features # MSRV without evicting the stable cache
python3 tools/ci/spec_coverage.py --check # docs/spec-coverage.md agrees with its data
python3 tools/ci/dispatch_arms.py --check # fixed_entity_label and decode_fixed_entity_body agree
```

CI also builds with `RUSTFLAGS="-D warnings"`, so a warning anywhere fails the Test job.
CI also builds with `RUSTFLAGS="-D warnings"`, so a warning anywhere fails the Test job. The Test job runs `tools/ci/run_test_targets.sh`, not a bare `cargo test`: it runs the `real`, `contract` and `hardening` targets of `tools/ci/test-targets.txt` and the documentation examples, never the `unit` ones and never the inline `#[cfg(test)]` suites in `src/`.

Single tests:
Single targets:

```sh
tools/ci/run_test_targets.sh --check-only # every tests/*.rs file is classified
cargo test --release --all-features --test canonical_corpus # one integration file (tests/<stem>.rs)
cargo test --release --all-features --test samples arc_r14_is_ac1014 # one test in it
cargo test --release --all-features --lib entities::line # lib tests by module path filter
UPDATE_GOLDENS=1 cargo test --release --test integration_svg_goldens # regenerate tests/golden/svg
```

The binaries (`dwg-info`, `dwg-corpus`, `dwg-dump`, `dwg-convert`, `dwg-to-dxf`, `dwg-to-gltf`, `dwg-write`) all have `required-features = ["cli"]`; plain `cargo build` builds none of them:
Expand Down Expand Up @@ -74,7 +74,7 @@ python3 -I tools/corpus/measure.py check --results r.json # every drawing

The `Corpus (public DWG)` job runs exactly this and fails on any count change in either direction, so a PR that changes a decoded, skipped or errored count must commit the regenerated baseline (`python3 -I tools/corpus/measure.py baseline --results r.json` after the `run` above, or the `corpus-measurement` artifact the job uploads). Streaming E2E reads its own pinned nextgis files (`tools/streaming/*-corpus.json`, cross-checked against `pins.tsv`).

The older real-file integration tests (`tests/samples.rs`, `block_names.rs`, `corpus_roundtrip.rs`, `r2013_entity_values.rs`) read a 19-file subset at `../../samples` relative to the crate root and **skip silently** when it is absent, so a green local `cargo test` says nothing about real drawings unless that directory exists. `fetch.py --legacy-samples ../../samples` writes exactly that subset there; from a worktree, `../../samples` resolves against the worktree's own path.
The older real-file integration tests (`tests/samples.rs`, `block_names.rs`, `corpus_roundtrip.rs`, `r2013_entity_values.rs`) read a 19-file subset at `../../samples` relative to the crate root and **skip silently** when it is absent, so a green run of them says nothing about real drawings unless that directory exists. `fetch.py --legacy-samples ../../samples` writes exactly that subset there; from a worktree, `../../samples` resolves against the worktree's own path.

Coverage numbers come from two places, and each says which: the headline tables in `README.md` and `STATUS.md` cite the corpus job's run on `main`, while older sections (STATUS.md's 2026-08-30 baseline, the evidence behind `closes` states in `docs/spec-coverage.md`) are dated local measurements of the 19-file subset. Never estimate or carry a number forward: re-measure, and cite the run or the command and date.

Expand All @@ -100,8 +100,8 @@ Invariants across the crate: `#![forbid(unsafe_code)]`; malformed input returns

- **Clean-room** (`CLEANROOM.md`): never consult or import LibreDWG, the ODA/Teigha SDK, Autodesk SDKs or any GPL DWG source. Algorithm-description comments in MIT, Apache or BSD projects may resolve a spec ambiguity only with a dated disclosure added to `CLEANROOM.md`. Every PR body carries the source-provenance declaration from the PR template.
- **Cite the spec by v5.4.1 number.** Objects are §20.4.x; older code still says §19.4.x (#106). `docs/spec-coverage.md` gives every section a state (`missing`, `decoder`, `partial`, `closes`, `complete`) and is generated: edit `docs/data/spec-coverage.tsv`, then run `python3 tools/ci/spec_coverage.py --write`. CI's Spec coverage job fails when a section has no row or the page and its data disagree. A state changes only in the PR that brings its evidence.
- **Evidence is real-file, not unit tests.** The PR template asks for the corpus records a change was measured on and the reproducing command, with no new unit tests; existing suites stay and must pass. A new format finding gets a probe under `examples/` (module doc saying what it proves and how to verify it) and a dated section in `ARCHITECTURE.md` with its evidence table.
- **New decoders** follow `docs/EXTENDING_DECODERS.md`. For a fixed-code entity: a type-code constant, a `DecodedEntity` variant, a `fixed_entity_label` entry and a `decode_fixed_entity_body` arm in `entities/dispatch.rs`. The label list and the body arms must match (`fixed_entity_label_covers_every_body_arm` checks it), and `checked_inline` applies the boundary check for you. Custom classes dispatch on their DXF class name instead. `templates/entity-decoder` is a cargo-generate skeleton.
- **Unit tests are banned** (the maintainer's standing directive, 2026-09-23), and evidence is real-file. Do not add one, and do not add a regression test for a fix: a unit-suite pass count is never evidence. The PR template asks for the corpus records a change was measured on and the reproducing command. `tools/ci/test-targets.txt` classes every `tests/*.rs` file as `real` (reads real drawings or the committed canonical fixtures), `cli`, `contract` (checks a published claim), `hardening` (replays inputs that crashed a parser) or `unit` (synthetic-only or property tests); the Test job runs all but `unit`, and `tools/ci/run_test_targets.sh` fails on a file the manifest does not class, so a new target needs a line there and a reason. The `unit` files stay as unrun source, and the inline `#[cfg(test)]` suites in `src/` are deleted area by area (git history is the archive): never add one, and never let a decoder PR re-grow one. A new format finding gets a probe under `examples/` (module doc saying what it proves and how to verify it) and a dated section in `ARCHITECTURE.md` with its evidence table.
- **New decoders** follow `docs/EXTENDING_DECODERS.md`. For a fixed-code entity: a type-code constant, a `DecodedEntity` variant, a `fixed_entity_label` entry and a `decode_fixed_entity_body` arm in `entities/dispatch.rs`. The label list and the body arms must match (`python3 tools/ci/dispatch_arms.py --check` reads both from the source, in CI's Spec coverage job), and `checked_inline` applies the boundary check for you. Custom classes dispatch on their DXF class name instead. `templates/entity-decoder` is a cargo-generate skeleton.
- Every `pub fn` in `src/` gets a `///` line naming its bit code, flag mask or spec section; the docs-rs workflow fails below 80% and the tree is at 100%.
- No absolute home paths (`/Users/<name>/`, `/home/<name>/`, `C:\Users\<name>\`) anywhere outside `.github/workflows/`: the PII guard fails on them. No PII in fixtures; use synthetic values.
- Conventional Commits with the scopes listed in CONTRIBUTING.md. `main` is squash-merge only, so one PR is one commit and one finding.
36 changes: 32 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,15 @@ The gate (identical to the required CI checks — run it before every push):
```sh
cargo fmt --all -- --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --release
tools/ci/run_test_targets.sh --release --all-features
cargo deny check all # cargo install cargo-deny
```

`tools/ci/run_test_targets.sh` runs the real-file, contract and hardening
targets of [`tools/ci/test-targets.txt`](tools/ci/test-targets.txt) and the
documentation examples, as CI's Test job does. Unit tests are banned; see
[Unit tests are banned](#unit-tests-are-banned) below.

Two soft gates you will also hit:

- **Docstrings.** The docs-rs workflow requires at least 80% of `pub fn` in
Expand Down Expand Up @@ -143,8 +148,8 @@ Open an issue (or a draft PR) before starting work on any of:

- Rust 2024 edition.
- `cargo fmt` before every commit.
- `cargo test --release` must pass. The CI in `.github/workflows/`
enforces this.
- `tools/ci/run_test_targets.sh --release --all-features` must pass. The
CI in `.github/workflows/` enforces this.
- **No `unsafe` in the library crate.** See above.
- **No panics in parsing paths.** Malformed input must return an
`Error`, never `panic!`. Defensive caps on loop iteration and
Expand All @@ -155,6 +160,29 @@ Open an issue (or a draft PR) before starting work on any of:
- **Every probe under `examples/`** gets a module-level doc comment
explaining *what fact it proves* and *how to verify* the result.

## Unit tests are banned

Do not add them, and do not add a regression test for a fix. A unit-suite
pass count is not evidence. Changes are verified end to end: against real
drawings and AutoCAD's own DXF exports of them, through the built binaries,
and by the contract checks that keep the published claims honest. The
evidence a pull request carries is the corpus records it was measured on and
the command that reproduces it.

`tools/ci/test-targets.txt` classes every file under `tests/` as `real`
(reads real drawings, or the committed canonical fixtures), `cli`, `contract`
(checks a published claim), `hardening` (replays inputs that crashed a
parser) or `unit` (synthetic-only or property tests).
`tools/ci/run_test_targets.sh` runs all but `unit`, in CI's Test job on every
OS, and fails on a file the manifest does not class, so a new target needs a
line there and a reason. The `unit` files stay as source and are not run; the
inline `#[cfg(test)]` suites in `src/` are being deleted area by area, and
git history is the archive. Targets that read the corpus skip cleanly when
`../../samples` is absent, as it is on every Test leg; the Corpus (public DWG)
job places the 19 pinned drawings there for the targets it lists. For a format
fact, the evidence is a probe under `examples/` and a dated section of
`ARCHITECTURE.md`.

## Commit messages

Conventional Commits format:
Expand Down Expand Up @@ -182,7 +210,7 @@ When you discover something new about the file format:
(byte offsets, observed values, variant count across versions).
3. If the finding is a decoding rule, show it on real files: name
the corpus records it changes and paste the coverage report from
before and after into the pull request. Don't add unit tests.
before and after into the pull request. Don't add tests.

This keeps every claim independently verifiable, which is the
whole point of open reverse-engineering work.
Expand Down
2 changes: 1 addition & 1 deletion RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ Before tagging, confirm:
```bash
cargo fmt --all -- --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --release --all-features
tools/ci/run_test_targets.sh --release --all-features
RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features
cargo deny check advisories bans licenses sources
cargo package --allow-dirty # dry-run the publish tarball
Expand Down
9 changes: 5 additions & 4 deletions docs/EXTENDING_DECODERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,8 +142,8 @@ Other kinds of record take other routes in the same file:
## Step 4 — Prove it on real files

The proof that a decoder is right is the corpus, not a synthetic
payload you built to match your own reading of the spec. Do not add
unit tests (the pull-request template asks for none).
payload you built to match your own reading of the spec. Unit tests are
banned (see `CONTRIBUTING.md`): do not add one, inline or under `tests/`.

1. Measure before and after with the coverage report on real
drawings:
Expand Down Expand Up @@ -209,11 +209,12 @@ count-driven decoder.
```bash
cargo fmt --all -- --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --release --all-features
tools/ci/run_test_targets.sh --release --all-features
python3 tools/ci/dispatch_arms.py --check
RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features
```

The real-file integration tests (`tests/samples.rs` and its
The real-file targets (`tests/samples.rs` and its
neighbours) look for the corpus at `../../samples` relative to the
crate and skip silently when it is absent, so a passing run only
covers real drawings if that directory exists.
Expand Down
Loading
Loading