Repository navigation
feat(competitors): aspose-cells-foss + zavora-xlsx adapters, Tier 4, mutation + calc lanes - #45
Merged
Merged
Conversation
…n + calc lanes
- aspose-cells-foss adapter (Python lane, 'aspose' extra): 11/22 green
- zavora-xlsx external-oracle adapter (cross-language lane) + Rust helper;
upstream 0.1.2 defects scored honestly (hyperlink rel corruption, no recalc)
- Tier 4 fidelity features (sheet_protection, page_setup, chart_anchor) with
openpyxl-structural fixtures; 22 scored features total
- wolfxl adapter ported to WolfXL 2.1.0 public API (19/22; CF-read and
print-title-write gaps recorded)
- Template-mutation lane: wall time, peak RSS, package preservation
- Calc tier: cache-free 133-formula financial DAG fixture, LibreOffice oracle,
strict pass/fail semantics
- CLI: 'excelbench mutation' and 'excelbench calc'; zavora in cross-language lane
- Snapshot artifacts: results-2026-09-08/{xlsx,mutation,calc,cross-language}
- Docs: DEC-023..DEC-026, README tables, METHODOLOGY, TRACKER, architecture
Verification: uv run pytest (1235 passed, 79 skipped, 6 xfailed); ruff clean.
…extra - base.unsupported_operation now annotated NoReturn (it always raises); downstream use is unreachable and mypy-clean without local asserts - restore pyumya mypy override clobbered by the aspose override entry; add aspose/aspose.* overrides for the untyped installed package - type the wolfxl adapter pivot/table fallbacks, calc suite status/reason locals, aspose merged-ranges return - guard optional module imports in modifiable.py with the registry pattern - pytest.importorskip for aspose-cells-foss when the extra is absent Verification: ruff clean; mypy 0 issues in 149 files; pytest 1235 passed, 79 skipped, 6 xfailed.
openpyxl stamps dcterms:modified with save time, so the committed fixture drifted on every test run that rebuilt it. The builder now rewrites the modified timestamp to the fixed epoch in its ZIP repack stage; the artifact and manifest are regenerated against the pinned bytes and the recorded template_sha256 matches the committed fixture again. The regenerable tier4_manifest_fragment.json merge byproduct is untracked. Verification: two builds 2s apart produce identical bytes; mutation suite rerun preserved scores (60/100/60/0) with matching artifact SHA; tests/test_mutation_suite.py 7 passed.
importorskip ran after the module-level adapter import, which raises ImportError when the extra is missing; the guard now precedes it so CI environments without the 'aspose' extra collect-skip the module.
CI runners have no soffice binary at the hardcoded homebrew path. Resolve the binary via PATH first (homebrew fallback kept), and skip the three LibreOffice-dependent tests (calc builder regeneration, engine smoke, mutation template PDF render) when it is absent. The wolfxl calc-engine assertion engages only on wolfxl >= 2.1, matching the calculate() API the engine requires. Verification: ruff clean; mypy 0 issues; pytest 1235 passed locally; with soffice hidden the two calc tests skip with reason and the mutation LO test carries the same guard.
The tier4 builder test regenerates the merge byproduct with a fresh timestamp on every suite run, dirtying the tree; manifest.json is the merged source of truth.
Adds 28 contract tests for paths CI can execute without LibreOffice, aspose-cells-foss, or the Rust helper: strict calc-tier status semantics with synthetic engines, expected-payload validation, cache-free fixture invariants, engine-unavailable reporting, mutation-suite control flow (unavailable/failed/integrity/passed rows, repeats guard, manifest parsing, time-output parsing, package integrity helpers), modifiable engine guards, mutation-report verdict branches, and Zavora adapter payload/error contracts. Harness fixes the tests exposed: - run_calc_suite now contains engine exceptions as failed rows instead of aborting the suite - LibreOfficeCalcEngine reports unavailable (not failed) when the binary is missing Verification: ruff clean; mypy 0 issues in 150 files; full suite 1263 passed, 79 skipped, 6 xfailed.
Adds seven round-trip tests exercising WolfxlAdapter through the public WolfXL API: value/boolean/blank round-trips, source-path bookkeeping, format/border/dimension/merge, formula, hyperlink, and structural features (freeze panes, named ranges, comments, tables). All operations verified identical on WolfXL 2.0.1 (PyPI) and 2.1.0 (local wheel), so the tests run ungated in CI. Adapter coverage: 1% -> 45% (261 of 586 statements). Verification: ruff clean; mypy 0 issues; full suite 1270 passed, 79 skipped, 6 xfailed; every test re-run against wolfxl 2.0.1 in a throwaway venv.
The test job previously installed only the dev extra, so wolfxl and aspose-cells-foss were absent on CI: the wolfxl adapter tests skipped and roughly a thousand adapter statements could never count toward coverage, pinning the suite at 63.8% against the 65% gate. Both extras are pure installs from PyPI (wolfxl 2.0.1 wheel, aspose-cells-foss py3-none-any wheel). Verified in a CI-parity venv (python 3.12, wolfxl 2.0.1, aspose-cells-foss 26.7.0, no soffice): 1270 passed, 79 skipped, 6 xfailed, exact coverage 66.43%.
The test job's new rust extra resolved to the lock's ancient wolfxl 0.1.0, whose Workbook lacks the remove() API the adapter and its round-trip tests rely on. Upgrades the single lock entry to 2.0.1 (plus its new defusedxml dependency); no other packages change. Verified in a venv synced exactly from the updated lock (python 3.12, wolfxl 2.0.1, aspose-cells-foss, no soffice): 1270 passed, 79 skipped, 6 xfailed, exact coverage 66.31%.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extends the scored matrix to 22 features and 14 Python adapters, adds two competitor adapters, and ships two new decision-surface lanes (template mutation, formula recalculation). Snapshot artifacts:
results-2026-09-08/{xlsx,mutation,calc,cross-language}.What's included
Adapters
aspose-cells-foss26.7 (Python lane, opt-inasposeextra): 11/22 green features; explicit unsupported surfaces for pivots/chart rendering/math evaluation.zavora-xlsx0.1.2 (cross-language lane only): Rust helper intools/external-oracles/zavoraover the existing external-oracle JSON contract. Upstream defects are scored, not hidden: mutate/open-save corrupts hyperlink relationships (mutation lane:integrity-failed, preservation 0), andrecalculate()writes no caches (calc lane: 0/133).Tier 4 fidelity features (
sheet_protection,page_setup,chart_anchor)generator: openpyxl-structural,excel_version: none) because no Excel app is available here; expectations are verifier-grade openpyxl readbacks. Documented in DEC-023.password, chartdata_ref/categories_ref) ride in the manifest and are stripped pre-compare.Template-mutation lane (
excelbench mutation)fixtures/mutation/template_corporate_model.xlsx, measured in isolated subprocesses: median wall time, median peak RSS, package-preservation score; outputs must reopen with the verifier or the engine isintegrity-failed.Calc tier (
excelbench calc)fixtures/calc/financial_model.xlsxis a 133-formula financial DAG that ships no cached values; the builder asserts cache-freedom. Oracle values come from a throwaway LibreOffice headless recalculation (version pinned in the artifact).failedwith matched/total (DEC-026).^power-operator family and dependents), aspose-cells-foss 25/133 (FOSS FormulaEvaluator has no value for 107 formulas), zavora 0/133.wolfxl adapter
Fidelity headline (14 Python adapters, 22 features)
Verification
uv run pytest: 1235 passed, 79 skipped, 6 xfaileduv run ruff check .: cleanuv run excelbench benchmark --tests fixtures/excel --output results-2026-09-08/xlsx --adapter ...(14 Python-lane adapters): rendered, 308 score rowsuv run excelbench cross-language-context --output results-2026-09-08/cross-language: zavora-xlsx write laneuv run excelbench mutation --output results-2026-09-08/mutation: 4 engines, results.json + READMEuv run excelbench calc --output results-2026-09-08/calc: 4 engines vs LibreOffice oraclebuild_tier4_fixtures.py,build_mutation_template.py,build_calc_fixture.py(cache-free assertion active)Notes for reviewers
docs/public-reporting.mdupdated accordingly.results-2026-09-08/calc/*/contains per-engine recalculated workbooks as evidence.Merge note
Do not merge before the WolfXL 2.1.0 gaps are triaged upstream if the public README headline is meant to stay at 18/18; otherwise this PR's headline intentionally reflects 2.1.0 as-is.
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.