Repo-level cross-converter interop matrix (hub-level testing)

[MikeNitsenko]

Problem

Every converter tests its own round trip (X → Ossie → X), so all suites can be green while converter pairs fail. Real cases found by manually chaining converters during Cube-converter development:

1. Silently empty output. A Databricks-authored Ossie document carries only DATABRICKS-dialect expressions; an exporter that required ANSI_SQL dropped every field and metric and emitted an empty target model — exit 0, loadable, nothing downstream noticed.
2. Mis-attributed aggregates. Running Cube → Ossie → Databricks → Ossie → Cube on the TPC-DS model, SUM(ss_ext_sales_price) came back placed on the customer cube — a measure over a column that cube does not have. It compiles (SQL is opaque to compilers) and reads the wrong table at query time.
3. Structural mismatches. A Databricks metric view cannot carry the source table's own key, so it silently vanishes on the way through; Databricks, Snowflake, and NVIDIA GSF reject the 2-part source other converters happily emit. Each converter's own tests pass; the pair fails.

Prior art: converters/cube/tools/interop_matrix.py — one-directional (Cube → Ossie → every other spoke), Cube-owned. This proposal promotes the idea to a repo-level facility owned by everyone.

Proposal

A repo-level harness (e.g. interop/) driven by one small manifest entry per converter — the CLIs genuinely differ today (export vs osi-to-msi vs no subcommand vs API-only GoodData), and a manifest normalizes over that without forcing a CLI migration first:

# interop/manifest.yaml (sketch)
cube:
  import: [ossie-cube, import]        # native -> Ossie
  export: [ossie-cube, export]        # Ossie -> native
  output: directory
  fixture: converters/cube/tests/fixtures/tpcds_cube
  toolchain: uv
  native_gate: [node, tools/cube_compile.js]   # optional, env-gated

databricks:
  import: [ossie-databricks, import]
  export: [ossie-databricks, export]
  output: file
  fixture: converters/databricks/tests/fixtures/...
  toolchain: uv

polaris:
  toolchain: maven   # Java converters enter via a Maven runner, or stay SKIP

Test tiers

• Tier A — Ossie acceptance. Every export must consume the canonical examples/*.yaml and every import's output: exit 0 and non-empty output.
• Tier B — pairwise chains. For each pair (A, B): A.fixture → Ossie → B.export → B.import → Ossie′, where Ossie′ must pass validation/validate.py and the JSON Schema.
• Tier C — native gates. Where a target has a real compiler/validator (Cube's compiler today, behind OSSIE_CUBE_REPO), run it on the chain result.

Grading

OK / LOSSY (n warns, m foreign-ext warns) / EMPTY / FAIL / SKIP (toolchain)

• EMPTY is a first-class failure, not a variant of OK — it is the trap in case 1 above.
• Foreign custom_extensions warning counts quantify what each converter's stash costs every other spoke — the number to watch when deciding whether something belongs in an extension at all.

CI

Report-only markdown matrix as a build artifact first. Once green, ratchet Tier A on examples/tpcds_semantic_model.yaml to blocking. Java converters (Polaris, Salesforce) stay SKIP until a Maven runner is wired.

Offer

We will contribute the harness by generalizing the existing Cube tool; each converter's owners only fill in their manifest entry.

Sent discussion: [DISCUSS] Repo-level cross-converter interop matrix on dev@ossie.apache.org but they don't appear on lists.apache.org for some reasons =(