Skip to content

Caltable: type each site's gains and D-terms from its feed type - #335

Draft
Lftw wants to merge 3 commits into
dev-backend-mixpolfrom
refactor/caltable-basis-dispatch
Draft

Lftw wants to merge 3 commits into
dev-backend-mixpolfrom
refactor/caltable-basis-dispatch

Conversation

@Lftw

@Lftw Lftw commented Oct 2, 2026

Copy link
Copy Markdown
Collaborator

What and why

Every site's gain and D-term tables were typed circular, whatever the station's feeds, so a linear station's X/Y gains were stored under rscale/lscale. Caltable now types each site from tarr['feed_type']:

ct.gains['ALMA']    # DTCAL_LIN:  (time, xscale, yscale)   for an 'xy' station
ct.gains['APEX']    # DTCAL_CIRC: (time, rscale, lscale)   for an 'rl' station
ct.gains[site]['p1scale']   # generic names work on every site

Internally the module reads and writes the generic p1scale/p2scale and d_p1/d_p2 names, so applycal, merge, pad_scans, scan_avg, enforce_positive and invert_gains are one code path for both bases. DTCAL_LIN/DTDTERM_LIN now survive save/load; the file format is unchanged, the basis rides on array.txt. make_jones types its simulated cal table per station instead of always circular.

Pure refactor, no new capability: this is the step before applycal learns to apply gains on any basis (next PR). Circular data is byte-identical throughout.

Design decisions worth discussing

  • ehc.caltable_dtypes(feed_type) is the single feed_type -> (DTCAL_*, DTDTERM_*) map. 'rl' and 'xy' only. Hybrid feeds raise like field_rotation_matrix does; '??' says to declare the feeds; 'lr'/'yx' now raise at construction (before, they were silently typed circular, which labelled p1 as R).
  • Columns map by name, not position. A (time, lscale, rscale) table used to come out with its feeds swapped. Tables with names that can't be placed in a slot raise instead of being stored and failing later.
  • The declared feed type wins over the field names. A circular-typed table on an 'xy' station is relabelled silently, because every solver still writes ehc.DTCAL whatever the basis, so r/l names carry no information. The one direction that warns is x/y names under an 'rl' station, since only basis-aware code writes those.
  • plot_gains/plot_compare_gains pol accepts R|L|X|Y|p1|p2|both, case-insensitive. A physical name from the wrong basis raises up front. The default is 'p1', which is 'R' on a circular site.
  • merge refuses the same site in two bases; disjoint site lists in different bases (ALMA in LIN, the rest in CIRC) merge as before.

Bugs found along the way

  • plot_gains(pol='both') passed the validator and then hit an unset local; it has never worked
  • plot_gains bound tmins = tmaxes = gmins = gmaxes = [] to one list, so the y-range was computed from the times as well as the gains
  • plot_compare_gains with any pol other than 'R'/'L' hit an unset local
  • load_caltable's fallback filename was missing a path separator
  • summary_plots.imgsum, modeling_utils.caltable_to_gains and the interactive dashboard read rscale/lscale directly, which a linear station no longer has; they now use the generic names

Deferred

  • applycal still forces switch_polrep('circ'); gains on any basis is the next PR, apply_dterms=True the one after
  • The solvers (self_cal, network_cal, polgains_cal, modeling_utils) still write ehc.DTCAL; the constructor relabels, so nothing breaks, but they should write the site's own dtype eventually
  • make_jones keeps its gainR/gainL variable names: the surrounding gain model is circular-specific (gain_RLratio, rlgaincal), so only the carrier variables would have been renamed

How to test

micromamba run -n jax-ehtim pytest tests/test_caltable.py tests/test_mixedpol.py tests/test_obs_simulate.py

142 + 231 + 95 pass. Sections 1-11 of test_caltable.py are the no-regression contract from #331; the one change there is test_pad_scans_keeps_linear_gain_dtype, whose station is now genuinely 'xy' rather than a LIN table on an 'rl' site, which dispatch makes self-contradictory. Section 12 is new. test_mixedpol.py gains the make_jones cal-table checks; they were confirmed to fail with the writer fix reverted.

Part of #212.

@codecov

codecov Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.73585% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 49.40%. Comparing base (571e00f) to head (ea7d82e).

Files with missing lines Patch % Lines
ehtim/caltable.py 89.00% 7 Missing and 4 partials ⚠️
ehtim/plotting/interactive.py 66.66% 1 Missing ⚠️
ehtim/plotting/summary_plots.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@                  Coverage Diff                   @@
##           dev-backend-mixpol     #335      +/-   ##
======================================================
+ Coverage               49.17%   49.40%   +0.22%     
======================================================
  Files                      54       54              
  Lines                   27241    27266      +25     
  Branches                 4662     4668       +6     
======================================================
+ Hits                    13397    13471      +74     
+ Misses                  12324    12262      -62     
- Partials                 1520     1533      +13     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Each site's gain and D-term table now takes its dtype from the site's
feed_type in the Caltable constructor, with a zero-copy view like Obsdata
uses for legacy data. This replaces the const_def helpers, so writers
keep writing DTCAL. A site's D-term table is copied into the caltable's
tarr as its time average, and applycal's tarr check skips the D-term
columns.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant