Skip to content

Stop copying train_mod2 into novel's model artifact - #47

Merged
rcannood merged 2 commits into
mainfrom
perf/novel-model-artifact-size
Jul 30, 2026
Merged

Stop copying train_mod2 into novel's model artifact#47
rcannood merged 2 commits into
mainfrom
perf/novel-model-artifact-size

Conversation

@rcannood

Copy link
Copy Markdown
Member

Describe your changes

novel_train writes a full copy of the training target into its model directory:

adata = input_train_mod2.copy()
...
adata.uns["model_dim"] = {"mod1": n_vars_mod1, "mod2": n_vars_mod2}
adata.write_h5ad(f"{par['output']}/train_mod2.h5ad", compression="gzip")

but novel_predict only ever touches uns on it:

mod2        = input_train_mod2.uns['modality']
n_vars_mod1 = input_train_mod2.uns["model_dim"]["mod1"]
n_vars_mod2 = input_train_mod2.uns["model_dim"]["mod2"]
rem_var     = input_train_mod2.uns["removed_vars"]

so the matrix, obs and var are written, published and re-read for nothing. Keeping just the uns:

modality:  ATAC
model_dim: {'mod1': 256, 'mod2': 1500}
removed_vars via .get(): None

artifact size old: 0.69 MB   (test fixture, 171 x 1500)
artifact size new: 0.53 MB

The fixture is small enough that h5ad overhead dominates; on the real datasets this is the difference between carrying a full expression matrix per model and not.

I did consider having novel_predict read --input_train_mod2 directly -- the wrapper already passes it -- but that argument is optional in comp_method_predict.yaml, so the model artifact staying self-describing seemed safer.

Part of a series of PRs coming out of a pre-run review of the benchmark.

Checklist before requesting a review

  • I have performed a self-review of my code

  • Check the correct box. Does this PR contain:

    • Breaking changes
    • New functionality
    • Major changes
    • Minor changes
    • Bug fixes
  • Proposed changes are described in the CHANGELOG.md

  • CI Tests succeed and look good!

rcannood added 2 commits July 29, 2026 14:13
novel_predict only reads uns from that file -- modality, model_dim and
removed_vars -- so there's no reason to carry the expression matrix,
obs and var along with it.
@rcannood
rcannood merged commit e0e3c90 into main Jul 30, 2026
1 check passed
@rcannood
rcannood deleted the perf/novel-model-artifact-size branch July 30, 2026 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant