Seed novel's internal validation split

CHANGELOG.md

@@ -42,6 +42,8 @@
 
 * `guanlab_dengkw_pm`: Only map the predictions back through the mod2 SVD when that SVD was actually fitted. When the target had fewer features than `n_mod2`, the method raised `NameError: embedder_mod2`. Unsupported modality pairs now exit 99 (non-applicable) instead of raising a `KeyError` (PR #32).
 
+* `novel_train`: Seed the fallback that picks the internal validation batches, and log which ones were picked. Both `bmmc_multiome` datasets take this path, so the checkpoint `novel` kept differed from run to run (PR #46).
+
 * `process_dataset`: Fall back to holding out a quarter of the batches when the dataset has no `obs["is_train"]`, rather than silently producing four empty h5ads. `obs["is_train"]` carries the NeurIPS 2021 competition split and stays optional; `obs["cell_type"]` is now declared and required (PR #28).
 
 * Fix the component paths, build paths and `rename_keys` separator in the helper scripts, which prevented `scripts/create_datasets/test_resources.sh` and both `run_test.sh` scripts from running at all (PR #22).

src/methods/novel/novel_train/config.vsh.yaml

@@ -7,6 +7,10 @@ arguments:
     description: Number of training epochs.
     info:
       test_default: 2
+  - name: "--seed"
+    type: "integer"
+    default: 1
+    description: "Seed for choosing the internal validation batches, when the dataset does not contain the batches used in the competition."
 resources:
   - path: script.py
     type: python_script

src/methods/novel/novel_train/script.py

@@ -24,7 +24,8 @@
   'input_train_mod1': 'resources_test/task_predict_modality/openproblems_neurips2021/bmmc_multiome/normal/train_mod1.h5ad',
   'input_train_mod2': 'resources_test/task_predict_modality/openproblems_neurips2021/bmmc_multiome/normal/train_mod2.h5ad',
   'output': 'resources_test/task_predict_modality/openproblems_neurips2021/bmmc_multiome/normal/models/novel',
-  'n_epochs': 100
+  'n_epochs': 100,
+  'seed': 1
 }
 meta = {
   'resources_dir': 'src/methods/novel',
@@ -77,7 +78,9 @@
 # if none of phase1_batch is in batch, sample 25% of batch categories rounded up
 if len(test_batches.intersection(set(batch))) == 0:
   all_batches = batch.cat.categories.tolist()
-  test_batches = set(np.random.choice(all_batches, math.ceil(len(all_batches) * 0.25), replace=False))
+  rng = np.random.default_rng(par['seed'])
+  test_batches = set(map(str, rng.choice(all_batches, math.ceil(len(all_batches) * 0.25), replace=False)))
+print(f"Using {sorted(test_batches)} as internal validation batches", flush=True)
 train_ix = [ k for k,v in enumerate(batch) if v not in test_batches ]
 test_ix = [ k for k,v in enumerate(batch) if v in test_batches ]