Skip to content

feat: matched random control, quieter readout, safe plugin installs - #213

Merged
KartikP merged 4 commits into
unified-model-interface-v2from
feat/newuser-ergonomics
Nov 5, 2026
Merged

KartikP merged 4 commits into
unified-model-interface-v2from
feat/newuser-ergonomics

Conversation

@KartikP

@KartikP KartikP commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Changes from a new user's review of UMI v2.

  • RandomSelection.matched_to(selection, seed=0): the matched random control in one call. It copies the layer, unit count, population and layout from a resolved functional selection.
  • Readout warning: the sklearn multi_class deprecation notice is silenced at fit only. The argument is still honoured below 1.8, and dropping it would move binary probabilities; environment_check already guards the version.
  • Plugin installs:
    • Installing a plugin's requirements now holds the shared pins (numpy, torch, transformers, ...) at their installed versions. A plugin that needs other versions raises PluginDependencyError naming the conflict ("the plugin asks for transformers==4.30.2; this environment holds transformers==4.57.6") and how to give it its own environment. Packages with no build for the platform, or that fail to build, get their own message.
    • Requirements lines that name Python's own modules (imp, collections) are skipped. They made 16 vision models (BiT-S, mobilenet_v2_*, vonegrcnn) fail to install anywhere.
    • In newenv mode, the plugin may still set any versions it needs.

Dry-run of all 630 vision and language plugins against the pinned environment:

  • 535 install without changing anything; 67 have no requirements.
  • 8 would move a pin (7 CvT models: transformers 4.57 to 4.30; lm1b: numpy 2). These are now refused with the message above.
  • 1 other moves only OpenCV (cv_18_dagger_408_pretrained) and still installs.
  • The other 19 cannot install at all: 16 were the built-in module names, now skipped; 3 have packages with no Mac build.

Note: the existing newenv runner (conda_score.sh) cannot work on v2 today. It installs vision into a fresh environment, and vision pins brainscore-core==2.4.0rc1, which is not on PyPI. The conflict message therefore suggests cloning the environment instead.

CI runs unified at KartikP/brainscore-unified feat/newuser-ergonomics, whose dependency test checks the new install command.

Tests: core suite passes locally, except tests that need conda on PATH or a checkout path without spaces (they fail the same way on base).

https://claude.ai/code/session_01WU1w1HjLtiEMf1v2LHdoKs

The argument is still honoured below sklearn 1.8 and dropping it would move binary probabilities; environment_check guards the version, so the notice was noise on every fit.

Claude-Session: https://claude.ai/code/session_01WU1w1HjLtiEMf1v2LHdoKs
Installing a plugin's requirements no longer moves numpy, torch, transformers or the other shared pins; a conflicting plugin raises PluginDependencyError naming the conflict and how to give it its own environment. Python's own module names (imp, collections) in requirements files are skipped.

Claude-Session: https://claude.ai/code/session_01WU1w1HjLtiEMf1v2LHdoKs
@KartikP
KartikP merged commit ae7c857 into unified-model-interface-v2 Nov 5, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant