Repository navigation
README: practical notes from a pre-release user study - #10
Merged
Merged
Conversation
Clustered labels need a grouped split; grouped folds need the same groups in propensity_split; low n_eff or auc near 1 means no usable estimate; features must be numeric; integer labels with a separate deployment file; exact lat/lon names; PYTHONPATH for a local --fit-model module. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Docs only (README, +12/-6). Two agents used the library as new users (Helmets + AlphaEarth; UCI Adult and wine quality). These notes cover the problems that mislead users but are too large to fix in code before release; the code fixes go to v0.1.1.
Added, each verified against the code or a repro:
Split(name, groups=...). On Helmets (2 rows per photo), the default row split gave known-inclusion bias -0.06 / 70% coverage vs -0.017 / 95% with a photo split. The bootstrap then resamples whole clusters by default.propensity_split: states the default (correction groups on labeled rows, each deployment row alone) and says to pass the same groups when folds are grouped. With 1 km block folds and the default, propensity AUC fell to 0.37.n_effbelow a few dozen oraucnear 1 as no usable estimate. Real Helmets printed n_eff 5, intervals [-0.04, 0.72], AUC 0.99, with no flag.could not convert string to float); rows with a missing value are dropped; integer labels with a separate deployment file read as3.0and amodelfitted on3is rejected;lat/lonmust be exactly these names.--fit-modelmodule in the current folder needsPYTHONPATH=.(checked).Review: wording, and the "few dozen"
n_effrule of thumb, which is a judgment call.🤖 Generated with Claude Code