Add Dummy's results on the eight largest Multiverse datasets - #32
Open
TonyBagnall wants to merge 1 commit into
Open
TonyBagnall wants to merge 1 commit into
TonyBagnall wants to merge 1 commit into
Conversation
DREAMERA, DREAMERV, InsectWingbeat, LenDB and the four S2Agri datasets, from the first-submission Dummy runs, now copied into D:/Results/Multiverse. They use the same configuration as the current Dummy (strategy prior, random_state 0), and the five whose data is held locally match the class labels exactly. The two full S2Agri prediction files hold 12 million rows each, too large to load through tsml-eval here, so the metrics were computed from counts of each (true, predicted) pair with the one probability row every Dummy row shares, using tsml-eval's metric definitions as sample-weighted scikit-learn calls. The same computation reproduces all 125 existing Dummy results on all seven metrics to within 1e-15. Existing rows are unchanged. No table changes, since none of the eight is complete for any other estimator. InsectWingbeat is added to DEFERRED_DATASETS: with a Dummy result it would otherwise be listed against every other estimator on the UEA page as not run outside Multiverse-core, which is wrong, as the paper's classifiers were run on it and did not finish. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DREAMERA, DREAMERV, InsectWingbeat, LenDB and the four S2Agri datasets, from the first-submission Dummy runs, now copied into D:/Results/Multiverse. They use the same configuration as the current Dummy (strategy prior, random_state 0), and the five whose data is held locally match the class labels exactly.
The two full S2Agri prediction files hold 12 million rows each, too large to load through tsml-eval here, so the metrics were computed from counts of each (true, predicted) pair with the one probability row every Dummy row shares, using tsml-eval's metric definitions as sample-weighted scikit-learn calls. The same computation reproduces all 125 existing Dummy results on all seven metrics to within 1e-15. Existing rows are unchanged.
No table changes, since none of the eight is complete for any other estimator. InsectWingbeat is added to DEFERRED_DATASETS: with a Dummy result it would otherwise be listed against every other estimator on the UEA page as not run outside Multiverse-core, which is wrong, as the paper's classifiers were run on it and did not finish.
Summary
Checklist
results/schema.md.