Skip to content

Add Dummy's results on the eight largest Multiverse datasets #32

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
TonyBagnall wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add Dummy's results on the eight largest Multiverse datasets #32

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions multiverse/experiments/tables.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -412,6 +412,12 @@ def _missing_by_estimator(frames, estimators, common, datasets):
"the series are length 8 and MRHydra requires at least 9, so the dataset "
"cannot complete while MRHydra is a column"
),
"InsectWingbeat": (
"25,000 training cases of 200 channels. Only the Dummy baseline has a "
"result on it: none of the classifiers in the Multiverse archive paper "
"finished it within the resource limits there, so it cannot enter a "
"table that needs every estimator"
),
"BenzeneConcentration_disc": (
"removed from Multiverse-core. The results here are on version 1, whose "
"PT08.S2 channel is a deterministic function of the target; version 2 "
Expand Down
8 changes: 8 additions & 0 deletions results/multiverse/Dummy/Dummy_accuracy.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ CharacterTrajectories,0.06476323119777158
CounterMovementJump,0.33519553072625696
Cricket,0.08333333333333333
CrowdSourced,0.5001764913519238
DREAMERA,0.3934409619021886
DREAMERV,0.5811267225074305
DuckDuckGeese,0.2
ERing,0.16666666666666666
EigenWorms,0.4198473282442748
Expand Down Expand Up @@ -53,6 +55,7 @@ IRDS-STR,0.9166666666666666
ImaginedFeetHands,0.5002583979328166
ImaginedOpenCloseFist,0.49947089947089945
InnerSpeech,0.235
InsectWingbeat,0.1
JapaneseVowels,0.08378378378378379
KERAAL-CTK,0.5
KERAAL-CTK-MC,0.5
Expand All @@ -70,6 +73,7 @@ KINECAL-GGFV,0.9230769230769231
KINECAL-QSEC,0.9444444444444444
KINECAL-QSEO,0.9411764705882353
LSST,0.3150851581508516
LenDB,0.5092767890950397
Libras,0.06666666666666667
LiveFuelMoistureContent_disc,0.8410596026490066
Locust2022,0.9111852076386784
Expand All @@ -92,6 +96,10 @@ PhonemeSpectra,0.025648672830301224
PhotoStimulation,0.4166666666666667
PronouncedSpeech,0.23
RacketSports,0.28289473684210525
S2Agri-10pc-17,0.6091353993993388
S2Agri-10pc-34,0.6091353993993388
S2Agri-17,0.5962028198994797
S2Agri-34,0.5962028198994797
SPHERE-WUS,0.5
STEW,0.5
SelfRegulationSCP1,0.5017064846416383
Expand Down
8 changes: 8 additions & 0 deletions results/multiverse/Dummy/Dummy_auroc.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ CharacterTrajectories,0.5
CounterMovementJump,0.5
Cricket,0.5
CrowdSourced,0.5
DREAMERA,0.5
DREAMERV,0.5
DuckDuckGeese,0.5
ERing,0.5
EigenWorms,0.5
Expand Down Expand Up @@ -53,6 +55,7 @@ IRDS-STR,0.5
ImaginedFeetHands,0.5
ImaginedOpenCloseFist,0.5
InnerSpeech,0.5
InsectWingbeat,0.5
JapaneseVowels,0.5
KERAAL-CTK,0.5
KERAAL-CTK-MC,0.5
Expand All @@ -70,6 +73,7 @@ KINECAL-GGFV,0.5
KINECAL-QSEC,0.5
KINECAL-QSEO,0.5
LSST,0.5
LenDB,0.5
Libras,0.5
LiveFuelMoistureContent_disc,0.5
Locust2022,0.5
Expand All @@ -92,6 +96,10 @@ PhonemeSpectra,0.5
PhotoStimulation,0.5
PronouncedSpeech,0.5
RacketSports,0.5
S2Agri-10pc-17,0.5
S2Agri-10pc-34,0.5
S2Agri-17,0.5
S2Agri-34,0.5
SPHERE-WUS,0.5
STEW,0.5
SelfRegulationSCP1,0.5
Expand Down
8 changes: 8 additions & 0 deletions results/multiverse/Dummy/Dummy_balacc.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ CharacterTrajectories,0.05
CounterMovementJump,0.3333333333333333
Cricket,0.08333333333333333
CrowdSourced,0.5
DREAMERA,0.5
DREAMERV,0.5
DuckDuckGeese,0.2
ERing,0.16666666666666666
EigenWorms,0.2
Expand Down Expand Up @@ -53,6 +55,7 @@ IRDS-STR,0.5
ImaginedFeetHands,0.5
ImaginedOpenCloseFist,0.5
InnerSpeech,0.25
InsectWingbeat,0.1
JapaneseVowels,0.1111111111111111
KERAAL-CTK,0.5
KERAAL-CTK-MC,0.3333333333333333
Expand All @@ -70,6 +73,7 @@ KINECAL-GGFV,0.5
KINECAL-QSEC,0.5
KINECAL-QSEO,0.5
LSST,0.07142857142857142
LenDB,0.5
Libras,0.06666666666666667
LiveFuelMoistureContent_disc,0.5
Locust2022,0.5
Expand All @@ -92,6 +96,10 @@ PhonemeSpectra,0.02564102564102564
PhotoStimulation,0.3333333333333333
PronouncedSpeech,0.25
RacketSports,0.25
S2Agri-10pc-17,0.058823529411764705
S2Agri-10pc-34,0.034482758620689655
S2Agri-17,0.058823529411764705
S2Agri-34,0.030303030303030304
SPHERE-WUS,0.5
STEW,0.5
SelfRegulationSCP1,0.5
Expand Down
8 changes: 8 additions & 0 deletions results/multiverse/Dummy/Dummy_f1.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ CharacterTrajectories,0.007878326358917932
CounterMovementJump,0.16829901124330893
Cricket,0.012820512820512822
CrowdSourced,0.0
DREAMERA,0.564704171413336
DREAMERV,0.0
DuckDuckGeese,0.06666666666666667
ERing,0.047619047619047616
EigenWorms,0.24829680702618406
Expand Down Expand Up @@ -53,6 +55,7 @@ IRDS-STR,0.0
ImaginedFeetHands,0.0
ImaginedOpenCloseFist,0.666196189131969
InnerSpeech,0.08943319838056679
InsectWingbeat,0.01818181818181818
JapaneseVowels,0.01295410123340298
KERAAL-CTK,0.0
KERAAL-CTK-MC,0.3333333333333333
Expand All @@ -70,6 +73,7 @@ KINECAL-GGFV,0.0
KINECAL-QSEC,0.0
KINECAL-QSEO,0.0
LSST,0.15098437735628223
LenDB,0.0
Libras,0.008333333333333333
LiveFuelMoistureContent_disc,0.0
Locust2022,0.0
Expand All @@ -92,6 +96,10 @@ PhonemeSpectra,0.0012828065503959901
PhotoStimulation,0.2450980392156863
PronouncedSpeech,0.08601626016260162
RacketSports,0.1247638326585695
S2Agri-10pc-17,0.4611742864396579
S2Agri-10pc-34,0.4611742864396579
S2Agri-17,0.4453792438212535
S2Agri-34,0.4453792438212535
SPHERE-WUS,0.0
STEW,0.6666666666666666
SelfRegulationSCP1,0.0
Expand Down
8 changes: 8 additions & 0 deletions results/multiverse/Dummy/Dummy_logloss.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ CharacterTrajectories,2.9873893422378415
CounterMovementJump,1.0986313429481223
Cricket,2.4849066497880004
CrowdSourced,0.6931471282693898
DREAMERA,0.7178228248184128
DREAMERV,0.6801109256783351
DuckDuckGeese,1.6094379124341003
ERing,1.791759469228055
EigenWorms,1.4718633766740754
Expand Down Expand Up @@ -53,6 +55,7 @@ IRDS-STR,0.29463965020934135
ImaginedFeetHands,0.6931478243680321
ImaginedOpenCloseFist,0.6932899710021728
InnerSpeech,1.3922348857254991
InsectWingbeat,2.302585092994046
JapaneseVowels,2.197224577336219
KERAAL-CTK,0.7266593439339404
KERAAL-CTK-MC,1.191181400090801
Expand All @@ -70,6 +73,7 @@ KINECAL-GGFV,0.27491925815268553
KINECAL-QSEC,0.2146132851891638
KINECAL-QSEO,0.22371807606583374
LSST,2.1321750076663113
LenDB,0.6930252160849788
Libras,2.70805020110221
LiveFuelMoistureContent_disc,0.43812361168597996
Locust2022,0.2997872434658028
Expand All @@ -92,6 +96,10 @@ PhonemeSpectra,3.6635616461296454
PhotoStimulation,1.0833754191909117
PronouncedSpeech,1.3893835552665177
RacketSports,1.3817041078077663
S2Agri-10pc-17,1.0603060277768492
S2Agri-10pc-34,1.313611111652872
S2Agri-17,1.0990464414887853
S2Agri-34,1.363923798049471
SPHERE-WUS,0.7465166183739191
STEW,0.6931471805599454
SelfRegulationSCP1,0.6931495567879038
Expand Down
8 changes: 8 additions & 0 deletions results/multiverse/Dummy/Dummy_sensitivity.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ CharacterTrajectories,0.06476323119777158
CounterMovementJump,0.33519553072625696
Cricket,0.08333333333333333
CrowdSourced,0.0
DREAMERA,1.0
DREAMERV,0.0
DuckDuckGeese,0.2
ERing,0.16666666666666666
EigenWorms,0.4198473282442748
Expand Down Expand Up @@ -53,6 +55,7 @@ IRDS-STR,0.0
ImaginedFeetHands,0.0
ImaginedOpenCloseFist,1.0
InnerSpeech,0.235
InsectWingbeat,0.1
JapaneseVowels,0.08378378378378379
KERAAL-CTK,0.0
KERAAL-CTK-MC,0.5
Expand All @@ -70,6 +73,7 @@ KINECAL-GGFV,0.0
KINECAL-QSEC,0.0
KINECAL-QSEO,0.0
LSST,0.3150851581508516
LenDB,0.0
Libras,0.06666666666666667
LiveFuelMoistureContent_disc,0.0
Locust2022,0.0
Expand All @@ -92,6 +96,10 @@ PhonemeSpectra,0.025648672830301224
PhotoStimulation,0.4166666666666667
PronouncedSpeech,0.23
RacketSports,0.28289473684210525
S2Agri-10pc-17,0.6091353993993388
S2Agri-10pc-34,0.6091353993993388
S2Agri-17,0.5962028198994797
S2Agri-34,0.5962028198994797
SPHERE-WUS,0.0
STEW,1.0
SelfRegulationSCP1,0.0
Expand Down
8 changes: 8 additions & 0 deletions results/multiverse/Dummy/Dummy_specificity.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ CharacterTrajectories,0.06476323119777158
CounterMovementJump,0.33519553072625696
Cricket,0.08333333333333333
CrowdSourced,1.0
DREAMERA,0.0
DREAMERV,1.0
DuckDuckGeese,0.2
ERing,0.16666666666666666
EigenWorms,0.4198473282442748
Expand Down Expand Up @@ -53,6 +55,7 @@ IRDS-STR,1.0
ImaginedFeetHands,1.0
ImaginedOpenCloseFist,0.0
InnerSpeech,0.235
InsectWingbeat,0.1
JapaneseVowels,0.08378378378378379
KERAAL-CTK,1.0
KERAAL-CTK-MC,0.5
Expand All @@ -70,6 +73,7 @@ KINECAL-GGFV,1.0
KINECAL-QSEC,1.0
KINECAL-QSEO,1.0
LSST,0.3150851581508516
LenDB,1.0
Libras,0.06666666666666667
LiveFuelMoistureContent_disc,1.0
Locust2022,1.0
Expand All @@ -92,6 +96,10 @@ PhonemeSpectra,0.025648672830301224
PhotoStimulation,0.4166666666666667
PronouncedSpeech,0.23
RacketSports,0.28289473684210525
S2Agri-10pc-17,0.6091353993993388
S2Agri-10pc-34,0.6091353993993388
S2Agri-17,0.5962028198994797
S2Agri-34,0.5962028198994797
SPHERE-WUS,1.0
STEW,0.0
SelfRegulationSCP1,1.0
Expand Down
Loading
Loading