You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Benchmark kNN discovery sensitivity to k and time-series length #55
Follow-up to #14 / #48 and the kNN estimator cleanup in #53.
The estimator-level experiments suggest that the Chebyshev kNN CMI estimate improves substantially as the sample size increases, but we have not yet characterized how that translates to causal-discovery performance.
I’d like to benchmark the discovery path across:
multiple k values,
multiple time-series lengths T,
multiple random seeds,
the Chebyshev kNN estimator used by the current API.
I’ll report graph-level metrics such as TP/FP/FN, precision/recall/F1, and runtime, and check whether recovery improves consistently as T increases or whether particular k values behave poorly.
I’ll base the experiments on the final kNN implementation once #53 is resolved, so the benchmark reflects the version that will actually ship.
Yes, I think it can. Once #56 and #53 are resolved, I can build #55 on top of the same benchmarks package and reuse the generator, scoring, and result output. Since #55 is a k/T sweep for the kNN discovery path rather than a Gaussian oCSE vs PCMCI comparison, I’d probably keep it as a separate benchmark entry point within the package.
Activity
kslote1 commentedon Oct 8, 2026
I opened a comment on #56
Can this benchmark fit within the new benchmarks package design?
chaewon-research commentedon Oct 8, 2026
Yes, I think it can. Once #56 and #53 are resolved, I can build #55 on top of the same benchmarks package and reuse the generator, scoring, and result output. Since #55 is a k/T sweep for the kNN discovery path rather than a Gaussian oCSE vs PCMCI comparison, I’d probably keep it as a separate benchmark entry point within the package.