Skip to content

Add symbolic (partition-based) entropy, MI and CMI estimators (#59) - #60

Open
ParitKansal wants to merge 3 commits into
Center-For-Complex-Systems-Science:mainfrom
ParitKansal:symbolic-cmi-estimator
Open

ParitKansal wants to merge 3 commits into
Center-For-Complex-Systems-Science:mainfrom
ParitKansal:symbolic-cmi-estimator

Conversation

@ParitKansal

@ParitKansal ParitKansal commented Oct 23, 2026 •

Copy link
Copy Markdown

Closes #59.

Adds symbolic (partition-based) estimators following Cafaro, Lord, Sun & Bollt, "Causation entropy from symbolic representations of dynamical systems", Chaos 25, 043106 (2015), arXiv:1507.07262.

What is added

  • symbolize (entropy.py): partitions each variable into n_symbols cells (Eq. 2–3 of the paper). Options: quantile (equal-frequency; edges placed between distinct values so tied values always share a cell), uniform (equal-width), precomputed (data already symbolic).
  • symbolic_entropy, symbolic_mutual_information, symbolic_conditional_mutual_information: plug-in estimators (Eq. 13–15), in nats like the other estimators.
  • method="symbolic" in conditional_mutual_information (new explicit args n_symbols, partition; no **kwargs).
  • information="symbolic" in discover_network (uses the default 2-symbol median split).
  • README and CHANGELOG entries.

Tests (24 new, tests/core/information/test_symbolic.py)

  • CMI matches Eq. (15) computed term by term; chain X→Z→Y gives I(X;Y|Z)=0.
  • Logistic map with the generating partition at 1/2 gives memoryless fair-coin symbols (the setting of Sec. IV of the paper).
  • Tied/count data (e.g. 70% zeros) keep both symbols.
  • discover_network(information="symbolic") recovers exactly the true edge X0→X1 at lag 1.
  • Full suite: 497 passed; black, isort and flake8 (CI settings) clean; 100% line coverage of the new functions.

Cross-checks against other libraries (random cases, differences at rounding level)

Quantity Reference Max abs. difference
Entropy pyitlib, infomeasure 4e-16
MI sklearn mutual_info_score, pyitlib 2e-15
CMI (incl. 2-column Z) pyitlib, infomeasure 4e-15
CMI tigramite CMIsymb 3e-15
Bins sklearn KBinsDiscretizer uniform identical; quantile equal-frequency (only exact-edge points differ)

Open questions for maintainers

  1. Should discover_network expose n_symbols/partition? That would mean threading two explicit parameters through the internal functions, so I left the default (2 symbols) for now.
  2. No small-sample bias correction (e.g. Miller–Madow); the permutation test uses the same estimator, so the bias partly cancels. Could be a follow-up.

…-For-Complex-Systems-Science#59)

Implements the symbolic estimators of Cafaro, Lord, Sun and Bollt (Chaos, 2015):
symbolize, symbolic_entropy, symbolic_mutual_information and
symbolic_conditional_mutual_information, available as method='symbolic' in
conditional_mutual_information and information='symbolic' in discover_network.
Adds 24 tests, README and CHANGELOG entries.
With partition='precomputed', NaN or inf input raised the misleading
'requires integer-valued symbols' error. The finite check now runs first,
so every partition type gives the same error. The test now checks the
message.
@ParitKansal
ParitKansal force-pushed the symbolic-cmi-estimator branch from 4b9ccb0 to 9d8bf54 Compare October 23, 2026 05:39

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

New Entropy Estimator for Symbolized Dynamics

1 participant