Repository navigation
Add split-graph DNC mechanism variants - #37
Merged
Merged
Conversation
This change adds configurable split-graph DNC read/write mechanisms: relational-mlp read scoring and a Kanerva-style closed-form write posterior, wired through controller config, CLI arguments, and model loading. It also adds graph traversal probes for relational and Q99 order-sensitivity checks, updates experiment CSV logging for Track A/B diagnostics, and preserves mechanism-aware memory resizing and link-matrix ablations.
Lower the minimum required `cell_size` for relational probes from 12 to 6 so tiny smoke-test configurations can run without aborting before training. The probe only needs three role channels, and the previous 4-dimension-per-channel requirement was unnecessarily restrictive for small fixtures.
Add a new `relational-residual` read variant that keeps the DNC cosine baseline at initialization by zero-initializing the learned residual output, and extend split-graph diagnostics with read-content entropy and max-weight metrics. Update the config and CLI/docs to expose the variant and document the Track A/B experiment logs and reader choices.
This change introduces a configurable `--dnc-read-residual-scale` for the relational-residual DNC read variant, preserving the cosine baseline at initialization while allowing the learned correction to be strengthened or weakened. The setting is threaded through config, training, checkpoint loading, and memory resizing, and diagnostics now report the residual RMS and residual-to-cosine RMS ratio. The README is updated with the new flag and experiment guidance.
This updates the default relational-residual read correction from 2.0 to 1.0 across config and runtime defaults, and clarifies the docs that the residual remains cosine-anchored and exact at initialization. It also notes that lower scales such as 0.5 are available for follow-up comparisons while keeping the baseline and Track A setup consistent.
Adds a configurable cap for relational DNC residual read scores, exposed through config and CLI, and threads it through model loading and memory resizing. The Q99 order-sensitivity check is also refactored into separate fact-order and query-order probes with clearer semantic metrics and post-training gating. This keeps the residual correction bounded while making the order-sensitivity diagnostics more interpretable and robust. Includes a small .gitignore update for tmp files.
Avoid zero-valued RMS denominators in residual score normalization during the first forward pass. This change adds a small epsilon before taking the square root for both residual and cosine RMS values, preventing NaNs and infinite gradients when the scorer is initialized to zero.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This change adds configurable split-graph DNC read/write mechanisms: relational-mlp read scoring and a Kanerva-style closed-form write posterior, wired through controller config, CLI arguments, and model loading. It also adds graph traversal probes for relational and Q99 order-sensitivity checks, updates experiment CSV logging for Track A/B diagnostics, and preserves mechanism-aware memory resizing and link-matrix ablations.