You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(neural): stop the Graph of Trust leaking trust on repeated evidence - #1028
build_edges/1 emits one edge per verified success. When the same evidence is recorded twice, the PageRank iteration in iterate_trust/4 sees parallel edges:
it divided a source's trust by the summed weight of all of its out-edges, duplicates included;
it credited each target with only the first matching edge (Enum.find).
So column sums fell below 1 and trust leaked, and how much leaked depended on how many times the evidence had been logged. Tripling an unchanged outcome log moved a recipe's trust from 0.38 to 0.59 and a bot's from 0.89 to 0.94.
I found this while checking the PageRank convergence theorem's premises for #1025. The leak does not break convergence, because the matrix stays column-substochastic, but it makes the scores depend on logging noise.
The fix
Rank over the set of edges (Enum.uniq before the adjacency is built), as the Jaccard cross-repo edges already do. Trust then depends on what was observed, not on how often it was logged. graph.edges itself is unchanged.
Credit every edge to a target, so the columns of the transition matrix sum to 1 even if parallel edges reappear.
This changes production trust scores. Wherever an outcome log holds repeated (recipe, repo, bot) successes, the scores will move.
The alternative would let repeated successes count with their multiplicity. That would also need the Jaccard cross-repo edges reweighted by count; otherwise duplication still shifts mass between edge kinds. I chose set semantics because the cross-repo edges already work that way. Say if you want multiplicity instead.
Validation
New test, recording the same evidence k times leaves every trust score unchanged:
fails on main (bot-qualification-fixture: once=0.889 thrice=0.939);
still fails with only the summing change (0.980 vs 1.0), because the Jaccard edges ignore multiplicity;
build_edges/1 emits one edge per verified success, so the same evidence
recorded twice arrives at the PageRank iteration as parallel edges. The
iteration divided a source's trust by the weight of all its out-edges,
duplicates included, but credited each target with only the first
matching edge (Enum.find). Column sums fell below 1 and trust leaked, and
how much leaked depended on how many times the evidence was logged.
Tripling an unchanged log moved a recipe's trust from 0.38 to 0.59 and a
bot's from 0.89 to 0.94.
Rank over the set of edges, as the Jaccard cross-repo edges already do,
so that trust depends on what was observed and not on how often it was
logged. Also credit every remaining edge to a target, so the columns of
the transition matrix sum to 1 even if parallel edges reappear.
New test: recording the same evidence three times leaves every trust
score unchanged. It fails on main, and also with only the summing
change; it passes with this commit. mix test: 2689 tests, 0 failures,
246 excluded.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0165rGu3tsKmJtevmrdMVyfJ
No actionable comments were generated in the recent review.
Summary
Summary by CodeRabbit
Bug Fixes
Trust scores now remain consistent when identical verified evidence is recorded multiple times, preventing repeated entries from changing the set of scored items or their scores.
Trust calculations now include all positive-weight connections between the same source and destination, rather than counting only one. This ensures scores reflect the full contribution of those connections.
Trust-score calculation now removes exact duplicate edges from adjacency and sums positive parallel-edge weights for incoming trust. A regression test compares scores from four verified observations with scores from those observations repeated three times.
Changes: Trust graph edge handling
Layer / File(s)
Summary
Edge handling and trust-score regression lib/neural/graph_of_trust.ex, test/graph_of_trust_qualification_test.exs
Adjacency construction removes exact duplicate edges. Incoming trust sums all positive-weight edges from a source to a node before applying outgoing-weight normalisation. The regression test checks that repeating verified observations does not change score keys or scores beyond a tolerance of 1.0e-9.
Issue 984 concerns deduplicating replays while retaining distinct verified attempts. This change removes identical trust-graph edges, but does not address the broader event-identity distinction.
Possibly related PRs
hyperpolymath/hypatia#1002: Changes trust graph construction and supported-node ranking, which produce the scores affected by this change.
hyperpolymath/hypatia#1034: Identifies parallel-edge trust leakage in GraphOfTrust, which this change also addresses.
hyperpolymath/hypatia#982: Prevents repeated fleet-row imports, which can otherwise add repeated outcome evidence to the trust graph.
Merge Risk: Minimal · up to 00de8
Repeated evidence no longer changes trust scores, and retained duplicate edges do not affect scores after restart. The change is ready to merge after normal checks.
Pre-merge checks: 5
Check name
Status
Explanation
Title check
The title clearly describes the fix for trust leakage caused by repeated evidence, which is the main change.
Description check
The description explains the trust leakage, the set-semantics fix, and the reported validation. It is directly related to the changeset.
Docstring Coverage
No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage
Explanation
No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 2 files.
Finishing Touches
Generate docstrings
Commit to this branch
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
File: No configuration file from this repository applied
Commit: Unavailable
Layers: Organization UI
Review profile: ASSERTIVE
Plan: Advanced
Run ID: 8910af7a-0694-4110-9852-91717897d45e
Commits
Reviewing files that changed from the base of the PR and between 142380e and 00de870.
Files selected for processing: 2
lib/neural/graph_of_trust.ex
test/graph_of_trust_qualification_test.exs
Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.
This security review couldn’t start because the paying account has exhausted its Codex allowance and has insufficient available credits. The payer can check their balance and add credits in the Codex usage dashboard, then try again. If you do not manage the paying account, contact this repository's admins.
This security review couldn’t start because the paying account has exhausted its Codex allowance and has insufficient available credits. The payer can check their balance and add credits in the Codex usage dashboard, then try again. If you do not manage the paying account, contact this repository's admins.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
The bug
build_edges/1emits one edge per verified success. When the same evidence is recorded twice, the PageRank iteration initerate_trust/4sees parallel edges:Enum.find).So column sums fell below 1 and trust leaked, and how much leaked depended on how many times the evidence had been logged. Tripling an unchanged outcome log moved a recipe's trust from 0.38 to 0.59 and a bot's from 0.89 to 0.94.
I found this while checking the PageRank convergence theorem's premises for #1025. The leak does not break convergence, because the matrix stays column-substochastic, but it makes the scores depend on logging noise.
The fix
Enum.uniqbefore the adjacency is built), as the Jaccard cross-repo edges already do. Trust then depends on what was observed, not on how often it was logged.graph.edgesitself is unchanged.This changes production trust scores. Wherever an outcome log holds repeated (recipe, repo, bot) successes, the scores will move.
The alternative would let repeated successes count with their multiplicity. That would also need the Jaccard cross-repo edges reweighted by count; otherwise duplication still shifts mass between edge kinds. I chose set semantics because the cross-repo edges already work that way. Say if you want multiplicity instead.
Validation
recording the same evidence k times leaves every trust score unchanged:bot-qualification-fixture: once=0.889 thrice=0.939);0.980vs1.0), because the Jaccard edges ignore multiplicity;mix test: 2689 tests, 0 failures, 246 excluded.mix format --check-formattedis clean.Generated by Claude Code