From 3cb4ccd2bbd9914283265b62d0b3da9e77e0127a Mon Sep 17 00:00:00 2001 From: Parshkov Date: Sun, 6 Sep 2026 19:54:31 -0700 Subject: [PATCH] Show the near miss instead of hiding it, and point the roadmap at the real problem Parshkov: the project has leaned toward the form of an idea-project about thinking rather than its substance and the substance of finding like-minded people. He is right, and the sharpest evidence is one line of code. `phrasing._discover` dropped every row the classifier called negative before saying anything to a person. So on 2026-09-06 someone was told "Nothing the engine calls a resonance" about a person who had independently built the same construction step for step -- meaning agreement 0.59, eleven ideas corresponding, the same cold start, the same reliance on organisations that already hold the histories. The product did not merely mis-score that pair. It hid the person and said the opposite of the truth. What rescued it was the chat assistant reading the raw rows and overruling the product. A product cannot depend on the assistant correcting it. Near misses are now shown, at the engine's own bar for two people being on the same subject, and said for what they are: the engine did not call this a resonance and may be wrong, here is what lines up, the judgement is yours. The template coincidence from the same run -- agreeing on 0.12 -- is still not offered, because showing everything would be as useless as showing nothing. VISION.md asked for exactly this and the code was doing the reverse: it made a judgement and withheld the evidence. The roadmap now opens with the thing that decides whether any of this matters: seven shared thoughts exist in the world. The engine has eighteen benchmark families, five verdicts, thirteen thresholds, four policy versions and seven ADRs; the corpus has seven thoughts. A matcher with nobody to match is not a product, and the matching is now good enough -- it found a real cross-domain twin between two people who had never met. With an irony worth naming in the file itself: the thought that produced that match was about a registry of employer conduct, and its author had already reasoned out the binding constraint -- while there are few reports there is no signal, so start from those who already hold the data. That is Resonance's own constraint, and it has never been applied to Resonance. The four questions about falsifying the engine are still there, below it, where they belong. 693 tests OK, both gates pass with gold unedited. Co-Authored-By: Claude Opus 5 --- ROADMAP.md | 48 ++++++++++++++++++++----- docs/STATUS.md | 24 +++++++++++++ src/product/phrasing.py | 53 +++++++++++++++++++++++++++- tests/test_same_subject_resonance.py | 46 ++++++++++++++++++++++++ 4 files changed, 162 insertions(+), 9 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index d3bd7b8..47b44c1 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,10 +1,42 @@ # What's next -The product is built and deployed. What follows is not more features — it is the -work that can still prove the idea wrong. In rough order of how much it would -change. +## The thing that decides everything -## 1. The embedding baseline +**Seven shared thoughts exist in the world.** That is the entire live corpus. + +Everything below is secondary to that, and it is worth saying plainly because +the project has spent most of its effort on the other side. The engine now has +eighteen benchmark families, five verdicts, thirteen thresholds, four policy +versions and seven ADRs. The corpus has seven thoughts. A matcher with nobody +to match is not a product, however good the matching is — and the matching is +now good enough: on 2026-09-06 it found a genuine cross-domain twin between two +people who had never met. + +There is an irony worth naming. The thought that produced that match was about +a registry of employer conduct, and its author had already reasoned out the +binding constraint: *while there are few reports there is no signal, so start +from those who already hold the data.* That is exactly Resonance's own +constraint, and it has never been applied to Resonance. + +So the first question is not "is the engine right?" It is **"where do the first +few hundred thoughts come from?"** Some honest options, none of them code: +seed from a community that already writes down what it is working on; invite +people around one narrow problem rather than in general; or accept that the +standing search — "we will tell you when someone arrives" — is the whole +product until the pool is large enough for search to return anything. + +Nothing below matters if that is not answered. + +## What a person actually gets, today + +Worth holding in view while reading the rest. A person shares a thought and, +almost always, hears that nobody matched yet. That is honest, and it is thin. +The parts that make it not-thin are the standing search and what is said when a +near miss turns up — not another decimal place on the classifier. + +## Then: can the idea be falsified? + +### 1. The embedding baseline The one experiment this project set itself and never ran. [`WHY_NOT.md`](WHY_NOT.md) rejects whole-thought embeddings, and @@ -22,7 +54,7 @@ If the baseline matches the engine, the structural machinery is not earning its complexity and that has to be said out loud. If it does not, this is the claim the project has never been entitled to make. -## 2. Human review of the benchmark gold +### 2. Human review of the benchmark gold Every gold label and every benchmark graph was authored by agents. Until a person has reviewed the 8 analogy families and the 8 template-coincidence @@ -32,7 +64,7 @@ negatives, `classification_accuracy = 1.0` means "no regression", not [ADR-0005](docs/decisions/ADR-0005-same-vocabulary-cross-domain-verdict.md) is explicitly waiting on this and must not be settled by moving a threshold. -## 3. Real thoughts +### 3. Real thoughts Every graph measured so far is authored, not extracted from a real conversation — and @@ -41,7 +73,7 @@ scoring 0.00–0.07 on real pairs, which is what the label encoder exists to fix A consented corpus of real extracted thoughts, with two-human gold, is the only thing that measures the product as used. -## 4. Scale +### 4. Scale Query time is linear from roughly 350 graphs upward — 148 ms at 176, 580 ms at 1408 — which is the second condition ADR-0004 names for reconsidering the @@ -49,7 +81,7 @@ concept channel. `ResonanceEngine._require_bound()` additionally re-hashes the whole corpus on every query. Not urgent at the current corpus size, and a wall at 10⁴. -## 5. Multilingual prose extraction +### 5. Multilingual prose extraction `src/extraction/cue.py` is English-only — not just its cue table, but its sentence splitting (it wants a capital Latin letter after the period), its diff --git a/docs/STATUS.md b/docs/STATUS.md index 4b6b0a3..78ef950 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -302,6 +302,30 @@ yields an honest empty graph, so the assistant must build the `thought` graph itself — which the contract now says plainly. Making the extractor multilingual is real work, not a patch, and is not attempted here. +## Evidence, not judgement (2026-09-06) + +A verdict is this engine's opinion. It was being treated as the truth about two +people: `phrasing._discover` dropped every row the classifier called +`negative`, so what a person heard was *"Nothing the engine calls a +resonance"* — said about someone who had independently built the same +construction step for step, with meaning agreement 0.59 and eleven ideas +corresponding. + +What rescued it that day was the chat assistant reading the raw rows and +overruling the product. A product should not depend on that. + +Near misses are now shown, at the engine's own bar for two people being on the +same subject (`T_SAME_SUBJECT_SEMANTIC`), and said for what they are: *"The +engine did not call any of these a resonance, and it may be wrong. These people +are talking about the same things as you, and what lines up is shown so the +judgement can be yours."* The template coincidence from the same run, agreeing +on 0.12, is still not offered — showing everything would be as useless as +showing nothing. + +This is what `VISION.md` already asked for and the code was not doing: +provide evidence, not social judgement, and let people decide with a better +signal. + ## Next falsification targets 1. A corpus of real extracted thoughts (consented) with two-human gold; compare engine 0.2 against a whole-thought embedding baseline. diff --git a/src/product/phrasing.py b/src/product/phrasing.py index 8e0ac40..0ad5ad4 100644 --- a/src/product/phrasing.py +++ b/src/product/phrasing.py @@ -22,6 +22,7 @@ import json from typing import Any, Callable, Mapping + Result = Mapping[str, Any] @@ -184,6 +185,22 @@ def _mark_resonances_seen(r: Result) -> str: return "Marked as seen. Nothing was sent to anyone." +# The engine's own bar for "these two are on the same subject". Copied rather +# than imported: `src.scoring` pulls in the alignment package, and this module +# is loaded while that package is still initialising. A test asserts the two +# stay equal, so the words a person reads cannot drift from the verdict they +# are told about. +SAME_SUBJECT_MEANING = 0.40 + + +def _semantic_of(row: Mapping[str, Any]) -> float: + """Agreement about meaning, whatever the verdict was.""" + try: + return max(0.0, min(1.0, float((row.get("scores") or {}).get("semantic", 0)))) + except (TypeError, ValueError): + return 0.0 + + def _discover(r: Result) -> str: rows = list(r.get("matches_in_backend_order") or []) people = [row for row in rows if not row.get("hard_rejection") @@ -193,7 +210,41 @@ def _discover(r: Result) -> str: # needs it too, so they do not wonder where the rest went. aside = str(r.get("shape_note") or "").strip() if not people: - if rows: + # A verdict is this engine's opinion, not the truth about two people. + # Rows it called `negative` used to be dropped here entirely, so a + # person was told "nothing the engine calls a resonance" about someone + # who had independently built the same construction step for step + # (2026-09-06: semantic agreement 0.59, every idea corresponding, and + # the classifier said no). Hiding that is a judgement; the project's + # whole claim is to show evidence and let the person decide. + # + # So near misses are shown when the engine's OWN bar for "these two are + # on the same subject" is met -- `T_SAME_SUBJECT_SEMANTIC` -- and said + # plainly for what they are. + near = [row for row in rows if not row.get("hard_rejection") + and _semantic_of(row) >= SAME_SUBJECT_MEANING] + near.sort(key=_semantic_of, reverse=True) + if near: + lines = [] + for index, row in enumerate(near[:5], start=1): + evidence = row.get("evidence") or {} + display = row.get("display") or {} + nodes = int(evidence.get("mapped_node_count") or 0) + links = int(evidence.get("preserved_relation_count") or 0) + conflicts = int(evidence.get("contradiction_count") or 0) + depth = (f"{_count(nodes, 'idea corresponds', 'ideas correspond')}, " + f"{_count(links, 'link kept', 'links kept')}") + if conflicts: + depth += f", {_count(conflicts, 'contradiction', 'contradictions')}" + lines.append(f"{index}. {row.get('person_pseudonym') or 'someone'} — " + f"meaning {_round(_semantic_of(row))}: {depth}" + + (f" (“{display.get('topic')}”)" if display.get("topic") else "")) + said = ("The engine did not call any of these a resonance, and it may be " + "wrong. These people are talking about the same things as you, and " + "what lines up is shown so the judgement can be yours: " + + " ".join(lines) + + " Ask for the working on any of them before deciding.") + elif rows: said = ("Nothing the engine calls a resonance. Some thoughts share a " "skeleton with yours, but not enough meaning for it to say they " "are the same reasoning — so nobody is being suggested to you.") diff --git a/tests/test_same_subject_resonance.py b/tests/test_same_subject_resonance.py index c4c1ce6..e7c9e05 100644 --- a/tests/test_same_subject_resonance.py +++ b/tests/test_same_subject_resonance.py @@ -116,3 +116,49 @@ def test_one_person_holds_a_piece_of_the_other_problem(self): if __name__ == "__main__": unittest.main() + + +class EvidenceNotJudgementTests(unittest.TestCase): + """A verdict is this engine's opinion, not the truth about two people. + + Rows the classifier called `negative` were dropped from what a person is + told, so on 2026-09-06 someone was told "nothing the engine calls a + resonance" about a person who had independently built the same + construction step for step. VISION.md says the opposite is the point: + provide evidence, not social judgement, and let people decide with a + better signal. + """ + + def _result(self, rows): + return {"matches_in_backend_order": rows, "rejected": [], "shape_note": ""} + + def _row(self, pseudonym, semantic, topic, verdict="negative", ideas=11, links=2): + return {"person_pseudonym": pseudonym, "mode_classification": verdict, + "hard_rejection": None, "scores": {"structural": 0.19, "semantic": semantic}, + "evidence": {"mapped_node_count": ideas, "preserved_relation_count": links, + "contradiction_count": 4}, + "display": {"topic": topic}} + + def test_a_near_miss_with_real_meaning_is_shown_not_hidden(self): + from src.product import phrasing + said = phrasing.say("resonance_discover", self._result([ + self._row("Fleet Minstrel", 0.59, "Shared landlord history before signing")])) + self.assertIn("Fleet Minstrel", said) + self.assertIn("Shared landlord history", said) + self.assertIn("may be wrong", said) + self.assertNotIn("nobody is being suggested", said) + + def test_a_coincidence_without_meaning_is_still_not_offered(self): + """Showing everything would be as useless as showing nothing: the + template coincidence measured that day agreed on 0.12.""" + from src.product import phrasing + said = phrasing.say("resonance_discover", self._result([ + self._row("Fleet Minstrel", 0.12, "Protected handover time on a ward")])) + self.assertNotIn("Protected handover time", said) + self.assertIn("Nothing the engine calls a resonance", said) + + def test_the_bar_is_the_engines_own(self): + """The words a person reads must not drift from the verdict they are + told about; `phrasing` copies the constant to avoid an import cycle.""" + from src.product import phrasing + self.assertEqual(phrasing.SAME_SUBJECT_MEANING, scoring.T_SAME_SUBJECT_SEMANTIC)