Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 40 additions & 8 deletions ROADMAP.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,42 @@
# What's next

The product is built and deployed. What follows is not more features — it is the
work that can still prove the idea wrong. In rough order of how much it would
change.
## The thing that decides everything

## 1. The embedding baseline
**Seven shared thoughts exist in the world.** That is the entire live corpus.

Everything below is secondary to that, and it is worth saying plainly because
the project has spent most of its effort on the other side. The engine now has
eighteen benchmark families, five verdicts, thirteen thresholds, four policy
versions and seven ADRs. The corpus has seven thoughts. A matcher with nobody
to match is not a product, however good the matching is — and the matching is
now good enough: on 2026-09-06 it found a genuine cross-domain twin between two
people who had never met.

There is an irony worth naming. The thought that produced that match was about
a registry of employer conduct, and its author had already reasoned out the
binding constraint: *while there are few reports there is no signal, so start
from those who already hold the data.* That is exactly Resonance's own
constraint, and it has never been applied to Resonance.

So the first question is not "is the engine right?" It is **"where do the first
few hundred thoughts come from?"** Some honest options, none of them code:
seed from a community that already writes down what it is working on; invite
people around one narrow problem rather than in general; or accept that the
standing search — "we will tell you when someone arrives" — is the whole
product until the pool is large enough for search to return anything.

Nothing below matters if that is not answered.

## What a person actually gets, today

Worth holding in view while reading the rest. A person shares a thought and,
almost always, hears that nobody matched yet. That is honest, and it is thin.
The parts that make it not-thin are the standing search and what is said when a
near miss turns up — not another decimal place on the classifier.

## Then: can the idea be falsified?

### 1. The embedding baseline

The one experiment this project set itself and never ran.
[`WHY_NOT.md`](WHY_NOT.md) rejects whole-thought embeddings, and
Expand All @@ -22,7 +54,7 @@ If the baseline matches the engine, the structural machinery is not earning its
complexity and that has to be said out loud. If it does not, this is the claim
the project has never been entitled to make.

## 2. Human review of the benchmark gold
### 2. Human review of the benchmark gold

Every gold label and every benchmark graph was authored by agents. Until a
person has reviewed the 8 analogy families and the 8 template-coincidence
Expand All @@ -32,7 +64,7 @@ negatives, `classification_accuracy = 1.0` means "no regression", not
[ADR-0005](docs/decisions/ADR-0005-same-vocabulary-cross-domain-verdict.md) is
explicitly waiting on this and must not be settled by moving a threshold.

## 3. Real thoughts
### 3. Real thoughts

Every graph measured so far is authored, not extracted from a real
conversation — and
Expand All @@ -41,15 +73,15 @@ scoring 0.00–0.07 on real pairs, which is what the label encoder exists to fix
A consented corpus of real extracted thoughts, with two-human gold, is the only
thing that measures the product as used.

## 4. Scale
### 4. Scale

Query time is linear from roughly 350 graphs upward — 148 ms at 176, 580 ms at
1408 — which is the second condition ADR-0004 names for reconsidering the
concept channel. `ResonanceEngine._require_bound()` additionally re-hashes the
whole corpus on every query. Not urgent at the current corpus size, and a wall
at 10⁴.

## 5. Multilingual prose extraction
### 5. Multilingual prose extraction

`src/extraction/cue.py` is English-only — not just its cue table, but its
sentence splitting (it wants a capital Latin letter after the period), its
Expand Down
24 changes: 24 additions & 0 deletions docs/STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -302,6 +302,30 @@ yields an honest empty graph, so the assistant must build the `thought` graph
itself — which the contract now says plainly. Making the extractor multilingual
is real work, not a patch, and is not attempted here.

## Evidence, not judgement (2026-09-06)

A verdict is this engine's opinion. It was being treated as the truth about two
people: `phrasing._discover` dropped every row the classifier called
`negative`, so what a person heard was *"Nothing the engine calls a
resonance"* — said about someone who had independently built the same
construction step for step, with meaning agreement 0.59 and eleven ideas
corresponding.

What rescued it that day was the chat assistant reading the raw rows and
overruling the product. A product should not depend on that.

Near misses are now shown, at the engine's own bar for two people being on the
same subject (`T_SAME_SUBJECT_SEMANTIC`), and said for what they are: *"The
engine did not call any of these a resonance, and it may be wrong. These people
are talking about the same things as you, and what lines up is shown so the
judgement can be yours."* The template coincidence from the same run, agreeing
on 0.12, is still not offered — showing everything would be as useless as
showing nothing.

This is what `VISION.md` already asked for and the code was not doing:
provide evidence, not social judgement, and let people decide with a better
signal.

## Next falsification targets

1. A corpus of real extracted thoughts (consented) with two-human gold; compare engine 0.2 against a whole-thought embedding baseline.
Expand Down
53 changes: 52 additions & 1 deletion src/product/phrasing.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import json
from typing import Any, Callable, Mapping


Result = Mapping[str, Any]


Expand Down Expand Up @@ -184,6 +185,22 @@ def _mark_resonances_seen(r: Result) -> str:
return "Marked as seen. Nothing was sent to anyone."


# The engine's own bar for "these two are on the same subject". Copied rather
# than imported: `src.scoring` pulls in the alignment package, and this module
# is loaded while that package is still initialising. A test asserts the two
# stay equal, so the words a person reads cannot drift from the verdict they
# are told about.
SAME_SUBJECT_MEANING = 0.40


def _semantic_of(row: Mapping[str, Any]) -> float:
"""Agreement about meaning, whatever the verdict was."""
try:
return max(0.0, min(1.0, float((row.get("scores") or {}).get("semantic", 0))))
except (TypeError, ValueError):
return 0.0


def _discover(r: Result) -> str:
rows = list(r.get("matches_in_backend_order") or [])
people = [row for row in rows if not row.get("hard_rejection")
Expand All @@ -193,7 +210,41 @@ def _discover(r: Result) -> str:
# needs it too, so they do not wonder where the rest went.
aside = str(r.get("shape_note") or "").strip()
if not people:
if rows:
# A verdict is this engine's opinion, not the truth about two people.
# Rows it called `negative` used to be dropped here entirely, so a
# person was told "nothing the engine calls a resonance" about someone
# who had independently built the same construction step for step
# (2026-09-06: semantic agreement 0.59, every idea corresponding, and
# the classifier said no). Hiding that is a judgement; the project's
# whole claim is to show evidence and let the person decide.
#
# So near misses are shown when the engine's OWN bar for "these two are
# on the same subject" is met -- `T_SAME_SUBJECT_SEMANTIC` -- and said
# plainly for what they are.
near = [row for row in rows if not row.get("hard_rejection")
and _semantic_of(row) >= SAME_SUBJECT_MEANING]
near.sort(key=_semantic_of, reverse=True)
if near:
lines = []
for index, row in enumerate(near[:5], start=1):
evidence = row.get("evidence") or {}
display = row.get("display") or {}
nodes = int(evidence.get("mapped_node_count") or 0)
links = int(evidence.get("preserved_relation_count") or 0)
conflicts = int(evidence.get("contradiction_count") or 0)
depth = (f"{_count(nodes, 'idea corresponds', 'ideas correspond')}, "
f"{_count(links, 'link kept', 'links kept')}")
if conflicts:
depth += f", {_count(conflicts, 'contradiction', 'contradictions')}"
lines.append(f"{index}. {row.get('person_pseudonym') or 'someone'} — "
f"meaning {_round(_semantic_of(row))}: {depth}"
+ (f" (“{display.get('topic')}”)" if display.get("topic") else ""))
said = ("The engine did not call any of these a resonance, and it may be "
"wrong. These people are talking about the same things as you, and "
"what lines up is shown so the judgement can be yours: "
+ " ".join(lines) +
" Ask for the working on any of them before deciding.")
elif rows:
said = ("Nothing the engine calls a resonance. Some thoughts share a "
"skeleton with yours, but not enough meaning for it to say they "
"are the same reasoning — so nobody is being suggested to you.")
Expand Down
46 changes: 46 additions & 0 deletions tests/test_same_subject_resonance.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,3 +116,49 @@ def test_one_person_holds_a_piece_of_the_other_problem(self):

if __name__ == "__main__":
unittest.main()


class EvidenceNotJudgementTests(unittest.TestCase):
"""A verdict is this engine's opinion, not the truth about two people.

Rows the classifier called `negative` were dropped from what a person is
told, so on 2026-09-06 someone was told "nothing the engine calls a
resonance" about a person who had independently built the same
construction step for step. VISION.md says the opposite is the point:
provide evidence, not social judgement, and let people decide with a
better signal.
"""

def _result(self, rows):
return {"matches_in_backend_order": rows, "rejected": [], "shape_note": ""}

def _row(self, pseudonym, semantic, topic, verdict="negative", ideas=11, links=2):
return {"person_pseudonym": pseudonym, "mode_classification": verdict,
"hard_rejection": None, "scores": {"structural": 0.19, "semantic": semantic},
"evidence": {"mapped_node_count": ideas, "preserved_relation_count": links,
"contradiction_count": 4},
"display": {"topic": topic}}

def test_a_near_miss_with_real_meaning_is_shown_not_hidden(self):
from src.product import phrasing
said = phrasing.say("resonance_discover", self._result([
self._row("Fleet Minstrel", 0.59, "Shared landlord history before signing")]))
self.assertIn("Fleet Minstrel", said)
self.assertIn("Shared landlord history", said)
self.assertIn("may be wrong", said)
self.assertNotIn("nobody is being suggested", said)

def test_a_coincidence_without_meaning_is_still_not_offered(self):
"""Showing everything would be as useless as showing nothing: the
template coincidence measured that day agreed on 0.12."""
from src.product import phrasing
said = phrasing.say("resonance_discover", self._result([
self._row("Fleet Minstrel", 0.12, "Protected handover time on a ward")]))
self.assertNotIn("Protected handover time", said)
self.assertIn("Nothing the engine calls a resonance", said)

def test_the_bar_is_the_engines_own(self):
"""The words a person reads must not drift from the verdict they are
told about; `phrasing` copies the constant to avoid an import cycle."""
from src.product import phrasing
self.assertEqual(phrasing.SAME_SUBJECT_MEANING, scoring.T_SAME_SUBJECT_SEMANTIC)