Skip to content

Answer in the user's language - #205

Merged
adamjohnwright merged 1 commit into
mainfrom
feat/answer-in-user-language
Sep 10, 2026
Merged

Answer in the user's language#205
adamjohnwright merged 1 commit into
mainfrom
feat/answer-in-user-language

Conversation

@adamjohnwright

Copy link
Copy Markdown
Contributor

Spec 004, stages 1 and 2. detected_language reaches the answer prompt in React-to-Me and Plant Reactome, as its own variable.

What was going wrong

Reproduced before fixing:

step result
question Quel role joue TP53 dans l'apoptose ?
detected French
rephrased for retrieval What role does TP53 play in apoptosis?
retrieval good ✓
answer English

Four steps of five already worked. chat_history still held the original French question and the model answered in English anyway — so it was never that the model couldn't tell; nothing asked it to reply in French.

Why a prompt variable and not #140's approach

create_retrieval_chain passes only input to the retriever:

retrieval_docs = (lambda x: x["input"]) | retriever

So a separate variable cannot reach BM25, the vector store, or the query expansion in front of them. #140 appended its instruction to input; measured through the whole retriever, that changes about half the fused documents.

A test asserts the retrieval query is byte-identical across four languages — exact, where a retrieval baseline would only show noise and cost an hour of API time.

The instruction wording is @bleedblack1's, kept because it gets the hard part right: SET, MAX and CAT are gene symbols and ordinary English words, and R-HSA-9612973 means nothing translated.

The adversarial review caught my plan contradicting my spec

FR-007 promised English questions "no additional prompt content", while the plan always passes the language — which does add a sentence to every English prompt. FR-007 was the wrong half: it now promises no extra model call and a byte-identical retrieval query, and states the added sentence plainly. Branching instead would leave the common path exercised only by users who never ask in English.

Verified against the Release95 bundle

answer R-HSA ids docs
French question French preserved 40
English question unchanged English preserved 40

Reverting the call site fails the test that covers it. 231 tests pass.

Worth watching: the French answer carried 2 R-HSA citations against English's 9. One sample, not a finding — but the kind of thing an answer-quality run should look at.

#140 and #125 close once this lands.

🤖 Generated with Claude Code

Spec 004, stages 1 and 2. detected_language reaches the answer prompt in
React-to-Me and Plant Reactome, as its own variable.

Reproduced before fixing: "Quel role joue TP53 dans l'apoptose ?" was
detected as French, rephrased to English, retrieved well -- and answered
in English. Four steps of five already worked; the fifth dropped the
value. chat_history still held the original French question and the
model answered in English anyway, so it was never that the model could
not tell.

The mechanism is the whole point. create_retrieval_chain passes only
`input` to the retriever --

    retrieval_docs = (lambda x: x["input"]) | retriever

-- so a separate prompt variable cannot reach BM25, the vector store, or
the query expansion in front of them. #140 appended the instruction to
`input` instead; measured through the whole retriever, that changes
about half the fused documents. A test asserts the retrieval query is
byte-identical across four languages, which is exact where a retrieval
baseline would only show noise.

The instruction's wording is @bleedblack1's, kept because it gets the
hard part right: gene symbols, protein names, pathway names, R-HSA ids
and URLs must survive untranslated. SET, MAX and CAT are gene symbols
and ordinary English words.

An adversarial review of the plan caught it contradicting the spec.
FR-007 promised English questions "no additional prompt content" while
the plan always passed the language -- which does add a sentence to
every English prompt. FR-007 was the wrong half: it now promises no
extra model call and a byte-identical retrieval query, and states the
added sentence plainly rather than hiding it. Branching instead would
leave the common path exercised only by non-English users.

Verified against the Release95 bundle. French question -> French answer,
R-HSA ids intact, 40 documents. English question -> unchanged English
answer, 40 documents. Reverting the call site fails the test that covers
it.

Worth watching: the French answer carried 2 R-HSA citations against the
English answer's 9. One sample, not a finding, but the kind of thing an
answer-quality run should look at.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@adamjohnwright
adamjohnwright merged commit 0454beb into main Sep 10, 2026
10 checks passed
@adamjohnwright
adamjohnwright deleted the feat/answer-in-user-language branch September 10, 2026 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant