Answer in the user's language - #205
Merged
Merged
Conversation
Spec 004, stages 1 and 2. detected_language reaches the answer prompt in
React-to-Me and Plant Reactome, as its own variable.
Reproduced before fixing: "Quel role joue TP53 dans l'apoptose ?" was
detected as French, rephrased to English, retrieved well -- and answered
in English. Four steps of five already worked; the fifth dropped the
value. chat_history still held the original French question and the
model answered in English anyway, so it was never that the model could
not tell.
The mechanism is the whole point. create_retrieval_chain passes only
`input` to the retriever --
retrieval_docs = (lambda x: x["input"]) | retriever
-- so a separate prompt variable cannot reach BM25, the vector store, or
the query expansion in front of them. #140 appended the instruction to
`input` instead; measured through the whole retriever, that changes
about half the fused documents. A test asserts the retrieval query is
byte-identical across four languages, which is exact where a retrieval
baseline would only show noise.
The instruction's wording is @bleedblack1's, kept because it gets the
hard part right: gene symbols, protein names, pathway names, R-HSA ids
and URLs must survive untranslated. SET, MAX and CAT are gene symbols
and ordinary English words.
An adversarial review of the plan caught it contradicting the spec.
FR-007 promised English questions "no additional prompt content" while
the plan always passed the language -- which does add a sentence to
every English prompt. FR-007 was the wrong half: it now promises no
extra model call and a byte-identical retrieval query, and states the
added sentence plainly rather than hiding it. Branching instead would
leave the common path exercised only by non-English users.
Verified against the Release95 bundle. French question -> French answer,
R-HSA ids intact, 40 documents. English question -> unchanged English
answer, 40 documents. Reverting the call site fails the test that covers
it.
Worth watching: the French answer carried 2 R-HSA citations against the
English answer's 9. One sample, not a finding, but the kind of thing an
answer-quality run should look at.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Sep 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Spec 004, stages 1 and 2.
detected_languagereaches the answer prompt in React-to-Me and Plant Reactome, as its own variable.What was going wrong
Reproduced before fixing:
Four steps of five already worked.
chat_historystill held the original French question and the model answered in English anyway — so it was never that the model couldn't tell; nothing asked it to reply in French.Why a prompt variable and not #140's approach
create_retrieval_chainpasses onlyinputto the retriever:So a separate variable cannot reach BM25, the vector store, or the query expansion in front of them. #140 appended its instruction to
input; measured through the whole retriever, that changes about half the fused documents.A test asserts the retrieval query is byte-identical across four languages — exact, where a retrieval baseline would only show noise and cost an hour of API time.
The instruction wording is @bleedblack1's, kept because it gets the hard part right:
SET,MAXandCATare gene symbols and ordinary English words, andR-HSA-9612973means nothing translated.The adversarial review caught my plan contradicting my spec
FR-007 promised English questions "no additional prompt content", while the plan always passes the language — which does add a sentence to every English prompt. FR-007 was the wrong half: it now promises no extra model call and a byte-identical retrieval query, and states the added sentence plainly. Branching instead would leave the common path exercised only by users who never ask in English.
Verified against the Release95 bundle
Reverting the call site fails the test that covers it. 231 tests pass.
Worth watching: the French answer carried 2
R-HSAcitations against English's 9. One sample, not a finding — but the kind of thing an answer-quality run should look at.#140 and #125 close once this lands.
🤖 Generated with Claude Code