Skip to content

Fix non-f-string ValueError in t5gemma _normalize_token - #753

Open
PratikDhanave wants to merge 1 commit into
google-deepmind:mainfrom
PratikDhanaveFork:fix/t5gemma-forbidden-token-fstring
Open

Fix non-f-string ValueError in t5gemma _normalize_token#753
PratikDhanave wants to merge 1 commit into
google-deepmind:mainfrom
PratikDhanaveFork:fix/t5gemma-forbidden-token-fstring

Conversation

@PratikDhanave

Copy link
Copy Markdown

In gemma/research/t5gemma/sampling.py, _normalize_token raises:

raise ValueError(
    'Invalid forbidden token: {token!r}. Forbidden tokens must map to'
    ' single token ids in the vocab.'
)

The first line uses {token!r} but is not an f-string, so the placeholder is emitted literally — the user sees Invalid forbidden token: {token!r} instead of the actual offending token. Add the f prefix so the token is interpolated. (The second concatenated line has no placeholders and correctly stays a plain string.)

The message used {token!r} but the literal lacked the f prefix, so the
offending token was never interpolated and the error read literally
'Invalid forbidden token: {token!r}'. Add the f prefix.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant