Repository navigation
Conversation
The cue text and annotation validators accepted only the six named references of the WebVTT syntax table and dropped the whole cue on anything else, such as &docling-project#39; or é. The tokenizer (WebVTT 6.4) hands both states to the HTML character reference algorithm, so read with html.unescape and keep a reference that does not resolve as literal text. Serialization still writes only the escapes it needs. A reference that resolves to a line terminator is still rejected, because the cue would be written back across two lines. Refs docling-project#919 Signed-off-by: Christopher Krah <48338417+chrikrah@users.noreply.github.com>
Contributor
|
✅ DCO Check Passed Thanks @chrikrah, all your commits are properly signed off. 🎉 |
Contributor
Merge Protections🔴 1 of 2 protections blocking · waiting on 👀 reviews
🔴 Require two reviewer for test updatesWaiting for
This rule is failing.When test data is updated, we require two reviewers
Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #919.
WebVTTFile.parsedropped any cue holding a reference outside the six names of the syntax table, such as'oré. Cue text and annotations now go throughhtml.unescape, as @ceberam laid out in the issue. An unknown reference or a bare&stays literal text._escapeis untouched.One addition:
is still rejected. It resolves to a line break, and the cue would then be written back across two lines.Reverting
webvtt.pyand keeping the tests failstest_webvtt_cue_html_character_references, andtest_vtt_cue_commponentsfor its two invalid-entity cases, which now assert literal text.ruff 0.16.10check and format are clean.Signed off for DCO. A cue holding
is still dropped with a warning, as before, and keeping it needs_escapeto write line breaks back. I'd leave that until a real file has one. Fine with you?