Repository navigation
fix(sefariasqlite): stop labelling Mishnah Berurah segments - #102
Merged
Merged
Conversation
Sefaria's Mishnah Berurah segments already open with their printed seif
katan marker, so the generated "(letter) " label doubled it ("(יד) (יד)
נהגו לתייג"), and gave fake numbers to the kuntresim that have none
(Mishnat Soferim in siman 36: "(טז) משנת סופרים", "(יז) תפילין...").
Simanim 494-529 mark the seif katan as "{א}", which read "(א) {א}".
Skip the generated label for this book and normalize a leading "{א}" to
"(א)". Line ids are unaffected: Sefaria lines are keyed on their heRef,
which is unchanged. Checked against the 2026-10-07 export: same 18,119
lines, no heRef difference, no doubled marker left.
Fixes #74
Fixes #73
Signed-off-by: Elie Gambache <elyahou.hadass@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Sefaria's Mishnah Berurah segments already start with their printed seif katan marker, so the generated
(letter)label:(יד) (יד) נהגו לתייג(כפילויות באותיות הס"ק במשנה ברורה #73)(טז) משנת סופרים,(יז) תפילין ומזוזות…(משנה ברורה, סימן לו - קונטרס דברי סופרים #74){א}, gave(א) {א}Fix
SefariaImportText.kt: list of books whose segments carry their own marker (onlyMishnah Berurah; elsewhere a leading(…)is קרי/כתיב,(ליקוט),(סעיף ה)… so no text heuristic), plus normalization of a leading{א}to(א).SefariaBookPayloadReader.kt: skip the generated label for those books and apply the normalization.SefariaSelfLabeledSegmentsTest: covers the book and a control book that still gets labels.ID stability
Sefaria line ids are keyed on
REF:+heRef, which this change doesn't touch. Running the reader on the 2026-10-07 export vs the current DB: same 18,119 lines, 0 heRef differences — the delta only upserts the content of ~17.4k Mishnah Berurah lines. No doubled marker or leading{left.Fixes #74
Fixes #73