Repository navigation
Conversation
…iter's own null also where the writer opened the run itself
## Description
When a client inserts text without a value inside a run of that value (uncommented text typed inside a comment, plain text inside bold), it restores the value after its text and relies on the item that ends the run. Rule 5 of `Cursor#format` makes a restore whose run another client's change opened write a null of its own right after that change's end, because that end can go while the restore stays: the change's undo deletes it, and a concurrent delete may drop it (a delete keeps a bounded number of the format items it leaves in a run of deleted content).
Two cases were left out, and in both the restored value ran on to the end of the document after a collaborator's concurrent delete dropped the run's end:
- **The writer opened the run itself.** `restatedEnd` skipped runs this client opened. A collaborator's delete that has not seen the restore can still drop that end. For example, a client comments a word and later, offline, types into it while a collaborator deletes text next to the commented word, which was deleted earlier. When the client comes back, the comment covers every later word on every client.
- **The run was opened by a write that also restored it.** For example, a comment written with uncommented text inside it, in one write. The run's end is then the end the restore wrote, but the check compared against the null before that text (the opener's `changeEnd`), so no own end was written.
Now a restore gets its own end after the end of any write in the chain that gave the run its value (the opener, or a restore in it: `runChain`, which replaces `runOpener`), whichever client wrote it. The extra null is written only where a restore is written, right after an end that already exists, so it adds at most one item per such insert, and the bounded collapse can remove it later like any other item (the growth-bound tests pass unchanged).
## Changeset
`.changeset/lexical-y-own-run-restore-end.md`: `@lexical/y` patch.
## Test plan
New `packages/lexical-y/src/__tests__/unit/OwnRunRestoreEnd.test.ts`, run in both client id orders:
- A comments "Kilo", then types uncommented "X" inside it; B deletes the word, then deletes the comment's opener and end with raw Yjs deletes (for B they bracket deleted content only), standing in for a concurrent deletion of that end such as the bounded collapse.
- The comment is written with uncommented "X" inside it, then A types uncommented "Y" inside it.
Each test expects nothing after the word to carry the comment.
`pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/OwnRunRestoreEnd.test.ts`
### Before
```
× ... ends where the comment ended when a collaborator's delete drops the comment's end (ids 1,2)
× ... ends where the comment ended when the comment was written with a hole in it (ids 1,2)
× ... ends where the comment ended when a collaborator's delete drops the comment's end (ids 2,1)
× ... ends where the comment ended when the comment was written with a hole in it (ids 2,1)
AssertionError: expected 'aa X bb[c1] cc[c1]' to be 'aa Xbb cc' // Object.is equality
AssertionError: expected 'aa Y bb[c1] cc[c1]' to be 'aa Ybb cc' // Object.is equality
Tests 4 failed (4)
```
With only the own-client exemption removed, the second test still fails in both orders. With only the chain change, the first does.
### After
```
Tests 4 passed (4)
```
- `pnpm exec vitest run --project unit packages/lexical-y`: 165 files passed (7483 tests, 46 expected fail), no failures.
- `pnpm run ci-check` passes.
- In a randomized long-session soak (2 people and an agent, 2 simulated hours), the four sessions that showed this spread now show none: 12, 8, 8 and 4 unexplained formatted words become 0. Twenty-four other sessions show the same results with and without the change.
- Extended fuzz (`FUZZ_SEED_START=50000 FUZZ_SEEDS=2000`): the same seven seeds fail with and without the change, with the same messages, so none of them comes from it.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
lexical
@lexical/a11y
@lexical/clipboard
@lexical/code
@lexical/code-core
@lexical/code-prism
@lexical/code-shiki
@lexical/compiler
@lexical/devtools-core
@lexical/dragon
@lexical/eslint-plugin
@lexical/extension
@lexical/fast-check
@lexical/file
@lexical/hashtag
@lexical/headless
@lexical/history
@lexical/html
@lexical/internal
@lexical/link
@lexical/list
@lexical/mark
@lexical/markdown
@lexical/mdast
@lexical/offset
@lexical/overflow
@lexical/plain-text
@lexical/react
@lexical/rich-text
@lexical/selection
@lexical/table
@lexical/tailwind
@lexical/text
@lexical/utils
@lexical/y
@lexical/yjs
commit: |
…nd a value an undo ends past other writes' run ends ## Description Depends on #9562 (based on its branch): a restore that relies on a run end of its own client's run gets its own end there. The undo here keeps such restores (case 1), so without #9562 a collaborator's deletes that collapse that run end away let the restore spread the value (a comment on a word, text typed inside it and undone offline, the word and its neighbours deleted by a collaborator: the comment covered the rest of the document). Cases where a bold, a comment or another format value spread onto text nobody gave it, on every client and in a fresh read: 1. An undo of text typed inside a run of another value deleted the run ends that text was written with. F types plain " mid" into bold "one two": the write ends the bold before " mid" and gives it back after it. E types plain " plain" right after "one": that text lands after F's null and writes no item of its own (its value matches). F undoes (whether or not it has seen E's text): the undo deleted F's null and F's restore with " mid", and " plain" turned bold (or, with a comment, commented). The same with a paragraph inserted between the halves of a bold run split by Enter, and with bold text typed into plain text (text typed before it inside its run lost the bold). A delete of " mid" by hand keeps those run ends (`Cursor#collapseGap`); the undo did not. The undo now keeps them (`neutralRunEnds`, through the undo manager's delete filter): the format items of its step next to content of its own that it deletes (past format items and content written there since: content whose Yjs origins lead to an item of the step), where deleting them would not change the value of content that was there before the step, or that the undo brings back (that value is the one the item of the key before them gives, past the run ends a collaborator's write put around text it typed inside the step's content, deleted since: those give nothing). Where it keeps a restore (the item that gives a run's value back after the text), it also keeps the null its write put right after the end of that run, which the restore relies on (`Cursor#format`, rule 5). Items of a format change (next to content the step did not write) are deleted as before, so an undo of a format change still removes it. Then, as a delete does, the run of items without live content the undo leaves is collapsed (`collapseUndone`, through `Cursor#collapseDeleted`), so text of another value typed and undone again and again at one place leaves a bounded number of items. It deletes only this client's own items there (`Cursor#collapsible`): another write's run end can be all a collaborator's text relies on once that collaborator undoes its own items. That collapse starts from the formats the editor shows, as the cursor it uses walks them: a collaborator's format a composition holds back is not counted (counting it lost an offline peer's bold or comment when the undo came while composing). 2. An insert's run ends inside another write's run restate that run's value: F's restore after " mid" gives the rest of A's bold back. A bolds "one two"; F types " mid" inside and deletes it (on main), or undoes it (with 1.); A undoes the bold: " two" stayed bold. After an undo or redo, where the restore of an insert that is gone gives back the value the content before that insert's null had, and the undo or redo changed that value, an item of this client's own now gives the content after it the new value (`restateRunEnds`), placed after what was written next to the restore since (text a collaborator typed right after it relies on its value and keeps it). Such a restore (`boundsOwnInsert`) is one of two items one insert wrote around text of its own, all of it deleted since, inserted before content that was there already, and giving back the value the key had where the insert went. A format change's closer ends its own value, and the items of a write that formats its own content (a document written in one go) open values of their own: both are left alone. The item is recorded on the step the undo or redo pushed, and the next redo or undo of that step deletes it before restating again (`unrestate`). An undo or redo and its follow-up writes are one operation: the steps it took and pushed are taken from the undo manager's events, by listeners the binding registers first (so a listener of the app that clears the history or throws does not skip the follow-up writes, and its error is thrown after them), what `restateRunEnds` writes is recorded on its step before it is written, and `undo()`/`redo()` asked for during the operation (from a listener) does nothing. `regressions/Formats.test.ts` L1B-1 expected C's text, typed over J's plain text after a commented word, to take the comments when J's typing was undone at the same time: that was case 1 (J's undo deleted the null C's text relied on). C's text now stays plain. ## Status against the randomized sessions that found these Sessions that change the format of a word no op touched, on this change and not on main with the same ops: - Fixed (pass now): L1B fmt 206163 (an underline undo took a document's own underline off a later word), L1B mark 324160 (gained bold), L1B mark 336261 (gained a link), L1B undoheavy 413112 (lost bold), N brand 1200036 and 1200107 (gained bold). - Still failing: L1B fmt 226508 (a word typed inside an italic run, after the restore of an insert that is gone, loses the italic when the run is undone: see the known regression above), L1B mix 143920 (an undo of an underline gives six words on another line the underline; under investigation). ## Changeset `.changeset/lexical-y-undo-run-ends.md` (`[fix]`). ## Test plan New `packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts` (all six client id orders where the order matters): bold and comment, a paragraph in a split bold run, bold text typed into plain text, the undoing client having seen the collaborator's text, a redo, bounds on the format items left by typing and undoing at one place (one value, and a new style each time), controls (no undo, a delete by hand), undo-redo-undo chains of the bold after a word typed inside it was undone or deleted, a third peer's text inside the collaborator's, a format change's closer next to undone text, the null a kept restore relies on surviving a collaborator's delete, `boundsOwnInsert` on an insert's run ends, a format change's and a document's written in one go, an undo of an underline next to a document's own run ends (its later underline stays), a comment's ends a collaborator's deletes drop (with #9562), an undo while a composition holds a collaborator's bold or comment, the restore an undo keeps when a collaborator's undone text inside left run ends, a collaborator's format over text an undo removes, text another peer typed right after a restore or before the run's next word, and listeners that undo again, clear the history or throw (of the undo manager and of the follow-up write). The comment-ends, composition and kept-restore cases pass on main and guard against regressions of this change. Known regression vs main on one shape (a randomized session): a word a collaborator typed inside a run, taking its value, after the restore of an insert that is gone, loses the value when the run's format is undone; on main it keeps it there because that restore keeps giving it. Without such a restore in front of it the word loses the value on main too (A italics "one two three", B types italic "X " inside, A undoes: "X " is plain on main), which is what this change gives. ### Before ``` $ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts Tests 59 failed | 52 passed (111) × bold: text typed after its null stays plain (6 of 6 id orders) × comment: text typed after its null stays uncommented (6 of 6) × a paragraph inserted into a bold run split by Enter (6 of 6) × bold text typed into plain text: text typed before it inside its run stays bold (6 of 6) × F has seen E's text when it undoes; then A undoes the bold (6 of 6) × text a third peer typed inside the collaborator's text stays plain too (2 of 2) × F types " mid" into A's bold and deletes it; A undoes the bold, redoes it, undoes it (6 of 6) × a redo puts the text back with its own value × the end a kept restore relies on stays ended when a collaborator deletes the run (6 of 6) × only an insert's own run ends restate a value (not a format change's) × the format items of a write that formats its own content do not restate a value (S9-9) × text another peer typed in the run's value after " mid" / before " two" keeps it when the run is undone (12) ✓ F types " mid" into A's bold and undos it; A undoes the bold, redoes it, undoes it (6 of 6) ✓ controls, bounds and stack listener cases AssertionError: expected 'one plain two*' to be 'one*| plain| two*' // Object.is equality AssertionError: expected 'one plain two[c1]' to be 'one[c1]| plain| two[c1]' // Object.is equality AssertionError: expected 'one* / plain* / two*' to be 'one* / plain / two*' // Object.is equality AssertionError: expected 'one B two' to be 'one| B*| two' // Object.is equality AssertionError: expected 'one two*' to be 'one two' // Object.is equality ``` ### After ``` $ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts Test Files 1 passed (1) Tests 111 passed (111) ``` The `@lexical/y` and `@lexical/mark` unit suites pass (on #9562), the randomized ones included (`Fuzz`, `FollowUpFuzz`, `ConcurrentShapesFuzz`, `NestedFuzz`, `SlotsFuzz`): ``` $ pnpm exec vitest run --project unit packages/lexical-y packages/lexical-mark Test Files 182 passed | 2 skipped (184) Tests 8755 passed | 46 expected fail | 4 skipped (8805) ``` `pnpm run ci-check` passes.
…l sits next to it and a collaborator collapses the run
## Description
With this pull request a restore writes a null of its own right after the run's end also where its own client wrote that end. One client can then hold two items with the same value in a row from two of its writes: the run's end from an earlier write, and the null of a later write. `Cursor#collapseGap` kept, of such a key, the last item and the last earlier item with the same value written by another client. So a collaborator's collapse kept the later null and deleted the earlier end, the author's concurrent undo of the later write deleted the null, and nothing ended the run. In a randomized session a link undone this way ran on over the following lines on every client. The same happened with a second author whose null was the last item.
The collapse now also keeps, of each of the two clients it keeps an item of there, the first item (lowest clock) with that value since the value last changed. That is at most two more items per key, so the bound holds.
## Test plan
New cases in `packages/lexical-y/src/__tests__/unit/OwnRunRestoreEnd.test.ts`: A comments "Kilometer" and types uncommented "X" and then "Y" inside it in two writes; offline, A undoes both while B deletes "ter" and then " bb"; after the sync nothing after "Kilome" carries the comment (both id orders). The same with B also typing "Z" inside the comment and undoing it while C deletes (four id orders).
`pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/OwnRunRestoreEnd.test.ts`
### Before
```
× S9-12: ends where the comment ended when its author undoes two writes inside it while a collaborator deletes past its end (ids 1,2)
× S9-12: ends where the comment ended when its author undoes two writes inside it while a collaborator deletes past its end (ids 2,1)
× S9-12: ends where the comment ended when two authors undo their writes inside it while a third deletes past its end (ids 1,2,3)
× S9-12: ends where the comment ended when two authors undo their writes inside it while a third deletes past its end (ids 3,2,1)
× S9-12: ends where the comment ended when two authors undo their writes inside it while a third deletes past its end (ids 2,1,3)
× S9-12: ends where the comment ended when two authors undo their writes inside it while a third deletes past its end (ids 1,3,2)
AssertionError: expected 'aa Kilome[c1] cc[c1] dd[c1]' to be 'aa Kilome[c1] cc dd' // Object.is equality
Tests 6 failed | 4 passed (10)
```
### After
```
Test Files 1 passed (1)
Tests 10 passed (10)
```
The new cases pass without this pull request (main does not write the author's own null there). The randomized session that found it (a formatting-heavy profile, an undo of a link) passes, and fails without this commit.
`pnpm exec vitest run --project unit packages/lexical-y packages/lexical-mark`:
```
Test Files 181 passed | 2 skipped (183)
Tests 8650 passed | 46 expected fail | 4 skipped (8700)
```
`pnpm run ci-check` passes. Extended fuzz (`FUZZ_SEED_START=1 FUZZ_SEEDS=3000`): only seed 2277 fails (`dup`), the same on main and on this branch without this commit.
…nd a value an undo ends past other writes' run ends ## Description Depends on #9562 (based on its branch): a restore that relies on a run end of its own client's run gets its own end there. The undo here keeps such restores (case 1), so without #9562 a collaborator's deletes that collapse that run end away let the restore spread the value (a comment on a word, text typed inside it and undone offline, the word and its neighbours deleted by a collaborator: the comment covered the rest of the document). Cases where a bold, a comment or another format value spread onto text nobody gave it, on every client and in a fresh read: 1. An undo of text typed inside a run of another value deleted the run ends that text was written with. F types plain " mid" into bold "one two": the write ends the bold before " mid" and gives it back after it. E types plain " plain" right after "one": that text lands after F's null and writes no item of its own (its value matches). F undoes (whether or not it has seen E's text): the undo deleted F's null and F's restore with " mid", and " plain" turned bold (or, with a comment, commented). The same with a paragraph inserted between the halves of a bold run split by Enter, and with bold text typed into plain text (text typed before it inside its run lost the bold). A delete of " mid" by hand keeps those run ends (`Cursor#collapseGap`); the undo did not. The undo now keeps them (`neutralRunEnds`, through the undo manager's delete filter): the format items of its step next to content of its own that it deletes (past format items and content written there since: content whose Yjs origins lead to an item of the step), where deleting them would not change the value of content that was there before the step, or that the undo brings back (that value is the one the item of the key before them gives, past the run ends a collaborator's write put around text it typed inside the step's content, deleted since: those give nothing). Where it keeps a restore (the item that gives a run's value back after the text), it also keeps the null its write put right after the end of that run, which the restore relies on (`Cursor#format`, rule 5). Items of a format change (next to content the step did not write) are deleted as before, so an undo of a format change still removes it. Then, as a delete does, the run of items without live content the undo leaves is collapsed (`collapseUndone`, through `Cursor#collapseDeleted`), so text of another value typed and undone again and again at one place leaves a bounded number of items. It deletes only this client's own items there (`Cursor#collapsible`): another write's run end can be all a collaborator's text relies on once that collaborator undoes its own items. That collapse starts from the formats the editor shows, as the cursor it uses walks them: a collaborator's format a composition holds back is not counted (counting it lost an offline peer's bold or comment when the undo came while composing). 2. An insert's run ends inside another write's run restate that run's value: F's restore after " mid" gives the rest of A's bold back. A bolds "one two"; F types " mid" inside and deletes it (on main), or undoes it (with 1.); A undoes the bold: " two" stayed bold. After an undo or redo, where the restore of an insert that is gone gives back the value the content before that insert's null had, and the undo or redo changed that value, an item of this client's own now gives the content after it the new value (`restateRunEnds`), placed after what was written next to the restore since (text a collaborator typed right after it relies on its value and keeps it). Such a restore (`boundsOwnInsert`) is one of two items one insert wrote around text of its own, all of it deleted since, inserted before content that was there already, and giving back the value the key had where the insert went. A format change's closer ends its own value, and the items of a write that formats its own content (a document written in one go) open values of their own: both are left alone. The item is recorded on the step the undo or redo pushed, and the next redo or undo of that step deletes it before restating again (`unrestate`). An undo or redo and its follow-up writes are one operation: the steps it took and pushed are taken from the undo manager's events, by listeners the binding registers first (so a listener of the app that clears the history or throws does not skip the follow-up writes, and its error is thrown after them), what `restateRunEnds` writes is recorded on its step before it is written, and `undo()`/`redo()` asked for during the operation (from a listener) does nothing. `regressions/Formats.test.ts` L1B-1 expected C's text, typed over J's plain text after a commented word, to take the comments when J's typing was undone at the same time: that was case 1 (J's undo deleted the null C's text relied on). C's text now stays plain. ## Status against the randomized sessions that found these Sessions that change the format of a word no op touched, on this change and not on main with the same ops: - Fixed (pass now): L1B fmt 206163 (an underline undo took a document's own underline off a later word), L1B mark 324160 (gained bold), L1B mark 336261 (gained a link), L1B undoheavy 413112 (lost bold), N brand 1200036 and 1200107 (gained bold). - Still failing: L1B fmt 226508 (a word typed inside an italic run, after the restore of an insert that is gone, loses the italic when the run is undone: see the known regression above), L1B mix 143920 (an undo of an underline gives six words on another line the underline; under investigation). ## Changeset `.changeset/lexical-y-undo-run-ends.md` (`[fix]`). ## Test plan New `packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts` (all six client id orders where the order matters): bold and comment, a paragraph in a split bold run, bold text typed into plain text, the undoing client having seen the collaborator's text, a redo, bounds on the format items left by typing and undoing at one place (one value, and a new style each time), controls (no undo, a delete by hand), undo-redo-undo chains of the bold after a word typed inside it was undone or deleted, a third peer's text inside the collaborator's, a format change's closer next to undone text, the null a kept restore relies on surviving a collaborator's delete, `boundsOwnInsert` on an insert's run ends, a format change's and a document's written in one go, an undo of an underline next to a document's own run ends (its later underline stays), a comment's ends a collaborator's deletes drop (with #9562), an undo while a composition holds a collaborator's bold or comment, the restore an undo keeps when a collaborator's undone text inside left run ends, a collaborator's format over text an undo removes, text another peer typed right after a restore or before the run's next word, and listeners that undo again, clear the history or throw (of the undo manager and of the follow-up write). The comment-ends, composition and kept-restore cases pass on main and guard against regressions of this change. Known regression vs main on one shape (a randomized session): a word a collaborator typed inside a run, taking its value, after the restore of an insert that is gone, loses the value when the run's format is undone; on main it keeps it there because that restore keeps giving it. Without such a restore in front of it the word loses the value on main too (A italics "one two three", B types italic "X " inside, A undoes: "X " is plain on main), which is what this change gives. ### Before ``` $ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts Tests 59 failed | 52 passed (111) × bold: text typed after its null stays plain (6 of 6 id orders) × comment: text typed after its null stays uncommented (6 of 6) × a paragraph inserted into a bold run split by Enter (6 of 6) × bold text typed into plain text: text typed before it inside its run stays bold (6 of 6) × F has seen E's text when it undoes; then A undoes the bold (6 of 6) × text a third peer typed inside the collaborator's text stays plain too (2 of 2) × F types " mid" into A's bold and deletes it; A undoes the bold, redoes it, undoes it (6 of 6) × a redo puts the text back with its own value × the end a kept restore relies on stays ended when a collaborator deletes the run (6 of 6) × only an insert's own run ends restate a value (not a format change's) × the format items of a write that formats its own content do not restate a value (S9-9) × text another peer typed in the run's value after " mid" / before " two" keeps it when the run is undone (12) ✓ F types " mid" into A's bold and undos it; A undoes the bold, redoes it, undoes it (6 of 6) ✓ controls, bounds and stack listener cases AssertionError: expected 'one plain two*' to be 'one*| plain| two*' // Object.is equality AssertionError: expected 'one plain two[c1]' to be 'one[c1]| plain| two[c1]' // Object.is equality AssertionError: expected 'one* / plain* / two*' to be 'one* / plain / two*' // Object.is equality AssertionError: expected 'one B two' to be 'one| B*| two' // Object.is equality AssertionError: expected 'one two*' to be 'one two' // Object.is equality ``` ### After ``` $ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts Test Files 1 passed (1) Tests 111 passed (111) ``` The `@lexical/y` and `@lexical/mark` unit suites pass (on #9562), the randomized ones included (`Fuzz`, `FollowUpFuzz`, `ConcurrentShapesFuzz`, `NestedFuzz`, `SlotsFuzz`): ``` $ pnpm exec vitest run --project unit packages/lexical-y packages/lexical-mark Test Files 182 passed | 2 skipped (184) Tests 8755 passed | 46 expected fail | 4 skipped (8805) ``` `pnpm run ci-check` passes.
…n a write, and keep the collapse as it was ## Description The previous commit made `Cursor#collapseGap` keep more items of a key (the first same-value item of each client it already kept one of). That brought back the comment spread after an offline return that this pull request fixes: in a randomized session, keeping a run's old end let the collapse drop the null of its own that a later restore had written after it, so the next restores there found no end of their chain to match and wrote none. This commit removes that collapse change. The link spread it addressed came from one write that restored a run at two places: each restore wrote a null of the writer's own right after the same end (rule 5 of `Cursor#format`). A collaborator's collapse keeps a key's last items, so it kept the two nulls and deleted the end, and the writer's undo of that write removed both nulls. Now a write puts at most one null of its own right after a given end (`Cursor#endedInThisWrite`): an undo removes the write's items together, so a second one never protects anything, and the end stays among the last items a collapse keeps. Not covered: two separate writes of one client that each restore inside its own run, both undone while a collaborator collapses past the run's end, still lose the end, because each write adds a null of its own there. Without this pull request no such null is written for the writer's own run. ## Test plan `packages/lexical-y/src/__tests__/unit/OwnRunRestoreEnd.test.ts` now holds one S9-12 case (both id orders) instead of the previous commit's six: A comments "Kilometer"; one write of A types uncommented "X" after "Kil" and "Y" after "Kilome"; offline, A undoes it while B deletes "ter" and then " bb"; after the sync nothing after "Kilome" carries the comment. The previous commit's two-write and two-author cases are the shape described above and are removed. `pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/OwnRunRestoreEnd.test.ts` ### Before ``` On c1a7909 (this pull request without its follow-up commits): × S9-12: ends where the comment ended when its author undoes a write with two inserts inside it while a collaborator deletes past its end (ids 1,2) × S9-12: ends where the comment ended when its author undoes a write with two inserts inside it while a collaborator deletes past its end (ids 2,1) AssertionError: expected 'aa Kilome[c1] cc[c1] dd[c1]' to be 'aa Kilome[c1] cc dd' // Object.is equality Tests 2 failed | 4 passed (6) ``` ### After ``` Test Files 1 passed (1) Tests 6 passed (6) ``` The new case passes on main. The randomized sessions: the link spread (formatting-heavy profile) passes; the four offline-return sessions this pull request fixes show no unexplained formatted words, against 12 for one of them with the previous commit. `pnpm exec vitest run --project unit packages/lexical-y packages/lexical-mark`: ``` Test Files 181 passed | 2 skipped (183) Tests 8646 passed | 46 expected fail | 4 skipped (8696) ``` `pnpm run ci-check` passes. Extended fuzz (`FUZZ_SEED_START=1 FUZZ_SEEDS=3000`): only seed 2277 fails (`dup`), the same on main.
There was a problem hiding this comment.
Thanks for the detailed write-up and the two-commit split. Both behaviours hold up: the four headline cases in OwnRunRestoreEnd.test.ts fail on main and pass here, and the S9-12 cases fail with only the first commit and pass with the collapseGap change, so the second commit is clearly needed by the first.
The own-run null can erase a collaborator's concurrent format that starts where the run ends (packages/lexical-y/src/internal/Cursor.ts:657, written at :608-613)
With the own-client exemption removed from restatedEnd, a client that writes a restore inside a run it opened itself now inserts a null right after the run's end, with that end as its origin and the following content as its right origin. A collaborator who formats text starting at that same end at the same time keeps the end and opens its value right after it (format rule 2, :1303-1316), with the same origin and right origin. Yjs breaks that tie by client id, so when the restoring client's id is the higher one, the null lands between the collaborator's opener and its text, and the collaborator's format disappears from its whole range on every peer. On aa Kilo bb cc, with A = client 2 and B = client 1:
| Concurrent edits | main |
this PR |
|---|---|---|
A bolds Kilo, later types plain X inside it; B bolds bb |
bb bold |
bb plain |
A bolds Kilo, later removes bold from il; B bolds bb |
bb bold |
bb plain |
A comments Kilo as c1, later types uncommented X inside it; B adds c1 to bb |
bb in c1 |
bb not in c1 |
With ids 1, 2 all three keep B's format. main already behaves this way when another client opened the run (B bolds Kilo, A types plain X inside it, B bolds bb): the placement rule 5 documents ("right after the end, not before it", Cursor.ts:1167-1171) protects text typed at the end of the run, not an opener placed there. This PR extends that to runs the writing client opened itself, such as editing inside formatting you made yourself. The Limits section of packages/lexical-website/docs/collaboration/y.md lists the concurrent edits that do not merge as a user might expect, and this one is not among them.
A test in the style of OwnRunRestoreEnd.test.ts (add $setSelection to the lexical import) fails here with ids [2, 1] and passes on main:
const $bold = (from: number, to: number) => () => {
const sel = $selectIn([0], from, to);
sel.format = (sel.anchor.getNode() as TextNode).getFormat();
sel.formatText('bold');
$setSelection(null);
};
test(`keeps a collaborator's bold that starts where the author's bold ends (ids ${ids})`, async () => {
const t = await setup(
() => $getRoot().append($p('aa Kilo bb cc'), $p('tail line')),
{ids, prepare},
);
t.a.update($bold(3, 7));
await t.sync();
t.offline();
t.a.update($bold(4, 6)); // A removes bold from "il" of its own bold word
t.b.update($bold(7, 10)); // B bolds " bb", starting where that word ends
await t.sync();
await expectConverged(t.peers);
expect(t.a.describe()).toContain('"o bb"{1}');
});Could the rule-5 null be placed, or followed up, so that it cannot separate another client's concurrent opener from its text? That would also fix the main case. If the trade-off is intended, please pin it with tests in both id orders and add it to the Limits list in y.md.
Smaller points
- The two headline tests drop the comment's end with raw
Item#deletecalls (dropCommentItems,OwnRunRestoreEnd.test.ts:63-83). If one of the soak sessions can be reduced to binding operations whose delete collapse drops the end, that case would show the binding reaching the state on its own. - The cases you deferred, three clients' items after the end (#9575) and reads past a pending struct (covered in
y.mdunder "Deliver each client's updates in order"), also happen onmain, so they are not blockers here.
| continue; | ||
| } | ||
| return valueOf(it) === null && changeEnd(doc, opener) === it ? it : null; | ||
| return valueOf(it) === null && chain.some(c => changeEnd(doc, c) === it) |
There was a problem hiding this comment.
With the own-client exemption gone, this now also returns the end of a run the writing client opened itself, and insertNegatedFormats writes a null right after it (:608-613). A collaborator's concurrent format that starts at that end opens its value right after the same end (format rule 2, :1303-1316), so the two items share an origin and a right origin and Yjs orders them by client id. When the restoring client's id is higher, the null sits between the collaborator's opener and its text and that format is lost on every peer: A bolds Kilo, then removes bold from il while B bolds bb, and bb ends up plain with ids [2, 1] (it stays bold on main). The main comment has a test for it.
…ndos lose as an expected failure
## Description
The previous commit gives a run end one null of the writer's own per write. When one client restores inside its own run in two separate writes, each write still adds a null of its own right after the run's end; a collaborator's collapse keeps those two nulls (a key's last items) and drops the end, and when the client undoes both writes the run runs on. Main does not write a null of the client's own for its own run, so main does not lose the end here: this is a gap of this pull request. Writing one own null per end across writes would leave a restore without its own end when only the earlier write is undone, so the case needs provenance or another collapse rule. It is pinned as `test.fails` so it is tracked and flips when fixed.
## Test plan
New `test.fails` case "S9-12b" in `packages/lexical-y/src/__tests__/unit/OwnRunRestoreEnd.test.ts` (both id orders): A comments "Kilometer", types uncommented "X" and then, in a second write, "Y" inside it; offline, A undoes both while B deletes "ter" and then " bb".
### Before
The same body as a plain test, on this branch (it fails for the stated reason):
```
AssertionError: expected 'aa Kilome[c1] cc[c1] dd[c1]' to be 'aa Kilome[c1] cc dd' // Object.is equality
Tests 2 failed | 6 skipped (8)
```
On main the plain test passes:
```
Tests 2 passed | 6 skipped (8)
```
### After
```
Test Files 1 passed (1)
Tests 6 passed | 2 expected fail (8)
```
There was a problem hiding this comment.
Thanks for the detailed write-up, the reproductions in both client-id orders, and for pinning the known gap instead of leaving it implicit. One regression needs a fix before this lands.
1. A restore link in runChain can take a later write's hole as the run's end (packages/lexical-y/src/internal/Cursor.ts:681, changeEnd at :152)
restatedEnd now accepts changeEnd(doc, c) for every link c of the chain. For a restore link, changeEnd returns the first item of the key, with another value, that the restore's client wrote after it in clock order. When that restore's own write put no rule-5 null (for example because its run ended at another client's hole), that item is the null opening the same client's next hole in the run, which is exactly the null the rule 5 comment says "is no such end" (Cursor.ts:1155-1157). Another client's restore in front of that hole then writes a null right after it; when the hole's author undoes that write, the hole's null and restore are deleted, the extra null stays, and the rest of the word loses the comment on every peer. No concurrency or offline step is needed. Added to OwnRunRestoreEnd.test.ts (it reuses words and $insertPlainAt), this passes on main and fails here:
test(`a later write's hole is not a run end (ids ${ids})`, async () => {
const t = await setup(
() => $getRoot().append($p('aa Kilometer bb cc'), $p('tail line')),
{ids, prepare},
);
t.a.update(() => void $addMarkID('c1', $selectIn([0], 3, 12)));
await t.sync();
t.b.update(() => $insertPlainAt(10, 'Y')); // after "Kilomet"
await t.sync();
t.a.update(() => $insertPlainAt(6, 'X')); // after "Kil"
await t.sync();
t.a.update(() => $insertPlainAt(9, 'Z')); // after "om", a second write
await t.sync();
t.b.update(() => $insertPlainAt(8, 'Q')); // after "o"
await t.sync();
t.a.undo(); // removes "Z"
await t.sync();
await expectConverged(t.peers);
// this branch: 'aa Kil[c1] X o[c1] Q m[c1] etY er[c1] bb cc'
expect(words(t.a)).toBe('aa Kil[c1] X o[c1] Q met[c1] Y er[c1] bb cc');
});The same steps with bold (formatText('bold') instead of the comment) leave "et" plain. After B types "Q", B's null sits right after A's null that opens the "Z" hole, because changeEnd of A's first restore is that null. Keeping the own-client change but going back to comparing only with the opener's end makes the case pass. One direction: do not accept a candidate end that its client follows with a restore of the run's value after its own text, or bound changeEnd on a restore link to that restore's own write.
2. S9-12b is a regression against main (OwnRunRestoreEnd.test.ts:228-260)
Its body passes on main; on this branch the comment runs on over cc dd and the next paragraph. The test.fails documents it well. Since #9575 covers the four-client variant (three nulls from two collaborators) that already happens on main, please link this two-client variant there or in a new issue so it stays tracked until provenance or another collapse rule lands.
Smaller test notes
- Both headline cases reach their precondition with raw Yjs deletes (
dropCommentItems). A case where the end goes through a collaborator's own deletes, as S9-12 does, would show the fix without the simulation. - The hole case only exercises the comment author's own restore. A collaborator typing inside a comment written with a hole would cover the other-client path the description says the chain also fixes.
The changeset follows CONTRIBUTING.md (patch, a one-line [fix] summary, and a Verify: paragraph).
| continue; | ||
| } | ||
| return valueOf(it) === null && changeEnd(doc, opener) === it ? it : null; | ||
| return valueOf(it) === null && chain.some(c => changeEnd(doc, c) === it) |
There was a problem hiding this comment.
must_fix: for a restore link of runChain, changeEnd returns the first item of the key its client wrote after it in clock order. When that restore's own write put no rule-5 null, that item is the null opening the same client's next hole, which rule 5 (:1155-1157) says is no end. Another client's restore in front of that hole then writes a null right after it, and the hole author's undo leaves that null ending the comment early: main keeps aa Kil[c1] X o[c1] Q met[c1] Y er[c1] bb cc, this branch gives aa Kil[c1] X o[c1] Q m[c1] etY er[c1] bb cc (steps in the review comment). One direction: reject a candidate end that its client follows with a restore of the run's value after its own text, or bound changeEnd on restore links to that restore's own write.
There was a problem hiding this comment.
Fixed in 83df52d: a candidate end that its client follows, in clock order, with its own text and a restore of the run's value opens a hole, not an end (opensHole), so no null is written after it. The review's test is in OwnRunRestoreEnd.test.ts ("a later write's hole is not a run end") and passes for a comment and for bold, in both client id orders.
| expect(words(t.a)).toBe('aa Kilometer[c1] bb cc dd'); | ||
| t.b.update(() => void $selectIn([0], 11, 14).removeText()); | ||
| t.b.update(() => void $selectIn([0], 11, 14).removeText()); | ||
| expect(words(t.b)).toBe('aa Kil[c1] X ome[c1] Y cc dd'); |
There was a problem hiding this comment.
This body passes on main, so S9-12b is a regression this change introduces rather than an existing gap. Worth linking from #9575 (the four-client variant, with three nulls from two collaborators, that already fails on main) or a new issue so it stays tracked.
There was a problem hiding this comment.
Accepted as a documented limit (rare, and it loses no text): the test.fails pin stays here, and a sentence in y.md's Limits covering it and #9575 comes with #9569. #9561, stacked on this PR, makes this case pass (its undo keeps the run ends of the text it removes) and turns the pin into a plain test.
…a restore relies on
## Description
Follow-up to this PR's own-run rule 5 (`restatedEnd` follows `runChain`).
For a restore link of the chain, `changeEnd` returns the first item of the
key, with another value, that the restore's client wrote after it in clock
order. Where that restore's own write put no rule-5 null (its run ended at
another client's item), that item can be the null opening the same
client's next hole in the run: its null, its text and its restore of the
run's value, written in a later write. Another client's restore in front of
that hole then wrote a null right after it, and when the hole's author
undid that later write, the hole's null and restore were deleted and the
extra null stayed, ending the run early on every peer: A comments
"Kilometer", B types "Y" after "Kilomet", A types "X" after "Kil" and then
"Z" after "om", B types "Q" after "o", A undoes "Z": "et" lost the comment
(or the bold, with bold).
A candidate end that its client follows, in clock order, with text of its
own and then a restore of the run's value is the null of a hole, not an end
(`opensHole`), and gets no null after it. Reported in review.
## Test plan
`OwnRunRestoreEnd.test.ts`: "a later write's hole is not a run end", for a
comment and for bold, in both client id orders (the expected strings are
main's results for the same steps).
### Before
```
$ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/OwnRunRestoreEnd.test.ts
Tests 4 failed | 6 passed | 2 expected fail (12)
AssertionError: expected 'aa Kil[c1] X o[c1] Q m[c1] etY er[c1]…' to be 'aa Kil[c1] X o[c1] Q met[c1] Y er[c1]…' // Object.is equality
AssertionError: expected 'aa |Kil*|X|o*|Q|m*|etY|er*| bb cc' to be 'aa |Kil*|X|o*|Q|met*|Y|er*| bb cc' // Object.is equality
```
### After
```
$ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/OwnRunRestoreEnd.test.ts
Tests 10 passed | 2 expected fail (12)
```
… or format can run on past its end when writes inside it are undone while another client deletes across its end ## Description A rare race is now a documented limit of `@lexical/y` (#9575). Writes inside a comment or a format are undone (two collaborators' writes; with #9562, also one client's two writes) while another client, which has not seen the undos, deletes text across the comment's end. Each write ended the run before its text and restored it after, leaving a null of its writer right after the run's end. The delete collapses the deleted content there, keeping those nulls but not the run's own end, and the undos then remove them, so the run carries on over the following words and into the next line. No text is lost. The Undo bullet of the Limits section now says so. `Races.test.ts` gets #9575's repro (four clients, both id orders) as a regular test asserting today's outcome and a `test.fails` asserting the fix, which passes once #9575 is fixed. Checked on main for a comment (`aa |Kilome cc dd[c1]` / `tail line[c1]`) and the same steps with bold instead of a comment (`aa |Kilome cc dd[b]` / `tail line[b]`), and on #9562's head for one client's two writes (its S9-12b: `aa Kilome[c1] cc[c1] dd[c1]` / `tail line[c1]`). ## Test plan ``` pnpm exec vitest run --project unit --reporter=verbose packages/lexical-y/src/__tests__/unit/Races.test.ts -t 9575 pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/Races.test.ts pnpm run ci-check ``` ### Before ``` $ vitest --run --project unit packages/lexical-y/src/__tests__/unit/Races.test.ts Tests 116 passed | 2 expected fail (118) ``` ### After ``` $ vitest --run --project unit --reporter=verbose packages/lexical-y/src/__tests__/unit/Races.test.ts -t 9575 ✓ ... undo of writes inside a comment ‖ delete across its end > #9575 as documented: the comment runs on over the next words and line (1,2,3,4) ✓ ... undo of writes inside a comment ‖ delete across its end > #9575: the comment ends where it ended (1,2,3,4) ✓ ... undo of writes inside a comment ‖ delete across its end > #9575 as documented: the comment runs on over the next words and line (4,3,2,1) ✓ ... undo of writes inside a comment ‖ delete across its end > #9575: the comment ends where it ended (4,3,2,1) Tests 2 passed | 2 expected fail | 118 skipped (122) $ vitest --run --project unit packages/lexical-y/src/__tests__/unit/Races.test.ts Tests 118 passed | 4 expected fail (122) $ pnpm run ci-check Found 0 errors ```
There was a problem hiding this comment.
Thanks for the detailed write-up, the reproductions in both client-id orders, and for pinning S9-12b and the #9589 tie as test.fails with tracking issues. One regression needs a fix before this lands.
opensHole can take a format change's real end for a hole (packages/lexical-y/src/internal/Cursor.ts:219-247, used by restatedEnd at :722-726)
opensHole walks the candidate end's client's structs in clock order and returns true when that client's next item of the key carries the run's value after one block of text. Nothing checks that this text and that item belong to the null's own write, although the comment at :214-217 says the scan covers that write's structs only. When the candidate is the end of a format change, and its client later typed plain text and then applied the same format elsewhere (bold a word, type, bold another word), the scan runs across those writes and returns true: restatedEnd rejects the real end and the restore gets no null of its own.
That switches rule 5 off for its original case, a run another client opened, which main handles (the S7-24 cases in regressions/Formats.test.ts). AGENTS.md's Backwards Compatibility section asks to keep current behavior intact, so this case should keep working. This test, added inside the ids loop of OwnRunRestoreEnd.test.ts, passes on main and fails on this branch in both orders, with the bold running over the rest of the document on every peer:
// B bolds "Kilo", types plain text, bolds "cc", then undoes all three,
// while A types plain "X" inside "Kilo".
test(`a restore in another client's run keeps its end after that client wrote the value again (ids ${ids})`, async () => {
const $bold = (from: number, to: number) => () => {
const selection = $selectIn([0], from, to);
selection.format = (selection.anchor.getNode() as TextNode).getFormat();
selection.formatText('bold');
$setSelection(null);
};
const t = await setup(
() => $getRoot().append($p('aa Kilo bb cc dd'), $p('tail line')),
{ids, prepare},
);
t.b.update($bold(3, 7));
await t.sync();
t.b.update(() => $insertPlainAt(10, ' zz'));
await t.sync();
t.b.update($bold(14, 16));
await t.sync();
t.offline();
t.b.undo();
t.b.undo();
t.b.undo();
t.a.update(() => $insertPlainAt(5, 'X'));
await t.sync();
await expectConverged(t.peers);
// this branch: paragraph("aa KiX" "lo bb cc dd"{1}) / paragraph("tail line"{1})
expect(t.a.describe()).toBe(
'paragraph("aa KiX" "lo"{1} " bb cc dd") / paragraph("tail line")',
);
});The same happens without the undos when Kilo's bold opener and end are dropped the way your first case drops the comment's ( bb zz cc turns bold), and the same sequence also leaves this PR's own-run fix off (A bolds Kilo, types, bolds cc, then types inside Kilo).
One direction: in opensHole, accept the key's restore only when it sits right after the text the scan passed, in the document (walking left from it over format and deleted items reaches that text), so an item a later write put elsewhere no longer counts. Please add the case above, and the dropped-end variant, to OwnRunRestoreEnd.test.ts.
Already tracked, not blocking here: S9-12b and the ids [2, 1] tie with a collaborator's bold that starts at the run's end are pinned and tracked (#9589, #9575, #9561), with the Limits text coming in #9569.
| if (next.content.key === key) { | ||
| return phase > 0 && equalFormatValue(next.content.value, value); | ||
| } | ||
| if (phase === 1) { | ||
| phase = 2; | ||
| } | ||
| } else if (next.countable) { | ||
| if (phase === 2) { | ||
| return false; | ||
| } | ||
| phase = 1; |
There was a problem hiding this comment.
Must fix: nothing here checks that the text and the key's next item belong to the null's own write. For a format change's end whose client later typed plain text and then applied the same value elsewhere (bold a word, type, bold another word), this returns true, so restatedEnd rejects the real end and a collaborator's restore inside that run gets no null of its own. main writes one there (rule 5's original case), and on this branch the bold then runs over the rest of the document on every peer. Accepting the restore only where it sits right after the scanned text in the document (walking left over format and deleted items) fixes it; the reproducer is in the review comment.
There was a problem hiding this comment.
Fixed in 5a34d5c, as suggested: the key's item now counts as the hole's restore only where it sits right after the text the scan passed (walking left from it over format items and deleted items reaches that text, followsText). OwnRunRestoreEnd.test.ts has your case, the dropped-end variant and the own-run variant, each in both client id orders. All six fail on b8c4175 and pass now. The 64-comment-id case still passes.
There was a problem hiding this comment.
Update: superseded by ac2c143. A randomized run-sequence fuzz still found shapes the hole scan got wrong (187 of 1,000 seeds against 9 on main), so rule 5 is narrowed back to main's (runOpener and that opener's change end), with one change: a restore inside a run its own client opened is no longer left out. opensHole, followsText and runChain are removed. Your three cases still pass in both orders, the fuzz now fails exactly main's seeds (RunSequenceUndo.test.ts, seed 87 = #9599), and the two cases the removed machinery covered beyond main are pinned as expected failures (main fails them too).
…re for no hole's restore
## Description
Follow-up to the previous commits (a later write's hole is not a run end).
`opensHole` walked the candidate end's client's structs in clock order and
took the client's next item of the key that gives the run's value back
after one block of its text for the restore of a hole. Nothing checked
that the item sat after that text in the document. When the candidate was
the end of a format change, and its client later typed plain text and then
applied the same format elsewhere (bold "Kilo", type " zz", bold "cc"),
the scan crossed those writes and found the later opener: `restatedEnd`
rejected the real end, a collaborator's restore inside that run got no
null of its own, and the bold ran over the rest of the document on every
peer. That turned rule 5 off for its original case, a run another client
opened, which `main` handles (S7-24 in `regressions/Formats.test.ts`), and
for this PR's own-run case in the same sequence. Reported in review.
The key's item now counts as the hole's restore only where it sits right
after the text the scan passed: walking left from it over format items
and deleted items reaches that text (`followsText`).
## Test plan
`OwnRunRestoreEnd.test.ts`, both client id orders: the review's case (B
bolds "Kilo", types plain " zz", bolds "cc", then undoes all three while A
types plain "X" inside "Kilo"), the same without the undos with B's bold
items around "Kilo" dropped, and the own-run variant (A bolds "Kilo",
types, bolds "cc", then types inside "Kilo" while B's copy drops the bold
items around it).
### Before
```
$ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/OwnRunRestoreEnd.test.ts -t "wrote the value again"
Tests 6 failed | 16 skipped (22)
AssertionError: expected 'paragraph("aa KiX" "lo bb cc dd"{1}) …' to be 'paragraph("aa KiX" "lo"{1} " bb cc dd…' // Object.is equality
AssertionError: expected '' to be ' bb zz |cc*| dd|tail line' // Object.is equality
```
### After
```
$ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/OwnRunRestoreEnd.test.ts
Tests 19 passed | 3 expected fail (22)
$ pnpm exec vitest run --project unit packages/lexical-y packages/lexical-mark
Test Files 181 passed | 2 skipped (183)
Tests 8659 passed | 49 expected fail | 4 skipped (8712)
```
The review's 64-comment-id case still passes (16/16). `pnpm run ci-check`
passes.
…nd a value an undo ends past other writes' run ends ## Description Depends on #9562 (based on its branch): a restore that relies on a run end of its own client's run gets its own end there. The undo here keeps such restores (case 1), so without #9562 a collaborator's deletes that collapse that run end away let the restore spread the value (a comment on a word, text typed inside it and undone offline, the word and its neighbours deleted by a collaborator: the comment covered the rest of the document). Cases where a bold, a comment or another format value spread onto text nobody gave it, on every client and in a fresh read: 1. An undo of text typed inside a run of another value deleted the run ends that text was written with. F types plain " mid" into bold "one two": the write ends the bold before " mid" and gives it back after it. E types plain " plain" right after "one": that text lands after F's null and writes no item of its own (its value matches). F undoes (whether or not it has seen E's text): the undo deleted F's null and F's restore with " mid", and " plain" turned bold (or, with a comment, commented). The same with a paragraph inserted between the halves of a bold run split by Enter, and with bold text typed into plain text (text typed before it inside its run lost the bold). A delete of " mid" by hand keeps those run ends (`Cursor#collapseGap`); the undo did not. The undo now keeps them (`neutralRunEnds`, through the undo manager's delete filter): the format items of its step next to content of its own that it deletes (past format items and content written there since: content whose Yjs origins lead to an item of the step), where deleting them would not change the value of content that was there before the step, or that the undo brings back (that value is the one the item of the key before them gives, past the run ends a collaborator's write put around text it typed inside the step's content, deleted since: those give nothing). Where it keeps a restore (the item that gives a run's value back after the text), it also keeps the null its write put right after the end of that run, which the restore relies on (`Cursor#format`, rule 5). Items of a format change (next to content the step did not write) are deleted as before, so an undo of a format change still removes it. Then, as a delete does, the run of items without live content the undo leaves is collapsed (`collapseUndone`, through `Cursor#collapseDeleted`), so text of another value typed and undone again and again at one place leaves a bounded number of items. It deletes only this client's own items there (`Cursor#collapsible`): another write's run end can be all a collaborator's text relies on once that collaborator undoes its own items. That collapse starts from the formats the editor shows, as the cursor it uses walks them: a collaborator's format a composition holds back is not counted (counting it lost an offline peer's bold or comment when the undo came while composing). 2. An insert's run ends inside another write's run restate that run's value: F's restore after " mid" gives the rest of A's bold back. A bolds "one two"; F types " mid" inside and deletes it (on main), or undoes it (with 1.); A undoes the bold: " two" stayed bold. After an undo or redo, where the restore of an insert that is gone gives back the value the content before that insert's null had, and the undo or redo changed that value, an item of this client's own now gives the content after it the new value (`restateRunEnds`), placed after what was written next to the restore since (text a collaborator typed right after it relies on its value and keeps it). Such a restore (`boundsOwnInsert`) is one of two items one insert wrote around text of its own, all of it deleted since, inserted before content that was there already, and giving back the value the key had where the insert went. A format change's closer ends its own value, and the items of a write that formats its own content (a document written in one go) open values of their own: both are left alone. The item is recorded on the step the undo or redo pushed, and the next redo or undo of that step deletes it before restating again (`unrestate`). An undo or redo and its follow-up writes are one operation: the steps it took and pushed are taken from the undo manager's events, by listeners the binding registers first (so a listener of the app that clears the history or throws does not skip the follow-up writes, and its error is thrown after them), what `restateRunEnds` writes is recorded on its step before it is written, and `undo()`/`redo()` asked for during the operation (from a listener) does nothing. `regressions/Formats.test.ts` L1B-1 expected C's text, typed over J's plain text after a commented word, to take the comments when J's typing was undone at the same time: that was case 1 (J's undo deleted the null C's text relied on). C's text now stays plain. ## Status against the randomized sessions that found these Sessions that change the format of a word no op touched, on this change and not on main with the same ops: - Fixed (pass now): L1B fmt 206163 (an underline undo took a document's own underline off a later word), L1B mark 324160 (gained bold), L1B mark 336261 (gained a link), L1B undoheavy 413112 (lost bold), N brand 1200036 and 1200107 (gained bold). - Still failing: L1B fmt 226508 (a word typed inside an italic run, after the restore of an insert that is gone, loses the italic when the run is undone: see the known regression above), L1B mix 143920 (an undo of an underline gives six words on another line the underline; under investigation). ## Changeset `.changeset/lexical-y-undo-run-ends.md` (`[fix]`). ## Test plan New `packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts` (all six client id orders where the order matters): bold and comment, a paragraph in a split bold run, bold text typed into plain text, the undoing client having seen the collaborator's text, a redo, bounds on the format items left by typing and undoing at one place (one value, and a new style each time), controls (no undo, a delete by hand), undo-redo-undo chains of the bold after a word typed inside it was undone or deleted, a third peer's text inside the collaborator's, a format change's closer next to undone text, the null a kept restore relies on surviving a collaborator's delete, `boundsOwnInsert` on an insert's run ends, a format change's and a document's written in one go, an undo of an underline next to a document's own run ends (its later underline stays), a comment's ends a collaborator's deletes drop (with #9562), an undo while a composition holds a collaborator's bold or comment, the restore an undo keeps when a collaborator's undone text inside left run ends, a collaborator's format over text an undo removes, text another peer typed right after a restore or before the run's next word, and listeners that undo again, clear the history or throw (of the undo manager and of the follow-up write). The comment-ends, composition and kept-restore cases pass on main and guard against regressions of this change. Known regression vs main on one shape (a randomized session): a word a collaborator typed inside a run, taking its value, after the restore of an insert that is gone, loses the value when the run's format is undone; on main it keeps it there because that restore keeps giving it. Without such a restore in front of it the word loses the value on main too (A italics "one two three", B types italic "X " inside, A undoes: "X " is plain on main), which is what this change gives. ### Before ``` $ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts Tests 59 failed | 52 passed (111) × bold: text typed after its null stays plain (6 of 6 id orders) × comment: text typed after its null stays uncommented (6 of 6) × a paragraph inserted into a bold run split by Enter (6 of 6) × bold text typed into plain text: text typed before it inside its run stays bold (6 of 6) × F has seen E's text when it undoes; then A undoes the bold (6 of 6) × text a third peer typed inside the collaborator's text stays plain too (2 of 2) × F types " mid" into A's bold and deletes it; A undoes the bold, redoes it, undoes it (6 of 6) × a redo puts the text back with its own value × the end a kept restore relies on stays ended when a collaborator deletes the run (6 of 6) × only an insert's own run ends restate a value (not a format change's) × the format items of a write that formats its own content do not restate a value (S9-9) × text another peer typed in the run's value after " mid" / before " two" keeps it when the run is undone (12) ✓ F types " mid" into A's bold and undos it; A undoes the bold, redoes it, undoes it (6 of 6) ✓ controls, bounds and stack listener cases AssertionError: expected 'one plain two*' to be 'one*| plain| two*' // Object.is equality AssertionError: expected 'one plain two[c1]' to be 'one[c1]| plain| two[c1]' // Object.is equality AssertionError: expected 'one* / plain* / two*' to be 'one* / plain / two*' // Object.is equality AssertionError: expected 'one B two' to be 'one| B*| two' // Object.is equality AssertionError: expected 'one two*' to be 'one two' // Object.is equality ``` ### After ``` $ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts Test Files 1 passed (1) Tests 111 passed (111) ``` The `@lexical/y` and `@lexical/mark` unit suites pass (on #9562), the randomized ones included (`Fuzz`, `FollowUpFuzz`, `ConcurrentShapesFuzz`, `NestedFuzz`, `SlotsFuzz`): ``` $ pnpm exec vitest run --project unit packages/lexical-y packages/lexical-mark Test Files 182 passed | 2 skipped (184) Tests 8755 passed | 46 expected fail | 4 skipped (8805) ``` `pnpm run ci-check` passes.
…ue an undo ends past seen text still there, and collapse an undo's runs in one pass
## Description
Follow-up to the run ends an undo keeps (case 1) and the values an undo
ends past other writes' run ends (case 2) in this PR.
1. Run ends an undo kept gave text a value it did not have before.
A gives "one two" a value (bold, italic, underline or a comment); F
types plain " mid" after "one"; both sync. Then F undoes its typing and
A undoes the value: F's undo kept its null and the restore after " mid",
and the restore gave " two" the value for everyone (nothing kept it
before this PR). The same with a block moved into a collaborator's
underlined lines: the undo of the move kept an opener of the moved
copy's underline (it gave the value the collaborator's run gave), and
the collaborator's undo of the underline left it without an end, so the
underline ran on over the lines after it.
- An undo no longer keeps a run end that gives the value the content
there has without it (from the value before the stretch and the items
kept before it): it changes nothing, and nothing relies on it
(`neutralRunEnds`). Leaving one out changes no value.
- Where the two undos are concurrent (A's undo has not seen F's), the
kept restore still gives " two" the value: a known limit of this PR,
#9593 (`main` passes that case, since its undo deletes the run ends
with the text, which loses the value of text a collaborator typed
next to it, the case this PR fixes). Settling it needs a write once a
client has seen both undos, which the binding may not make while it
applies a remote update, nor on a later change of its own (it would
change text that change did not touch). `UndoRunEnds.test.ts` pins
it as an expected failure, in two client id orders.
- No run end is kept inside one non-mergeable text node (a token,
segmented or code highlight node, whose segment key goes on across it
once the undo is done: `insideSegments`), and no node's type or
segment key is kept: content there is that node's, which no one
else's text joins, and keeping an end there split the node, whose
mode or state then covered only part of it (an agent's format of one
half took characters of an untouched word; a suggestion state was
lost from part of a code word; a link ran on over untouched words).
A style is a run like a format and is kept.
2. Where Yjs skips a step a collaborator emptied and undoes the next one
in the same call, the kept run ends were computed for the skipped step
only, and the undo deleted the real step's run ends: they are now
computed for the step whose items the undo deletes (`followStep`).
3. Type, delete, undo, undo. `undoing` held the items of the step as
recorded, but the undo manager deletes the live copies an undo of a
delete made of them (`followRedone`), so `neutralRunEnds` saw no
content of the step next to its run ends and deleted them: text a
collaborator typed after the null turned bold. The undo's items now
include those copies, found the way the undo manager finds them.
4. An undo now ends its value also past the restore of text it has seen
that is still there: A bolds "one two", F types plain " mid" inside, A
(having seen it) undoes the bold, and " two" stayed bold. The run ends
of an insert now restate a value whether its text is still there or
gone (`boundsOwnInsert`). Text typed inside the range that the undo has
not seen still keeps the old value after it, as y.md's Limits say (the
sentence now says that exactly).
5. `collapseUndone` walked back to the start of a deleted item's run and
rescanned the run for each deleted item: quadratic in a long run (the
undo of a pasted formatted text, about 1 s for 4,800 words). It now
walks each container once, left to right. The documents it leaves are
byte-identical to the old walk's on every undo of `UndoRunEnds`,
`Fuzz`, `FollowUpFuzz`, `ConcurrentShapesFuzz`, `NestedFuzz`,
`SlotsFuzz` and `UndoSelection` (the encoded state after each collapse,
compared run by run).
S9-12b in `OwnRunRestoreEnd.test.ts` (#9562 pins it as an expected
failure: a comment's end that a collaborator's collapse drops, after its
author's two writes inside it are undone) passes with this PR's undo,
which keeps the run ends of the text it removes: it is a plain test here.
Docs: `undo()` / `redo()` (comment and y.md) say they return `false`,
doing nothing, when called during another undo or redo; y.md's Undo limit
names the unseen case only, and says that a word a collaborator typed
inside the range with its value, right after text someone typed inside it
and removed again, loses the value with the range when it is undone, as
text typed anywhere else inside the range does (it kept it before, through
the restore of the removed text). The changeset names the #9593 limit.
## Test plan
New cases in `packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts`
("run ends an undo kept, after changes made since"): concurrent undos of
the value and of the typing for bold, italic, underline and a comment, each
in six client id orders, then a redo of the value; the same with text a
collaborator typed after the null (stays plain, and the redo gives " two"
its bold back); type, delete, undo, undo; the undo of a block moved into a
collaborator's underline during its undo; the #9593 case (concurrent
undos, pinned as an expected failure); after those concurrent undos, a
redo of the bold and then of the typing gives " mid" back plain (six
orders; it passes on the first commit, and an earlier version of this
follow-up failed it); an undo that skips a step a
collaborator emptied; an undo of text put inside a segmented or
token text node, commented or not, leaves the node whole; a style, as
bold; an undo ending its value past
seen text still there, and its unseen counterpart as a control of the
documented limit; the undo of a long formatted paste counting item link
reads (n, 2n, 4n). The expected strings were checked against main where
main has no case 1 (the concurrent undos with redo, the moved block, the
control and the walk count pass on main).
### Before
```
$ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts
Tests 56 failed | 123 passed | 2 expected fail (181)
AssertionError: expected ' two' to be '' // Object.is equality
AssertionError: expected 'one plain two*' to be 'one plain two' // Object.is equality
AssertionError: expected 'one plain two*' to be 'one*| plain| two*' // Object.is equality
AssertionError: expected 'xx_| yy / one / two_ / three' to be 'xx_| yy / one / two / three' // Object.is equality
AssertionError: expected 'one mid two*' to be 'one mid two' // Object.is equality
AssertionError: expected 3.9225617279005305 to be less than 2.5
AssertionError: expected false to be true // Object.is equality
```
(The undos of the value and of the typing, the skipped step, the text
node modes, the style, the collaborator's text, type delete undo undo, the
moved block, the seen restore, the walk count, and the `boundsOwnInsert`
assertion on an insert whose text is there.)
### After
```
$ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts
Tests 179 passed | 2 expected fail (181)
$ pnpm exec vitest run --project unit packages/lexical-y packages/lexical-mark
Test Files 182 passed | 2 skipped (184)
Tests 8840 passed | 49 expected fail | 4 skipped (8893)
```
The randomized sessions that found the moved-block case (L1B mix 143920),
the text node mode case (L1B catalog tab-heavy 3500310), the link case
(L1B catalog undo-heavy 3301123) and the code word state case (L2 catalog
4000092) pass, alone and merged with the other open changes; they failed
on this PR's previous head and pass on #9562. The randomized sessions
that found a write while applying a remote update and lost
tokens in an IME session in an earlier version of this follow-up (L1B
tab-heavy 3500073, 3500177, 3500479; mix 3100412, 3100480, 3100495,
3101172, 3101324; fmt 3900538, 3900574, 3900645, 3900699, 3900792,
3901108, 3901847; undo-heavy 3300430, 3300727; L1C imestrict 3600025,
3600062) show no flag that main does not show.
`pnpm run ci-check` passes.
… too ## Description The 4 ownership scan cells marked for #9578 (A writes a token or segmented node and text right before B's first one; B undoes all three writes) also pass with #9561: on main with #9562 and #9561 applied they report as passing unexpectedly, and with #9562 alone they still fail. The comment over their `test.fails` now names both, so the marks are dropped by whichever of the two lands first. ## Test plan ``` pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/matrix/OwnershipScans.test.ts ``` ### Before Comment change only. On main `cad9015`: ``` Tests 220 passed | 4 expected fail (224) ``` ### After On main `cad9015`, unchanged: ``` Tests 220 passed | 4 expected fail (224) ``` On main with #9562 `5a34d5c` and #9561 `4724c9d`, the 4 marked cells pass: ``` Tests 4 failed | 220 passed (224) (the 4 are "Expect test to fail") ```
…s included, and drop the hole scan
## Description
Follow-up to the previous commits. They generalized rule 5 (a restore's
own null after the end of the run it relies on) beyond main's: the end of
any write in the run's chain counted (`runChain`), and a scan of the
writing client's later structs (`opensHole`, `followsText`) told a hole's
null from a real end. Each fix to that scan opened another shape: a
randomized two-client run-sequence fuzz (a client formats a run, types
plain text, formats a second run or its own just-typed word, then undoes
one to three of those writes while a collaborator types plain text inside
the first run) failed 187 of 1,000 seeds on the previous head, against 9
on `main`, and a word outside both runs took the format.
Rule 5 is now `main`'s with one change, the case this PR is for (S9-5): a
restore inside a run its own client opened is no longer left out. The run
is the one `runOpener` finds and its end is that opener's change end, as
on `main`; `runChain`, `opensHole` and `followsText` are gone. One own
null per end per write (`endedInThisWrite`) stays.
What this gives up from the previous commits, now pinned as expected
failures and also failing on `main`: a comment written with uncommented
text inside it, whose run ends with its restore's end rather than its
opener's ("written with a hole in it"), and the hole case with 64 comment
ids on the run (the one-id cases pass). S9-12b and the #9589 tie stay
pinned as before.
## Test plan
`RunSequenceUndo.test.ts` (new): the run-sequence fuzz, 200 seeds; only
seed 87 fails, which also fails on `main` (#9599). It covers the soak's
minimal shapes (seed 4: the second run on the writer's own just-typed
word, one undo; seed 8: three undos, the last after the collaborator's
text arrived). `OwnRunRestoreEnd.test.ts`: the review's three cases pass
in both orders; the two cases above are pinned.
### Before
```
$ RS_N=1000 pnpm exec vitest run --project unit <run-sequence fuzz, 1,000 seeds>
HITS 187/1000
```
### After
```
$ RS_N=1000 pnpm exec vitest run --project unit <run-sequence fuzz, 1,000 seeds>
HITS 9/1000 (main: 9/1000, the same seeds)
$ pnpm exec vitest run --project unit packages/lexical-y packages/lexical-mark
Test Files 182 passed | 2 skipped (184)
Tests 8656 passed | 53 expected fail | 4 skipped (8713)
```
The catalog's ownership cells (`OwnershipScans`, #9588) all pass.
`pnpm run ci-check` passes.
…nd a value an undo ends past other writes' run ends ## Description Depends on #9562 (based on its branch): a restore that relies on a run end of its own client's run gets its own end there. The undo here keeps such restores (case 1), so without #9562 a collaborator's deletes that collapse that run end away let the restore spread the value (a comment on a word, text typed inside it and undone offline, the word and its neighbours deleted by a collaborator: the comment covered the rest of the document). Cases where a bold, a comment or another format value spread onto text nobody gave it, on every client and in a fresh read: 1. An undo of text typed inside a run of another value deleted the run ends that text was written with. F types plain " mid" into bold "one two": the write ends the bold before " mid" and gives it back after it. E types plain " plain" right after "one": that text lands after F's null and writes no item of its own (its value matches). F undoes (whether or not it has seen E's text): the undo deleted F's null and F's restore with " mid", and " plain" turned bold (or, with a comment, commented). The same with a paragraph inserted between the halves of a bold run split by Enter, and with bold text typed into plain text (text typed before it inside its run lost the bold). A delete of " mid" by hand keeps those run ends (`Cursor#collapseGap`); the undo did not. The undo now keeps them (`neutralRunEnds`, through the undo manager's delete filter): the format items of its step next to content of its own that it deletes (past format items and content written there since: content whose Yjs origins lead to an item of the step), where deleting them would not change the value of content that was there before the step, or that the undo brings back (that value is the one the item of the key before them gives, past the run ends a collaborator's write put around text it typed inside the step's content, deleted since: those give nothing). Where it keeps a restore (the item that gives a run's value back after the text), it also keeps the null its write put right after the end of that run, which the restore relies on (`Cursor#format`, rule 5). Items of a format change (next to content the step did not write) are deleted as before, so an undo of a format change still removes it. Then, as a delete does, the run of items without live content the undo leaves is collapsed (`collapseUndone`, through `Cursor#collapseDeleted`), so text of another value typed and undone again and again at one place leaves a bounded number of items. It deletes only this client's own items there (`Cursor#collapsible`): another write's run end can be all a collaborator's text relies on once that collaborator undoes its own items. That collapse starts from the formats the editor shows, as the cursor it uses walks them: a collaborator's format a composition holds back is not counted (counting it lost an offline peer's bold or comment when the undo came while composing). 2. An insert's run ends inside another write's run restate that run's value: F's restore after " mid" gives the rest of A's bold back. A bolds "one two"; F types " mid" inside and deletes it (on main), or undoes it (with 1.); A undoes the bold: " two" stayed bold. After an undo or redo, where the restore of an insert that is gone gives back the value the content before that insert's null had, and the undo or redo changed that value, an item of this client's own now gives the content after it the new value (`restateRunEnds`), placed after what was written next to the restore since (text a collaborator typed right after it relies on its value and keeps it). Such a restore (`boundsOwnInsert`) is one of two items one insert wrote around text of its own, all of it deleted since, inserted before content that was there already, and giving back the value the key had where the insert went. A format change's closer ends its own value, and the items of a write that formats its own content (a document written in one go) open values of their own: both are left alone. The item is recorded on the step the undo or redo pushed, and the next redo or undo of that step deletes it before restating again (`unrestate`). An undo or redo and its follow-up writes are one operation: the steps it took and pushed are taken from the undo manager's events, by listeners the binding registers first (so a listener of the app that clears the history or throws does not skip the follow-up writes, and its error is thrown after them), what `restateRunEnds` writes is recorded on its step before it is written, and `undo()`/`redo()` asked for during the operation (from a listener) does nothing. `regressions/Formats.test.ts` L1B-1 expected C's text, typed over J's plain text after a commented word, to take the comments when J's typing was undone at the same time: that was case 1 (J's undo deleted the null C's text relied on). C's text now stays plain. ## Status against the randomized sessions that found these Sessions that change the format of a word no op touched, on this change and not on main with the same ops: - Fixed (pass now): L1B fmt 206163 (an underline undo took a document's own underline off a later word), L1B mark 324160 (gained bold), L1B mark 336261 (gained a link), L1B undoheavy 413112 (lost bold), N brand 1200036 and 1200107 (gained bold). - Still failing: L1B fmt 226508 (a word typed inside an italic run, after the restore of an insert that is gone, loses the italic when the run is undone: see the known regression above), L1B mix 143920 (an undo of an underline gives six words on another line the underline; under investigation). ## Changeset `.changeset/lexical-y-undo-run-ends.md` (`[fix]`). ## Test plan New `packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts` (all six client id orders where the order matters): bold and comment, a paragraph in a split bold run, bold text typed into plain text, the undoing client having seen the collaborator's text, a redo, bounds on the format items left by typing and undoing at one place (one value, and a new style each time), controls (no undo, a delete by hand), undo-redo-undo chains of the bold after a word typed inside it was undone or deleted, a third peer's text inside the collaborator's, a format change's closer next to undone text, the null a kept restore relies on surviving a collaborator's delete, `boundsOwnInsert` on an insert's run ends, a format change's and a document's written in one go, an undo of an underline next to a document's own run ends (its later underline stays), a comment's ends a collaborator's deletes drop (with #9562), an undo while a composition holds a collaborator's bold or comment, the restore an undo keeps when a collaborator's undone text inside left run ends, a collaborator's format over text an undo removes, text another peer typed right after a restore or before the run's next word, and listeners that undo again, clear the history or throw (of the undo manager and of the follow-up write). The comment-ends, composition and kept-restore cases pass on main and guard against regressions of this change. Known regression vs main on one shape (a randomized session): a word a collaborator typed inside a run, taking its value, after the restore of an insert that is gone, loses the value when the run's format is undone; on main it keeps it there because that restore keeps giving it. Without such a restore in front of it the word loses the value on main too (A italics "one two three", B types italic "X " inside, A undoes: "X " is plain on main), which is what this change gives. ### Before ``` $ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts Tests 59 failed | 52 passed (111) × bold: text typed after its null stays plain (6 of 6 id orders) × comment: text typed after its null stays uncommented (6 of 6) × a paragraph inserted into a bold run split by Enter (6 of 6) × bold text typed into plain text: text typed before it inside its run stays bold (6 of 6) × F has seen E's text when it undoes; then A undoes the bold (6 of 6) × text a third peer typed inside the collaborator's text stays plain too (2 of 2) × F types " mid" into A's bold and deletes it; A undoes the bold, redoes it, undoes it (6 of 6) × a redo puts the text back with its own value × the end a kept restore relies on stays ended when a collaborator deletes the run (6 of 6) × only an insert's own run ends restate a value (not a format change's) × the format items of a write that formats its own content do not restate a value (S9-9) × text another peer typed in the run's value after " mid" / before " two" keeps it when the run is undone (12) ✓ F types " mid" into A's bold and undos it; A undoes the bold, redoes it, undoes it (6 of 6) ✓ controls, bounds and stack listener cases AssertionError: expected 'one plain two*' to be 'one*| plain| two*' // Object.is equality AssertionError: expected 'one plain two[c1]' to be 'one[c1]| plain| two[c1]' // Object.is equality AssertionError: expected 'one* / plain* / two*' to be 'one* / plain / two*' // Object.is equality AssertionError: expected 'one B two' to be 'one| B*| two' // Object.is equality AssertionError: expected 'one two*' to be 'one two' // Object.is equality ``` ### After ``` $ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts Test Files 1 passed (1) Tests 111 passed (111) ``` The `@lexical/y` and `@lexical/mark` unit suites pass (on #9562), the randomized ones included (`Fuzz`, `FollowUpFuzz`, `ConcurrentShapesFuzz`, `NestedFuzz`, `SlotsFuzz`): ``` $ pnpm exec vitest run --project unit packages/lexical-y packages/lexical-mark Test Files 182 passed | 2 skipped (184) Tests 8755 passed | 46 expected fail | 4 skipped (8805) ``` `pnpm run ci-check` passes.
…ue an undo ends past seen text still there, and collapse an undo's runs in one pass
## Description
Follow-up to the run ends an undo keeps (case 1) and the values an undo
ends past other writes' run ends (case 2) in this PR.
1. Run ends an undo kept gave text a value it did not have before.
A gives "one two" a value (bold, italic, underline or a comment); F
types plain " mid" after "one"; both sync. Then F undoes its typing and
A undoes the value: F's undo kept its null and the restore after " mid",
and the restore gave " two" the value for everyone (nothing kept it
before this PR). The same with a block moved into a collaborator's
underlined lines: the undo of the move kept an opener of the moved
copy's underline (it gave the value the collaborator's run gave), and
the collaborator's undo of the underline left it without an end, so the
underline ran on over the lines after it.
- An undo no longer keeps a run end that gives the value the content
there has without it (from the value before the stretch and the items
kept before it): it changes nothing, and nothing relies on it
(`neutralRunEnds`). Leaving one out changes no value.
- Where the two undos are concurrent (A's undo has not seen F's), the
kept restore still gives " two" the value: a known limit of this PR,
#9593 (`main` passes that case, since its undo deletes the run ends
with the text, which loses the value of text a collaborator typed
next to it, the case this PR fixes). Settling it needs a write once a
client has seen both undos, which the binding may not make while it
applies a remote update, nor on a later change of its own (it would
change text that change did not touch). `UndoRunEnds.test.ts` pins
it as an expected failure, in two client id orders.
- No run end is kept inside one non-mergeable text node (a token,
segmented or code highlight node, whose segment key goes on across it
once the undo is done: `insideSegments`), and no node's type or
segment key is kept: content there is that node's, which no one
else's text joins, and keeping an end there split the node, whose
mode or state then covered only part of it (an agent's format of one
half took characters of an untouched word; a suggestion state was
lost from part of a code word; a link ran on over untouched words).
A style is a run like a format and is kept.
2. Where Yjs skips a step a collaborator emptied and undoes the next one
in the same call, the kept run ends were computed for the skipped step
only, and the undo deleted the real step's run ends: they are now
computed for the step whose items the undo deletes (`followStep`).
3. Type, delete, undo, undo. `undoing` held the items of the step as
recorded, but the undo manager deletes the live copies an undo of a
delete made of them (`followRedone`), so `neutralRunEnds` saw no
content of the step next to its run ends and deleted them: text a
collaborator typed after the null turned bold. The undo's items now
include those copies, found the way the undo manager finds them.
4. An undo now ends its value also past the restore of text it has seen
that is still there: A bolds "one two", F types plain " mid" inside, A
(having seen it) undoes the bold, and " two" stayed bold. The run ends
of an insert now restate a value whether its text is still there or
gone (`boundsOwnInsert`). Text typed inside the range that the undo has
not seen still keeps the old value after it, as y.md's Limits say (the
sentence now says that exactly).
5. An undo of one step that formats a run, types text and formats that
text (a format, plain " zz", a format of " zz") keeps the run ends of
its own typed text (1.), and the format change's closers after the
first run were then deleted: the walk that keeps a closer where the
content after it would otherwise take a value (`isKeptByUndo`) counted
only the step's deleted items as the step's, so the kept run ends of
" zz" made it delete them, and a collaborator's restore inside the
first run (plain "X" typed inside it, seen before the undo) lost its
end and gave its value to the untouched words after it. A run end the
undo keeps now counts as gone there too.
6. `collapseUndone` walked back to the start of a deleted item's run and
rescanned the run for each deleted item: quadratic in a long run (the
undo of a pasted formatted text, about 1 s for 4,800 words). It now
walks each container once, left to right. The documents it leaves are
byte-identical to the old walk's on every undo of `UndoRunEnds`,
`Fuzz`, `FollowUpFuzz`, `ConcurrentShapesFuzz`, `NestedFuzz`,
`SlotsFuzz` and `UndoSelection` (the encoded state after each collapse,
compared run by run).
S9-12b in `OwnRunRestoreEnd.test.ts` (#9562 pins it as an expected
failure: a comment's end that a collaborator's collapse drops, after its
author's two writes inside it are undone) passes with this PR's undo,
which keeps the run ends of the text it removes: it is a plain test here.
Docs: `undo()` / `redo()` (comment and y.md) say they return `false`,
doing nothing, when called during another undo or redo; y.md's Undo limit
names the unseen case only, and says that a word a collaborator typed
inside the range with its value, right after text someone typed inside it
and removed again, loses the value with the range when it is undone, as
text typed anywhere else inside the range does (it kept it before, through
the restore of the removed text). The changeset names the #9593 limit.
## Test plan
New cases in `packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts`
("run ends an undo kept, after changes made since"): concurrent undos of
the value and of the typing for bold, italic, underline and a comment, each
in six client id orders, then a redo of the value; the same with text a
collaborator typed after the null (stays plain, and the redo gives " two"
its bold back); type, delete, undo, undo; the undo of a block moved into a
collaborator's underline during its undo; the #9593 case (concurrent
undos, pinned as an expected failure); after those concurrent undos, a
redo of the bold and then of the typing gives " mid" back plain (six
orders; it passes on the first commit, and an earlier version of this
follow-up failed it); an undo that skips a step a
collaborator emptied; `RunSequenceUndo.test.ts` (from #9562) fails only
on seed 87, as on main (#9599); an undo of text put inside a segmented or
token text node, commented or not, leaves the node whole; a style, as
bold; an undo ending its value past
seen text still there, and its unseen counterpart as a control of the
documented limit; the undo of a long formatted paste counting item link
reads (n, 2n, 4n). The expected strings were checked against main where
main has no case 1 (the concurrent undos with redo, the moved block, the
control and the walk count pass on main).
### Before
```
$ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts
Tests 56 failed | 123 passed | 2 expected fail (181)
AssertionError: expected ' two' to be '' // Object.is equality
AssertionError: expected 'one plain two*' to be 'one plain two' // Object.is equality
AssertionError: expected 'one plain two*' to be 'one*| plain| two*' // Object.is equality
AssertionError: expected 'xx_| yy / one / two_ / three' to be 'xx_| yy / one / two / three' // Object.is equality
AssertionError: expected 'one mid two*' to be 'one mid two' // Object.is equality
AssertionError: expected 3.9225617279005305 to be less than 2.5
AssertionError: expected false to be true // Object.is equality
```
(The undos of the value and of the typing, the skipped step, the text
node modes, the style, the collaborator's text, type delete undo undo, the
moved block, the seen restore, the walk count, and the `boundsOwnInsert`
assertion on an insert whose text is there.)
### After
```
$ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts
Tests 179 passed | 2 expected fail (181)
$ pnpm exec vitest run --project unit packages/lexical-y packages/lexical-mark
Test Files 182 passed | 2 skipped (184)
Tests 8837 passed | 53 expected fail | 4 skipped (8894)
```
The randomized sessions that found the moved-block case (L1B mix 143920),
the text node mode case (L1B catalog tab-heavy 3500310), the link case
(L1B catalog undo-heavy 3301123) and the code word state case (L2 catalog
4000092) pass, alone and merged with the other open changes; they failed
on this PR's previous head and pass on #9562. The randomized sessions
that found a write while applying a remote update and lost
tokens in an IME session in an earlier version of this follow-up (L1B
tab-heavy 3500073, 3500177, 3500479; mix 3100412, 3100480, 3100495,
3101172, 3101324; fmt 3900538, 3900574, 3900645, 3900699, 3900792,
3901108, 3901847; undo-heavy 3300430, 3300727; L1C imestrict 3600025,
3600062) show no flag that main does not show.
`pnpm run ci-check` passes.
… a second shape of the #9589 tie ## Description Two review findings on the narrowed rule, both kept as tests: - **More than 64 structs in the comment change (not a regression).** An author's restore inside its own comment run gets its own end only when rule 5 finds where the comment change ended, and `changeEnd`'s scan of the author's structs stops at 64. With 1, 16 or 64 comment ids on the range the restore now ends after the run in both client-id orders; with 128 it gets no end and every comment spreads to the rest of the document, as on `main` (which fails all of these cells). The 128-id cells are `test.fails` naming #9603. - **A second shape of the #9589 tie (a regression against `main`, accepted with #9589).** The author types plain text inside its own bold run while a collaborator bolds from inside that run past its end: the author's own-run null after the run's end and the collaborator's opener there share their origins, so with the author's client id higher the collaborator's bold after the run's end is lost on every client (no text is lost, the clients converge, and the bold comes back if the author undoes). `main` keeps it, since it writes no own-run null. The ids `[2, 1]` cell is `test.fails` naming #9589; `[1, 2]` is a regular test. ## Test plan ### Before The new cells did not exist; on `main` (cad9015) the 16 comment-count cells all fail (comments spread), and on this branch before this commit the reviewers' repros failed at 128 ids and with ids `[2, 1]` as described. ### After ``` $ vitest run --project unit packages/lexical-y/src/__tests__/unit/OwnRunRestoreEnd.test.ts Test Files 1 passed (1) Tests 22 passed | 10 expected fail (32) ``` `prettier`, `eslint --no-cache` and `tsc -p tsconfig.test.json` are clean on the file.
…9562 only: drop the concurrent-undo limit, and extend the run-end format limit to a format applied from inside the run ## Description #9561 is not landing with this, so the Limits section describes main, with #9562 for the sentences that say so. - **Undo: both undo at once.** The sentence added for #9593 (a format staying on text after a collaborator's typing when both undo at once) describes a case that only #9561 produces; main gives `paragraph("one two")`. It is removed. The format-undo sentence that main has (the text after a collaborator's typing inside the range keeps the format, whether the undoing client saw that text or not) and its `Races.test.ts` cases are unchanged and still describe main. - **A collaborator's format after a run's end (#9589, #9603).** The sentence now also covers a format applied from inside the run past its end, and says that the format comes back when the edit inside the run is undone: with `Kilo` bold, one client types plain text inside `Kilo` (or unbolds `il`) while another bolds `lo bb`, and with some client id orders ` bb` stays plain. No text is lost. Checked with `aa Kilo bb cc dd`, A bolds `Kilo` and syncs, then concurrently A types plain `X` after `Ki` (or unbolds `il`) while B bolds the range named below; then A undoes. Ids `[1, 2]` and `[2, 1]` (A = first id): | B bolds | main `cad9015` | main + #9562 `ac2c143` | after A's undo | |---|---|---|---| | ` bb` (7-10), A types `X` | ` bb` bold | ` bb` plain with `[2, 1]` | ` bb` bold everywhere | | ` bb` (7-10), A unbolds `il` | ` bb` bold | ` bb` plain with `[2, 1]` | ` bb` bold everywhere | | `lo bb` (5-10), A types `X` | ` bb` bold | ` bb` plain with `[2, 1]` | ` bb` bold everywhere | | `ilo bb` (4-10), A types `X` | ` bb` bold | ` bb` plain with `[2, 1]` | ` bb` bold everywhere | | `o bb cc` (6-13), A types `X` | ` bb cc` bold | ` bb cc` plain with `[2, 1]` | ` bb cc` bold everywhere | ## Test plan ``` pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/Races.test.ts pnpm run ci-check ``` ### Before ``` $ vitest --run --project unit packages/lexical-y/src/__tests__/unit/Races.test.ts Tests 118 passed | 5 expected fail (123) ``` ### After ``` $ vitest --run --project unit packages/lexical-y/src/__tests__/unit/Races.test.ts Tests 118 passed | 5 expected fail (123) $ pnpm run ci-check Changeset check passed: 31 pending changeset(s) valid. Found 0 errors ```
There was a problem hiding this comment.
Thanks for the thorough write-up and for pinning every known limit with a tracking issue. The own-run end does fix the offline-return spread. One case needs attention before this can merge.
Undoing a single write inside your own comment can now spread the comment over the rest of the document (packages/lexical-y/src/internal/Cursor.ts:669, through collapseGap at line 990)
Because restatedEnd no longer skips runs the writer opened, the writer's restore puts a null right after its own comment's end. Suppose a collaborator who has already seen that write deletes past the end, and another comment's items fall in the gap that delete collapses. collapseGap keeps the new null, which is the key's last item, and deletes the comment's own end. Its rule for keeping an earlier same-valued item only applies when another client wrote that item. The author's concurrent undo then removes the null, so nothing ends the comment. main keeps the end here. With the harness from OwnRunRestoreEnd.test.ts, the case fails in both client-id orders:
// $p('aa Kilo bb cc dd'), $p('tail line')
t.a.update(() => void $addMarkID('c1', $selectIn([0], 3, 7))); // A comments "Kilo"
t.b.update(() => void $addMarkID('c2', $selectIn([0], 8, 10))); // "bb" has another comment
await t.sync();
t.a.update(() => $insertPlainAt(6, 'X')); // one write: "KilXo"
await t.sync();
t.offline();
t.a.undo();
t.b.update(() => void $selectIn([0], 7, 8).removeText()); // B deletes "o"
t.b.update(() => void $selectIn([0], 7, 8).removeText()); // then the space
await t.sync();
// main: "aa Kil[c1] bb[c2] cc dd" / "tail line"
// this branch: "aa Kil[c1] bb[c1,c2] cc[c1] dd[c1]" / "tail[c1] line[c1]"Deleting bb instead of the space gives the same result. So does deleting bb cc when the second comment is on cc. This is one write, so S9-12b and the Limits sentence proposed in #9569 (one client's two writes, or two clients' writes) do not cover it.
Two directions keep the end beside the new null. One is a collapse rule like 89e03fd's, which keeps each client's first same-valued item. The other is to also treat a same-client item from another write (for example, one with a non-adjacent clock) as protected at line 990. The PR description notes that 89e03fd brought back the offline spread in one session, so this is your call. If this case is accepted as a limit instead, please pin it beside S9-12 (OwnRunRestoreEnd.test.ts:254) with an issue and add it to the Limits text.
Optional, non-blocking:
- The comments at
Cursor.ts:539-542,637-646and1128-1131still say rule 5 applies to a run "another client's change opened".OwnRunRestoreEnd.test.ts:326-329still describes the removed hole scan. - S9-12b and the #9589 tie are documented as accepted. Their Limits text lives in #9569, so it would be good to land that together with this change.
- The PR description still describes
runChainand the original four-test plan. AGENTS.md asks the description, which becomes the commit message, to carry the real Before and After output of the final change.
| @@ -643,9 +667,6 @@ export class Cursor { | |||
| return null; | |||
| } | |||
| const opener = runOpener(doc, source); | |||
There was a problem hiding this comment.
With the own-client exemption gone, a restore inside the writer's own comment puts a null right after the comment's end. If a collaborator who saw that write deletes past the end with another comment's items in the gap, collapseGap keeps that null (the key's last item) and deletes the end: its same-value rule at line 990 protects only another client's item. The writer's concurrent undo then removes the null, and the comment runs on over every later word and line. main keeps the end. The review comment has a reproduction in both id orders (comment on Kilo and on bb; A types X inside Kilo; A undoes while B deletes o, then the space).
There was a problem hiding this comment.
Thanks: confirmed, and it is the third regression against main this extension of rule 5 has had. The PR is parked as a draft; the "Parked (10/30)" section at the top of the description covers this case (one write, a delete past the end collapsing a gap that holds another comment's items, collapseGap keeping only another client's same-valued item), the two earlier regressions, and what a redesign needs (a provenance marker for the end a restore relies on, or a different approach to S9-5). The cases, this one included, move to main as tests in #9605, with the ones main fails as expected failures.
| $insertPlainAt(6, 'X'); | ||
| }); | ||
| await t.sync(); | ||
| expect(words(t.a)).toBe('aa Kil[c1] X ome[c1] Y ter[c1] bb cc dd'); |
There was a problem hiding this comment.
One null per end is enough here only while the run's own items are the gap's last units. With a comment on the next word, a single write is enough to lose the end: A undoes X while B deletes o and the space, and the comment spreads. Could that case sit beside this one, either as a test or as a pinned test.fails with an issue?
|
|
||
| /** | ||
| * For a restore of `key` to `value` written at the cursor: where the run it | ||
| * restates was opened by another client's change (see runOpener; the item |
There was a problem hiding this comment.
Nit: this comment, the rule 5 text (1128-1131) and insertNegatedFormats' comment (539-542) still say the run was opened by another client's change. Since this PR, rule 5 also applies to the writer's own runs.
…out #9562: the run-on case needs two collaborators' writes, and the lost format a run another client formatted ## Description #9562 is parked, so the Limits text must describe `main` without it. Two sentences held only with #9562, which extended rule 5 to runs the writing client opened itself: - A comment or format running on past its end when writes inside it are undone while another client deletes across its end: on `main` one client's two writes inside a comment (S9-12b, the author's own; or a collaborator's two writes) end the comment where it ended, and only two collaborators' writes run on (#9575, `Races.test.ts`). The sentence now names two collaborators' writes. - A format a collaborator applies at a run's end lost while another client edits inside the run (#9589): on `main` that happens where the run was formatted by a client other than the one editing inside it (bolded by one client, edited inside by a second, ` bb` bolded by the first or a third), in some client id orders, as with #9562; where the editing client formatted the run itself, `main` keeps ` bb` bold. The sentence now says so. ## Test plan Checked on `main` (2fe8056) with the package's harness: two writes of one collaborator inside a comment, undone while another client deletes `ter` and ` bb`, leave `aa |Kilome[c1]| cc dd` / `tail line` in all six client id orders, and #9575's `Races.test.ts` cells keep the documented run-on (both orders). With `Kilo` bolded by B and A typing `X` inside it (or unbolding `il`) while B or C bolds ` bb` (or `lo bb`), ` bb` ends plain for ids `[2,1,3]`, `[3,1,2]` and `[3,2,1]` (B) or `[2,3,1]`, `[3,1,2]`, `[3,2,1]` (C), and is bold again after A's undo; with A bolding `Kilo` itself, ` bb` stays bold in every order. ``` $ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/Races.test.ts Tests 118 passed | 5 expected fail (123) ``` `pnpm run prettier` passes.
… from #9562, with the ones main fails as expected failures ## Description Tests only. #9562 (a restore inside a run its own client opened gets a null of its own after the run's end) is parked; its cases move to main here, so they stay whatever lands later. - `OwnRunRestoreEnd.test.ts`: the cases where the writer opened the run itself (a comment's restore whose end a collaborator's delete drops; a comment written with a hole in it; the same after the writer wrote the value again; 64 comment ids on the run; 1, 16, 64 and 128 comment ids dropped with the word) fail on main and are `test.fails`, tracked in #9609. The cases #9562 regressed and main handles are plain tests: S9-12 and S9-12b, the review's cases of a restore in another client's run after that client wrote the value again, a later write's hole (comment and bold), and the #9589 tie where the author opened the run (a collaborator's bold starting at, or running past, the author's bold run's end stays). - `OwnCommentGapCollapse.test.ts`: one write inside a comment its author opened, undone while a collaborator who has seen it deletes past the comment's end across another comment; the comment ends where it ended. Main passes; #9562 fails it in both id orders. - `RunSequenceUndo.test.ts`: the 200-seed two-client run-sequence fuzz; only seed 87 fails, as on main (#9599). ## Test plan ``` $ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/OwnRunRestoreEnd.test.ts packages/lexical-y/src/__tests__/unit/OwnCommentGapCollapse.test.ts packages/lexical-y/src/__tests__/unit/RunSequenceUndo.test.ts packages/lexical-y/src/__tests__/unit/UndoRunEnds.test.ts packages/lexical-y/src/__tests__/unit/RunSequenceCases.test.ts Tests 124 passed | 97 expected fail (221) ``` The same on main at 2fe8056 (the three new files: 21 passed, 16 expected fail). On #9562's branch `OwnCommentGapCollapse` fails 4 of 4: `expected 'aa Kil[c1] bb[c1,c2] cc[c1] dd[c1]' not to match /(bb|cc|dd)\[c1/`. `pnpm run ci-check` passes.
…lapse case
## Description
`OwnCommentGapCollapse.test.ts` checked only that `bb`, `cc` and `dd` carry
no `c1` and that the second line is untouched: losing the comment on `Kil`
(or `c2` on `bb`) passed too, so a mutation that skips A's comment of
`Kilo` passed all four cells. It now asserts the whole first line as main
gives it: `aa Kil[c1] bb[c2] cc dd` when B deletes `o` and the space,
`aa Kil[c1] cc dd` when it deletes `o` and ` bb`. Reported in review.
## Test plan
```
$ pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/OwnCommentGapCollapse.test.ts
Tests 4 passed (4)
```
With A's comment of `Kilo` removed from the test, all four cells fail now:
```
AssertionError: expected 'aa Kil bb[c2] cc dd' to be 'aa Kil[c1] bb[c2] cc dd' // Object.is equality
AssertionError: expected 'aa Kil cc dd' to be 'aa Kil[c1] cc dd' // Object.is equality
```
On #9562's branch all four still fail. `pnpm run ci-check` passes.
Parked (10/30)
This PR is parked as a draft. The branch and its tests stay as they are. Its goal, S9-5, is tracked in #9609; the tests move to main in #9605, with the cases main fails as expected failures.
Why. Rule 5 (a restore writes a null of its own right after the end of the run it relies on) was extended to runs the writing client opened itself, to fix S9-5: a comment's restore after uncommented text its author typed inside it, whose end a long offline peer's delete drops, ran on over the rest of the document. Each attempt to make that extension safe regressed a case main handles, each found in review, each fix reopening an earlier case:
opensHole) took a format change's real end for a hole when its client later typed plain text and applied the same format elsewhere (bold a word, type, bold another word), so a collaborator's restore inside that run got no null and the bold ran to the end of the document (S7-24's original case).collapseGapkeeps the new null (the key's last item) and deletes the comment's own end, and the author's concurrent undo removes the null: the comment covers the rest of the document. Main keeps the end there.The collapse problem.
Cursor#collapseGapkeeps, besides each key's last item, the last earlier item with the same value only where another client wrote it (otherClient, aroundCursor.ts:990). An own-run null from a different write of the same client is the same client, so it wins over the run's own end, which is deleted. Protecting a same-client item from another write there (or keeping each client's first same-valued item, as 89e03fd did) brought back the offline spread in one session.What a redesign needs. The binding cannot tell, from the document alone, which write an item of the same client belongs to or which end a restore relies on: every rule here inferred it from clocks and positions, and each inference had a counterexample. Either a provenance marker (the restore records the end it relies on, so a collapse can keep exactly that end), or a different approach to S9-5 that does not add a second null of the same client next to the run's end (for example, the offline peer's delete keeping the run ends a later restore relies on).
Current state (narrowed after review,
ac2c143a+6ff45373)The change is now
main's rule 5 with two differences, inCursor.tsonly:restatedEndno longer skips runs the writing client opened itself (the offline-return spread described below: a restore inside one's own comment run gets its own end after the run).endedInThisWrite).The hole-scanning machinery added in
83df52d3,b8c41752and5a34d5cc(runChain,opensHole,followsText) is removed: a randomized run-sequence fuzz (one client writes a format, plain text, then the same format; a collaborator types plain text inside the first run while the writer undoes) failed 187 of 1,000 seeds with it againstmain's 9. The narrowed rule fails exactlymain's 9 seeds;RunSequenceUndo.test.ts(200 seeds) expects exactlymain's failing seed. The bullets for those three commits further down describe what they did then; their scan is gone.Known limits, each pinned in
OwnRunRestoreEnd.test.ts:changeEndfinds where the comment change ended, and its scan of the author's structs stops at 64: with 1, 16 or 64 comment ids the restore now ends after the run in both client-id orders; with 128 it gets none and every comment spreads to the rest of the document, as onmain(which fails all of these cells).test.failsnaming Bug: [lexical-y] A restore inside a run whose format change has more than 64 items gets no end of its own, so the format spreads to the rest of the document #9603.main, accepted). With the author's client id higher, a collaborator's format that starts at the end of the author's own run, or runs from inside it past its end while the author types plain text inside it, is lost after the run's end on every client: the author's own-run null and the collaborator's opener share their origins. No text is lost, the clients converge, and the format comes back if the author undoes.mainkeeps the collaborator's format there because it writes no own-run null; it is accepted because without that null the author's offline restore lets the comment spread over every later word, on every client.test.failsfor ids[2, 1]naming Bug: [lexical-y] A collaborator's format that starts where your own format run ends is lost when you change part of that run at the same time #9589, regular tests for[1, 2]; [lexical-website][lexical-y] Documentation Update: Say which list conversions are moves, and add an undo race and a join race to the @lexical/y limits #9569 documents both shapes.Reviewed on
6ff45373by gpt-5.6-sol (clean, with #9603's bound and the #9589 tie accepted; independent matrices over one/two inserts, same/other-client ownership, both id orders and 1/16/64/128 ids). An independent non-GPT adversarial review found the second #9589 shape, now pinned.Description
When a client inserts text without a value inside a run of that value (uncommented text typed inside a comment, plain text inside bold), it restores the value after its text and relies on the item that ends the run. Rule 5 of
Cursor#formatmakes a restore whose run another client's change opened write a null of its own right after that change's end, because that end can go while the restore stays: the change's undo deletes it, and a concurrent delete may drop it (a delete keeps a bounded number of the format items it leaves in a run of deleted content).Two cases were left out, and in both the restored value ran on to the end of the document after a collaborator's concurrent delete dropped the run's end:
restatedEndskipped runs this client opened. A collaborator's delete that has not seen the restore can still drop that end. For example, a client comments a word and later, offline, types into it while a collaborator deletes text next to the commented word, which was deleted earlier. When the client comes back, the comment covers every later word on every client.changeEnd), so no own end was written.Now a restore gets its own end after the end of any write in the chain that gave the run its value (the opener, or a restore in it:
runChain, which replacesrunOpener), whichever client wrote it. The extra null is written only where a restore is written, right after an end that already exists, so it adds at most one item per such insert, and the bounded collapse can remove it later like any other item (the growth-bound tests pass unchanged).Changeset
.changeset/lexical-y-own-run-restore-end.md:@lexical/ypatch.Test plan
New
packages/lexical-y/src/__tests__/unit/OwnRunRestoreEnd.test.ts, run in both client id orders:Each test expects nothing after the word to carry the comment.
pnpm exec vitest run --project unit packages/lexical-y/src/__tests__/unit/OwnRunRestoreEnd.test.tsBefore
With only the own-client exemption removed, the second test still fails in both orders. With only the chain change, the first does.
After
The follow-up commits add one case to this file for the fix described under "Randomized sessions found after opening" (it fails on this branch without them and passes on main); the file now has 8 tests: 6 pass and the 2 orders of S9-12b are expected failures (the known gap below).
pnpm exec vitest run --project unit packages/lexical-y: 165 files passed (7483 tests, 46 expected fail), no failures. With the follow-up commits,packages/lexical-yandpackages/lexical-mark: 181 files passed (8646 tests, 46 expected fail).pnpm run ci-checkpasses.FUZZ_SEED_START=50000 FUZZ_SEEDS=2000): the same seven seeds fail with and without the change, with the same messages, so none of them comes from it.Randomized sessions found after opening
A link undone spreads onto untouched words (fixed for one write, follow-up commits). One write restored the link's run at two places, and each restore put a null of the writer's own right after the same end. A collaborator's
Cursor#collapseGapkeeps a key's last items, so it kept the two nulls and deleted the end, and the writer's undo of that write removed both nulls: nothing ended the run. A write now puts at most one null of its own after a given end (Cursor#endedInThisWrite). A first attempt that instead made the collapse keep more items was reverted in the next commit: it brought back the offline-return spread this change fixes in one of the four sessions above. The newOwnRunRestoreEnd.test.tscase (one write, two inserts) fails without it; the session passes. Known gap: two separate writes of one client that each restore inside its own run and are both undone while a collaborator collapses past the run's end still lose the end (each write adds a null of its own there; main writes none for the writer's own run). It is pinned as thetest.failscase "S9-12b" inOwnRunRestoreEnd.test.ts(its body passes on main). The same loss with three clients' nulls after one end also happens on main: Bug: [lexical-y] A comment spreads onto later words when two collaborators undo text they typed inside it while a third deletes past its end #9575. Both are accepted as documented limits (rare, and no text is lost): the Undo bullet of@lexical/y's Limits gets a sentence covering them in [lexical-website][lexical-y] Documentation Update: Say which list conversions are moves, and add an undo race and a join race to the @lexical/y limits #9569 ("A comment or format can run on past its end when writes inside it are undone ... while another client deletes text across its end"), and the S9-12btest.failspin stays. [lexical-y] Bug Fix: Keep the run ends of text an undo removes, and end a value an undo ends past other writes' run ends #9561, stacked on this PR, makes S9-12b pass.An editor that differs from a fresh read of its own document (two sessions): not a binding divergence. In both, the client applied an update past a struct still pending on a missing dependency. Yjs 14 integrates it past the gap and shows it, but a load of that client's own encoding leaves those structs pending too, so the soak's fresh read lacked them until the dependency arrived; then both agree. Without this change the same happens for any update that arrives past a pending one: on main, plain Yjs (C inserts "C"; B, which has it, inserts "b" after it and then "X" at the start; J applies B's two updates only) gives
j="Xhello world"against a reload of its encoding"hello world", and both read"XhelloCb world"once C's update arrives. This is Structs past a clock gap: stuck pending structs, and a doc that differs from its own encoding (v14) yjs/yjs#824 (a doc that differs from its own encoding past a clock gap); with Fix pending structs around clock gaps in the struct store yjs/yjs#827 the reload also reads"Xhello world". This change makes it more frequent only because its null's origin is the run's end, which can be far from the cursor.y.mdalready documents it ("Deliver each client's updates in order"). A soak oracle should compare an editor with a fresh read only once its doc holds no pending structs (or run on a Yjs with Refactor isAttached method #827).A comment spread over untouched words in a 1-hour, 3-person session: not reproduced with this change's current stack. On the identical operation trace (2,249 ops) it appears only with an earlier revision of [lexical-y] Bug Fix: Keep the run ends of text an undo removes, and end a value an undo ends past other writes' run ends #9561 merged on top. This change alone, this change with [lexical-y] Bug Fix: Keep the run ends of text an undo removes, and end a value an undo ends past other writes' run ends #9561's current head, and the same integration with [lexical-y] Bug Fix: Keep the run ends of text an undo removes, and end a value an undo ends past other writes' run ends #9561's current head leave no spread.
A later write's hole taken as the run's end (review follow-up,
83df52d3; scan removed inac2c143a).restatedEndaccepted, for a restore link of the run's chain, the first later item of the key the restore's client wrote; when that restore put no null of its own, that item is the null opening the same client's next hole, which rule 5 says is no end. Another client's restore in front of that hole then wrote a null right after it, and the hole author's undo ended the comment early (aa Kil[c1] X o[c1] Q m[c1] etY er[c1]instead of main's... Q met[c1] Y er[c1]; with bold, "et" lost the format). A candidate end that its client follows with its own text and a restore of the run's value now opens a hole and is not an end. The review's case is inOwnRunRestoreEnd.test.tsfor a comment and for bold, in both id orders: 4 failed before; the file now has 10 passing tests and the 2 S9-12b expected failures.A hole missed when many comments share the range (review follow-up,
b8c41752; scan removed inac2c143a). The check above scanned at most 64 items after a candidate end, so with 64 comment ids on the range it missed the hole and an undo removed the comments from untouched text. It now scans the write's own items by phase (format items, then text, then format items) with no fixed cap; a 64-comment case is inOwnRunRestoreEnd.test.ts(it fails on83df52d3).A collaborator's format starting where your own run ends (accepted limit, Bug: [lexical-y] A collaborator's format that starts where your own format run ends is lost when you change part of that run at the same time #9589). If A removes a format from part of its own run while B applies it to text starting at that run's end, A's own rule-5 null and B's opener tie in Yjs's order; with A's client id higher, B's text loses the format on every client (no text is lost;
mainkeeps it in this own-run case, and has the same tie for runs another client opened). Placement cannot win that tie without giving up what rule 5 protects. It is pinned inOwnRunRestoreEnd.test.ts(ids[2, 1]astest.failsnaming Bug: [lexical-y] A collaborator's format that starts where your own format run ends is lost when you change part of that run at the same time #9589, ids[1, 2]as a regular test), the follow-up repair is Bug: [lexical-y] A collaborator's format that starts where your own format run ends is lost when you change part of that run at the same time #9589, and [lexical-website][lexical-y] Documentation Update: Say which list conversions are moves, and add an undo race and a join race to the @lexical/y limits #9569 adds it to the Limits next to S9-12b and Bug: [lexical-y] A comment spreads onto later words when two collaborators undo text they typed inside it while a third deletes past its end #9575.