Repository navigation
[lexical][lexical-extension][lexical-html][lexical-website][lexical-y] Feature: Block handles, short ids to show blocks to a model - #9531
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
lexical
@lexical/a11y
@lexical/clipboard
@lexical/code
@lexical/code-core
@lexical/code-prism
@lexical/code-shiki
@lexical/compiler
@lexical/devtools-core
@lexical/dragon
@lexical/eslint-plugin
@lexical/extension
@lexical/fast-check
@lexical/file
@lexical/hashtag
@lexical/headless
@lexical/history
@lexical/html
@lexical/internal
@lexical/link
@lexical/list
@lexical/mark
@lexical/markdown
@lexical/mdast
@lexical/offset
@lexical/overflow
@lexical/plain-text
@lexical/react
@lexical/rich-text
@lexical/selection
@lexical/table
@lexical/tailwind
@lexical/text
@lexical/utils
@lexical/y
@lexical/yjs
commit: |
|
It's probably worth noting in |
| } | ||
| } | ||
|
|
||
| describe('block handles: property', () => { |
There was a problem hiding this comment.
we probably want to add an environment variable gate here too, if for no other reason than it spews stuff to the console every unit test run. It's probably also somewhat expensive though.
We do also have fast-check available so that could be used for property tests instead of DIY every time
There was a problem hiding this comment.
Done in a9cee6f. The test is now a fast-check property: random documents of ids (random, shorter than 4 characters, sharing a prefix with an earlier id) and batches of edits, with the same invariants checked after every update. The default run is small (15 documents, about 150 ms) and prints nothing; BLOCK_HANDLE_PROPERTY_RUNS runs more. The 1,000-block typo-rate measurement is skipped unless BLOCK_HANDLE_TYPO_DOCS is set, and only then prints. fast-check is added to the root devDependencies at the range @lexical/fast-check already uses.
| const mode = | ||
| $getExtensionOutput(BlockIdHTMLExtension).export.peek(); |
There was a problem hiding this comment.
This kind of thing could be done with disabledForSession instead of installing a no-op middleware that runs for every element even when it's not enabled
There was a problem hiding this comment.
Done in 2652eec. The $exportDOM override now passes disabledForSession, so while export is off the export session compiles without it and nothing runs per element. Turning export back on (true or 'handle') brings it back in the next session. A new test checks the session config and the output through false, handle, true and false again.
d82d91c to
8850d18
Compare
…dles, short ids to show blocks to a model
## Description
`BlockIdExtension` gives blocks 21-character random ids, which is what a
document needs (unique without coordination, safe in JSON, HTML, paste and
collaboration) but costly in a prompt: about 14 o200k tokens per id, read and
copied back by the model for every block.
This adds **handles**: the shortest prefix of a block's id that no other
block's id starts with, at least 4 characters long, like a short git hash.
`V1StGXR8_Z5jdHi6B-myT` is shown to the model as `V1St`. Handles are derived
from the ids when they are read and never stored, so nothing in the document
changes and nothing has to persist between an agent's turns.
All of it is additive and `@experimental`:
- `@lexical/extension`, next to `$getBlockById`:
- `$getBlockHandle(node): string | null`: the node's handle, or `null` for
a node that `$getBlockById` would not find by its id.
- `$getBlockHandles(): Map<id, handle>`: every block's handle.
- `$resolveBlockHandle(handle): BlockHandleResult`, for untrusted input: a
discriminated union instead of a throw, `{status: 'found', node}`,
`{status: 'not-found', handle}`, `{status: 'ambiguous', handle,
candidates}` (the matching blocks' current handles, sorted, at most 10)
or `{status: 'invalid', handle}` (not 1 to 64 base64url characters, or
not a string).
- `MIN_BLOCK_HANDLE_LENGTH` (4), and the handles in `BlockIdOutput`.
- `@lexical/html`: `BlockIdHTMLConfig.export` widens from `boolean` to
`boolean | 'handle'`. `'handle'` writes `data-block-handle`
(`BLOCK_HANDLE_ATTRIBUTE`) instead of `data-block-id`. Import reads only
`data-block-id`, so a handle is never taken for an id.
Rules:
- An id shorter than 4 characters, or one that another id starts with, is
its own handle. An exact id always wins: with `b1` and `b12`, `b1` finds
`b1`. A whole id is accepted wherever a handle is.
- A prefix longer than the current handle still finds its block, as in git,
so a handle read while another id shared more of its prefix keeps working
after that block is deleted. A prefix shorter than 4 characters finds
nothing unless it is a whole id.
- Handles are computed from the same blocks as `$getBlockById`: the indexed
ids of the last committed state, all at once (sort, then compare each id
with its neighbours, O(n log n)) and cached until a commit changes the set
of ids. While the extension is disabled, each call scans the document, as
`$getBlockById` does.
- A handle names whichever block's id starts with it now. After remote or
local changes, a handle from an earlier read resolves as `not-found` or
`ambiguous`; a prefix freed by a deletion can be taken by a new block
(about one chance in 16.7 million per new random id), and the docs show
how to check the id read where an edit must reach exactly that block.
Nothing released changes: stored ids, the id format, `$getBlockById` (still
an exact match), `export: true` / `false` and HTML import behave as before.
TypeScript and Flow types are updated together.
Discovery is in the JSDoc of `$getBlockById`, `BlockIdExtension` and
`BlockIdHTMLConfig.export`, and in a new "Showing blocks to a model" section
of the Block IDs page: the round trip, the four statuses and what to tell
the model for each, the collaboration caveat, handles in HTML, and the token
cost.
## Changeset
`.changeset/block-handles.md`: `[feature]` for `@lexical/extension` and
`@lexical/html`.
## Test plan
New tests: `BlockIdExtension.handles.test.ts` (the shortest-prefix rule and
the 4-character minimum, ids shorter than 4, ids that prefix others, counter
and `b` + base36 ids, exact ids, invalid and non-string input, deleted
blocks, a new block sharing the prefix making an old handle ambiguous with
the right candidates and longer next handles, blocks named in the same
update, earlier states, duplicates awaiting repair, the disabled extension,
headless and JSON-only use), `BlockIdExtension.handles.property.test.ts`
(random documents with random and adversarial ids under random edits: every
handle equals the brute-force one and resolves to its own block, and no
one-character typo resolves to another block unless it is that block's id
or starts with its current handle), `BlockIdExtension.handlesCache.test.ts`
(one handle build per set of ids), new `export: 'handle'` tests in
`BlockIdHTMLExtension.test.ts` (export, `true`/`false` unchanged, import
ignores `data-block-handle`, round trip of export, edit by handle and
resolve), the docs examples in `BlockIdDocs.test.ts`, and an opt-in perf
test.
### Before
On `main`, with the new tests:
```
$ npx vitest run --project unit packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handles.test.ts packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handles.property.test.ts packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handlesCache.test.ts packages/lexical-html/src/__tests__/unit/BlockIdHTMLExtension.test.ts packages/lexical-html/src/__tests__/unit/BlockIdDocs.test.ts
Error: Cannot find module '../../internal/blockHandles' imported from .../BlockIdExtension.handles.test.ts
TypeError: (0 , $getBlockHandles) is not a function
TypeError: (0 , $getBlockHandle) is not a function
FAIL BlockIdHTMLExtension.test.ts > BlockIdHTMLExtension: export: 'handle' > exports data-block-handle instead of data-block-id
AssertionError: expected '<h2 data-block-id="V1StGXR8_Z5jdHi6B-…' to be '<h2 data-block-handle="V1StG"><span s…' // Object.is equality
Test Files 5 failed (5)
Tests 17 failed | 18 passed (35)
```
(The `export: true`/`false` and import tests pass on `main` too: that
behaviour is unchanged.)
### After
```
$ npx vitest run --project unit (same files)
✓ BlockIdHTMLExtension.test.ts (14 tests)
✓ BlockIdExtension.handlesCache.test.ts (1 test)
✓ BlockIdExtension.handles.test.ts (29 tests)
✓ BlockIdDocs.test.ts (8 tests)
✓ BlockIdExtension.handles.property.test.ts (13 tests)
Test Files 5 passed (5)
Tests 65 passed (65)
$ BLOCK_HANDLE_PROPERTY_SEEDS=300 BLOCK_HANDLE_TYPO_DOCS=30 npx vitest run --project unit packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handles.property.test.ts
block handle typo rate: 440 of 17158649 one-character typos (30 × 1000 blocks) named another block: 2.56e-5 (432 of them exactly that block's handle)
Test Files 1 passed (1)
Tests 301 passed (301)
$ npx vitest run --project unit packages/lexical-extension packages/lexical-html packages/lexical/
Test Files 145 passed | 2 skipped (147)
Tests 3016 passed | 2 skipped (3018)
$ npx vitest run --project scripts-unit scripts/__tests__/unit/treeShakingSource.test.ts
Tests 374 passed (374)
$ pnpm run ci-check
Changeset check passed: 23 pending changeset(s) valid.
Found 0 errors
$ pnpm -C packages/lexical-website run build
[SUCCESS] Generated static files in "build".
```
Performance, 10,000 blocks of random ids (Apple silicon, Node 24, medians of
15):
```
$ BLOCK_ID_PERF=1 npx vitest run --project unit packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handles.perf.test.ts
load (setEditorState) without BlockIdExtension: 13.10 ms
load with it (builds the $getBlockById index): 30.97 ms
index share of the load: 17.87 ms
first $getBlockHandle after a commit (builds all handles): 4.80 ms
$getBlockHandle for every block, cached: 7.68 ms
$getBlockHandles(), cached: 6.35 ms
$resolveBlockHandle for every handle: 11.92 ms
$getBlockById for every id: 4.89 ms
handle lengths: {"4":10013,"5":2}
```
Token cost, o200k (`gpt-tokenizer`, cross-checked with `js-tiktoken`, which
agreed on all 20,000 ids), 20 documents of 1,000 random ids:
| | Bare | In `data-block-…="…"` | In a JSON string |
| --- | --- | --- | --- |
| 21-character id | 14.42 | 15.41 | 15.44 |
| Handle | 2.92 | 3.91 | 3.94 |
…: Resolve block handles against the read the model was shown, and edit blocks in place
## Description
Follow-up from review of the block handles change:
- **Handles resolved against the wrong state.** In the documented
`@lexical/y` server pattern (`editYState` on the state the model read,
then merge), resolving a handle inside the edit always found its block, so
`not-found` and `ambiguous` never fired for changes made since the read.
Resolving it against the latest state alone was not enough either: with
ids `b123` and `b123456`, deleting `b123` made the handle `b123` name the
surviving `b123456`, and a prefix freed by a deletion could be taken by a
block added since.
`$resolveBlockHandle(handle, {read})` takes the handles the model was shown
(the map `$getBlockHandles()` returned at the read). The handle then names
the block it named in that read: `found` while that block still exists in
the state being read, `not-found` once it is deleted, and never `ambiguous`
because of a block added since. Typos fail against the read's ids. The map
is read as it is on each call, with no cached index. The docs, the JSDoc
and the `@lexical/y` server recipe now do this: resolve in a `readYState` of
the stored state with `{read: shown}`, then apply the edit to the read by
id and merge. `y-server.md` points at it.
- **Edits replaced the block.** The examples applied the model's edit with
`result.node.replace(...)`. That re-creates the block: with `@lexical/y`,
text a collaborator typed in it meanwhile no longer stays where they typed
it ("Revenue grew about 6%." merged as "about Revenue grew 6%."). The
examples now change the block in place. The Block IDs page defines
`$setBlockText(block, text)`, which changes only the characters that
differ in the block's own text nodes and returns `false` for a block whose
text is not just its text nodes (line breaks, nested blocks).
- **Too-short handles were reported as deleted blocks.** A string shorter
than `MIN_BLOCK_HANDLE_LENGTH` that is not a whole id was `not-found`. It is
now `{status: 'invalid', reason: 'too-short'}`, and every `invalid` result
carries `reason: 'too-short' | 'malformed'`, so the suggested feedback can
say what is wrong.
TypeScript and Flow are updated together (`BlockHandleOptions`, the
`reason` field), and so is the changeset. The API has not been released yet.
## Changeset
`.changeset/block-handles.md` is updated: `$resolveBlockHandle(handle,
{read})`, and the `invalid` reasons.
## Test plan
New and updated tests:
- `BlockIdExtension.handles.test.ts`: resolution against a read (a deleted
block with a surviving prefix holder, a block added since, a freed prefix,
typos, a map changed between calls), and the too-short and malformed
reasons.
- `BlockIdExtension.handles.property.test.ts`: every step checks that the
previous step's handles, resolved with that read, name the same blocks or
are not found.
- `BlockHandles.test.ts` (`@lexical/y`): the documented server recipe end to
end. Concurrent typing is kept in place, a deleted block is `not-found`
(also with a surviving prefix holder), a block added since is not taken,
resolving inside the edit of the read always finds, and `replace()`
displaces the typing.
- `BlockIdDocs.test.ts`: the new docs examples, and `$setBlockText` cases
(formatted runs, node boundaries, refusing line breaks and lists).
### Before
The new tests on the previous commit of this PR:
```
$ npx vitest run --project unit packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handles.test.ts packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handles.property.test.ts packages/lexical-html/src/__tests__/unit/BlockIdDocs.test.ts packages/lexical-y/src/__tests__/unit/BlockHandles.test.ts
FAIL BlockHandles.test.ts > block handles with @lexical/y > resolving against the stored state, applying to the read in place
- Expected
+ Received
[
null,
"not-found",
- "not-found",
null,
+ "ambiguous",
]
FAIL BlockIdExtension.handles.test.ts > $resolveBlockHandle with the handles of a read > a block deleted since the read is not found, even if another id now starts with the handle
AssertionError: expected { id: 'b123456', status: 'found' } to deeply equal { handle: 'b123', status: 'not-found' }
FAIL BlockIdExtension.handles.test.ts > $resolveBlockHandle > a typo is not found; a prefix shorter than 4 characters is too short
AssertionError: expected { handle: '9xQ', status: 'not-found' } to deeply equal { handle: '9xQ', …(2) }
FAIL BlockIdExtension.handles.property.test.ts > block handles: property > seed 11
AssertionError: seed 11 step 10 ml2iFhmo: expected 'found' to be 'not-found'
Test Files 4 failed (4)
Tests 20 failed | 38 passed (58)
```
(In the `@lexical/y` case the third edit, meant for the deleted `b123`, was
applied to `b123456`.)
### After
```
$ npx vitest run --project unit packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handles*.ts packages/lexical-html/src/__tests__/unit/BlockId*.ts packages/lexical-y/src/__tests__/unit/BlockHandles.test.ts
Tests 73 passed | 1 skipped (74)
$ BLOCK_HANDLE_PROPERTY_SEEDS=200 npx vitest run --project unit packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handles.property.test.ts
Tests 201 passed (201)
$ npx vitest run --project unit packages/lexical-extension packages/lexical-html packages/lexical/ packages/lexical-y/src/__tests__/unit/BlockHandles.test.ts
Test Files 146 passed | 2 skipped (148)
Tests 3024 passed | 2 skipped (3026)
$ npx vitest run --project scripts-unit scripts/__tests__/unit/treeShakingSource.test.ts
Tests 375 passed (375)
$ pnpm run ci-check
Found 0 errors
```
…the HTML handle example to the export's handles, and check a block's shape before $setBlockText
## Description
Two examples that review of this PR found unsafe:
- **The `BlockIdHTMLConfig.export` JSDoc** resolved a handle from exported
HTML without `{read}`, then removed the node. Here is how that goes wrong:
1. The model is shown `9xQe1111` with the handle `9xQe`.
2. That block is deleted, and a block `9xQe2222` takes the prefix.
3. The example deletes `9xQe2222`, a block the model never saw.
The example now captures `$getBlockHandles()` in the same read as
`$generateHtmlFromNodes`, resolves the answer with
`$resolveBlockHandle(handle, {read: shown})`, and edits the block in place
with `$setBlockText` instead of removing it. The "Handles in HTML" section
of the Block IDs page shows the same capture.
- **`$setBlockText` on the Block IDs page** used text equality as its
structure check: the block's concatenated text-node text equal to its
`getTextContent()`. That does not prove the block holds only text. A list
with one item and a paragraph with an inline decorator that has no text
both passed, so they were rewritten instead of refused. The helper now
checks the block's shape before changing anything: every child must be a
text node, or an inline element (such as a link) that holds only text, at
any depth. Line breaks, decorators (even ones without text), nested
blocks and occupied named slots (at any level) are refused.
The test copy of the helper moves to
`packages/lexical-html/src/__tests__/utils/setBlockText.ts`, shared by the
docs tests and the HTML tests. A test now checks that it, the docs page's
copy and the `@lexical/y` test's copy are identical.
## Test plan
- `BlockIdHTMLExtension.test.ts`:
- The round trip follows the JSDoc: export and handles in one read, a
pinned resolve, an in-place edit that keeps the same node.
- New regression for a prefix freed by a deletion. The unpinned resolve
finds `9xQe2222`, the pinned resolve is `not-found`, and `9xQe2222` is
left as it was.
- `BlockIdDocs.test.ts`:
- A one-item list, a paragraph with an empty inline decorator, a block
whose slot holds a line break, and an inline link that hosts a slot are
all refused, and the editor state is unchanged.
- Text inside a link is edited in place.
- The three copies of the helper are compared.
### Before
The new refusal test against the previous helper:
```
$ npx vitest run --project unit packages/lexical-html/src/__tests__/unit/BlockIdDocs.test.ts -t refuses
FAIL BlockIdDocs.test.ts > Block IDs documentation examples > Showing blocks to a model: $setBlockText refuses blocks that are not just text
AssertionError: expected true to be false // Object.is equality
❯ editor.update.discrete packages/lexical-html/src/__tests__/unit/BlockIdDocs.test.ts:512:53
Tests 1 failed | 11 skipped (12)
```
(Line 512 is `expect($setBlockText(oneItem, 'rewritten')).toBe(false)`.) The
previous JSDoc's unpinned resolve finds `9xQe2222`. The new HTML regression
asserts this before it shows that the pinned resolve is `not-found`.
### After
```
$ npx vitest run --project unit packages/lexical-html/src/__tests__/unit/BlockId*.ts packages/lexical-y/src/__tests__/unit/BlockHandles.test.ts packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handles*.ts
Tests 77 passed | 1 skipped (78)
$ npx vitest run --project unit packages/lexical-extension packages/lexical-html packages/lexical/ packages/lexical-y/src/__tests__/unit/BlockHandles.test.ts
Test Files 146 passed | 2 skipped (148)
Tests 3028 passed | 2 skipped (3030)
$ npx vitest run --project scripts-unit scripts/__tests__/unit/treeShakingSource.test.ts
Tests 375 passed (375)
$ pnpm run ci-check
Found 0 errors
$ pnpm -C packages/lexical-website run build
[SUCCESS] Generated static files in "build".
```
8850d18 to
cc03df0
Compare
…h fast-check, and gate its long runs
## Description
Review feedback: the block handle property test printed a typo-rate line on
every unit run and ran a 1,000-block measurement each time. It also used a
hand-written random generator where the repo already uses fast-check.
The test is now a fast-check property. It generates documents of ids
(random 21-character ids, ids shorter than 4 characters, and ids that share
a prefix of 1 to 10 characters with an earlier id) and a sequence of
updates, each of one to three edits: inserts that bring such an id,
deletes, replaces and moves. It checks the same invariants as before after
every update:
- every handle equals the brute-force shortest unique prefix;
- every handle and every whole id resolves to its own block;
- the previous read's handles, resolved with that read, name the same
blocks, or are `not-found` once those blocks are deleted;
- no one-character typo of a handle names another block, except through
that block's id or a prefix at least as long as its current handle.
Failures shrink to a minimal document and edit list. A fixed example
repeats an id while 32 short ids are taken: the fallback `generateId`,
which only names such copies, makes ids longer than any generated one, so
it never collides with them.
The default run is small (15 documents, about 150 ms) and prints nothing.
`BLOCK_HANDLE_PROPERTY_RUNS` runs more, as the other property tests'
`*_SEEDS` variables do. The typo-rate measurement is skipped unless
`BLOCK_HANDLE_TYPO_DOCS` is set, and only then prints the rate.
`fast-check` is added to the root `devDependencies`, at the range
`@lexical/fast-check` already uses.
## Test plan
### Before
```
$ npx vitest run --project unit packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handles.property.test.ts
stdout | packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handles.property.test.ts > block handles: property > one-character typo rate, 1000-block documents of random 21-character ids
block handle typo rate: 18 of 571952 one-character typos (1 × 1000 blocks) named another block: 3.15e-5 (16 of them exactly that block's handle)
Tests 13 passed (13)
Duration 2.37s
```
### After
```
$ npx vitest run --project unit packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handles.property.test.ts
Tests 1 passed | 1 skipped (2)
Duration 703ms
$ BLOCK_HANDLE_PROPERTY_RUNS=1000 npx vitest run --project unit packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handles.property.test.ts
✓ handles, resolution and typo safety under random ids and edits 17364ms
Tests 1 passed | 1 skipped (2)
$ BLOCK_HANDLE_TYPO_DOCS=3 npx vitest run --project unit packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handles.property.test.ts
block handle typo rate: 43 of 1715851 one-character typos (3 × 1000 blocks) named another block: 2.51e-5 (42 of them exactly that block's handle)
```
… out of export sessions while export is off
## Description
Review feedback: `BlockIdHTMLExtension` installed an `$exportDOM`
middleware for every node. That middleware ran for each exported element
even with `export: false`, only to return early.
The override now passes `disabledForSession`, which is read once at the
start of each export session. While the `export` signal is `false`, the
session compiles without the override, so nothing runs for each element.
Turning `export` back on (`true` or `'handle'`) brings the override back in
the next session. The output is unchanged in every mode.
## Changeset
`.changeset/block-id-html-export-session.md`: `[perf]` for `@lexical/html`.
## Test plan
New test in `BlockIdHTMLExtension.test.ts`. With `export: false`, an export
session uses a render config compiled without the override
(`$getSessionDOMRenderConfig(editor) !== editor._config.dom`). Switching
the signal to `'handle'`, `true` and back to `false` switches the session
config and the output with it.
### Before
```
$ npx vitest run --project unit packages/lexical-html/src/__tests__/unit/BlockIdHTMLExtension.test.ts -t "every export session"
AssertionError: expected true to be false // Object.is equality
❯ packages/lexical-html/src/__tests__/unit/BlockIdHTMLExtension.test.ts:355:38
```
### After
```
$ npx vitest run --project unit packages/lexical-html/src/__tests__/unit/BlockIdHTMLExtension.test.ts
Tests 16 passed (16)
$ npx vitest run --project scripts-unit scripts/__tests__/unit/treeShakingSource.test.ts
Tests 375 passed (375)
```
…short block handles need ids with a random prefix ## Description Review feedback: handles are the shortest unique prefix of each id, so they are short only when ids differ in their first characters. An id that starts with a timestamp or a version breaks that. A UUIDv7 in its standard layout, for example, starts with a millisecond time and a version digit, so ids made close together share their first characters and get handles much longer than the usual 4 characters. This is now noted in `BlockIdConfig.generateId`'s JSDoc and on the Block IDs page: in the `generateId` configuration entry, and next to the typo-rate paragraph of "Showing blocks to a model". Both recommend keeping the default or putting the random part first. ## Test plan Documentation only. `pnpm run ci-check` passes, and the website builds (`pnpm -C packages/lexical-website run build`) with broken anchors set to throw.
Added in 2a68d09: |
There was a problem hiding this comment.
Thanks for this. The handle computation, pinning resolution to the read the model was shown, and the tests (the fast-check property test and the @lexical/y merge test especially) are careful work. The earlier review follow-ups (fast-check, disabledForSession, the generateId note) are all in.
Please fix before merging
1. $setBlockText throws on a block that contains a tab. This is packages/lexical-website/docs/extensions/block-ids.md:278, with copies in packages/lexical-html/src/__tests__/utils/setBlockText.ts:82 and packages/lexical-y/src/__tests__/unit/BlockHandles.test.ts:109. $holdsOnlyText accepts every $isTextNode child. TabNode is a TextNode, but TabNode.spliceText throws for anything except a no-op or re-inserting \t (packages/lexical/src/nodes/LexicalTabNode.ts:107-118). Step 3 of "Showing blocks to a model", run on a paragraph Name:\tvalue, gives:
| Model's text | Result |
|---|---|
Name:\tvalues |
edited in place |
Name:\tMy value (insert right after the tab) |
editor.update throws TabNode does not support spliceText; no feedback; block unchanged |
Name: value |
throws |
Name:value |
throws |
The Block IDs page says the helper "Returns false, changing nothing, unless the block holds only text", and the JSDoc of $getBlockById and BlockIdHTMLConfig.export sends readers to it. A copied recipe will therefore throw on ordinary pasted content, including in the @lexical/y recipe, where editYState rethrows whatever $edit throws. Refusing tabs in all three copies would fix it, either with $isTextNode(child) && !$isTabNode(child) or with child.isSimpleText(), which also refuses token-mode and subclassed text nodes. Please also add the case to the refusal test in BlockIdDocs.test.ts.
Worth considering
2. Text added at a link's edge goes into the link (block-ids.md:250-257). An insertion at a node boundary goes into the node before it, and an insertion at the start of the block into the first node, even when that node is inside a LinkNode. LinkNode.canInsertTextBefore() and canInsertTextAfter() return false, and RangeSelection.insertText uses that to keep typing at a link's edge outside the link (packages/lexical/src/LexicalSelection.ts:430-443). With this helper, Read [the plan] edited to Read the plan before Friday. gives the link text the plan before Friday., and [Plan] is ready. edited to The Plan is ready. gives The Plan.
3. Handles cost quadratic time while BlockIdExtension is disabled (packages/lexical-extension/src/BlockIdExtension.ts:1597-1613). In that mode every $getBlockHandle call walks the document and rebuilds the table, and both the export: 'handle' override and the documented per-block read call it once per block. Under jsdom, exporting 2,000 blocks with export: 'handle' took about 5.5 s with the extension disabled, against about 73 ms with it enabled (or with export: true). Caching the scan per editor state during a read would keep this linear. At least, the docs could say that an export scans the document once per block.
4. Nit: BlockIdHTMLConfig.export and its output signal widen to boolean | 'handle', so code that types the signal as Signal<boolean> no longer typechecks. That is fine for an @experimental option, but one line in .changeset/block-handles.md would tell callers.
Everything else I checked holds up. export: true and export: false produce the same HTML as before, import never reads data-block-handle, the TypeScript and Flow declarations match, and both changesets follow .changeset/README.md.
| .getChildren() | ||
| .every( | ||
| child => | ||
| $isTextNode(child) || |
There was a problem hiding this comment.
TabNode passes this check (it extends TextNode), but TabNode.spliceText throws unless it is a no-op or re-inserts \t. On Name:\tvalue, the model texts Name:\tMy value, Name: value and Name:value all make the documented editor.update throw TabNode does not support spliceText, where this helper promises to return false. Refusing tabs here, with !$isTabNode(child) or child.isSimpleText(), in all three copies would fix it.
There was a problem hiding this comment.
Fixed in 938acc7. $holdsOnlyText now accepts only isSimpleText() text nodes, so tabs, tokens, segmented text and text node subclasses are refused with false, and the block is left unchanged. All three copies are updated, and a test checks they stay identical. BlockIdDocs.test.ts adds a tab paragraph and a token paragraph to the refusal cases.
| ) { | ||
| // The node where the change starts (for an insert at a node boundary, | ||
| // the node before it): delete and insert there. | ||
| node.spliceText(from, Math.max(to - from, 0), inserted); |
There was a problem hiding this comment.
At a boundary this inserts into the node before it, or into the first node at the start of the block, even inside a LinkNode. LinkNode returns false from canInsertTextBefore/canInsertTextAfter, and insertText uses that to keep typing at a link's edge outside the link. Read [the plan] → Read the plan before Friday. leaves the link text the plan before Friday.. Consider putting boundary text in a text node outside an inline element that refuses it.
There was a problem hiding this comment.
Fixed in 938acc7. A pure insert at the edge of inline elements that refuse text there (canInsertTextAfter/canInsertTextBefore returns false) now goes beside the outermost such element, the way typing does. For a link inside a mark, that means beside the mark. The text goes into the adjacent simple text node, or into a new one. Tests cover a link in the middle of a block, at its start, at its end and inside a mark, plus an insert inside the link, which stays in the link.
| // document order, as $getBlockById finds it. | ||
| const {$isIdentifiedBlock} = runtime.config; | ||
| const firstOfId = new Map<string, LexicalNode>(); | ||
| $walk(null, node => { |
There was a problem hiding this comment.
While the extension is disabled, this walk and the table build run on every $getBlockHandle call, and export: 'handle' calls it once per block. Exporting 2,000 blocks took about 5.5 s here under jsdom, against about 73 ms with the extension enabled. Caching the scan per editor state during a read, when that state is frozen, would keep it linear.
There was a problem hiding this comment.
Fixed in 938acc7. While the extension is disabled, handles now come from one scan of the last committed state, cached until the next commit, and lookups are verified against the state being read. That matches enabled mode, which uses the index of the last committed state. For a repeated id, the lookup returns its document-first live holder, as $getBlockById does. An export: 'handle' export of 2,000 blocks (jsdom) now takes 33 ms with the extension disabled, against 39 ms enabled. $getBlockHandle on each of 10,000 blocks takes 8 ms disabled, against 10 ms enabled. The changeset nit is in too: export widens to boolean | 'handle'.
…: Hash block ids into handles, so any id format gets short, typo-safe handles; fix $setBlockText on tabs and link edges, and disabled-mode cost
## Description
This addresses the code-review bot's review of this PR, plus a gap the
review did not cover.
- **Handles were only as good as the ids' entropy.** A handle was the
shortest unique prefix of the id itself. Ids that share their first
characters therefore got long handles, and when those ids are sequential, a
one-character typo named a neighbouring real block. That is the risk handles
were meant to remove. It hit `@lexical/y`'s derived ids
(`y<client>_<clock>`, which every line written by a client without
`BlockIdExtension` shows) and UUIDv7 ids. On 1,000-block documents:
| Ids | Handle length, mean (before → after) | One-character typos naming another block (before → after) |
| --- | --- | --- |
| Random, 21 characters | 4.0 → 4.0 | 2.8e-5 → 2.3e-5 |
| `@lexical/y` derived | 15.2 → 4.0 | 1.2e-2 → 1.4e-5 |
| UUIDv7, ms apart | 14.7 → 4.0 | 4.4e-3 → 3.9e-5 |
A handle is now the shortest prefix, at least 4 characters long, of a
64-bit hash of the id (`handleKey`: a cyrb53-style mix in base64url, 11
characters, most significant bits first) that no other id's hash starts
with. Handles are opaque: `V1StGXR8_Z5jdHi6B-myT` shows as `2bNo`.
Resolution compares a handle against the hashes of the document's ids, and
with `{read}`, against those of the read. A string longer than 11
characters (longer than any handle) is looked up as a whole id, so the
default 21-character ids still work. A shorter whole id can only be passed
by its handle. Hash collisions only make handles longer, and ambiguity is
already reported. `@lexical/y` needs no change. The docs, the JSDoc, the
changeset and the `generateId` note are updated: any id format now gets
short handles.
- **`$setBlockText` threw on a block with a tab.** `TabNode.spliceText`
throws for anything but a no-op. The helper on the Block IDs page, and its
test copies, now accept only `isSimpleText()` text nodes, so tabs, tokens,
segmented text and text node subclasses are refused with `false`.
- **Text added at a link's edge went into the link.** A pure insert at the
edge of an inline element that refuses text there
(`canInsertTextBefore`/`canInsertTextAfter` is false, as for `LinkNode`)
now goes beside the outermost such element (a link inside a mark: beside
the mark), as typing does: into the adjacent simple text node, or into a new
text node.
- **Handles cost quadratic time while `BlockIdExtension` was disabled.**
Each `$getBlockHandle` call scanned the document and rebuilt the table.
While disabled, handles now come from one scan of the last committed state,
cached until the next commit, and lookups are verified against the state
being read: a repeated id's document-first live holder, as
`$getBlockById` finds it. This matches enabled mode, where they come from
the index of the last committed state. Reading the handle of each of 2,000 blocks drops
from seconds (the review measured 5.5 s for an `export: 'handle'` HTML
export) to about 2 ms.
- **Changeset:** one line saying that `export` widens to
`boolean | 'handle'`.
## Test plan
- `BlockIdExtension.handles.test.ts`, rewritten for hashed handles:
- `handleKey` values, brute-force handles, and 4–5 character handles for
counter, derived, short and UUIDv7 ids;
- ambiguity, the candidate cap, read pinning and the freed-handle cases,
with ids picked for colliding hashes;
- typo safety on UUIDv7, derived-like and counter documents;
- the whole-id cutoff;
- committed-state handles while disabled, and a repeated id's holders
removed or reordered while disabled.
- `BlockHandles.test.ts` (`@lexical/y`):
- 400 lines written without `BlockIdExtension` and read with it show
derived ids. Their handles are 4–5 characters, all resolve, and typos
stay safe.
- The server recipe is updated to hashed handles.
- `BlockIdDocs.test.ts`: tab and token refusal, link-edge inserts (also a
link inside a mark), and the helper copies kept in sync, besides the
updated examples.
- `BlockIdExtension.handles.property.test.ts`: brute-force handles over hash
keys. The gated measurement covers random, derived and UUIDv7 ids.
### Before
The new tests on the previous head of this PR:
```
$ npx vitest run --project unit packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handles.test.ts packages/lexical-html/src/__tests__/unit/BlockIdDocs.test.ts packages/lexical-y/src/__tests__/unit/BlockHandles.test.ts
FAIL BlockHandles.test.ts > block handles of derived ids > lines written without BlockIdExtension, read with it: derived ids get short, typo-safe handles
AssertionError: expected 13 to be 4 // Object.is equality
FAIL BlockIdDocs.test.ts > Block IDs documentation examples > Showing blocks to a model: $setBlockText refuses special text nodes
AssertionError: expected true to be false // Object.is equality
FAIL BlockIdDocs.test.ts > Block IDs documentation examples > Showing blocks to a model: text added at a link edge goes outside the link
AssertionError: mid → See the plans for Q3.: expected [ [ 'text', 'See ' ], …(2) ] to deeply equal [ [ 'text', 'See ' ], …(2) ]
FAIL BlockIdExtension.handles.test.ts > handle computation (pure) > handleKey: 64 bits of a hash of the id in base64url
TypeError: handleKey is not a function
Test Files 3 failed (3)
Tests 37 failed | 16 passed (53)
```
### After
```
$ npx vitest run --project unit packages/lexical-extension packages/lexical-html packages/lexical/ packages/lexical-y/src/__tests__/unit/BlockHandles.test.ts
Test Files 146 passed | 2 skipped (148)
Tests 3024 passed | 4 skipped (3028)
$ BLOCK_HANDLE_TYPO_DOCS=30 npx vitest run --project unit packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handles.property.test.ts
block handles, 30 × 1000 blocks per shape:
derived: handle length mean 4.00, max 5; 412 of 17158755 one-character typos named another block: 2.40e-5 (406 of them exactly that block's handle)
random: handle length mean 4.00, max 5; 472 of 17159033 one-character typos named another block: 2.75e-5 (462 of them exactly that block's handle)
uuidv7: handle length mean 4.00, max 4; 445 of 17158557 one-character typos named another block: 2.59e-5 (440 of them exactly that block's handle)
$ BLOCK_ID_PERF=1 npx vitest run --project unit packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handles.perf.test.ts
2000 blocks, enabled (index): $getBlockHandle for every block in one read: 1.72 ms
2000 blocks, disabled (scan): $getBlockHandle for every block in one read: 1.95 ms
10000 blocks, enabled (index): $getBlockHandle for every block in one read: 10.26 ms
10000 blocks, disabled (scan): $getBlockHandle for every block in one read: 8.03 ms
10000 derived ids: $getBlockHandles() after a commit 49.53 ms, handle lengths {"4":9999,"5":6}
$ npx vitest run --project scripts-unit scripts/__tests__/unit/treeShakingSource.test.ts
Tests 375 passed (375)
$ pnpm run ci-check
Found 0 errors
```
An `export: 'handle'` HTML export of 2,000 blocks (jsdom): 39 ms enabled
and 33 ms disabled. The review measured the disabled case at about 5.5 s
before.
…near time when many blocks share an id
## Description
With `BlockIdExtension` disabled, a repeated id persists, because no repair
runs. Reading the handle of every block (one `$getBlockHandle` per block, as
an `export: 'handle'` HTML export does) was then quadratic: each call looked
up the id, and the lookup checked every holder of the id to find the
document-first live one. With 40, 80 and 160 blocks sharing one id, a read
cost 1,680, 6,560 and 25,920 `$isIdentifiedBlock` calls, against 120, 240 and
480 for unique ids.
The lookup behind handles is now remembered per id for each frozen state it
is asked about, so it runs once per id in a read of a committed state. While
the active state is not frozen (an update), every call still looks up the
id. The memo lives in the handle table, which is rebuilt when the index
changes (enabled), or in the scan of each committed state (disabled), so it
never outlives what it was computed from. It needs the active EditorState,
which `lexical` does not expose, so `lexical` gains an internal export,
`INTERNAL_getActiveEditorState` (like `INTERNAL_$isBlock`, it is not part of
the public types).
## Test plan
New `BlockIdExtension.handlesCost.test.ts` counts `$isIdentifiedBlock` calls,
with no timing, while reading every block's handle for 40, 80 and 160
blocks. It covers the extension disabled and enabled, with unique ids and
with one repeated id, and requires the calls to grow linearly.
### Before
```
$ npx vitest run --project unit packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handlesCost.test.ts
✓ disabled, unique ids
× disabled, one id repeated
✓ enabled, unique ids
✓ enabled, one id repeated
AssertionError: expected 25760 to be less than or equal to 14260.000000000002
Tests 1 failed | 3 passed (4)
```
The review's own repro, which counts calls for 40, 80 and 160 holders of one
id:
```
REVIEW_DUPLICATE_HANDLES true [{"n":40,"calls":1680,…},{"n":80,"calls":6560,…},{"n":160,"calls":25920,…}]
```
### After
```
$ npx vitest run --project unit packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handlesCost.test.ts
Tests 4 passed (4)
REVIEW_DUPLICATE_HANDLES false [{"n":40,"calls":120,"handles":40},{"n":80,"calls":240,"handles":80},{"n":160,"calls":480,"handles":160}]
REVIEW_DUPLICATE_HANDLES true [{"n":40,"calls":120,"handles":1},{"n":80,"calls":240,"handles":1},{"n":160,"calls":480,"handles":1}]
$ npx vitest run --project unit packages/lexical-extension packages/lexical-html packages/lexical/ packages/lexical-y/src/__tests__/unit/BlockHandles.test.ts
Test Files 147 passed | 2 skipped (149)
Tests 3028 passed | 4 skipped (3032)
$ npx vitest run --project scripts-unit scripts/__tests__/unit/treeShakingSource.test.ts
Tests 375 passed (375)
$ pnpm run ci-check
Found 0 errors
```
## Description The previous commit added the internal export `INTERNAL_getActiveEditorState(): EditorState | null` to `lexical`'s TypeScript source, but not to `packages/lexical/flow/Lexical.js.flow`. This adds its Flow declaration, nullable and marked `@internal`, so the TypeScript and Flow declarations stay in sync. ## Test plan ### After ``` $ pnpm run flow Running Flow... Found 0 errors $ pnpm run ci-check Found 0 errors ```
There was a problem hiding this comment.
Thanks for the follow-ups. Hashing ids into handles keeps them at 4 or 5 characters for @lexical/y derived ids, UUIDv7s and counters; $setBlockText now refuses tabs and tokens and keeps text added at a link's edge outside the link; and reading every handle stays linear with the extension disabled. Each of these comes with a test that fails without the change.
Open question for maintainers: how should the export type change ship?
BlockIdHTMLConfig.export and the extension's export output signal shipped as boolean in v0.53.0. This PR widens both to boolean | 'handle' (packages/lexical-html/src/BlockIdHTMLExtension.ts:97; packages/lexical-html/flow/LexicalHtmlBlockIdHTMLExtension.js.flow:18 and :26), so code written against the released types stops typechecking:
const {export: exportSignal} = getExtensionDependencyFromEditor(
editor,
BlockIdHTMLExtension,
).output;
const on: boolean = exportSignal.value; // TS2322 with this PR
const signal: Signal<boolean> = exportSignal; // TS2322 with this PR.changeset/README.md and CONTRIBUTING.md treat a change that makes callers change their code, types included, as [breaking], with a minor bump and Migration: and Detect: paragraphs; AGENTS.md prefers additive, optional parameters; and the Block IDs page says these APIs may change in a minor release. .changeset/block-handles.md currently files the change as a patch [feature] with a Types: note. Should export stay boolean, with handles behind a separate option, or should this ship as a [breaking] changeset?
Smaller suggestions
packages/lexical-website/docs/extensions/block-ids.md:375-377: "each call still checks every block for changes, so read every block's handle with one$getBlockHandles()" predates the disabled-mode cache. The scan is now kept until the next commit, as the$getBlockHandleJSDoc says, so reading each block's handle is fine.block-ids.md:143-144says nothing has to be kept between an agent's turns, while the round trip right below asks the app to keep the handles it showed, for{read}.- The Changesets job reports no changeset for
lexical, whose only change is the@internalINTERNAL_getActiveEditorStateexport. That has no effect on people who use Lexical, so a maintainer can apply theskip-changesetlabel (CONTRIBUTING.md, "Changesets"). - Nit: the typo clause in
packages/lexical-extension/src/__tests__/unit/BlockIdExtension.handles.property.test.ts:235-238cannot fail for a resolver that agrees with the handle table; the rate assertion inBlockIdExtension.handles.test.ts:881is what backs the typo-safety claim, so the property test's header could describe that clause as a consistency check.
|
|
||
| A handle costs about 3 to 4 o200k tokens where a 21-character id costs about 14. Handles are derived from the stored ids and never stored; ids, `$getBlockById`, `export: true` and HTML import are unchanged, and import never reads `data-block-handle`. | ||
|
|
||
| Types: `BlockIdHTMLConfig.export` (and the `export` output signal) widens from `boolean` to `boolean | 'handle'`; code that reads it as a boolean should treat `'handle'` as on. |
There was a problem hiding this comment.
This widens a type that shipped as boolean in v0.53.0, so code such as const on: boolean = output.export.value or const s: Signal<boolean> = output.export stops typechecking. .changeset/README.md and CONTRIBUTING.md file a change that makes callers change their code, types included, as [breaking] with a minor bump and Migration:/Detect:, and the Block IDs page says these APIs may change in a minor release. Could export stay boolean, with handles behind a separate option? Otherwise this entry would need to become [breaking]; that choice is for the maintainers.
There was a problem hiding this comment.
Done in 392b1a1. export stays exactly as it shipped in v0.53.0: a boolean, with a Signal<boolean> output, in both TypeScript and Flow. Handles now sit behind a separate, optional handles?: boolean (default false). With export on, it writes data-block-handle instead of data-block-id; on its own, it does nothing. Its output is a Signal<boolean>. A new type test, checked by tsc-test in ci-check, compiles const on: boolean = output.export.value, const s: Signal<boolean> = output.export and a whole {export: true, import: true} config. It fails if export is widened again. The changeset no longer mentions a widened type.
| While the extension is disabled, they come from a scan of the document, kept | ||
| while the document has not changed; each call still checks every block for | ||
| changes, so read every block's handle with one `$getBlockHandles()`. |
There was a problem hiding this comment.
This predates the disabled-mode cache: the scan is now kept until the next commit and lookups in a read are remembered per state, so reading each block's handle costs about the same disabled as enabled (the $getBlockHandle JSDoc already says "cached until the next commit"). Suggestion: "While the extension is disabled, they come from a scan of the last committed state, kept until the next commit."
There was a problem hiding this comment.
Done in 392b1a1. The page now says: "While the extension is disabled, they come from a scan of the last committed state, kept until the next commit, so reading each block's handle costs about the same either way."
| are derived from them when they are read and are never stored, so nothing | ||
| has to be kept between an agent's turns. |
There was a problem hiding this comment.
With {read}, the app keeps the handles it showed (or the state they came from) until the answer is applied, as the round trip below says ("export handles (and keep them)"), so "nothing has to be kept between an agent's turns" reads as a contradiction. Maybe: "nothing is stored in the document".
There was a problem hiding this comment.
Done in 392b1a1. It now reads "handles are derived from them when they are read, so nothing is stored in the document."
…ckIdHTMLConfig.export a boolean, and turn on handles with a separate handles option
## Description
Review question: `BlockIdHTMLConfig.export` and `BlockIdHTMLExtension`'s
`export` output signal shipped as `boolean` in v0.53.0. This PR had widened
both to `boolean | 'handle'`, so code typed against the released types (for
example `const on: boolean = output.export.value`, or
`const s: Signal<boolean> = output.export`) stopped type-checking.
`export` now stays exactly as it shipped, a `boolean` with a
`Signal<boolean>` output, in both TypeScript and Flow. Handles get their own
option instead. `handles?: boolean` (default `false`) makes the export write
`data-block-handle` instead of `data-block-id` while `export` is on; on its
own, it does nothing. It is optional in the config type, so a whole
`BlockIdHTMLConfig` written for v0.53.0 still type-checks, and its output
signal is a `Signal<boolean>`. The export override is still left out of
export sessions while `export` is off.
The docs, JSDoc and changeset now describe `handles: true`, and the
changeset no longer mentions a widened type. This commit also takes the
review's smaller suggestions:
- the Block IDs page now says that reading each block's handle costs about
the same with the extension disabled;
- the page says "nothing is stored in the document" rather than "nothing has
to be kept between an agent's turns";
- the property test's header describes its typo clause as a consistency
check.
## Test plan
- New type test in `BlockIdHTMLExtension.test.ts`, type-checked by
`pnpm run tsc-test` as part of `ci-check`. Code written against v0.53.0
still compiles: `const on: boolean = output.export.value`,
`const s: Signal<boolean> = output.export`, and a whole
`{export: true, import: true}` config. `expectTypeOf` pins the `export`,
`handles` and `import` signals to `Signal<boolean>`.
- The session-gating test now also checks that `handles` alone does not turn
the export on.
- The other HTML tests use `handles: true`.
### Before
With `export` widened to `boolean | 'handle'`, as on the previous head, the
new type test does not compile:
```
$ npx tsc -p tsconfig.test.json --noEmit
packages/lexical-html/src/__tests__/unit/BlockIdHTMLExtension.test.ts(379,11): error TS2322: Type 'string | boolean' is not assignable to type 'boolean'.
packages/lexical-html/src/__tests__/unit/BlockIdHTMLExtension.test.ts(380,11): error TS2322: Type 'Signal<boolean | "handle">' is not assignable to type 'Signal<boolean>'.
```
### After
```
$ pnpm run ci-check
Found 0 errors
$ npx vitest run --project unit packages/lexical-extension packages/lexical-html packages/lexical/ packages/lexical-y/src/__tests__/unit/BlockHandles.test.ts
Test Files 147 passed | 2 skipped (149)
Tests 3029 passed | 4 skipped (3033)
$ npx vitest run --project scripts-unit scripts/__tests__/unit/treeShakingSource.test.ts
Tests 375 passed (375)
```
There was a problem hiding this comment.
This looks good to me. Keeping BlockIdHTMLConfig.export a boolean and moving handles behind an optional handles flag means code written against v0.53.0 keeps compiling (the new type test in BlockIdHTMLExtension.test.ts checks exactly that), and the hashed handles, {read} pinning and the in-place $setBlockText recipe each come with tests that fail without them.
A few optional, non-blocking suggestions:
$setBlockTextand the local caret (packages/lexical-website/docs/extensions/block-ids.md:229, with copies inpackages/lexical-html/src/__tests__/utils/setBlockText.tsandpackages/lexical-y/src/__tests__/unit/BlockHandles.test.ts). The helper rewrites text nodes withspliceTextbut leaves the editor'sRangeSelectionat its old offsets. In a live editor (step 3 of the round trip), a caret placed before "4%" inRevenue grew 4% in Q3.sits after "Total revenue" once the answerTotal revenue grew 4% in Q3.is applied, and a shorter answer can leave the offset past the end of the text node. Shifting text points that follow the change by the length difference, or one sentence on the page saying the helper does not move the selection, would cover it; since the page offers this helper as the way to apply a model's edit, it seems worth saying either way.- No changeset for
lexical. The only change inpackages/lexicalis the@internalINTERNAL_getActiveEditorStateexport, which does not affect people who use Lexical, so per.changeset/README.mda maintainer can apply theskip-changesetlabel instead of adding an entry. - Changeset summary length.
.changeset/block-handles.mdputs the whole API in one 830-character[feature]line..changeset/README.mddescribes a one-line summary followed by paragraphs that are copied into the changelog verbatim, so a short summary (for example[feature] Block handles: short, typo-safe names for showing blocks to a model) with the functions and thehandlesoption in the next paragraph would read better.
| if (start === deleteEnd) { | ||
| $insertText(node, from, inserted); | ||
| } else { | ||
| node.spliceText(from, to - from, inserted); |
There was a problem hiding this comment.
Optional: this splice (and the deletes and $insertText around it) leaves the editor's RangeSelection where it was. In a live editor, a caret before "4%" in Revenue grew 4% in Q3. ends up after "Total revenue" once Total revenue grew 4% in Q3. is applied, and a shorter answer can leave the offset past the end of the text node. Shifting text points after the change by the length difference (points inside the replaced span to its end), or a sentence on the page saying the helper does not move the selection, would cover it.
| '@lexical/html': patch | ||
| --- | ||
|
|
||
| [feature] Block handles for showing blocks to a model: `$getBlockHandle(node)` gives a block a short opaque handle, the shortest prefix (at least 4 characters, like a short git hash) of a 64-bit hash of its id that no other block's hash starts with, so handles stay short and typo-safe whatever the ids look like, `$getBlockHandles()` gives every block's, and `$resolveBlockHandle(handle, {read})` finds the block again (with `read`, the handles the model was shown, the block it named in that read, while it exists), returning `found`, `not-found`, `ambiguous` (with the candidates' handles) or `invalid` (`too-short` or `malformed`) instead of throwing; `BlockIdHTMLExtension` takes a new `handles` option (default `false`) that, with `export` on, writes `data-block-handle` (`BLOCK_HANDLE_ATTRIBUTE`) instead of `data-block-id` |
There was a problem hiding this comment.
Optional: this [feature] line is 830 characters, and the changelog shows it as a single bullet. .changeset/README.md describes a one-line summary followed by paragraphs that are copied verbatim, so something like [feature] Block handles: short, typo-safe names for showing blocks to a model, with the functions and the handles option in the next paragraph, would read better.
Important
exportstays aboolean(ninth commit,392b1a1ab). The HTML option is nowhandles: true, separate fromexportand optional.export: 'handle'and theboolean | 'handle'widening described in older sections below are gone:BlockIdHTMLConfig.exportand itsSignal<boolean>output are unchanged from v0.53.0.Note
Updated in the seventh commit (
938acc73b): handles are now the shortest unique prefix of a 64-bit hash of the id (opaque, for exampleV1StGXR8_Z5jdHi6B-myT→2bNo), not of the id itself. Any id format (random, UUIDv7,@lexical/yderived ids, counters) then gets 4–5 character, typo-safe handles. A string longer than 11 characters is taken as a whole id. Sections below that describe handles as id prefixes predate this change; see "Review follow-ups (seventh commit)".Description
BlockIdExtensiongives blocks 21-character random ids, which is what adocument needs (unique without coordination, safe in JSON, HTML, paste and
collaboration) but costly in a prompt: about 14 o200k tokens per id, read and
copied back by the model for every block.
This adds handles: the shortest prefix of a block's id that no other
block's id starts with, at least 4 characters long, like a short git hash.
V1StGXR8_Z5jdHi6B-myTis shown to the model asV1St. Handles are derivedfrom the ids when they are read and never stored, so nothing in the document
changes and nothing has to persist between an agent's turns.
All of it is additive and
@experimental:@lexical/extension, next to$getBlockById:$getBlockHandle(node): string | null: the node's handle, ornullfora node that
$getBlockByIdwould not find by its id.$getBlockHandles(): Map<id, handle>: every block's handle.$resolveBlockHandle(handle, {read?}): BlockHandleResult, for untrustedinput: a discriminated union instead of a throw,
{status: 'found', node},{status: 'not-found', handle},{status: 'ambiguous', handle, candidates}(the matching blocks' handles, sorted, at most 10) or{status: 'invalid', handle, reason: 'too-short' | 'malformed'}. Withread, the handles the model was shown ($getBlockHandles()at theread), a handle names the block it named in that read:
foundwhilethat block exists,
not-foundonce it is deleted.MIN_BLOCK_HANDLE_LENGTH(4), and the handles inBlockIdOutput.@lexical/html:BlockIdHTMLConfig.exportwidens frombooleantoboolean | 'handle'.'handle'writesdata-block-handle(
BLOCK_HANDLE_ATTRIBUTE) instead ofdata-block-id. Import reads onlydata-block-id, so a handle is never taken for an id.Rules:
its own handle. An exact id always wins: with
b1andb12,b1findsb1. A whole id is accepted wherever a handle is.so a handle read while another id shared more of its prefix keeps working
after that block is deleted. A prefix shorter than 4 characters is
invalid('too-short') unless it is a whole id.$getBlockById: the indexedids of the last committed state, all at once (sort, then compare each id
with its neighbours, O(n log n)) and cached until a commit changes the set
of ids. While the extension is disabled, each call scans the document, as
$getBlockByIddoes.local changes, a handle from an earlier read resolves as
not-foundorambiguous; a prefix freed by a deletion can be taken by a new block(about one chance in 16.7 million per new random id), and the docs show
how to check the id read where an edit must reach exactly that block.
Nothing released changes: stored ids, the id format,
$getBlockById(stillan exact match),
export: true/falseand HTML import behave as before.TypeScript and Flow types are updated together.
Discovery is in the JSDoc of
$getBlockById,BlockIdExtensionandBlockIdHTMLConfig.export, and in a new "Showing blocks to a model" sectionof the Block IDs page: the round trip, the four statuses and what to tell
the model for each, the collaboration caveat, handles in HTML, and the token
cost.
Follow-up (second commit)
Follow-up from review of the block handles change:
Handles resolved against the wrong state. In the documented
@lexical/yserver pattern (editYStateon the state the model read,then merge), resolving a handle inside the edit always found its block, so
not-foundandambiguousnever fired for changes made since the read.Resolving it against the latest state alone was not enough either: with
ids
b123andb123456, deletingb123made the handleb123name thesurviving
b123456, and a prefix freed by a deletion could be taken by ablock added since.
$resolveBlockHandle(handle, {read})takes the handles the model was shown(the map
$getBlockHandles()returned at the read). The handle then namesthe block it named in that read:
foundwhile that block still exists inthe state being read,
not-foundonce it is deleted, and neverambiguousbecause of a block added since. Typos fail against the read's ids. The map
is read as it is on each call, with no cached index. The docs, the JSDoc
and the
@lexical/yserver recipe now do this: resolve in areadYStateofthe stored state with
{read: shown}, then apply the edit to the read byid and merge.
y-server.mdpoints at it.Edits replaced the block. The examples applied the model's edit with
result.node.replace(...). That re-creates the block: with@lexical/y,text a collaborator typed in it meanwhile no longer stays where they typed
it ("Revenue grew about 6%." merged as "about Revenue grew 6%."). The
examples now change the block in place. The Block IDs page defines
$setBlockText(block, text), which changes only the characters thatdiffer in the block's own text nodes and returns
falsefor a block whosetext is not just its text nodes (line breaks, nested blocks).
Too-short handles were reported as deleted blocks. A string shorter
than
MIN_BLOCK_HANDLE_LENGTHthat is not a whole id wasnot-found. It isnow
{status: 'invalid', reason: 'too-short'}, and everyinvalidresultcarries
reason: 'too-short' | 'malformed', so the suggested feedback cansay what is wrong.
TypeScript and Flow are updated together (
BlockHandleOptions, thereasonfield), and so is the changeset. The API has not been released yet.Follow-up (third commit)
Two examples that review of this PR found unsafe:
The
BlockIdHTMLConfig.exportJSDoc resolved a handle from exportedHTML without
{read}, then removed the node. Here is how that goes wrong:9xQe1111with the handle9xQe.9xQe2222takes the prefix.9xQe2222, a block the model never saw.The example now captures
$getBlockHandles()in the same read as$generateHtmlFromNodes, resolves the answer with$resolveBlockHandle(handle, {read: shown}), and edits the block in placewith
$setBlockTextinstead of removing it. The "Handles in HTML" sectionof the Block IDs page shows the same capture.
$setBlockTexton the Block IDs page used text equality as itsstructure check: the block's concatenated text-node text equal to its
getTextContent(). That does not prove the block holds only text. A listwith one item and a paragraph with an inline decorator that has no text
both passed, so they were rewritten instead of refused. The helper now
checks the block's shape before changing anything: every child must be a
text node, or an inline element (such as a link) that holds only text, at
any depth. Line breaks, decorators (even ones without text), nested
blocks and occupied named slots (at any level) are refused.
The test copy of the helper moves to
packages/lexical-html/src/__tests__/utils/setBlockText.ts, shared by thedocs tests and the HTML tests. A test now checks that it, the docs page's
copy and the
@lexical/ytest's copy are identical.Changeset
.changeset/block-handles.md:[feature]for@lexical/extensionand@lexical/html.Test plan
New tests:
BlockIdExtension.handles.test.ts(the shortest-prefix rule andthe 4-character minimum, ids shorter than 4, ids that prefix others, counter
and
b+ base36 ids, exact ids, invalid and non-string input, deletedblocks, a new block sharing the prefix making an old handle ambiguous with
the right candidates and longer next handles, blocks named in the same
update, earlier states, duplicates awaiting repair, the disabled extension,
headless and JSON-only use),
BlockIdExtension.handles.property.test.ts(random documents with random and adversarial ids under random edits: every
handle equals the brute-force one and resolves to its own block, and no
one-character typo resolves to another block unless it is that block's id
or starts with its current handle),
BlockIdExtension.handlesCache.test.ts(one handle build per set of ids), new
export: 'handle'tests inBlockIdHTMLExtension.test.ts(export,true/falseunchanged, importignores
data-block-handle, round trip of export, edit by handle andresolve), the docs examples in
BlockIdDocs.test.ts, and an opt-in perftest.
Before
On
main, with the new tests:(The
export: true/falseand import tests pass onmaintoo: thatbehaviour is unchanged.)
After
Performance, 10,000 blocks of random ids (Apple silicon, Node 24, medians of
15):
Token cost, o200k (
gpt-tokenizer, cross-checked withjs-tiktoken, whichagreed on all 20,000 ids), 20 documents of 1,000 random ids:
data-block-…="…"Follow-up (second commit)
Before
The new tests on the previous commit of this PR:
(In the
@lexical/ycase the third edit, meant for the deletedb123, wasapplied to
b123456.)After
Follow-up (third commit)
BlockIdHTMLExtension.test.ts:pinned resolve, an in-place edit that keeps the same node.
finds
9xQe2222, the pinned resolve isnot-found, and9xQe2222isleft as it was.
BlockIdDocs.test.ts:whose slot holds a line break, and an inline link that hosts a slot are
all refused, and the editor state is unchanged.
Before
The new refusal test against the previous helper:
(Line 512 is
expect($setBlockText(oneItem, 'rewritten')).toBe(false).) Theprevious JSDoc's unpinned resolve finds
9xQe2222. The new HTML regressionasserts this before it shows that the pinned resolve is
not-found.After
Review follow-ups (fourth to sixth commits)
a9cee6fa7: the block handle property test is rewritten with fast-check. The default run is small and prints nothing.BLOCK_HANDLE_PROPERTY_RUNSruns more documents, and the typo-rate measurement runs only withBLOCK_HANDLE_TYPO_DOCSset.fast-checkis added to the root devDependencies.2652eec19:BlockIdHTMLExtension's export override usesdisabledForSession, so nothing runs per element whileexportis off. Changeset:.changeset/block-id-html-export-session.md.2a68d09dc:generateId's JSDoc and the Block IDs page note that short handles need ids with random first characters, so timestamp-first ids such as UUIDv7 are a poor fit.pnpm run ci-checkpasses. The@lexical/extension,@lexical/htmlandlexicalunit suites plus the@lexical/yhandle test: 3017 passed, 3 skipped. The tree-shaking source test passes, a 1,000-run property check passes, and the website builds.Review follow-ups (seventh commit)
This addresses the code-review bot's review of this PR, plus a gap the
review did not cover.
Handles were only as good as the ids' entropy. A handle was the
shortest unique prefix of the id itself. Ids that share their first
characters therefore got long handles, and when those ids are sequential, a
one-character typo named a neighbouring real block. That is the risk handles
were meant to remove. It hit
@lexical/y's derived ids(
y<client>_<clock>, which every line written by a client withoutBlockIdExtensionshows) and UUIDv7 ids. On 1,000-block documents:@lexical/yderivedA handle is now the shortest prefix, at least 4 characters long, of a
64-bit hash of the id (
handleKey: a cyrb53-style mix in base64url, 11characters, most significant bits first) that no other id's hash starts
with. Handles are opaque:
V1StGXR8_Z5jdHi6B-myTshows as2bNo.Resolution compares a handle against the hashes of the document's ids, and
with
{read}, against those of the read. A string longer than 11characters (longer than any handle) is looked up as a whole id, so the
default 21-character ids still work. A shorter whole id can only be passed
by its handle. Hash collisions only make handles longer, and ambiguity is
already reported.
@lexical/yneeds no change. The docs, the JSDoc, thechangeset and the
generateIdnote are updated: any id format now getsshort handles.
$setBlockTextthrew on a block with a tab.TabNode.spliceTextthrows for anything but a no-op. The helper on the Block IDs page, and its
test copies, now accept only
isSimpleText()text nodes, so tabs, tokens,segmented text and text node subclasses are refused with
false.Text added at a link's edge went into the link. A pure insert at the
edge of an inline element that refuses text there
(
canInsertTextBefore/canInsertTextAfteris false, as forLinkNode)now goes beside the outermost such element (a link inside a mark: beside
the mark), as typing does: into the adjacent simple text node, or into a new
text node.
Handles cost quadratic time while
BlockIdExtensionwas disabled.Each
$getBlockHandlecall scanned the document and rebuilt the table.While disabled, handles now come from one scan of the last committed state,
cached until the next commit, and lookups are verified against the state
being read: a repeated id's document-first live holder, as
$getBlockByIdfinds it. This matches enabled mode, where they come fromthe index of the last committed state. Reading the handle of each of 2,000 blocks drops
from seconds (the review measured 5.5 s for an
export: 'handle'HTMLexport) to about 2 ms.
Changeset: one line saying that
exportwidens toboolean | 'handle'.Test plan (seventh commit)
BlockIdExtension.handles.test.ts, rewritten for hashed handles:handleKeyvalues, brute-force handles, and 4–5 character handles forcounter, derived, short and UUIDv7 ids;
with ids picked for colliding hashes;
removed or reordered while disabled.
BlockHandles.test.ts(@lexical/y):BlockIdExtensionand read with it showderived ids. Their handles are 4–5 characters, all resolve, and typos
stay safe.
BlockIdDocs.test.ts: tab and token refusal, link-edge inserts (also alink inside a mark), and the helper copies kept in sync, besides the
updated examples.
BlockIdExtension.handles.property.test.ts: brute-force handles over hashkeys. The gated measurement covers random, derived and UUIDv7 ids.
Before
The new tests on the previous head of this PR:
After
An
export: 'handle'HTML export of 2,000 blocks (jsdom): 39 ms enabledand 33 ms disabled. The review measured the disabled case at about 5.5 s
before.
Review follow-up (eighth commit,
c0da46ec2)With
BlockIdExtensiondisabled, a repeated id persists, because no repairruns. Reading the handle of every block (one
$getBlockHandleper block, asan
export: 'handle'HTML export does) was then quadratic: each call lookedup the id, and the lookup checked every holder of the id to find the
document-first live one. With 40, 80 and 160 blocks sharing one id, a read
cost 1,680, 6,560 and 25,920
$isIdentifiedBlockcalls, against 120, 240 and480 for unique ids.
The lookup behind handles is now remembered per id for each frozen state it
is asked about, so it runs once per id in a read of a committed state. While
the active state is not frozen (an update), every call still looks up the
id. The memo lives in the handle table, which is rebuilt when the index
changes (enabled), or in the scan of each committed state (disabled), so it
never outlives what it was computed from. It needs the active EditorState,
which
lexicaldoes not expose, solexicalgains an internal export,INTERNAL_getActiveEditorState(likeINTERNAL_$isBlock, it is not part ofthe public types).
A new call-counting test (
BlockIdExtension.handlesCost.test.ts) fails without the fix (25,760 calls against a linear bound of 14,260) and passes with it. The review's repro now counts 120, 240 and 480 calls for 40, 80 and 160 holders of one id.Review follow-up (ninth commit,
392b1a1ab)Review question:
BlockIdHTMLConfig.exportandBlockIdHTMLExtension'sexportoutput signal shipped asbooleanin v0.53.0. This PR had widenedboth to
boolean | 'handle', so code typed against the released types (forexample
const on: boolean = output.export.value, orconst s: Signal<boolean> = output.export) stopped type-checking.exportnow stays exactly as it shipped, abooleanwith aSignal<boolean>output, in both TypeScript and Flow. Handles get their ownoption instead.
handles?: boolean(defaultfalse) makes the export writedata-block-handleinstead ofdata-block-idwhileexportis on; on itsown, it does nothing. It is optional in the config type, so a whole
BlockIdHTMLConfigwritten for v0.53.0 still type-checks, and its outputsignal is a
Signal<boolean>. The export override is still left out ofexport sessions while
exportis off.The docs, JSDoc and changeset now describe
handles: true, and thechangeset no longer mentions a widened type. This commit also takes the
review's smaller suggestions:
the same with the extension disabled;
to be kept between an agent's turns";
check.