Skip to content

perf(web_core): check updateComponents incrementally under deferred graph checks - #3327

Open
nan-yu wants to merge 12 commits into
mainfrom
perf/ts-core-incremental-component-checks
Open

nan-yu wants to merge 12 commits into
mainfrom
perf/ts-core-incremental-component-checks

Conversation

@nan-yu

@nan-yu nan-yu commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Part of #3315

Under a ValidationConfig that defers the graph checks (it allows dangling references, and orphans or a missing root, as the agent's RendererState configures it), the web core MessageProcessor copied every component of the surface into a candidate model and checked the whole surface on every updateComponents. A surface sent as n messages of one component each therefore took time that grew with n squared. This PR checks each message against the components it changes, following the Dart approach from #3314, and adds a runner for the conformance timing cases so the TypeScript packages can check this.

It implements requirement 1 of "Cost of a response" in the consolidated blueprint (#3283, a2ui_agent.blueprint.md) and item 4 of the agent use of MessageProcessor in a2ui_core.blueprint.md. Once this merges, the TypeScript half of entry 2 of "Where the SDKs differ" can be removed.

What changed

  1. component-graph-bound.ts is a port of the Dart ComponentGraphBound (component_graph_bound.dart). It records each component's references, the components that reference it, and an upper bound on the longest chain below it. admits(batch) updates the record with a batch and raises the bounds of the components above the ones the batch changed. If no bound passes maxDepth, the graph has no cycle and no chain that is too deep. A self-reference, or a bound past the limit, means the record cannot show that the batch passes.
  2. validation-helpers.ts extracts the component graph, candidate topology, and composition constraint validation helpers out of message-processor.ts. Under a deferring config, validateComponentGraph checks a batch against the record left by the previous message. The cycle and depth checks go through admits. The allowedParents and allowedChildren checks run only on the components whose parents, children or types the batch changes (compositionHolds). A component of the batch is checked against its parents and all of its children. A child it references before or after the batch, outside the batch, is checked against its parents only. A component outside the batch that references components of the batch is checked against only those children, as the Python core's holding index does. A container whose catalog declares allowedChildren, receiving one child per message, is then checked against one child per message rather than all of them. Duplicate ids and the per-component catalog checks run on the batch as before. The record is saved only after the batch has been applied.
  3. validateComponentGraph falls back to the whole-surface check in four cases: there is no record yet, the components changed outside the processor since the record was saved, the record cannot show that the batch passes, or a component in the batch has no type or comes from a catalog that declares no components (the topology and composition checks read references differently there). A message that fails therefore reports the same error as on main, and a message is still checked whole before it changes anything. A ComponentsWatcher counts changes through the components model's onCreated, onDeleted and onUpdated events. It is attached only under a deferring config.
  4. Strict configs and processors without a config run the same checks as before. They add no listeners to the components model, and a2ui-lit-element.test.ts checks this. One shortcut applies to every config: the composition check returns early when no catalog that the surface's components come from declares allowedParents or allowedChildren, because the check would find nothing in that case. To find a component added from outside the processor with a catalog of its own, the early return still looks at each component's catalog. That costs one set lookup per component, much less than the reference extraction it skips.
  5. typescript/core/web_core/tests/timing/timing.mjs runs the measure_growth and measure_overhead actions of conformance_schema.json. It reads a suite, writes a PayloadTemplate at a size, runs each operation once untimed and then five times in turn (collecting garbage first under --expose-gc), and compares the medians against max_ratio.
  6. web_core runs conformance/core/message_processor_scaling.yaml with yarn test:timing. yarn test does not include it. A new ci-timing job in web_ci.yml runs test:timing for each workspace in its matrix, which holds @a2ui/web_core for now. The default conformance harness still skips measure_growth, with a reason that points to test:timing. The composition case of the suite is main's test_scaling_process_single_component_messages_composition_v10, added by test(conformance): cover reference reading and composition cost in the core suites #3344 with its catalog composition_catalog_v1_0.json: a Column names n children, which then arrive one per message, at n=1000 and 4000. It is the scenario of the allowedChildren case this PR first added, so after merging main that case and its allowed_children_catalog_v1_0.json fixture are dropped, the same way in perf(python/core): check updateComponents incrementally under deferred graph checks #3326. The shared case replaces the web-core-only composition_scaling_test.mjs, which timed the same scenario. Python and TypeScript both pass it (TypeScript ratio about 2.5 at 1000 and 4000).

Numbers

These were measured on the same machine with Node 22.22.1, using the median of five runs.

test_scaling_process_single_component_messages_relaxed_v10 (yarn test:timing), at 200 and 800:

  • before: 136.6 ms and 1890.9 ms, ratio 13.84 (fails)
  • after: 12.3 ms and 52.7 ms, ratio 4.29

test_scaling_process_single_component_messages_v10 (no config), at 200 and 800:

  • before: 54.0 ms and 571.6 ms, ratio 10.58 (fails)
  • after: 8.9 ms and 41.8 ms, ratio 4.68

Before this change the no-config case was quadratic too, because the composition check walked every component on every message even when no catalog declares any constraints.

test_scaling_process_single_component_messages_composition_v10 (a Column whose catalog declares allowedChildren receives one child per message, relaxed config), at 1000 and 4000, as measured in review. "Before" forces the whole-surface composition check on every message:

  • before: ratio 17.4 (fails)
  • after: ratio 3.8

n updateComponents messages of one Text each on one surface, after a root Column that names all n children, with each message passed to its own processMessages call, at n = 100, 200 and 400:

  • no config, before: 21.4 ms, 65.4 ms and 169.3 ms
  • no config, after: 9.0 ms, 13.5 ms and 22.9 ms
  • relaxed, before: 42.2 ms, 135.4 ms and 420.2 ms
  • relaxed, after: 8.3 ms, 13.1 ms and 23.5 ms
  • agent config (RendererState: dangling references, missing root and orphans allowed, unknown elements rejected), before: 39.1 ms, 125.6 ms and 406.9 ms
  • agent config, after: 7.1 ms, 13.3 ms and 18.9 ms

Testing

  1. component-graph-bound.test.ts covers the bound itself and processor cases where only the incremental check would catch the error: a cycle closed by a later message, a self-reference, a chain past maxDepth built over several messages, a reference added to the surface outside the processor, a child placed under a disallowed parent, a parent that changes type, and a component left without the parent its type requires. It also runs a differential test on random message sequences, under three deferring configs and with and without composition constraints. Each sequence is checked against a processor whose record is dropped before every message, and the two must accept and reject the same messages with the same errors and end up holding the same components. I checked that removing the parents or the previous children from the set of affected components makes these tests fail.
    A second group ports the named composition cases of the Dart core's incremental check (perf(dart/core): check composition against what an update changes #3345), on Dart's catalog, under its two configs: a child that arrives after its container, a container that arrives after its child, a child added to a container in a later message, a container rewritten to drop a child it disallows, a held child whose type changes, a holder its child's allowedParents omits, and a root the surface may not hold. Each step is sent to the incremental processor and to one that checks the whole surface, and both must give the same error class, code and message and hold the same components and catalogs. A random comparison runs over two catalogs that declare different constraints on the same type names, with each component drawn from either. Making compositionHolds always pass, or dropping its holding or moved checks, makes these tests fail (23, 9 and 9 failures).
  2. yarn test (1014 tests at d768f18, conformance harness included), yarn test:timing (4 passed), yarn lint (no errors) and yarn format:check in typescript/core/web_core.
  3. The timing runner's own unit tests, which yarn test runs, advance a fake clock passed as compareMedianTimes's now option instead of timing real work, so their times are exact. They passed 30 of 30 runs with 72 busy-loop processes on 24 cores, and 30 of 30 idle.
  4. yarn test in @a2ui/markdown-it, @a2ui/angular and @a2ui/agent, and yarn test:unit in @a2ui/lit and @a2ui/react. After the allowedChildren change, and again at d768f18, I re-ran @a2ui/agent only (764 passed at d768f18).

…:timing

Add @a2ui/conformance-timing, a private workspace that runs the
measure_growth and measure_overhead cases of the conformance suites:
it reads a suite, writes a case's payload at a size, times each
operation after one untimed run and compares the medians. A package
supplies the operation to time, so typescript/agents/a2ui_agent can
run its scaling suites through it too.

web_core runs conformance/core/message_processor_scaling.yaml with
yarn test:timing, which yarn test leaves out, and the Web CI workflow
runs it in a job of its own. Both cases fail on main: processing a
surface sent as one message per component takes time that grows with
the square of the surface.

Signed-off-by: Nan Yu <nanyu@google.com>
…raph checks

Under a validation config that allows dangling references and orphans
or a missing root, as the agent's RendererState configures it,
MessageProcessor no longer copies every component of the surface into
a candidate model and checks the whole surface on each updateComponents.
It keeps what the last check found (each component's references, its
parents, and a bound on the longest chain below it), as the Dart
SurfaceComponentsModel does since #3314, and checks cycles, depth and
the composition constraints of the components the message affects.
When the record cannot show the message passes, or the components
changed outside the processor, it checks the whole surface as before,
so a failing message reports the same error. Strict configs and no
config keep the whole-surface checks; the composition check returns
early when no catalog involved declares allowedParents or
allowedChildren, which it would not check anyway.

Signed-off-by: Nan Yu <nanyu@google.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new private workspace package, @a2ui/conformance-timing, to run performance and timing conformance tests for TypeScript packages. It optimizes the MessageProcessor to perform incremental topology and composition checks on updated components rather than re-validating the entire surface, reducing the processing time from quadratic to linear under deferred validation configurations. To support this, ComponentGraphBound and ComponentsWatcher are introduced. Feedback suggests optimizing the DFS implementation in ComponentGraphBound.of to avoid O(N^2) worst-case child lookups by tracking the child index on the stack, achieving linear O(V + E) time complexity, and replacing a non-null assertion with a safer nullish coalescing operator.

Comment thread typescript/core/web_core/src/state/component-graph-bound.ts Outdated
publish_npm added every internal workspace dependency, including
devDependencies, to the publish set. web_core now has a devDependency on
the private @a2ui/conformance-timing workspace, so the script tried to
version-check and publish it. Private packages are never published, so
skip them when they are development-only dependencies and fail clearly
if a published package needs one at runtime.

Signed-off-by: Nan Yu <nanyu@google.com>
@nan-yu
nan-yu requested a review from polina-c October 20, 2026 16:37
nan-yu added a commit that referenced this pull request Oct 20, 2026
…te where the SDKs differ

- a2ui_core: RendererState's atomic application undoes a rejected message from
  an undo log rather than rolling back to a copy. Requirement 4 now states only
  what a processor must do under a ValidationConfig that defers the graph
  checks, with no SDK status.
- a2ui_core: the agent example shows the agent using RendererState (apply, then
  checkTopology) and, separately, the MessageProcessor calls RendererState makes
  inside, since an agent does not call MessageProcessor itself.
- a2ui_agent: "Where the SDKs differ" assumes the pending Python (#3326,
  #3268-#3271) and TypeScript (#3327, #3278, #3279) pull requests. Both meet the
  three requirements of "Cost of a response"; the remaining entries are Dart's,
  plus Python being the only SDK with a v0.8 state.

Signed-off-by: Nan Yu <nanyu@google.com>
nan-yu added a commit that referenced this pull request Oct 20, 2026
…se-stream

#3327 moved the shared timing runner from the @a2ui/conformance-timing
workspace into typescript/core/web_core/tests/timing/timing.mjs. The agent's
timing tests now import it from there (a test-only import), and the agent
drops its devDependency on the removed workspace.
@nan-yu
nan-yu added this pull request to stack #3342 October 20, 2026 17:57

@polina-c polina-c left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked out the branch and ran yarn test in web_core (993 pass) and yarn test:timing (ratios 4.09 and 4.73). I also tried the incremental path on cases the timing suite doesn't cover.

The bound and the fallback to the whole-surface check look right, and the differential tests cover what I would have asked for. One case is still quadratic: when a catalog declares allowedChildren, a message that adds a child to a large container reads every child of that container. The inline comment has the numbers. Python #3326 is linear on the same case.

This PR and #3326 both rewrite the same sentence of conformance/README.md, and each names the other SDK as the one with no runner. Whichever lands second should drop that example.

🤖 Agent-generated

Comment thread typescript/core/web_core/src/state/validation-helpers.ts Outdated
…eferred graph checks

A component outside the batch that references a component of the batch
is now checked against only the children of the batch it references, as
the Python core does with its holding index, so a container whose
catalog declares allowedChildren, receiving one child per message, is
no longer checked against all its children each time. Adds an SDK-local
timing case for it under yarn test:timing.

docs(conformance): end the timing runner sentence without naming the
SDK that has no runner, so #3326 and #3327 make the same change there.
nan-yu added a commit that referenced this pull request Oct 20, 2026
Restores the known component ids in validate_component_integrity,
analyze_topology and SurfaceComponentsModel.get_child_references, so the
legacy v0.8 heuristic reads the same references as on main, and restores
_AnyId for ComponentGraphBound, whose topology edges over-approximate
the ones the whole-surface check reads. Composition keeps reading the
references without known ids, as on main, in a map of its own. Keeps the
cleanup of empty parent sets from 7980442.

Drops the reference_map branch that read an array of ComponentId items
as a list of references; it changed the reference map under every
config and is not needed for the incremental check.

Adds tests for the v0.8 usageHint cases from review, a Slots type to the
differential test, and a performance test for a container with
allowedChildren receiving one child per message.

docs(conformance): end the timing runner sentence without naming the
SDK that has no runner, so #3326 and #3327 make the same change there.
…mposition cases

Adds the allowedChildren case to conformance/core/message_processor_scaling.yaml
(a Column whose catalog allows only Text children, receiving one child per
message under the relaxed config, at 500 and 2000 children) with its catalog
fixture, so the Python, TypeScript and Dart cores time the same payload. It
replaces tests/timing/composition_scaling_test.mjs, which timed the same
scenario.

Ports the named composition cases of the Dart core's incremental check
(#3345) as tests comparing the incremental and whole-surface checks, and
adds a random comparison over two catalogs that both declare constraints
on the same type names.

Signed-off-by: Nan Yu <nanyu@google.com>
@nan-yu

nan-yu commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks for the review. The allowedChildren composition check is now linear: 986aa41 ported Python's holding split (2000/8000 ratio 13.5 → 3.0). In 177b3ad the case you suggested is in core/message_processor_scaling.yaml with its fixture, at 500/2000, and the TypeScript, Python (#3326) and Dart (main, after #3345) cores pass it (ratios 2.70, 3.72 and 3.96; 13.27, 11.40 and 14.49 with the whole-surface check forced on every message). I also ported Dart's named composition cases as differential tests. About conformance/README.md: this PR and #3326 each rewrite the trailing sentence of the timing paragraph, one for yarn test:timing and one for pytest -m performance, so whichever merges second will be rebased to keep both sentences.

Signed-off-by: Nan Yu <nanyu@google.com>

# Conflicts:
#	conformance/core/message_processor_scaling.yaml
@nan-yu
nan-yu requested a review from polina-c October 20, 2026 20:46

@polina-c polina-c left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pulled 3a3a4cb and ran yarn test (1010 pass), yarn test:timing (4 pass), yarn lint (no errors) and yarn format:check in web_core, and yarn test in @a2ui/agent (764 pass).

The allowedChildren case from my last review is fixed. On test_scaling_process_single_component_messages_composition_v10 at 1000 and 4000, forcing the whole-surface check before every message gives a ratio of 17.4. This branch gives 3.8.

I also compared this branch with a processor that drops its record before every message, on random input. It used two catalogs with different constraints on the same type names, batches of up to three components, and four deferring configs. Every so often a component was added, removed or changed outside the processor. Over 144,000 steps, about 70% of them with a recorded bound, the errors and the components held matched every time.

One thing to fix before merge: the runner's own tests in tests/timing/timing.test.mjs time real spins inside yarn test, and they fail when the machine is busy (inline). The other two comments are small doc fixes.

Two things in the PR description:

  • "The shared case still lets it replaces the web-core-only composition_scaling_test.mjs" is missing words.

  • The "Numbers" tables would be easier to read as a line naming the measurement and sizes, then a before/after list. For example:

    test_scaling_process_single_component_messages_relaxed_v10, median of five runs at 200 and 800:

    • before: 136.6 ms and 1890.9 ms, ratio 13.84 (fails)
    • after: 12.3 ms and 52.7 ms, ratio 4.29

🤖 Agent-generated

Comment thread typescript/core/web_core/tests/timing/timing.test.mjs
Comment thread typescript/core/web_core/README.md Outdated
Comment thread typescript/core/web_core/CHANGELOG.md Outdated
compareMedianTimes takes the clock as a `now` option, defaulting to
performance.now(), and measureGrowth and measureOverhead pass it through.
The runner's unit tests, which `yarn test` runs alongside every other
workspace's tests, advance a fake clock instead of spinning, so their
times are exact under any load. Adds tests for the median and for
measureOverhead.

Updates the README and CHANGELOG now that tests/timing holds no timing
cases of the web core's own.

Signed-off-by: Nan Yu <nanyu@google.com>
Signed-off-by: Nan Yu <nanyu@google.com>

# Conflicts:
#	typescript/core/web_core/CHANGELOG.md
@nan-yu
nan-yu requested a review from polina-c October 20, 2026 23:12

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants