Repository navigation
perf(dart/core): check composition against what an update changes - #3345
Conversation
Under a validation config that defers the graph checks, a batch whose graph check started from what the check before found is checked against only the composition edges it changes, as the Python and TypeScript cores do with their holding index: its own components against all their children, and each component outside the batch against the components of the batch it holds. The shortcut applies while only the default catalog declares constraints, and a batch it rejects is checked again against the whole surface, so the errors raised are unchanged. Adds tests comparing the incremental and whole-surface checks, and a performance-tagged timing test for a container whose catalog declares allowedChildren, receiving one child per message. Signed-off-by: Nan Yu <nanyu@google.com>
There was a problem hiding this comment.
Code Review
This pull request optimizes the composition constraint checks in MessageProcessor under a deferred validation config. Instead of checking the entire graph, it now incrementally checks only the edges changed by an update batch, significantly improving performance when children are added one by one. The changes include refactoring checkCompositionConstraints to extract helper validation methods, introducing compositionHoldsAfter to verify changed edges, updating checkComponentsUpdate to return the graph bound, and moving timing test utilities to a shared support file. Comprehensive unit and performance tests have been added to verify correctness and scaling. I have no feedback to provide as there are no review comments.
Main now has #3345, which makes the Dart composition check incremental under a deferring validation config. Take its Dart core files in place of this branch's own fix, which did the same. The shared composition timing case passes with it.
…mposition cases Adds the allowedChildren case to conformance/core/message_processor_scaling.yaml (a Column whose catalog allows only Text children, receiving one child per message under the relaxed config, at 500 and 2000 children) with its catalog fixture, so the Python, TypeScript and Dart cores time the same payload. It replaces tests/timing/composition_scaling_test.mjs, which timed the same scenario. Ports the named composition cases of the Dart core's incremental check (#3345) as tests comparing the incremental and whole-surface checks, and adds a random comparison over two catalogs that both declare constraints on the same type names. Signed-off-by: Nan Yu <nanyu@google.com>
Under a deferred validation config, a batch the incremental composition check rejects is now checked again against the whole surface, which decides and raises the error it reports, as the Dart (#3345) and TypeScript cores do. The error no longer depends on the incremental check visiting components in surface order, so it no longer sorts them or rereads a holder's other children after a failure. Adds the allowedChildren case to conformance/core/message_processor_scaling.yaml (a Column whose catalog allows only Text children, receiving one child per message under the relaxed config, at 500 and 2000 children) with its catalog fixture, so the Python, TypeScript and Dart cores time the same payload. It replaces test_children_arriving_one_per_message_scale_linearly, which timed the same scenario. Ports the named composition cases of the Dart core's incremental check as tests comparing the incremental and whole-surface checks, and adds a random comparison over two catalogs that both declare constraints on the same type names. Signed-off-by: Nan Yu <nanyu@google.com>
What was quadratic
Under a
ValidationConfigthat defers the graph checks (dangling references allowed, plus orphans or a missing root allowed, as an agent's renderer state configures it), the Dart core already checks eachupdateComponentsagainst what it changes, throughComponentGraphBound(#3314, renamed in #3328). The composition check did not. When any catalog of the surface declaresallowedParentsorallowedChildren,MessageProcessorserialized every stored component and checked every edge of the surface on each message. AColumnwhose catalog declaresallowedChildren, listing all its children up front and then receiving one child per message, cost time quadratic in the number of children.The fix
When the graph check started from what the check before found, the surface passed the composition check of the batch before, so only the edges this batch changes need checking. This follows the
holdingsplit of the Python (#3326) and TypeScript (#3327) cores:The Dart composition check works edge by edge, with no check that a component has a parent, so a child that a batch component referenced before, or references now, needs no separate check of its parents: the edges that remain are checked with the batch component, and a dropped edge cannot fail.
The shortcut is taken only when the default catalog is the only one declaring constraints. The constraints are then that catalog's, and every earlier check applied them first, so the surface passed them. Otherwise, or when the graph check walked the whole graph, the whole surface is checked as before. A batch the shortcut rejects is checked again against the whole surface, so the error raised, its message and its path are those of the whole-surface check. Strict configs are unaffected.
checkComponentsUpdatenow returns the bound when it used it, andComponentGraphBoundgainsparentsOf. The edge and root checks ofcheckCompositionConstraintsmove into helpers shared with the newcompositionHoldsAfter.Tests
composition_constraints_test.dartsends each batch to two processors under the same deferred config (missing root allowed, and orphans allowed). The second has a component touched after each batch, so it always checks the whole surface. The two must raise the same error (type, code, path, message) and hold the same components. Cases: a child that arrives after its container, a container that arrives after its child, a child added to a container in a later message, a container rewritten to drop a disallowed child, a held child whose type changes, a holder that a child'sallowedParentsomits, a root the surface may not hold, and 200 random sequences of 20 batches per config. Skipping the holder edges, or the batch components' checks, makes several of these fail.composition_scaling_test.dartis aperformance-tagged test, run bydart test -P performance, timing theallowedChildrenColumncase at 500 and 2000 children with a maximum ratio of 8. The timing helpers move from the shared scaling suite's test intotest/support/timing.dartso both use them.Median times (5 runs each, 3 runs of the test):
The case is not added to the shared
conformance/core/message_processor_scaling.yaml, since the Python and TypeScript cores on main would fail it. Once #3326, #3327 and this PR land, it can move into the shared suite.