Repository navigation
test(conformance): cover reference reading and composition cost in the core suites - #3344
Merged
Merged
Conversation
…e core suites Add v0.8 cases for a plain string property that names a component id, v0.9 and v1.0 cases for an array property whose items are ComponentId, and a timing case for processing messages with a catalog that declares allowedParents and allowedChildren. List the cases the Python and Dart cores fail on main as known gaps.
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces several new conformance tests across different protocol versions (v0.8, v0.9, and v1.0) to validate topology rules, specifically ensuring that plain string properties are not incorrectly treated as child references and that array properties containing ComponentId items are properly validated. Additionally, a new scaling performance test and its corresponding test catalog have been added to measure message processing under composition constraints. Finally, the Dart and Python test suites have been updated to skip these new tests where they are currently known gaps. As there are no review comments, I have no additional feedback to provide.
…hanges MessageProcessor checked every reference on the surface against the composition constraints for each updateComponents, so a surface sent as n messages took time in proportion to n squared once a catalog declared allowedParents or allowedChildren. It now keeps the references the last check passed and checks a batch against the root, the references from its components and the references into them, falling back to the whole-surface check when the components changed outside the processor, a batch moves a component to another catalog, or the reference fields or constraints change. The composition timing case now passes.
Main now has #3345, which makes the Dart composition check incremental under a deferring validation config. Take its Dart core files in place of this branch's own fix, which did the same. The shared composition timing case passes with it.
polina-c
marked this pull request as ready for review
October 20, 2026 19:46
nan-yu
approved these changes
Oct 20, 2026
nan-yu
added a commit
that referenced
this pull request
Oct 20, 2026
#3344 added a dangling reference case and a reachable child case for arrays of ComponentId to the v0.9 and v1.0 validator suites. Drop this branch's duplicates of those two and keep its partial dangling and circular reference cases. Python now passes main's dangling cases, so they leave its skip list, and Kotlin passes main's v0.9 dangling case, which checks only the missing id. Signed-off-by: Nan Yu <nanyu@google.com>
This was referenced Oct 20, 2026
nan-yu
added a commit
that referenced
this pull request
Oct 20, 2026
… test entry covers the fix - Move the partial-dangling and circular ComponentId array cases to just after #3344's dangling and child-reachable cases in validator_v0_9.yaml and validator_v1_0.yaml, so the four cases sit together. The cases are unchanged. - test_reference_map.py: say that `external` passed before the fix (only `#/` pointers are resolved) and that `slots` is the shape a loaded catalog has, which the fix covers. Signed-off-by: Nan Yu <nanyu@google.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds conformance cases for three behaviours that came up in the reviews of #3326 and #3327. No suite covers them today. The Dart core passes the new timing case since #3345. Each SDK's results are below, and the cases an SDK still fails on
mainare listed as known gaps in its harness.The cases
core/validator_v0_8.yamlgets two v0.8 counterparts oftest_v09_topology_plain_string_property_not_treated_as_child_ref. In both, aTexthasusageHint: h1and another component has the idh1.test_v08_topology_plain_string_property_not_treated_as_child_refexpects the error for the orphanh1.test_v08_topology_plain_string_property_does_not_close_cycleexpectsroot -> h1 -> titleto pass. IfusageHintwere read as a reference, it would close a cycle.core/validator_v0_9.yamlandcore/validator_v1_0.yamleach get two cases. Both use a customStackwhoseslotsproperty is an array ofComponentId. The property name matches no naming convention.test_v09_topology_component_id_array_dangling_reference_errorand its v1.0 counterpart expect a dangling-reference error when the only item names no component.test_v09_topology_component_id_array_child_reachableand its v1.0 counterpart expect the component an item names to be reachable from the root.core/message_processor_scaling.yamlgetstest_scaling_process_single_component_messages_composition_v10. It times n messages of oneTexteach into a rootColumnthat names all n, under therelaxedconfig.conformance/test_data/catalogs/composition_catalog_v1_0.json. ItsColumndeclaresallowedParentsandallowedChildren, and itsTextdeclaresallowedParents.Results
Every harness in the table reads the shared suites from
conformance/, so the new cases run without harness changes. In Dart:dart/a2ui_core/test/conformance/validator_conformance_test.dartrunsvalidator_v0_9.yamlandvalidator_v1_0.yaml, so the newComponentIdarray cases run indart test.dart/a2ui_core/test/conformance/message_processor_scaling_conformance_test.dartrunsmessage_processor_scaling.yaml, so the timing case runs indart test -P performance. Thedart_a2ui_core (stable, performance)job of Flutter CI runs that.Python is the only harness this PR changes, to list the cases it fails on
mainas known gaps.ComponentIdarraymainmainmaina2ui_coredoes not implement v0.8The timing numbers are median times on the same machine, at 1000 and 4000:
#3345 adds a Dart-only timing test for the same container, and says the case can move into the shared suite once #3326 and #3327 land. This PR adds it to the shared suite now. The Python core harness skips the whole suite on
main, and the TypeScript harness has no timing runner there, so the case runs only in Dart until those PRs land. Each of them then needs to pass it or list it.The #3345 shortcut applies only under a validation config that defers the graph checks. A processor with no config, or a strict one, still checks every reference on the surface for each message once a catalog declares constraints. The shared case uses the
relaxedconfig, so it doesn't cover that path.Known gaps
Python core: on
main, the topology checks read references using the ids the surface holds.ComponentIdcounts only when one of its items names a held component.perf(python/core): check updateComponents incrementally under deferred graph checks #3326 reads both from the catalog alone and passes all four cases, so it should drop the four
SKIP_TEST_NAMESentries added here.TypeScript web core:
mainskips the timing case, so nothing is listed. perf(web_core): check updateComponents incrementally under deferred graph checks #3327 adds a timing runner, which fails this case until it checks a parent'sallowedChildrenonly against the components of the batch, as perf(python/core): check updateComponents incrementally under deferred graph checks #3326 does.This PR is a draft because the v0.8 cases decide a question that #3326 raises: whether a property that is not a reference field becomes one when its value names a component. They follow the v0.9 and v1.0 cases and #3326: it does not. If
main's reading is intended for v0.8, those two cases should go.Testing
uv run pytest python/a2ui_core/tests/test_conformance.py conformance/tests, after mergingmain: 642 passed, 17 skipped.dart/a2ui_core, after mergingmain,dart testpasses (1826 tests).dart test -P performancepasses, the new case included, in three runs out of three.dart testindart/a2ui_agentpasses (1105 tests).node tests/conformance/conformance_test.mjsintypescript/core/web_core: passes.swift test --filter ValidatorConformanceTestsinswift/core: passes../gradlew test --tests '*ConformanceTest*'inkotlin/agent_sdk_legacy: 173 tests pass.🤖 Agent-generated