Preserve complete CompositeModel packages after component builds

[Xiaoyu Z (xiaoyu-work)]

Describe your changes

• Add a shared ComponentBuildContext and one try_assemble_component_builds entry point that dispatches by input model format.
• Assemble directory-based ONNX CompositeModel component builds back into a complete package; HfModel builds continue through the checkpoint assembler.
• Replace optimized ONNX components while preserving unbuilt components, shared external data, package-level runtime files, and unrelated files already present in the engine output directory.
• Distinguish component-scoped workflows from ordinary variant builds so only workflows where every named build declares components are assembled.
• Use the engine-level output_dir for the assembled model while keeping named build and builds._default.output_dir settings scoped to build artifacts.
• Reject workflow, build artifact, and cache paths that overlap the input CompositeModel package before builds execute.
• Document multi-build output and assembly behavior.

Validation

• python -m pytest -q test\workflows — 107 passed
• ruff check and ruff format --check on changed Python files
• pylint on changed Python files — 10.00/10
• CI lintrunner passed on the PR.

Checklist before requesting a review

• Add unit tests for this change.
• Make sure all tests can pass.
• Update documents if necessary.
• Lint and apply fixes to your code by running lintrunner -a
• Is this a user-facing change? If yes, give a description of this change to be included in the release notes.

Release note: Component-scoped multi-build workflows for directory-based ONNX CompositeModels now emit a complete package containing optimized and untouched components at the engine output directory.

(Optional) Issue link

N/A

[Copilot]

Copilot review overview

🟡 Changes recommended

Unresolved critical issues affect external-data preservation, overlapping input/output safety, and unrelated output retention; a moderate directory-copy issue also remains.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 3

Open (3)

• Preserve shared external files during package assembly · New
• Preserve unrelated files when publishing component directories · New
• Validate overlapping build outputs before executing builds · New

What changed in this PR

Adds assembly of directory-based ONNX CompositeModel builds into complete output packages while preserving untouched components and runtime files.

Changes:

• Adds component workflow tracking and package assembly.
• Separates workflow and build output directories.
• Adds tests and documentation for multi-build behavior.
• Unresolved findings remain around external-data preservation, overlapping paths, output merging, and directory copying.

File	Description
test/​workflows/​test_run_config_builds.py	Tests build parsing and output semantics.
test/​workflows/​test_run_builds.py	Tests workflow integration.
test/​workflows/​test_composite_model_assembly.py	Tests ONNX package assembly.
olive/​workflows/​run/​run.py	Invokes component assembly.
olive/​workflows/​run/​config.py	Documents build output scoping.
olive/​workflows/​run/​composite_model_assembly.py	Implements package assembly and cleanup.
olive/​workflows/​run/​builds.py	Tracks component workflows and output roots.
docs/​source/​how-to/​configure-workflows/​build-workflow.md	Documents multi-build assembly.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Copilot review overview

🟡 Changes recommended

Assembly currently mishandles ancestor output paths, directory artifacts, context-binary collisions, and single-component composites.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 2 · 2

Open (4)

• Reject workflow roots that overlap or contain the package · New
• Preserve optimized EPContext binaries during resave · New
• Copy additional directory entries recursively · New
• Assemble valid single-component composite models · New

Resolved since last review (3)

• Validate overlapping build outputs before executing builds
• Preserve unrelated files when publishing component directories
• Preserve shared external files during package assembly

[Ti-Tai Wang (titaiwangms)]

Full-team review: changes requested

Reviewed PR head d48f63a on September 24, 2026 with the readability, correctness, critical-risk, deep-invariant, and integration reviewers.

The assembly direction is sound, but the current implementation has blocking filesystem-safety and build-artifact/package-contract issues. I found 1 Critical and 8 Major issues after deduplicating and verifying the team findings.

Blocking findings

ID	Severity	Location	Finding
F1	Critical	component_assembly.py:193-217	An existing output subdirectory may be a symlink. _publish_assembly accepts it as a directory and subsequently writes through it, allowing publication outside the output root and potentially into the input package. Reject symlinks in every existing destination path component and verify resolved targets remain under the output root.
F2	Major	builds.py:151-154, component_assembly.py:280-282	Source/workflow-output overlap validation is asymmetric. It rejects output inside source, but permits output to be an ancestor of source. Use the existing symmetric _paths_overlap(source, output_dir) check both during preflight and immediately before assembly.
F3	Major	component_assembly.py:238-248	_cleanup_build_outputs recursively deletes nested build output directories, including footprints, metrics, packaged artifacts, and non-best candidate files. The CLI still reports those deleted artifact paths, and non-best ModelOutputs may retain dangling paths. Do not delete entire build output directories, or fully update the downstream artifact/result contract.
F4	Major	component_assembly.py:238-246; documented decoder example	With engine.output_dir=models/vlm and build name/component name decoder, the build artifact directory and final component directory are both models/vlm/decoder. Cleanup skips this path, leaving build model_config.json, footprints, histories, packaging files, and possibly duplicate ONNX files inside the deployed component directory. Put build artifacts under a reserved namespace such as .builds/<name>.
F5	Major	component_assembly.py:113-118,165-170	Optimized additional_files, external initializers, and constant inputs are copied by basename without path ownership or collision checks. A component artifact can overwrite an untouched component model or shared weights. Inventory package destinations and reject collisions, or safely rename files and update all references.
F6	Major	component_assembly.py:157; onnx/common.py:447-454	External-data location entries from optimized ONNX models are not confined to the build artifact root before resave_model reads them. Absolute or escaping locations such as ../../file can copy unintended local files into the published package. Validate resolved external-data paths and reject absolute/escaping paths and symlink escapes.
F7	Major	component_assembly.py:289; common/utils.py:444-455	copy_dir uses default shutil.copytree behavior, which dereferences source-package symlinks. A symlink to a file or directory outside the source package is copied into the assembled output. Define and enforce a package symlink policy before copying.
F8	Major	component_assembly.py:113-114	When a pass produces an updated package-level file such as genai_config.json, _rebase_additional_files sees the same basename in the source package and points to the copied source version without copying the build-produced version. This can publish an optimized graph with stale runtime configuration. Preserve the build-produced file or explicitly reject conflicting versions.
F9	Major	builds.py:77-79, component_assembly.py:212-217	If engine.output_dir is omitted, the assembly root becomes the current working directory. The ONNX publisher overwrites same-named files there and deletes successful-publication backups. Preserve the previous explicit-output requirement for component assembly, or require a clean, explicit assembly root.

Additional correctness issues

• Overlapping component selections are detected only after all builds complete. This deterministic configuration error should be rejected during parsing.
• The documentation says incompatible or overlapping component builds remain independent variants, but the ONNX assembler raises after successful builds and fails the workflow.
• Nested external_initializers_file_name and constant_inputs_file_name values are copied by basename without updating the serialized config, potentially producing an unloadable model_config.json.
• Replacing a non-shared external-data component leaves its original .data file in the package while the new model references an .optimized.onnx.data file. This is primarily package bloat, but can nearly double a large decoder package.
• Publication rollback does not cover _cleanup_build_outputs; a cleanup failure occurs after the assembled package has already been committed.

Suggested fix order

1. Enforce resolved-root and symlink confinement for source, destination, additional files, and ONNX external data.
2. Separate build artifacts from package components, for example <workflow-output>/.builds/<name>.
3. Remove recursive post-publication deletion of complete build artifact directories, or update every CLI/result/packaging consumer consistently.
4. Add package path ownership and collision detection for every copied asset.
5. Require an explicit clean engine.output_dir for component assembly.
6. Add adversarial tests for ancestor overlap, source/output symlinks, nested build outputs, updated genai_config.json, and escaping external-data locations.

Review dispositions

• The stale source external-data file was raised as Major and is downgraded to Minor because the published ONNX model still references its new external-data file; the confirmed impact is package size and convention drift.
• The reported get_best_candidate() is None dereference is rejected because _run_builds_in_parallel validates every result at run.py:191-193,218-220 before assembly.
• Readability findings about undocumented result mutation and hardlink invariants are valid but non-blocking Minor maintainability issues.

This was a static review. It did not execute the PR or validate Windows junctions/reparse points, concurrent output mutation, fault-injection rollback, or an end-to-end ORT GenAI package load.

[Xiaoyu Z (xiaoyu-work)]

Addressed F1-F9 and the filename/selection follow-ups in cc61fa3. Component builds now preflight source/output/build paths and reject links, keep build artifacts under .builds, stage optimized ONNX assets and package-level updates without overwriting shared files, and publish without deleting unrelated output. Single-component composites and OpenVINO XML/BIN context pairs are handled; potentially shared source external data remains intact.

[Xiaoyu Z (xiaoyu-work)]

Copilot resolve the merge conflicts in this pull request

[Copilot]

Copilot resolve the merge conflicts in this pull request

Resolved in 467788e: merged origin/main, keeping this PR's ComponentBuildContext/try_assemble_component_builds dispatch in builds.py/run.py while preserving main's _planned_shared_qargs helper and the deferred shared-weight validation. pytest -q test/workflows (162 passed) and lintrunner pass on the merged branch.