Skip to content

ci migrate: resolve referenced paths and preserve YAML readability#533

Open
robstolarz wants to merge 1 commit into
mainfrom
rob/dep-5306-migrate-polish
Open

ci migrate: resolve referenced paths and preserve YAML readability#533
robstolarz wants to merge 1 commit into
mainfrom
rob/dep-5306-migrate-polish

Conversation

@robstolarz

@robstolarz robstolarz commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Summary

depot ci migrate mirrors a repo's GitHub Actions setup (workflows and composite actions) from .github/ into .depot/ and rewrites the internal .github/ path references to point at the copies. Four rough edges showed up when migrating a real, non-trivial setup: some referenced files were left behind, some paths were rewritten that shouldn't have been, and the re-serialized YAML lost readability.

This tightens the path-rewriting engine around a single rule and fixes all four. The rule:

A .github/(actions|workflows)/X reference is rewritten to .depot/ only when X was actually copied into .depot/ and the reference is a plain filesystem path — not a git-materialization directive, and not a pattern embedded in a regex or escaped string.

What changed

Referenced non-YAML siblings are now copied. Workflows routinely call helper scripts and read config/templates that live next to them under .github/workflows/. Those weren't copied, so after migration the references dangled. CopyWorkflowSiblings mirrors the non-YAML assets into .depot/workflows/, preserving relative layout and file mode and skipping symlinks. The copied files join the migrated-workflows allow-list, so references to them get rewritten consistently with the workflow YAML itself.

Path rewriting no longer fires inside regexes or escaped string literals. A composite action's source code may legitimately match the repo's real .github directory via a regex like /\.github\/actions\// — rewriting that to .depot silently breaks the match. Previously plain-path fixtures were rewritten but escaped forms in sources were treated inconsistently. The matcher now treats a backslash-escaped separator (\/) or dot (\.) as a path boundary the same way a plain one is, so escaped forms are classified — URL, remote owner/repo ref, or a longer non-path token — exactly like their plain counterparts, and left alone unless they name a genuinely-migrated path. The conservative direction: when in doubt, don't rewrite.

Sparse-checkout inputs are augmented, not rewritten. Rewriting an actions/checkout sparse-checkout entry from .github/ to .depot/ would stop git from materializing .github/ at all — breaking any step that still reads a script living there (only .github/actions and the selected workflows move; everything else stays). Because migrate copies rather than moves, .github/ still exists, so the checkout is made a superset: the .depot/ path is added alongside the kept .github/ entry. Checkout steps are recognized structurally (uses: actions/checkout + with.sparse-checkout, both block and list forms) rather than by blind scalar-walking, and gitignore-style ! negation patterns are mirrored so an explicit exclusion isn't silently re-included by the added parent.

Readable run: blocks stay readable. gopkg.in/yaml.v3 refuses block style for any scalar whose line ends in whitespace and silently falls back to a single-line, double-quoted "\n"-escaped string — flattening a readable run: | block into an unreadable one. Trailing whitespace on a POSIX shell command line is inert, so it's trimmed before encoding to keep the block style. This is deliberately narrow: only genuine step run: fields under a bash/sh shell are touched, and never on lines where trailing whitespace is load-bearing — heredoc bodies, escaped line continuations (POSIX \, PowerShell `, cmd ^), or text spanning an open quote. Action inputs that happen to be named run, and cmd/PowerShell/python steps, are left byte-exact.

Testing

  • go test ./pkg/ci/... ./pkg/cmd/ci/... — all green, including new cases for sibling copying (layout + mode + overwrite), escaped-context path classification (regex, escaped URL, escaped remote ref, escaped nested-sibling tail), the sparse-checkout superset (block + list forms, negation mirroring), and block-scalar preservation (heredoc, quote span, multi-shell continuations, with.run input, shell: cmd).
  • gofmt clean; go build ./... clean.
  • Local codex review --base origin/main run before push: no blocking correctness issues.

Refs DEP-5306.

🤖 Generated with Claude Code


Note

Medium Risk
Changes alter migrated workflow execution (checkout paths, shell run text, and path rewrites in YAML and copied assets); risk is mitigated by extensive tests but behavior is subtle for partial migrations and sparse-checkout ordering.

Overview
depot ci migrate workflows now mirrors non-YAML assets under .github/workflows/ into .depot/workflows/ via CopyWorkflowSiblings, registers copied paths on the partial-migration allow-list, and rewrites references in those files so helper scripts and configs keep working after migration.

The transform pipeline augments actions/checkout sparse-checkout inputs instead of rewriting .github/ entries away: each migrated .github/(actions|workflows) pattern keeps its original line and gains a .depot/ sibling (including ! negations and order-sensitive gitignore semantics), with partial migrations only mirroring paths that actually exist.

Path rewriting is stricter: escaped slashes/dots in regexes and string literals are classified like plain paths when appropriate, while URLs, remote refs, and partial workflow tails (including scripts\/build.sh) are handled more accurately; sanitizeRunBlockScalars trims trailing whitespace on POSIX run: | blocks so yaml.v3 keeps block style without touching cmd/PowerShell, heredocs, continuations, or non-step inputs.

Reviewed by Cursor Bugbot for commit c404ea7. Bugbot is set up for automated code reviews on this repo. Configure here.

@linear-code

linear-code Bot commented Jul 9, 2026

Copy link
Copy Markdown

DEP-5306

Comment thread pkg/ci/transform/transform.go

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 7b332b6. Configure here.

Comment thread pkg/ci/transform/transform.go
Migrating a GitHub Actions setup into .depot/ left a few references
dangling and reflowed some YAML that was fine as written. This tightens
the path-rewriting engine around a single rule and fixes four papercuts
that share it.

The rule: a `.github/(actions|workflows)/X` reference is rewritten to
`.depot/` only when X was actually copied into `.depot/` and the
reference is a plain filesystem path — not a git-materialization
directive and not a pattern embedded in a regex or escaped string.

- Referenced non-YAML siblings are now copied. Workflows often call
  helper scripts and read config files that live next to them under
  .github/workflows/. CopyWorkflowSiblings mirrors those assets into
  .depot/workflows/ (preserving layout and mode, skipping symlinks), and
  the copied files join the migrated-workflows allow-list so references
  to them rewrite consistently with the workflow YAML itself.

- Path rewriting no longer fires inside regexes or escaped string
  literals. A source that matches the repo's real .github via a regex
  like /\.github\/actions\// means it; rewriting it would break the
  match. The matcher now treats a backslash-escaped separator or dot as
  a boundary the same way a plain one is, so escaped forms are
  classified (URL, remote ref, longer token) exactly like their plain
  counterparts and left alone unless they name a genuinely-migrated path.

- Sparse-checkout inputs are augmented, not rewritten. Rewriting an
  actions/checkout sparse-checkout entry from .github/ to .depot/ would
  stop git from materializing .github/, breaking steps that still read
  scripts there. Because migrate copies rather than moves, .github/
  still exists, so the checkout is made a superset: the .depot/ path is
  added alongside the kept .github/ entry. Checkout steps are recognized
  structurally (uses: actions/checkout + with.sparse-checkout), and
  gitignore-style "!" negations are mirrored so an exclusion isn't
  silently re-included.

- Readable run blocks stay readable. yaml.v3 refuses block style for any
  scalar whose line ends in whitespace and silently flattens `run: |`
  into a single "\n"-escaped string. Trailing whitespace on a POSIX
  shell command line is inert, so it is trimmed to keep the block style
  — but only for genuine step run: fields under a bash/sh shell, and
  never on lines where the whitespace is load-bearing (heredoc bodies,
  line continuations, or text spanning an open quote). Action inputs
  that happen to be named run, and cmd/PowerShell/python steps, are left
  byte-exact.

Refs DEP-5306.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@robstolarz robstolarz force-pushed the rob/dep-5306-migrate-polish branch from 39262a4 to c404ea7 Compare July 10, 2026 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant