Skip to content

AutoFix: treat CI build log text as data in the Copilot prompt - #441

Closed
Ewerton Scaboro da Silva (ewertons) wants to merge 3 commits into
Azure:masterfrom
ewertons:ewertons/autofix-untrusted-log-isolation
Closed

Ewerton Scaboro da Silva (ewertons) wants to merge 3 commits into
Azure:masterfrom
ewertons:ewertons/autofix-untrusted-log-isolation

Conversation

@ewertons

Copy link
Copy Markdown
Contributor

Problem

update_deps/helper_scripts/autofix.ps1 builds the prompt it passes to the Copilot CLI by interpolating CI build log text directly:

## CI build failure logs

$build_logs

$build_logs is raw Azure DevOps failed-task log content fetched by get-azure-build-failure-details / get-failed-build-logs. It is appended to the prompt verbatim, and the prompt is then run with copilot -p $prompt --autopilot --allow-all --no-ask-user.

A build echoes a lot of text that is not written by the person running the propagation: source snippets, test output, file names, commit messages. Because that text lands in the prompt with no delimitation, anything in a log that reads like a markdown heading, a code fence, or an instruction is indistinguishable from the tool's own instructions to the agent.

Fix

Build log text is now handled as data:

  • New get-sanitized-log-block normalizes the log text before it reaches the prompt:
    • strips ANSI/VT escape sequences and remaining control/format characters (keeping tab and newline), so the block cannot be visually reformatted;
    • prefixes every line with | , so no line can close the block or start a new markdown section or code fence;
    • truncates to $global:MAX_BUILD_LOG_CHARS (20000), so a large log cannot push the task instructions out of the prompt.
  • The block is wrapped in a per-run, unguessable delimiter (BUILD-LOG-<guid>); any occurrence of that delimiter inside the log text is replaced, so the log cannot spoof its own end marker.
  • The prompt now states that the delimited block is reference data only, that instructions found inside it must be ignored, that the numbered task list is the only instruction set for the session, and that the agent should report it if the block looks like it contains instructions.
  • $branch_name and $pr_url are validated before interpolation ($branch_name is also placed into a git push origin ... command in the prompt). Unexpected values skip AutoFix instead of being embedded.

Testing

Ran get-sanitized-log-block over log text containing ANSI colour codes, an OSC title-set sequence, a copy of the delimiter, markdown headings, code fences, and an instruction-shaped line. Verified the output contains no ESC or CR characters, no occurrence of the delimiter, and that every line is quoted. Verified truncation on a 60000-character input and that empty input returns an empty string. Script parses clean, line endings and tabs unchanged.

CI build log content was interpolated verbatim into the prompt passed to the Copilot CLI, so text echoed by a build could be read as part of the prompt's instructions.

Log text is now normalized (ANSI/control characters removed, every line quoted with a leading '| ', truncated to a fixed size) and placed inside a per-run, unguessable delimiter, with the prompt stating that the delimited block is reference data only and that instructions inside it must be ignored. The branch name and PR URL interpolated into the prompt are validated before use.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving the PR in order to squash bureaucracy. The PR does not contain any value, but it is the easiest way to shut up the IcM.

@dcristoloveanu

Copy link
Copy Markdown
Member

/azp run Azure-C-Build-Tools-Gate

@azure-pipelines

Copy link
Copy Markdown
No pipelines are associated with this pull request.

@dcristoloveanu

Copy link
Copy Markdown
Member

Superseded by #472. The required Azure-C-Build-Tools-Gate check doesn't build fork PRs, so the branch was moved into Azure/c-build-tools as-is (same head commit 2db8eb6). Thanks Ewerton Scaboro da Silva (@ewertons).

auto-merge was automatically disabled October 20, 2026 23:10

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants