Repository navigation
Archive CLI starter validation diagnostics before artifact upload #22348
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Ankit Jain (radical)
wants to merge
3
commits into
main
Choose a base branch
from
cif/22347-r1-main-fix
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+288
−4
Open
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| <# | ||
| .SYNOPSIS | ||
| Archives Aspire CLI starter validation diagnostics for artifact upload. | ||
|
|
||
| .DESCRIPTION | ||
| actions/upload-artifact first enumerates every path under the upload root and then calls | ||
| fs.stat on each result. It does not tolerate an entry that disappears in between and fails with: | ||
|
|
||
| ENOENT: no such file or directory, stat '<root>/aspire-go-starter/AspireCliGoStarterSmoke/frontend/node_modules' | ||
|
|
||
| The generated starter projects contain node_modules trees. A JavaScript package installer, such | ||
| as an interrupted `frontend-installer` npm install, can still rewrite or delete them after | ||
| `aspire stop` returns. Upload a completed tar archive instead. tar excludes node_modules without | ||
| traversing them and stores symbolic links without following them. Hidden files and directories | ||
| remain excluded, matching upload-artifact's default. The archive contains the validation root | ||
| directory and its retained contents. | ||
|
|
||
| .PARAMETER SourceRoot | ||
| The starter validation root that contains generated projects, diagnostics, and CLI logs. | ||
|
|
||
| .PARAMETER DestinationRoot | ||
| The directory in which to create starter-validation.tar for upload. | ||
| #> | ||
| [CmdletBinding()] | ||
| param( | ||
| [Parameter(Mandatory = $true)] | ||
| [string]$SourceRoot, | ||
|
|
||
| [Parameter(Mandatory = $true)] | ||
| [string]$DestinationRoot | ||
| ) | ||
|
|
||
| $ErrorActionPreference = 'Stop' | ||
| $PSNativeCommandUseErrorActionPreference = $false | ||
|
|
||
| $archivePath = Join-Path $DestinationRoot 'starter-validation.tar' | ||
| $temporaryArchivePath = "$archivePath.partial" | ||
| foreach ($path in @($archivePath, $temporaryArchivePath)) | ||
| { | ||
| if (Test-Path -LiteralPath $path) | ||
| { | ||
| Remove-Item -LiteralPath $path -Force | ||
| } | ||
| } | ||
|
|
||
| if (-not (Test-Path -LiteralPath $SourceRoot -PathType Container)) | ||
| { | ||
| Write-Host "No starter validation diagnostics found at '$SourceRoot'." | ||
| return | ||
| } | ||
|
|
||
| New-Item -ItemType Directory -Path $DestinationRoot -Force | Out-Null | ||
| try | ||
| { | ||
| # Both GNU tar and BSD/Windows tar exclude this directory name at any depth without entering it. | ||
| # Do not dereference links or ignore tar failures: errors in retained diagnostics must stay visible. | ||
| # Archive the named root from its parent: using "." lets BSD tar's hidden-file pattern exclude | ||
| # the entire "./" tree. ".?*" excludes hidden names at any depth, including "..name". | ||
| # https://github.com/actions/upload-artifact#uploading-hidden-files | ||
| $sourceDirectory = Get-Item -LiteralPath $SourceRoot | ||
| & tar --exclude=node_modules '--exclude=.?*' -cf $temporaryArchivePath -C $sourceDirectory.Parent.FullName $sourceDirectory.Name | ||
| if ($LASTEXITCODE -ne 0) | ||
| { | ||
| throw "Archiving starter validation diagnostics failed with tar exit code $LASTEXITCODE." | ||
| } | ||
|
|
||
| Move-Item -LiteralPath $temporaryArchivePath -Destination $archivePath -Force | ||
| } | ||
| finally | ||
| { | ||
| if (Test-Path -LiteralPath $temporaryArchivePath) | ||
| { | ||
| Remove-Item -LiteralPath $temporaryArchivePath -Force | ||
| } | ||
| } | ||
|
|
||
| Write-Host "Archived starter validation diagnostics from '$SourceRoot' to '$archivePath'." |
190 changes: 190 additions & 0 deletions
190
tests/Infrastructure.Tests/PowerShellScripts/StageCliStarterValidationDiagnosticsTests.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,190 @@ | ||
| // Licensed to the .NET Foundation under one or more agreements. | ||
| // The .NET Foundation licenses this file to you under the MIT license. | ||
|
|
||
| using System.Formats.Tar; | ||
| using Aspire.TestUtilities; | ||
| using Xunit; | ||
|
|
||
| namespace Infrastructure.Tests; | ||
|
|
||
| /// <summary> | ||
| /// Tests for eng/scripts/stage-cli-starter-validation-diagnostics.ps1 | ||
| /// </summary> | ||
| public sealed class StageCliStarterValidationDiagnosticsTests : IDisposable | ||
| { | ||
| private readonly TemporaryWorkspace _workspace; | ||
| private readonly string _scriptPath; | ||
| private readonly ITestOutputHelper _output; | ||
|
|
||
| public StageCliStarterValidationDiagnosticsTests(ITestOutputHelper output) | ||
| { | ||
| _output = output; | ||
| _workspace = TemporaryWorkspace.Create(output); | ||
| _scriptPath = Path.Combine(RepoRoot.Path, "eng", "scripts", "stage-cli-starter-validation-diagnostics.ps1"); | ||
| } | ||
|
|
||
| public void Dispose() => _workspace.Dispose(); | ||
|
|
||
| [Fact] | ||
| [RequiresTools(["pwsh", "tar"])] | ||
| public async Task ArchivesDiagnosticsWithoutNodeModulesOrHiddenFiles() | ||
| { | ||
| var sourceRoot = Path.Combine(_workspace.Path, "validation with spaces"); | ||
| var destinationRoot = Path.Combine(_workspace.Path, "upload with spaces"); | ||
| WriteFile(sourceRoot, "cli-logs/cli.log"); | ||
| WriteFile(sourceRoot, ".aspire/settings.json"); | ||
| WriteFile(sourceRoot, ".env"); | ||
| WriteFile(sourceRoot, "..settings"); | ||
| WriteFile(sourceRoot, "aspire-go-starter/AspireCliGoStarterSmoke/.aspire/settings.json"); | ||
| WriteFile(sourceRoot, "aspire-go-starter/AspireCliGoStarterSmoke/frontend/.env"); | ||
| WriteFile(sourceRoot, "aspire-go-starter/diagnostics/aspire-stop.log"); | ||
| WriteFile(sourceRoot, "aspire-go-starter/AspireCliGoStarterSmoke/apphost.go"); | ||
| WriteFile(sourceRoot, "aspire-go-starter/AspireCliGoStarterSmoke/frontend/package.json"); | ||
| WriteFile(sourceRoot, "aspire-go-starter/AspireCliGoStarterSmoke/frontend/node_modules/@babel/core/package.json"); | ||
| WriteFile(sourceRoot, "aspire-ts-starter/AspireCliTsStarterSmoke/node_modules/vite/package.json"); | ||
| WriteFile(destinationRoot, "starter-validation.tar"); | ||
|
|
||
| var result = await RunScript(sourceRoot, destinationRoot); | ||
|
|
||
| result.EnsureSuccessful(); | ||
| Assert.Equal( | ||
| [ | ||
| "aspire-go-starter/AspireCliGoStarterSmoke/apphost.go", | ||
| "aspire-go-starter/AspireCliGoStarterSmoke/frontend/package.json", | ||
| "aspire-go-starter/diagnostics/aspire-stop.log", | ||
| "cli-logs/cli.log" | ||
| ], | ||
| ReadArchiveFiles(destinationRoot, sourceRoot)); | ||
| Assert.Equal(["starter-validation.tar"], GetRelativeEntries(destinationRoot)); | ||
| } | ||
|
|
||
| [Fact] | ||
| [RequiresTools(["pwsh", "tar"])] | ||
| public async Task ArchivesLinksWithoutFollowingTargets() | ||
| { | ||
| Assert.SkipWhen(OperatingSystem.IsWindows(), "Creating symbolic links requires extra privileges on Windows."); | ||
|
|
||
| var sourceRoot = Path.Combine(_workspace.Path, "validation"); | ||
| var destinationRoot = Path.Combine(_workspace.Path, "upload"); | ||
| var outsideDirectory = Path.Combine(_workspace.Path, "outside"); | ||
| WriteFile(sourceRoot, "aspire-ts-starter/diagnostics/aspire-stop.log"); | ||
| WriteFile(outsideDirectory, "secret.txt"); | ||
|
|
||
| File.CreateSymbolicLink( | ||
| Path.Combine(sourceRoot, "aspire-ts-starter", "removed.log"), | ||
| Path.Combine(_workspace.Path, "missing.log")); | ||
| Directory.CreateSymbolicLink(Path.Combine(sourceRoot, "aspire-ts-starter", "linked"), outsideDirectory); | ||
|
|
||
| var result = await RunScript(sourceRoot, destinationRoot); | ||
|
|
||
| result.EnsureSuccessful(); | ||
| Assert.Equal(["aspire-ts-starter/diagnostics/aspire-stop.log"], ReadArchiveFiles(destinationRoot, sourceRoot)); | ||
| using var archive = File.OpenRead(Path.Combine(destinationRoot, "starter-validation.tar")); | ||
| using var reader = new TarReader(archive); | ||
| List<string> links = []; | ||
| while (reader.GetNextEntry() is { } entry) | ||
| { | ||
| if (entry.EntryType is TarEntryType.SymbolicLink) | ||
| { | ||
| links.Add(entry.Name); | ||
| } | ||
| } | ||
| Assert.Equal(["validation/aspire-ts-starter/linked", "validation/aspire-ts-starter/removed.log"], links.Order(StringComparer.Ordinal)); | ||
| } | ||
|
|
||
| [Fact] | ||
| [RequiresTools(["pwsh"])] | ||
| public async Task MissingSourceLeavesNothingToUpload() | ||
| { | ||
| var destinationRoot = Path.Combine(_workspace.Path, "upload"); | ||
| WriteFile(destinationRoot, "starter-validation.tar"); | ||
| WriteFile(destinationRoot, "starter-validation.tar.partial"); | ||
|
|
||
| var result = await RunScript(Path.Combine(_workspace.Path, "missing"), destinationRoot); | ||
|
|
||
| result.EnsureSuccessful(); | ||
| Assert.Empty(Directory.EnumerateFileSystemEntries(destinationRoot)); | ||
| } | ||
|
|
||
| [Theory] | ||
| [InlineData(1)] | ||
| [InlineData(2)] | ||
| [RequiresTools(["pwsh"])] | ||
| public async Task ArchiveFailureLeavesNothingToUpload(int tarExitCode) | ||
| { | ||
| var sourceRoot = Path.Combine(_workspace.Path, "validation"); | ||
| var destinationRoot = Path.Combine(_workspace.Path, "upload"); | ||
| WriteFile(sourceRoot, "diagnostics/aspire-stop.log"); | ||
| WriteFile(destinationRoot, "starter-validation.tar"); | ||
|
|
||
| // A native command may leave partial output before failing. Exercise the production script | ||
| // with that failure without relying on permissions, disk exhaustion, or a timed filesystem race. | ||
| var wrapper = Path.Combine(_workspace.Path, "fail-tar.ps1"); | ||
| await File.WriteAllTextAsync(wrapper, """ | ||
| param($Script, $SourceRoot, $DestinationRoot, $TarExitCode) | ||
| function tar | ||
| { | ||
| [System.IO.File]::WriteAllText((Join-Path $DestinationRoot 'starter-validation.tar.partial'), 'incomplete') | ||
| $global:LASTEXITCODE = $TarExitCode | ||
| } | ||
| & $Script -SourceRoot $SourceRoot -DestinationRoot $DestinationRoot | ||
| """); | ||
| using var cmd = new PowerShellCommand(wrapper, _output).WithTimeout(TimeSpan.FromMinutes(2)); | ||
| var result = await cmd.ExecuteAsync( | ||
| "-Script", $"\"{_scriptPath}\"", | ||
| "-SourceRoot", $"\"{sourceRoot}\"", | ||
| "-DestinationRoot", $"\"{destinationRoot}\"", | ||
| "-TarExitCode", tarExitCode.ToString()); | ||
|
|
||
| Assert.NotEqual(0, result.ExitCode); | ||
| Assert.Contains($"tar exit code {tarExitCode}", result.Output); | ||
| Assert.Empty(Directory.EnumerateFileSystemEntries(destinationRoot)); | ||
| } | ||
|
|
||
| private async Task<CommandResult> RunScript(string sourceRoot, string destinationRoot) | ||
| { | ||
| using var cmd = new PowerShellCommand(_scriptPath, _output) | ||
| .WithTimeout(TimeSpan.FromMinutes(2)); | ||
|
|
||
| return await cmd.ExecuteAsync( | ||
| "-SourceRoot", $"\"{sourceRoot}\"", | ||
| "-DestinationRoot", $"\"{destinationRoot}\""); | ||
| } | ||
|
|
||
| private static void WriteFile(string root, string relativePath) | ||
| { | ||
| var path = Path.Combine(root, relativePath); | ||
| Directory.CreateDirectory(Path.GetDirectoryName(path)!); | ||
| File.WriteAllText(path, relativePath); | ||
| } | ||
|
|
||
| private static string[] GetRelativeEntries(string root) => | ||
| Directory.EnumerateFiles(root, "*", SearchOption.AllDirectories) | ||
| .Select(path => Path.GetRelativePath(root, path).Replace(Path.DirectorySeparatorChar, '/')) | ||
| .Order(StringComparer.Ordinal) | ||
| .ToArray(); | ||
|
|
||
| private static string[] ReadArchiveFiles(string root, string sourceRoot) | ||
| { | ||
| using var archive = File.OpenRead(Path.Combine(root, "starter-validation.tar")); | ||
| using var reader = new TarReader(archive); | ||
| List<string> files = []; | ||
| while (reader.GetNextEntry() is { } entry) | ||
| { | ||
| if (entry.EntryType is not (TarEntryType.RegularFile or TarEntryType.V7RegularFile)) | ||
| { | ||
| continue; | ||
| } | ||
|
|
||
| // Entries include the validation directory, for example "validation/cli-logs/cli.log". | ||
| var prefix = Path.GetFileName(sourceRoot) + "/"; | ||
| Assert.StartsWith(prefix, entry.Name, StringComparison.Ordinal); | ||
| var name = entry.Name[prefix.Length..]; | ||
| using var content = new StreamReader(entry.DataStream!); | ||
| Assert.Equal(name, content.ReadToEnd()); | ||
| files.Add(name); | ||
| } | ||
|
|
||
| return files.Order(StringComparer.Ordinal).ToArray(); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.