diff --git a/utilities/onboarding-tool/Onboarding-Tooling-Requirements.md b/utilities/onboarding-tool/Onboarding-Tooling-Requirements.md index e7cc8b2d..d52a1d49 100644 --- a/utilities/onboarding-tool/Onboarding-Tooling-Requirements.md +++ b/utilities/onboarding-tool/Onboarding-Tooling-Requirements.md @@ -172,6 +172,10 @@ CIDRs, BYO storage model, deploying identity, the policy set, and mandatory `rbac.humanAccessMode` (`permanent-role-assignments` or `customer-jit-managed`). The JIT mode also requires `rbac.jitEligibilityDurationDays` as a whole number from 7 through 3650. +The control-plane region is independent of the workload region. The VNet/subnets and all +resources in the managed resource groups (MRGs) must share `workloadRegion`. Stage 1 validates +an existing VNet against that region; Stage 4 checks deployment subnet regions before submission. + FR1.2 [P0] Embedded subnet-sizing validation. Validate each Discovery subnet against the required prefix and confirm all fit inside the provided VNet without overlap: diff --git a/utilities/onboarding-tool/script-specs/config.schema.json b/utilities/onboarding-tool/script-specs/config.schema.json index 317450db..7e4ddc40 100644 --- a/utilities/onboarding-tool/script-specs/config.schema.json +++ b/utilities/onboarding-tool/script-specs/config.schema.json @@ -49,10 +49,12 @@ ] }, "controlPlaneRegion": { - "type": "string" + "type": "string", + "description": "Region for Discovery control-plane resources. Can differ from workloadRegion." }, "workloadRegion": { - "type": "string" + "type": "string", + "description": "Region for the VNet/subnets and all managed-resource-group resources." }, "systemSku": { "type": "string", diff --git a/utilities/onboarding-tool/script-specs/stage1-planning/stage1_plan.md b/utilities/onboarding-tool/script-specs/stage1-planning/stage1_plan.md index 6a41ee58..2aaca794 100644 --- a/utilities/onboarding-tool/script-specs/stage1-planning/stage1_plan.md +++ b/utilities/onboarding-tool/script-specs/stage1-planning/stage1_plan.md @@ -37,6 +37,10 @@ Confirm the subscription is allowlisted for Discovery, record the RP-registratio ### capture_fields — FR1.1 Capture and normalize control-plane region, workload region, VNet CIDR, per-service subnet CIDRs, BYO storage model, deploying identity, and policy set. Normalize regions to Azure short names and CIDRs to canonical form; flag any required field left blank. + +For `byo-existing`, validate the existing VNet against `workloadRegion`, not `controlPlaneRegion`. +The VNet/subnets and all managed-resource-group resources must share the workload region; +the Discovery control-plane resources can use a different region. - Missing field: complete the corresponding cell in the planning sheet. ### project_storage — FR1.1 diff --git a/utilities/onboarding-tool/script-specs/stage4-deployment/stage4_deploy.md b/utilities/onboarding-tool/script-specs/stage4-deployment/stage4_deploy.md index c45b88b2..c764d5b1 100644 --- a/utilities/onboarding-tool/script-specs/stage4-deployment/stage4_deploy.md +++ b/utilities/onboarding-tool/script-specs/stage4-deployment/stage4_deploy.md @@ -59,9 +59,9 @@ to activate the role. Eligible-but-inactive does not fail the completed deployme principals and managed identities keep standard assignments. Other assignment failures are Warn. The workspace `publicNetworkAccess` comes from `workspace.publicNetworkAccess` (default `Enabled`) and is applied on the bicep deploy, the direct workspace PUT, and the recovery re-PUT. The immutable `NetworkIsolation` tag comes from `workspace.networkIsolation` (default `true`) and is applied by both Bicep and direct workspace deployment. -Region co-location (pre-flight): the supercomputer `location` (from `controlPlaneRegion`) must equal the region of the BYO VNet that holds the managedcluster/nodepool subnets. Discovery validates the node pool `SubnetId` against the supercomputer's primary region, and `discovery.overridemrgregion` (workloadRegion) does not relax that check. The chosen region must also be a Discovery-creatable region: creation is rejected in some regions (e.g. eastus2 returns "Creation of new Supercomputer resources is not supported in region") even though the ARM provider location list advertises them; eastus is confirmed creatable. So the BYO VNet must live in a creatable region. The script resolves the VNet region from the subnet id and fails fast with a RegionMismatch remediation before the PUT. Region is immutable, so a mismatch requires setting `controlPlaneRegion` to a creatable VNet region, deleting the failed supercomputer, and re-running (a plain re-PUT cannot move it). -- RegionMismatch on SubnetId: set controlPlaneRegion = the managedcluster/nodepool VNet region (which must itself be a Discovery-creatable region); delete the failed supercomputer and re-run. -- Creation not supported in region: the region is not Discovery-creatable for this resource type; place the BYO VNet and set controlPlaneRegion to a creatable region (eastus confirmed; eastus2 rejected despite being advertised). +Region alignment (pre-flight): Discovery resource `location` comes from `controlPlaneRegion` and can differ from the VNet region. All resources in the managed resource groups must share `workloadRegion`, set by `discovery.overridemrgregion` on the supercomputer and workspace. The script resolves each deployment subnet's VNet region and checks it against `workloadRegion` before either Bicep or ARM submission. An unreadable VNet or a mismatch returns `deploy-network-region` and stops deployment. +- RegionMismatch on SubnetId: set `workloadRegion` to the VNet/subnet region, or use subnets in `workloadRegion`. Existing managed resources may need to be recreated to change region. +- Creation not supported in region: select a Discovery-creatable `controlPlaneRegion` for the affected resource type. This does not require moving the VNet or changing `workloadRegion`. ### true_state_detection — FR4.3 Allow 15-30 minutes for the supercomputer and up to about an hour for the workspace; plan one to two hours overall. Deployment/recovery callers use a 120-minute resource-state polling timeout per resource; this is not a guaranteed completion time. @@ -73,8 +73,8 @@ Catch the case where the ARM resource list shows Succeeded while a Foundry accou ### error_remediation_engine — FR4.4 Match the ARM error signature and emit the specific remediation: - `InsufficientQuota` (Tokens Per Minute) on a managed-RG model deployment → not enough free gpt-5.4 GlobalStandard TPM in the workload region (workspace needs 250K, the chat model 200K) → free or request quota, then re-run (Stage 1 `model-quota` catches this up front). -- `Creation of new X resources is not supported in region` → the target region is not Discovery-creatable for that resource type (ARM advertises more locations than the service allows) → deploy into a creatable region (eastus confirmed; eastus2 rejected) and place the BYO VNet there so the supercomputer stays co-regional with its node pool subnet. -- `RegionMismatch` on supercomputer/nodePool `SubnetId` → supercomputer location differs from the BYO VNet/subnet region → set controlPlaneRegion to the VNet region so the supercomputer is co-regional with its subnets, delete the failed supercomputer, and re-run (region is immutable). +- `Creation of new X resources is not supported in region` → the target control-plane region is not Discovery-creatable for that resource type → select a supported `controlPlaneRegion`; keep the VNet and MRG resources aligned in `workloadRegion`. +- `RegionMismatch` on supercomputer/nodePool `SubnetId` → the MRG workload region differs from the VNet/subnet region → align `workloadRegion` and the VNet; `controlPlaneRegion` can differ. - `AKSCapacityHeavyUsage` / `InternalServerError` on AKS create → region capacity for API Server VNet Integration → retry, or deploy workload in a region with capacity and global-peer to the firewall. - Repeated `AZFWApplicationRule` deny on packages.microsoft.com → firewall source scoped to the wrong CIDR → add the real spoke CIDR and allow the bootstrap FQDNs. - 409 `FlagMustBeSetForRestore` → Foundry account soft-deleted then recreated → set Restore=true, or purge and wait before re-PUT. diff --git a/utilities/onboarding-tool/scripts/USER-GUIDE.md b/utilities/onboarding-tool/scripts/USER-GUIDE.md index cdd18a47..586f0c52 100644 --- a/utilities/onboarding-tool/scripts/USER-GUIDE.md +++ b/utilities/onboarding-tool/scripts/USER-GUIDE.md @@ -247,11 +247,11 @@ Notes next to each field explain what to enter. A few that trip people up: | `network.model` | Choose one of the options described above. Only `byo-existing` skips Stage 2 network-template provisioning. | | `network.networkResourceGroup` | The resource group holding the existing VNet for `byo-existing`, or the target group for tool-provisioned networking. Stage 2 creates/uses the target network group for provisioning models. | | `network.dnsResourceGroup` | Resource group containing the private DNS zones linked to the Discovery VNet. Leave it blank when DNS zones live in `network.networkResourceGroup`; specify it for centralized DNS in another resource group. Stage 2 creates/reuses the required zones and exact VNet links there and includes this RG in its RBAC targets. | -| `network.vnetName` | For `byo-existing`, enter the existing VNet name (Stage 1 checks it exists and is in `controlPlaneRegion`). For `managed`, `byo` or `byo-spoke`, enter the VNet name to deploy; blank defaults to `vnet-`. | +| `network.vnetName` | For `byo-existing`, enter the existing VNet name (Stage 1 checks it exists and is in `workloadRegion`). For `managed`, `byo` or `byo-spoke`, enter the VNet name to deploy; blank defaults to `vnet-`. | | `network.existingSubnets` | BYO form: list existing subnet CIDRs outside the planned Discovery subnets so address planning can check for overlaps. This field does not make a provisioning model preserve unlisted subnets. | | `network.outboundType` | Select `UserDefinedRouting` for customer firewall/NVA egress, or `LoadBalancer` for AKS load-balancer-based egress. | | `network.nvaNextHop` | For UDR, enter the actual firewall/NVA private IP. It is not a NAT Gateway public IP. Not required for `LoadBalancer`. | -| `controlPlaneRegion` / `workloadRegion` | Use a supported production region: **East US**, **Sweden Central**, or **UK South**. Keep resources for one deployment in the same region unless an approved architecture requires otherwise. | +| `controlPlaneRegion` / `workloadRegion` | Use a supported production region: **East US**, **Sweden Central**, or **UK South**. `controlPlaneRegion` places the Discovery control-plane resources and can differ from `workloadRegion`. The VNet/subnets and all resources in the managed resource groups (MRGs) must be in `workloadRegion`. | | `names.*` | The resource names (workspace, project, supercomputer, node pool, etc.). Keep them unique in the subscription. | | `storage.account` | Required for every project, independently of Bookshelf. The name of the storage account you pre-created (prerequisite 8); keep `storage.model=byo`. Stage 1 fails if the account doesn't exist. Stage 2 adds the Blob CORS rules and private endpoint. The account denies public network traffic, so users who open project outputs in Discovery Studio need network access to it: from the VNet (VPN or ExpressRoute), or through a client IP rule you add. | | `workspace.publicNetworkAccess` | `Enabled` (default) or `Disabled`. With `Disabled`, the workspace data plane (agents, investigations, conversations) answers only from inside the VNet, so Stage 5 must run from a runner with private network access. | @@ -260,8 +260,8 @@ Notes next to each field explain what to enter. A few that trip people up: Notes: - **Policies are collected live** from the subscription at runtime — you do not list deny/audit policies in the form. Add a `POLICY` row only to declare or override one. -- For **`byo-existing`**, `controlPlaneRegion` must equal the region of the existing VNet that holds - the `managedcluster` / `nodepool` subnets. +- For **`byo-existing`**, `workloadRegion` must equal the existing VNet region. + `controlPlaneRegion` does not need to match it. - For **`byo-existing`**, Stage 2 skips the VNet/network-template deployment. Before Stage 3, make sure that: - Each SUBNET row's CIDR matches an existing subnet. Stage 1 maps rows to subnet names by CIDR and fails `byo-subnets` on a mismatch. diff --git a/utilities/onboarding-tool/scripts/stage1-planning/stage1_plan.ps1 b/utilities/onboarding-tool/scripts/stage1-planning/stage1_plan.ps1 index 749d3396..0b064133 100755 --- a/utilities/onboarding-tool/scripts/stage1-planning/stage1_plan.ps1 +++ b/utilities/onboarding-tool/scripts/stage1-planning/stage1_plan.ps1 @@ -714,10 +714,10 @@ function Invoke-Stage1Checks { $existing = Get-AzJson -Args @('network', 'vnet', 'show', '-g', $byoRg, '-n', $byoVnet, '--subscription', $Cfg.subscriptionId) -AllowFail if ($existing) { $loc = [string]$existing.location - $status = if ($loc -and $Cfg.controlPlaneRegion -and $loc -ne $Cfg.controlPlaneRegion) { 'Fail' } else { 'Pass' } + $status = if ($loc -and $Cfg.workloadRegion -and $loc -ne $Cfg.workloadRegion) { 'Fail' } else { 'Pass' } $R.Add((New-CheckResult -Id 'byo-vnet' -Name 'Existing VNet (byo-existing)' -Status $status -Fr 'FR1.2' ` -Detail "VNet '$byoVnet' found in '$byoRg' (location=$loc, address space=$(@($existing.addressSpace.addressPrefixes) -join ','))." ` - -Remediation ($status -eq 'Fail' ? "Set controlPlaneRegion/workloadRegion to '$loc' (the VNet's region) or use a VNet in '$($Cfg.controlPlaneRegion)'." : '') ` + -Remediation ($status -eq 'Fail' ? "Set workloadRegion to '$loc' (the VNet's region) or use a VNet in '$($Cfg.workloadRegion)'. controlPlaneRegion can differ." : '') ` -Data @{ vnetId = $existing.id; location = $loc })) # Customer subnets keep their own names; later stages default to snet-, so # record each existing subnet's real name, matched by the planned CIDR. diff --git a/utilities/onboarding-tool/scripts/stage4-deployment/discovery-platform.bicep b/utilities/onboarding-tool/scripts/stage4-deployment/discovery-platform.bicep index 97f7f20d..30ca008d 100644 --- a/utilities/onboarding-tool/scripts/stage4-deployment/discovery-platform.bicep +++ b/utilities/onboarding-tool/scripts/stage4-deployment/discovery-platform.bicep @@ -6,10 +6,10 @@ targetScope = 'resourceGroup' -@description('Region for all Discovery resources. Must equal the BYO VNet/subnet region.') +@description('Control-plane region for Discovery resources. Can differ from the VNet/subnet and managed-resource-group region.') param location string -@description('workloadRegion tag (discovery.overridemrgregion) for the managed RGs.') +@description('Region for all managed-resource-group resources, set through discovery.overridemrgregion. Must match the VNet/subnet region.') param workloadRegion string = location param supercomputerName string diff --git a/utilities/onboarding-tool/scripts/stage4-deployment/discovery-platform.json b/utilities/onboarding-tool/scripts/stage4-deployment/discovery-platform.json index 5de586d3..987f8d3b 100644 --- a/utilities/onboarding-tool/scripts/stage4-deployment/discovery-platform.json +++ b/utilities/onboarding-tool/scripts/stage4-deployment/discovery-platform.json @@ -4,22 +4,22 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.41.2.15936", - "templateHash": "3453505155457192370" + "version": "0.39.26.7824", + "templateHash": "12946762802184486479" } }, "parameters": { "location": { "type": "string", "metadata": { - "description": "Region for all Discovery resources. Must equal the BYO VNet/subnet region." + "description": "Control-plane region for Discovery resources. Can differ from the VNet/subnet and managed-resource-group region." } }, "workloadRegion": { "type": "string", "defaultValue": "[parameters('location')]", "metadata": { - "description": "workloadRegion tag (discovery.overridemrgregion) for the managed RGs." + "description": "Region for all managed-resource-group resources, set through discovery.overridemrgregion. Must match the VNet/subnet region." } }, "supercomputerName": { diff --git a/utilities/onboarding-tool/scripts/stage4-deployment/stage4_deploy.ps1 b/utilities/onboarding-tool/scripts/stage4-deployment/stage4_deploy.ps1 index b53f6eaa..965fa031 100755 --- a/utilities/onboarding-tool/scripts/stage4-deployment/stage4_deploy.ps1 +++ b/utilities/onboarding-tool/scripts/stage4-deployment/stage4_deploy.ps1 @@ -97,13 +97,13 @@ function Resolve-DiscoveryError { [pscustomobject]@{ Signature = 'Creation not supported in region' RootCause = 'the target region is not a Discovery-creatable region for this resource type (the ARM-advertised location list can be broader than what the service allows for creation)' - Remediation = 'deploy into a Discovery-creatable region (for supercomputers/storageContainers eastus is confirmed creatable; eastus2 is rejected despite being advertised) and place the BYO VNet/subnets in that same region so the supercomputer stays co-regional with its node pool subnet' + Remediation = 'set controlPlaneRegion to a Discovery-creatable region for the affected resource type; independently, keep the VNet/subnets and all managed-resource-group resources in workloadRegion' Match = { param($t) $t -match '(?is)Creation of new .* resources is not supported in region|is not supported in region' } } [pscustomobject]@{ Signature = 'RegionMismatch on supercomputer/nodePool SubnetId' - RootCause = 'supercomputer location (controlPlaneRegion) differs from the BYO VNet/subnet region' - Remediation = 'set controlPlaneRegion to the region of the managedcluster/nodepool VNet so the supercomputer is co-regional with its subnets, then delete the failed supercomputer and re-run (region is immutable)' + RootCause = 'the managed-resource-group workload region differs from the BYO VNet/subnet region' + Remediation = 'set workloadRegion to the VNet/subnet region, or use a VNet in workloadRegion; all managed-resource-group resources must share that region. controlPlaneRegion can differ. Existing managed resources may need to be recreated to change region' Match = { param($t) $t -match '(?is)RegionMismatch|Region mismatch for dependent resource|must be in the same region as the primary resource' } } [pscustomobject]@{ @@ -414,6 +414,27 @@ function Get-VNetRegionFromSubnetId { return $loc.Trim() } +function Get-Stage4NetworkRegionFailures { + param([object]$Config, [string]$ResourceGroup) + + $region = [string]$Config.workloadRegion + $roles = @('managedcluster', 'nodepool', 'agent-containerapp', 'workspace-containerapp', 'search-containerapp', 'private-endpoints') + $outboundType = Get-ObjProp -Object $Config.network -Name 'outboundType' + if (-not $outboundType -or $outboundType -eq 'UserDefinedRouting') { $roles += 'spare' } + foreach ($role in $roles) { + $subnetId = Get-SubnetIdByRole -Config $Config -Role $role -ResourceGroup $ResourceGroup + if (-not $subnetId) { continue } + $actual = Get-VNetRegionFromSubnetId -SubnetId $subnetId + if (-not $actual -or $actual -ne $region) { + $detail = if ($actual) { "RegionMismatch workloadRegion=$region actual=$actual" } else { "Could not resolve VNet region for workloadRegion=$region" } + New-CheckResult -Id 'deploy-network-region' -Name 'workload and VNet region alignment' -Status 'Fail' -Fr 'FR4.1' ` + -Detail "$detail role=$role subnet=$subnetId provisioningState=NotSubmitted" ` + -Remediation "Verify the VNet is readable and in workloadRegion ($region). Set workloadRegion to the VNet region or use subnets in workloadRegion; all managed-resource-group resources must share that region. controlPlaneRegion can differ." ` + -Data ([pscustomobject]@{ expected = $region; actual = $actual; role = $role; subnetId = $subnetId }) + } + } +} + function Get-ManagedIdentityId { param([object]$Config, [string]$ResourceGroup) foreach ($containerName in @('managedIdentity', 'workspaceIdentity', 'identity')) { @@ -882,22 +903,16 @@ $chatModelId = "$workspaceId/chatModelDeployments/gpt-5-4" $storageContainerId = New-DiscoveryId -SubscriptionId $subscriptionId -ResourceGroup $resourceGroup -TypePath 'storageContainers' -NamePath $storageContainerName $projectId = "$workspaceId/projects/$projectName" -# Region co-location pre-flight (applies to both Bicep and ARM paths). The supercomputer must be -# co-regional with its managedcluster/nodepool subnets; discovery.overridemrgregion does not relax it. -if (-not $Recovery -and $location) { - $preflightSubnet = Get-SubnetIdByRole -Config $cfg -Role 'managedcluster' -ResourceGroup $resourceGroup - if ($preflightSubnet) { - $preflightRegion = Get-VNetRegionFromSubnetId -SubnetId $preflightSubnet - if ($preflightRegion -and $preflightRegion -ne $location) { - $results.Add((New-CheckResult -Id 'deploy-supercomputer' -Name 'supercomputer deployment' -Status 'Fail' -Fr 'FR4.1' ` - -Detail "resource=$supercomputerId provisioningState=NotSubmitted RegionMismatch expected=$location actual=$preflightRegion subnet=$preflightSubnet" ` - -Remediation "controlPlaneRegion ($location) must match the managedcluster/nodepool VNet region ($preflightRegion); the supercomputer and its node pool subnet must be co-regional. Set controlPlaneRegion=$preflightRegion, delete the failed supercomputer, and re-run (region is immutable)." ` - -Data ([pscustomobject]@{ expected = $location; actual = $preflightRegion; subnetId = $preflightSubnet }))) - if ($PassThru) { return $results.ToArray() } - $null = Write-OnboardingReport -Results $results.ToArray() -Title 'Stage 4 · deploy order (FR4.1-FR4.2)' -JsonPath $JsonPath - Complete-Stage -Results $results.ToArray() - } +# Validate workload/VNet alignment before either Bicep or ARM submits resources. +if (-not $Recovery) { + $regionFailures = @(Get-Stage4NetworkRegionFailures -Config $cfg -ResourceGroup $resourceGroup) + if ($regionFailures.Count) { + $results.AddRange($regionFailures) + if ($PassThru) { return $results.ToArray() } + $null = Write-OnboardingReport -Results $results.ToArray() -Title 'Stage 4 · deploy order (FR4.1-FR4.2)' -JsonPath $JsonPath + Complete-Stage -Results $results.ToArray() } + $preflightSubnet = Get-SubnetIdByRole -Config $cfg -Role 'managedcluster' -ResourceGroup $resourceGroup # Subnet-topology pre-flight. Supercomputer system nodes join the managedcluster subnet, so it must # carry no delegation; with UserDefinedRouting the API server needs a separate management subnet # ('spare' role / managementSubnet) delegated to Microsoft.ContainerService/managedClusters. @@ -985,23 +1000,11 @@ if (-not $BicepPath) { if (-not $location) { $scMissing += 'controlPlaneRegion' } if (-not $managedIdentityId) { $scMissing += 'managedIdentity.id or names.managedIdentity' } if (-not $scSubnet) { $scMissing += 'managedcluster subnet id/name' } - $regionMismatch = $null - if ($location -and $scSubnet) { - $subnetRegion = Get-VNetRegionFromSubnetId -SubnetId $scSubnet - if ($subnetRegion -and $subnetRegion -ne $location) { - $regionMismatch = [pscustomobject]@{ expected = $location; actual = $subnetRegion; subnetId = $scSubnet } - } - } if ($scMissing.Count) { $results.Add((New-CheckResult -Id 'deploy-supercomputer' -Name 'supercomputer deployment' -Status 'Fail' -Fr 'FR4.1' ` -Detail "resource=$supercomputerId provisioningState=NotSubmitted missing=$($scMissing -join ',')" ` -Remediation 'Supply the missing config values or provide -BicepPath/-ParametersPath for the platform template.' ` -Data ([pscustomobject]@{ resource = $supercomputerId; provisioningState = 'NotSubmitted'; missing = $scMissing }))) - } elseif ($regionMismatch) { - $results.Add((New-CheckResult -Id 'deploy-supercomputer' -Name 'supercomputer deployment' -Status 'Fail' -Fr 'FR4.1' ` - -Detail "resource=$supercomputerId provisioningState=NotSubmitted RegionMismatch expected=$($regionMismatch.expected) actual=$($regionMismatch.actual) subnet=$($regionMismatch.subnetId)" ` - -Remediation "controlPlaneRegion ($($regionMismatch.expected)) must match the managedcluster/nodepool VNet region ($($regionMismatch.actual)); the supercomputer and its node pool subnet must be co-regional. Set controlPlaneRegion=$($regionMismatch.actual), delete the failed supercomputer, and re-run (region is immutable)." ` - -Data $regionMismatch)) } else { $scNet = Get-ObjProp -Object $cfg -Name 'network' $scOutboundType = Get-ObjProp -Object $scNet -Name 'outboundType' diff --git a/utilities/onboarding-tool/tests/Stage1Config.Tests.ps1 b/utilities/onboarding-tool/tests/Stage1Config.Tests.ps1 index b0caa852..68bdd9e8 100644 --- a/utilities/onboarding-tool/tests/Stage1Config.Tests.ps1 +++ b/utilities/onboarding-tool/tests/Stage1Config.Tests.ps1 @@ -9,7 +9,7 @@ $ast = [System.Management.Automation.Language.Parser]::ParseFile( [ref]$errors ) if ($errors.Count) { throw ($errors.Message -join [Environment]::NewLine) } -foreach ($name in @('Read-XlsxSheet', 'Split-CellRef', 'ConvertFrom-ConfigSheet', 'ConvertTo-ConfigModel', 'Merge-ConfigDefaults', 'Get-RequiredVmSkuResult')) { +foreach ($name in @('Read-XlsxSheet', 'Split-CellRef', 'ConvertFrom-ConfigSheet', 'ConvertTo-ConfigModel', 'Merge-ConfigDefaults', 'Get-RequiredVmSkuResult', 'Test-Field', 'Get-Stage1PropertyValue', 'Get-LivePolicyAssignments', 'Invoke-Stage1Checks')) { $functionAst = $ast.Find({ param($node) $node -is [System.Management.Automation.Language.FunctionDefinitionAst] -and $node.Name -eq $name @@ -17,6 +17,56 @@ foreach ($name in @('Read-XlsxSheet', 'Split-CellRef', 'ConvertFrom-ConfigSheet' if (-not $functionAst) { throw "$name was not found in $stage1Path." } Invoke-Expression $functionAst.Extent.Text } +foreach ($statement in $ast.EndBlock.Statements) { + if ($statement -is [System.Management.Automation.Language.AssignmentStatementAst] -and + $statement.Left -is [System.Management.Automation.Language.VariableExpressionAst] -and + $statement.Left.VariablePath.UserPath -like 'script:*') { + Invoke-Expression $statement.Extent.Text + } +} + +Describe 'Stage 1 existing VNet region alignment' { + BeforeEach { + $configPath = Join-Path (Split-Path -Parent $stage1Path) '..\examples\sample-config.json' + $config = Get-Content -LiteralPath $configPath -Raw | ConvertFrom-Json + $defaults = ConvertTo-ConfigModel -Parsed ([pscustomobject]@{ Scalars = @{}; Subnets = @(); Policies = @(); Waivers = @() }) | + ConvertTo-Json -Depth 20 | ConvertFrom-Json + $config = Merge-ConfigDefaults -Config $config -Defaults $defaults + $config.network.model = 'byo-existing' + $script:existingVnet = [pscustomobject]@{ + id = '/subscriptions/sub/resourceGroups/network-rg/providers/Microsoft.Network/virtualNetworks/vnet' + location = 'uksouth' + addressSpace = [pscustomobject]@{ addressPrefixes = @($config.network.vnetCidr) } + subnets = @($config.network.subnets | ForEach-Object { + [pscustomobject]@{ name = "snet-$($_.role)"; addressPrefix = $_.cidr } + }) + } + Mock Get-AzJson { $null } + Mock Get-AzJson { [pscustomobject]@{ id = 'sub' } } -ParameterFilter { $Args[0] -eq 'account' } + Mock Get-AzJson { $script:existingVnet } -ParameterFilter { $Args[0] -eq 'network' } + Mock Get-VcpuQuotaResult {} + Mock Get-ModelQuotaResult {} + Mock Get-LivePolicyAssignments { @() } + } + + It 'checks the workload region independently of the control plane' -TestCases @( + @{ Control = 'eastus'; Workload = 'uksouth'; Expected = 'Pass' }, + @{ Control = 'uksouth'; Workload = 'uksouth'; Expected = 'Pass' }, + @{ Control = 'uksouth'; Workload = 'eastus'; Expected = 'Fail' } + ) { + param($Control, $Workload, $Expected) + $config.controlPlaneRegion = $Control + $config.workloadRegion = $Workload + + $result = Invoke-Stage1Checks -Cfg $config | Where-Object id -eq 'byo-vnet' + + $result.status | Should Be $Expected + if ($Expected -eq 'Fail') { + $result.remediation | Should Match "Set workloadRegion to 'uksouth'" + $result.remediation | Should Not Match 'Set controlPlaneRegion' + } + } +} Describe 'Stage 1 required VM SKU availability' { It 'pins the live SKU catalog query to the configured subscription' { diff --git a/utilities/onboarding-tool/tests/Stage4State.Tests.ps1 b/utilities/onboarding-tool/tests/Stage4State.Tests.ps1 index 281c88a1..fbb343ef 100644 --- a/utilities/onboarding-tool/tests/Stage4State.Tests.ps1 +++ b/utilities/onboarding-tool/tests/Stage4State.Tests.ps1 @@ -13,6 +13,15 @@ if ($errors.Count) { throw ($errors.Message -join [Environment]::NewLine) } foreach ($name in @( 'Get-WorkspacePublicNetworkAccess', 'Get-WorkspaceNetworkIsolation', + 'Get-ObjProp', + 'Get-ConfigName', + 'Get-Stage4VNetName', + 'Get-SubnetIdByRole', + 'Get-VNetRegionFromSubnetId', + 'Get-Stage4NetworkRegionFailures', + 'Get-ManagedIdentityId', + 'Get-StorageAccountId', + 'New-Stage4BicepParamFile', 'Resolve-DiscoveryError', 'Get-DiscoveryArmState', 'Get-FoundryAccountState', @@ -26,6 +35,78 @@ foreach ($name in @( Invoke-Expression $functionAst.Extent.Text } +Describe 'Stage 4 workload and VNet region alignment' { + BeforeEach { + $configPath = Join-Path (Split-Path -Parent $stage4Path) '..\examples\sample-config.json' + $config = Get-Content -LiteralPath $configPath -Raw | ConvertFrom-Json + $config.controlPlaneRegion = 'eastus' + $config.workloadRegion = 'uksouth' + Mock Get-VNetRegionFromSubnetId { 'uksouth' } + } + + It 'allows separate control-plane and workload regions' { + @(Get-Stage4NetworkRegionFailures -Config $config -ResourceGroup $config.resourceGroup).Count | Should Be 0 + } + + It 'preserves same-region deployments' { + $config.controlPlaneRegion = 'uksouth' + + @(Get-Stage4NetworkRegionFailures -Config $config -ResourceGroup $config.resourceGroup).Count | Should Be 0 + } + + It 'rejects a subnet in the control-plane region instead of the workload region' -TestCases @( + @{ Role = 'managedcluster' }, + @{ Role = 'nodepool' }, + @{ Role = 'agent-containerapp' }, + @{ Role = 'workspace-containerapp' }, + @{ Role = 'search-containerapp' }, + @{ Role = 'private-endpoints' }, + @{ Role = 'spare' } + ) { + param($Role) + $script:mismatchedRole = $Role + Mock Get-VNetRegionFromSubnetId { 'eastus' } -ParameterFilter { $SubnetId -like "*/snet-$script:mismatchedRole" } + + $failures = @(Get-Stage4NetworkRegionFailures -Config $config -ResourceGroup $config.resourceGroup) + + $failures.Count | Should Be 1 + $failures[0].status | Should Be 'Fail' + $failures[0].data.role | Should Be $Role + $failures[0].data.expected | Should Be 'uksouth' + $failures[0].data.actual | Should Be 'eastus' + $failures[0].remediation | Should Match 'controlPlaneRegion can differ' + } + + It 'fails explicitly when the VNet region cannot be read' { + Mock Get-VNetRegionFromSubnetId { '' } + + $failures = @(Get-Stage4NetworkRegionFailures -Config $config -ResourceGroup $config.resourceGroup) + + ($failures.Count -gt 0) | Should Be $true + $failures[0].detail | Should Match 'Could not resolve VNet region' + } + + It 'does not validate the unused management subnet with LoadBalancer egress' { + $config.network.outboundType = 'LoadBalancer' + Mock Get-VNetRegionFromSubnetId { 'eastus' } -ParameterFilter { $SubnetId -like '*/snet-spare' } + + @(Get-Stage4NetworkRegionFailures -Config $config -ResourceGroup $config.resourceGroup).Count | Should Be 0 + } + + It 'keeps the control-plane location and MRG workload region separate in Bicep parameters' { + $path = New-Stage4BicepParamFile -Config $config -ResourceGroup $config.resourceGroup -Location $config.controlPlaneRegion + try { + $parameters = Get-Content -LiteralPath $path -Raw | ConvertFrom-Json + + $parameters.parameters.location.value | Should Be 'eastus' + $parameters.parameters.workloadRegion.value | Should Be 'uksouth' + } + finally { + Remove-Item -LiteralPath $path -Force + } + } +} + Describe 'Stage 4 workspace configuration defaults' { It 'uses secure network-isolation and explicit public-access defaults' { $config = [pscustomobject]@{} @@ -132,6 +213,13 @@ Describe 'Resolve-DiscoveryError' { Should Match 'RegionMismatch' } + It 'does not require changing the control-plane region to resolve a subnet-region mismatch' { + $result = Resolve-DiscoveryError -ErrorText 'RegionMismatch for dependent resource SubnetId' + + $result.remediation | Should Match 'set workloadRegion' + $result.remediation | Should Not Match 'set controlPlaneRegion' + } + It 'maps policy failures by affected resource type' { (Resolve-DiscoveryError -ErrorText 'RequestDisallowedByPolicy Microsoft.CognitiveServices/accounts publicNetworkAccess').errorSignature | Should Match 'Foundry'