From 3284fb3f5c2f352574b052923a6f81aa17aed145 Mon Sep 17 00:00:00 2001 From: Ayushh Garg <259261949+ayushhgarg-work@users.noreply.github.com> Date: Wed, 11 Nov 2026 19:11:31 +0530 Subject: [PATCH] Update AI Gateway Bicep sample to standalone aigateways resources Use the Standard SKU, compatible preview child operations and payloads, a callable model alias, returned runtime hostname, and scoped permission guidance. Regenerate the sample ARM template and update its parameters and documentation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../01-connections/ai-gateway-tier/README.md | 120 +++++++++++++++--- .../ai-gateway-tier/azuredeploy.json | 95 +++++++------- .../ai-gateway-tier/connection.bicep | 4 +- .../01-connections/ai-gateway-tier/main.bicep | 65 ++++++---- .../ai-gateway-tier/metadata.json | 6 +- .../samples/connection.parameters.json | 2 +- .../ai-gateway-tier/samples/parameters.json | 2 - 7 files changed, 199 insertions(+), 95 deletions(-) diff --git a/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/README.md b/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/README.md index fc38bb401..1468cdd43 100644 --- a/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/README.md +++ b/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/README.md @@ -1,5 +1,5 @@ --- -description: AI Gateway tier (preview) hub-and-spoke module for the Foundry golden path. main.bicep deploys a centralized AI Gateway tier (Azure API Management AIGateway SKU) that fronts a Foundry model over managed identity, with a token-limit policy and a runtime key; connection.bicep attaches a consumer (spoke) project to it with an "Admin-connected models" ModelGateway connection, same subscription or cross-subscription. An agent then references the model as / via the Responses API. +description: AI Gateway tier (preview) hub-and-spoke module for the Foundry golden path. main.bicep deploys a standalone Microsoft.ApiManagement/aigateways resource that fronts a Foundry model over managed identity, with a callable model alias, a token-limit policy, and a runtime key; connection.bicep attaches a consumer (spoke) project to it with an "Admin-connected models" ModelGateway connection, same subscription or cross-subscription. An agent then references the model as / via the Responses API. page_type: sample products: - azure @@ -21,29 +21,109 @@ Foundry projects/subscriptions. | File | Role | Run | |------|------|-----| -| [`main.bicep`](./main.bicep) | **Hub** — Foundry account + hub project + model + AI Gateway tier (`AIGateway` SKU) + managed-identity provider + token-limit policy + runtime key | Once (the admin) | +| [`main.bicep`](./main.bicep) | **Hub** — Foundry account + hub project + model + standalone AI Gateway (`Microsoft.ApiManagement/aigateways`) + managed-identity provider + callable model alias + token-limit policy + runtime key | Once (the admin) | | [`connection.bicep`](./connection.bicep) | **Spoke** — an "Admin-connected models" (`ModelGateway` + `ApiKey`) connection on an existing project, pointed at the hub | Per consumer project (same or cross-subscription) | > [!IMPORTANT] -> The AI Gateway tier is a **release-gated public preview**. The gateway is -> `Microsoft.ApiManagement/service` with the **`AIGateway` SKU** (`2025-09-01-preview`), which -> deploys only where it is enabled — currently **East US 2** and **Sweden Central**. It sits on its -> own **`gatewayLocation`** so the account/model can live in another region. `az bicep build` emits -> **BCP081** warnings for the preview `modelProviders`/`models`/`apiKeys` types (expected). +> AI Gateway is a **release-gated public preview**. This sample uses the standalone +> `Microsoft.ApiManagement/aigateways` resource with API version **`2026-11-01`**, not an API +> Management `service`. Its workspace, provider, model, alias, and API-key operations use +> **`2026-09-01-preview`** with the corresponding preview payloads. It deploys only where the +> preview is enabled — currently **East US 2** and **Sweden Central**. It sits on its own +> **`gatewayLocation`** so the account/model can live in another region. +> +> This is a different Azure resource type, **not an in-place upgrade** of an existing +> `Microsoft.ApiManagement/service`. Existing APIM resources are not removed by this template. +> The obsolete `publisherEmail` and `publisherName` parameters have been removed. + +The runtime URL comes from the gateway's returned `properties.frontend.defaultHostname`; do not +assume it is `.azure-api.net`. Bicep type catalogs that do not yet include the +resource/API-version combinations can emit **BCP081** warnings and cannot validate those resource +bodies. + +The standalone gateway uses **`sku.name: 'Standard'`**. The former APIM `AIGateway` SKU name is +rejected by standalone gateway creation with `InvalidSkuName`. + +## API versions and compatibility + +The published November contract is valid, but its availability must be checked **per operation**. +A successful parent gateway GET does not establish support for provider/model/key creation or +`listSecrets`. + +| Operation | Version used by this sample | Compatibility check in Sweden Central (2026-11-11) | +|-----------|-----------------------------|----------------------------------------------------| +| Parent gateway | `2026-11-01` | GET returned HTTP 200; isolated gateway creation succeeded with SKU `Standard`. | +| Default workspace GET | `2026-09-01-preview` | HTTP 200; November returned HTTP 400 `MissingOrIncorrectVersionParameter`. | +| Provider PUT / GET | `2026-09-01-preview` | Temporary provider creation returned HTTP 201; GET preserved managed-identity authentication. November PUT and GET rejected the version. | +| Model PUT / GET | `2026-09-01-preview` | Temporary registration returned HTTP 201; GET preserved both endpoint paths and the token policy. November PUT and GET rejected the version. | +| Callable model alias | `2026-09-01-preview` | Isolated alias deployment succeeded; GET returned both runtime endpoints. Alias name matches the caller's model name. | +| Runtime key PUT / GET | `2026-09-01-preview` | Temporary key creation returned HTTP 201; an existing key's GET returned HTTP 200. November PUT and GET rejected the version. | +| Runtime key `listSecrets` | `2026-09-01-preview` | HTTP 200 with both generated keys; November returned HTTP 400. Key values were not logged. | + +Keep the API version and payload together: the tested preview provider/model bodies use +`displayName` and full endpoint paths (`/openai/v1/chat/completions`, `/openai/v1/responses`). +The November contract uses `name`; its examples use endpoint identifiers such as `chatCompletions`. +The token policy's `counterKey` remains an array containing `identity`; this was verified on +read-back, not reverted to the older string form. + +Registered models are backend assets; the gateway also needs a **callable alias**. `gatewayAlias` +maps the unchanged caller model name to `gatewayModel`, using the preview routing order +`selectionOrder: ['priority']`. Registration alone left the isolated runtime endpoints returning +HTTP 404; publishing the alias made both endpoints callable. + +The temporary child probes were deleted. An isolated gateway and connection were retained for the +live checks below. Availability can vary by rollout; recheck the target subscription and region +before changing either version or payload. + +## Live validation result + +The 2026-11-11 Sweden Central check reused an existing account, project, and `gpt-5-mini` +deployment (`2025-08-07`), rather than creating another backend. + +| Check | Result | +|-------|--------| +| Compile, generated ARM consistency, Azure validation, and scoped what-if | **PASS** | +| Isolated gateway, provider/model/alias/key, scoped Foundry User assignment, and project connection | **PASS** | +| Direct Chat Completions and Responses inference | **PASS** — HTTP 200 and non-empty expected model text. | +| Gateway token policy at 100 TPM | **PASS** — a bounded burst returned HTTP 429 with `Token limit is exceeded`; invalid keys returned HTTP 401. | +| Existing backend, existing connections, and previous gateway unchanged | **PASS** — baseline comparisons matched, excluding the intended new connection. | +| Prompt-agent inference through the project connection | **BLOCKED** — the test tenant reached its directory-object quota during agent identity creation. Connection configuration and stored credentials were verified, but are not an inference pass. | + +This does not validate the default `gpt-5.4` deployment or creation of a new Foundry backend. +Subscription Owner access does not resolve a tenant directory-object quota; that requires tenant +administration or the support contact in the service error. ## Prerequisites -1. **Azure CLI** logged in (`az login`) and the **`AIGateway` preview enabled** in East US 2 / Sweden Central. +1. **Azure CLI** logged in (`az login`) and the **AI Gateway preview enabled** in East US 2 / Sweden Central. 2. Quota for the model (`gpt-5.4`, `GlobalStandard`) in `location`. -3. Admin roles — see [Personas & RBAC](../../golden-path/README.md#personas--rbac-roles): **Foundry - Account Owner** on the resource **+ API Management Service Contributor** on the resource group. +3. Scoped Azure permissions as described below. GitHub repository access does not grant Azure + deployment or role-assignment permissions. 4. `pip install "azure-ai-projects>=2.0.0" azure-identity` for the agent step. +### Required permissions + +| Identity / task | Scope | Required access | +|-----------------|-------|-----------------| +| Hub deployer: create/update the Foundry account, model, project, gateway, and child resources | Hub resource group | **Contributor**, or a custom equivalent covering the required `Microsoft.CognitiveServices/accounts/*`, `Microsoft.ApiManagement/aigateways/*`, and deployment operations. | +| Hub deployer: grant the gateway access to Foundry | Created Foundry account, inherited from the hub resource group for a first deployment | `Microsoft.Authorization/roleAssignments/write`, for example **Role Based Access Control Administrator**. Contributor alone does not grant this permission. Administrators can constrain the grant to the **Foundry User** role and managed-identity principals. | +| Gateway system-assigned identity | Hub Foundry account | **Foundry User** (`53ca6127-db72-4b80-b1b0-d745d6d5456d`), assigned by `foundryUserRole` in `main.bicep`. Keep this assignment for keyless backend access. | +| Operator retrieving the runtime key | Gateway | `Microsoft.ApiManagement/aigateways/apiKeys/listSecrets/action`, included in the hub deployer's Contributor access. Reader alone is insufficient. | +| Spoke connection deployer | Spoke resource group | **Contributor**, or equivalent project-connection write and deployment permissions. | +| SDK caller creating/invoking the prompt agent | Spoke Foundry project | **Foundry User**. Management-plane Contributor access does not by itself grant this data-plane access. | + +**API Management Service Contributor is not sufficient for this gateway:** it grants +`Microsoft.ApiManagement/service/*`, not `Microsoft.ApiManagement/aigateways/*`. The hub +deployer needs both resource-deployment access and the scoped role-assignment access above +(or equivalent inherited permissions). Activate eligible PIM roles before deploying. +Creating a new resource group additionally requires resource-group creation permission at +subscription scope; an administrator can instead create the test resource group for you. + ## 1. Deploy the hub (once) ```powershell $rg = "" -$loc = "eastus2" # eastus2 or swedencentral (AIGateway preview regions) +$loc = "eastus2" # eastus2 or swedencentral (AI Gateway preview regions) az group create --name $rg --location $loc az deployment group create --resource-group $rg --template-file main.bicep --parameters "@samples/parameters.json" @@ -53,7 +133,7 @@ Capture the outputs the spoke connection needs: ```powershell $gwUrl = az deployment group show -g $rg -n main --query properties.outputs.gatewayModelsBaseUrl.value -o tsv -$gwName = az deployment group show -g $rg -n main --query properties.outputs.gatewayName.value -o tsv +$gwId = az deployment group show -g $rg -n main --query properties.outputs.gatewayId.value -o tsv ``` ## 2. Get the gateway runtime key @@ -62,9 +142,8 @@ The key is intentionally **not** a deployment output. Read it from the gateway ( `apiKeys/default`): ```powershell -$sub = (az account show --query id -o tsv) $key = az rest --method post ` - --url "https://management.azure.com/subscriptions/$sub/resourceGroups/$rg/providers/Microsoft.ApiManagement/service/$gwName/apiKeys/default/listSecrets?api-version=2025-09-01-preview" ` + --url "https://management.azure.com$gwId/apiKeys/default/listSecrets?api-version=2026-09-01-preview" ` --query primaryKey -o tsv ``` @@ -89,18 +168,22 @@ a **prompt agent** bound to `/`, invoked through the (a gateway-connected model does **not** resolve through the classic Assistants API). Point the SDK at the spoke project endpoint (the connection deployment's `projectEndpoint` output). +For interactive **direct gateway** inference, open the [AI Gateway portal](https://ai.gateway.azure.com), +select your gateway, open **Catalog**, select the model alias, choose **Chat completions** and a +runtime key, then send a short prompt. A 429 with `Token limit is exceeded` means the low sample +budget is throttling; wait for its retry interval before sending another prompt. + ## Parameters — `main.bicep` (hub) | Parameter | Default | Description | |-----------|---------|-------------| | `location` | `eastus2` | Region for the account, project, and model. Any region; can differ from `gatewayLocation`. | -| `gatewayLocation` | `eastus2` | Region for the AI Gateway tier (`AIGateway` SKU) — `eastus2` or `swedencentral`. | +| `gatewayLocation` | `eastus2` | Region for the standalone AI Gateway — `eastus2` or `swedencentral`. | | `aiServicesName` | `hub` | Base name for the hub account (a unique suffix is appended). | | `projectName` | `hub-project` | Hub project (admin workspace) created under the account. | | `modelName` / `modelFormat` / `modelVersion` | `gpt-5.4` / `OpenAI` / `2026-03-05` | Model deployed on the hub and imported into the gateway. | | `modelSkuName` / `modelCapacity` | `GlobalStandard` / `1` | Deployment SKU and capacity. Default `1` keeps the sample deployable anywhere; raise it for real workloads. | -| `gatewayName` | auto | Globally unique AI Gateway name (`.azure-api.net`). Auto-generated if empty. | -| `publisherEmail` / `publisherName` | `noreply@example.com` / `AI Gateway tier hub` | Required by the API Management service. | +| `gatewayName` | auto | AI Gateway resource name: 3–24 lowercase letters, digits or hyphens; start and end with a letter or digit. Auto-generated if empty. Use the returned runtime hostname. | | `tokensPerMinute` | `100` | Token-limit policy budget per caller identity. | ## Parameters — `connection.bicep` (spoke) @@ -120,4 +203,7 @@ at the spoke project endpoint (the connection deployment's `projectEndpoint` out - [Foundry golden path](../../golden-path/README.md) — this module is BYOM variant 5. - [Bring your own model to Foundry Agent Service](https://learn.microsoft.com/azure/foundry/agents/how-to/ai-gateway) — prompt agent + Responses API. - [Manage models and tools on the AI Gateway](https://learn.microsoft.com/azure/api-management/ai-gateway-manage-models-tools). +- [AI Gateway ARM API contract (`2026-11-01`)](https://github.com/Azure/azure-rest-api-specs/blob/main/specification/apimanagement/resource-manager/Microsoft.ApiManagement/AIGateway/stable/2026-11-01/openapi.json). +- [AI Gateway ARM API contract (`2026-09-01-preview`)](https://github.com/Azure/azure-rest-api-specs/blob/main/specification/apimanagement/resource-manager/Microsoft.ApiManagement/AIGateway/preview/2026-09-01-preview/openapi.json) — the published preview specification covers the parent resource, not the child operation payloads tested above. +- [Assign Azure roles](https://learn.microsoft.com/azure/role-based-access-control/role-assignments-portal) — prerequisites for the gateway-to-Foundry role assignment. - [Role-based access control for Microsoft Foundry](https://learn.microsoft.com/azure/foundry/concepts/rbac-foundry). diff --git a/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/azuredeploy.json b/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/azuredeploy.json index f63c151b4..dfaedd4a7 100644 --- a/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/azuredeploy.json +++ b/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/azuredeploy.json @@ -4,8 +4,8 @@ "metadata": { "_generator": { "name": "bicep", - "version": "0.46.1.21595", - "templateHash": "2257742962386746331" + "version": "0.49.2.45890", + "templateHash": "1224238146732485709" } }, "parameters": { @@ -53,7 +53,7 @@ "swedencentral" ], "metadata": { - "description": "Region for the AI Gateway tier (AIGateway SKU), release-gated to eastus2 / swedencentral. Separate from location so the account and model can deploy in another region while the gateway stays supported." + "description": "Region for the standalone AI Gateway, release-gated to eastus2 / swedencentral. Separate from location so the account and model can deploy in another region while the gateway stays supported." } }, "modelName": { @@ -94,22 +94,9 @@ "gatewayName": { "type": "string", "defaultValue": "", + "maxLength": 24, "metadata": { - "description": "Globally unique AI Gateway name. Resolves to .azure-api.net. Leave empty to auto-generate." - } - }, - "publisherEmail": { - "type": "string", - "defaultValue": "noreply@example.com", - "metadata": { - "description": "Publisher email required by the API Management service at create time." - } - }, - "publisherName": { - "type": "string", - "defaultValue": "AI Gateway tier hub", - "metadata": { - "description": "Publisher organization name required by the API Management service at create time." + "description": "AI Gateway resource name: 3-24 lowercase letters, digits or hyphens, starting and ending with a letter or digit. The service returns its runtime hostname. Leave empty to auto-generate." } }, "tokensPerMinute": { @@ -188,40 +175,36 @@ ] }, { - "type": "Microsoft.ApiManagement/service", - "apiVersion": "2025-09-01-preview", + "type": "Microsoft.ApiManagement/aigateways", + "apiVersion": "2026-11-01", "name": "[variables('effectiveGatewayName')]", "location": "[parameters('gatewayLocation')]", "identity": { "type": "SystemAssigned" }, "sku": { - "name": "AIGateway", - "capacity": 1 + "name": "Standard" }, - "properties": { - "publisherEmail": "[parameters('publisherEmail')]", - "publisherName": "[parameters('publisherName')]" - } + "properties": {} }, { "type": "Microsoft.Authorization/roleAssignments", "apiVersion": "2022-04-01", "scope": "[resourceId('Microsoft.CognitiveServices/accounts', variables('accountName'))]", - "name": "[guid(resourceId('Microsoft.CognitiveServices/accounts', variables('accountName')), resourceId('Microsoft.ApiManagement/service', variables('effectiveGatewayName')), variables('foundryUserRoleId'))]", + "name": "[guid(resourceId('Microsoft.CognitiveServices/accounts', variables('accountName')), resourceId('Microsoft.ApiManagement/aigateways', variables('effectiveGatewayName')), variables('foundryUserRoleId'))]", "properties": { - "principalId": "[reference(resourceId('Microsoft.ApiManagement/service', variables('effectiveGatewayName')), '2025-09-01-preview', 'full').identity.principalId]", + "principalId": "[reference(resourceId('Microsoft.ApiManagement/aigateways', variables('effectiveGatewayName')), '2026-11-01', 'full').identity.principalId]", "principalType": "ServicePrincipal", "roleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', variables('foundryUserRoleId'))]" }, "dependsOn": [ "[resourceId('Microsoft.CognitiveServices/accounts', variables('accountName'))]", - "[resourceId('Microsoft.ApiManagement/service', variables('effectiveGatewayName'))]" + "[resourceId('Microsoft.ApiManagement/aigateways', variables('effectiveGatewayName'))]" ] }, { - "type": "Microsoft.ApiManagement/service/workspaces/modelProviders", - "apiVersion": "2025-09-01-preview", + "type": "Microsoft.ApiManagement/aigateways/workspaces/modelProviders", + "apiVersion": "2026-09-01-preview", "name": "[format('{0}/{1}/{2}', variables('effectiveGatewayName'), 'default', 'foundry')]", "properties": { "kind": "Foundry", @@ -241,18 +224,17 @@ }, "dependsOn": [ "[resourceId('Microsoft.CognitiveServices/accounts', variables('accountName'))]", - "[resourceId('Microsoft.ApiManagement/service', variables('effectiveGatewayName'))]", - "[extensionResourceId(resourceId('Microsoft.CognitiveServices/accounts', variables('accountName')), 'Microsoft.Authorization/roleAssignments', guid(resourceId('Microsoft.CognitiveServices/accounts', variables('accountName')), resourceId('Microsoft.ApiManagement/service', variables('effectiveGatewayName')), variables('foundryUserRoleId')))]", + "[resourceId('Microsoft.ApiManagement/aigateways', variables('effectiveGatewayName'))]", + "[extensionResourceId(resourceId('Microsoft.CognitiveServices/accounts', variables('accountName')), 'Microsoft.Authorization/roleAssignments', guid(resourceId('Microsoft.CognitiveServices/accounts', variables('accountName')), resourceId('Microsoft.ApiManagement/aigateways', variables('effectiveGatewayName')), variables('foundryUserRoleId')))]", "[resourceId('Microsoft.CognitiveServices/accounts/deployments', variables('accountName'), parameters('modelName'))]" ] }, { - "type": "Microsoft.ApiManagement/service/workspaces/modelProviders/models", - "apiVersion": "2025-09-01-preview", + "type": "Microsoft.ApiManagement/aigateways/workspaces/modelProviders/models", + "apiVersion": "2026-09-01-preview", "name": "[format('{0}/{1}/{2}/{3}', variables('effectiveGatewayName'), 'default', 'foundry', parameters('modelName'))]", "properties": { "displayName": "[parameters('modelName')]", - "apiFormat": "OpenAIChatCompletions", "supportedEndpoints": [ "/openai/v1/chat/completions", "/openai/v1/responses" @@ -267,24 +249,47 @@ "type": "tokenLimit", "period": "minute", "count": "[parameters('tokensPerMinute')]", - "counterKey": "Identity" + "counterKey": [ + "identity" + ] } ] }, "dependsOn": [ - "[resourceId('Microsoft.ApiManagement/service/workspaces/modelProviders', variables('effectiveGatewayName'), 'default', 'foundry')]", + "[resourceId('Microsoft.ApiManagement/aigateways/workspaces/modelProviders', variables('effectiveGatewayName'), 'default', 'foundry')]", "[resourceId('Microsoft.CognitiveServices/accounts/deployments', variables('accountName'), parameters('modelName'))]" ] }, { - "type": "Microsoft.ApiManagement/service/apiKeys", - "apiVersion": "2025-09-01-preview", + "type": "Microsoft.ApiManagement/aigateways/workspaces/aliases", + "apiVersion": "2026-09-01-preview", + "name": "[format('{0}/{1}/{2}', variables('effectiveGatewayName'), 'default', replace(parameters('modelName'), '.', '-'))]", + "properties": { + "aliasName": "[parameters('modelName')]", + "backendModels": [ + { + "resourceId": "[resourceId('Microsoft.ApiManagement/aigateways/workspaces/modelProviders/models', variables('effectiveGatewayName'), 'default', 'foundry', parameters('modelName'))]", + "priority": 0 + } + ], + "selectionOrder": [ + "priority" + ] + }, + "dependsOn": [ + "[resourceId('Microsoft.ApiManagement/aigateways', variables('effectiveGatewayName'))]", + "[resourceId('Microsoft.ApiManagement/aigateways/workspaces/modelProviders/models', variables('effectiveGatewayName'), 'default', 'foundry', parameters('modelName'))]" + ] + }, + { + "type": "Microsoft.ApiManagement/aigateways/apiKeys", + "apiVersion": "2026-09-01-preview", "name": "[format('{0}/{1}', variables('effectiveGatewayName'), 'default')]", "properties": { "displayName": "AI Gateway tier hub runtime key" }, "dependsOn": [ - "[resourceId('Microsoft.ApiManagement/service', variables('effectiveGatewayName'))]" + "[resourceId('Microsoft.ApiManagement/aigateways', variables('effectiveGatewayName'))]" ] } ], @@ -313,9 +318,13 @@ "type": "string", "value": "[variables('effectiveGatewayName')]" }, + "gatewayId": { + "type": "string", + "value": "[resourceId('Microsoft.ApiManagement/aigateways', variables('effectiveGatewayName'))]" + }, "gatewayModelsBaseUrl": { "type": "string", - "value": "[format('{0}/default/models/openai/v1', reference(resourceId('Microsoft.ApiManagement/service', variables('effectiveGatewayName')), '2025-09-01-preview').gatewayUrl)]" + "value": "[format('https://{0}/default/models/openai/v1', reference(resourceId('Microsoft.ApiManagement/aigateways', variables('effectiveGatewayName')), '2026-11-01').frontend.defaultHostname)]" }, "modelName": { "type": "string", @@ -331,7 +340,7 @@ }, "gatewayPrincipalId": { "type": "string", - "value": "[reference(resourceId('Microsoft.ApiManagement/service', variables('effectiveGatewayName')), '2025-09-01-preview', 'full').identity.principalId]" + "value": "[reference(resourceId('Microsoft.ApiManagement/aigateways', variables('effectiveGatewayName')), '2026-11-01', 'full').identity.principalId]" }, "location": { "type": "string", diff --git a/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/connection.bicep b/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/connection.bicep index 7c991f08f..a92426f62 100644 --- a/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/connection.bicep +++ b/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/connection.bicep @@ -13,7 +13,7 @@ once from the hub (it is intentionally NOT emitted as a deployment output): az rest --method post \ - --url "https://management.azure.com/apiKeys/default/listSecrets?api-version=2025-09-01-preview" \ + --url "https://management.azure.com/apiKeys/default/listSecrets?api-version=2026-09-01-preview" \ --query primaryKey -o tsv Verified against the same connection shape as the all-in-one AI Gateway tier @@ -26,7 +26,7 @@ targetScope = 'resourceGroup' @description('Resource ID of the EXISTING spoke project to attach (from golden-path steps 1-2). The account and project must live in THIS resource group.') param projectResourceId string -@description('Hub gateway OpenAI base URL — the hub deployment output gatewayModelsBaseUrl (e.g. https://.azure-api.net/default/models/openai/v1).') +@description('Hub gateway OpenAI base URL — the hub deployment output gatewayModelsBaseUrl (e.g. https:///default/models/openai/v1).') param gatewayModelsBaseUrl string @description('Runtime key for the hub gateway. Retrieve it from the hub gateway (listSecrets on its apiKeys/default) and pass it here; works cross-subscription.') diff --git a/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/main.bicep b/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/main.bicep index d0fae8dc8..06508c4f6 100644 --- a/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/main.bicep +++ b/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/main.bicep @@ -12,7 +12,7 @@ This template creates the HUB only: - Foundry account (AIServices, system-assigned identity, keyless backend) + a hub project (admin workspace) + a gpt model deployment. - - AI Gateway tier instance (Microsoft.ApiManagement/service, AIGateway SKU) with a + - Standalone AI Gateway instance (Microsoft.ApiManagement/aigateways) with a system-assigned identity and a Foundry model provider that imports the account over MANAGED IDENTITY (the gateway MI gets Foundry User on the account). - The model registered on the gateway with a token-limit policy, plus a runtime key. @@ -21,7 +21,7 @@ gatewayName + gatewayModelsBaseUrl + modelName), then create a prompt agent that references the model as / via the Responses API. - Release-gated preview: the AIGateway SKU deploys only where it is enabled + Release-gated preview: AI Gateway deploys only where it is enabled (East US 2, Sweden Central) — kept on its own gatewayLocation so the account and model can live elsewhere (e.g. a CI region). @@ -57,7 +57,7 @@ param location string = 'eastus2' 'eastus2' 'swedencentral' ]) -@description('Region for the AI Gateway tier (AIGateway SKU), release-gated to eastus2 / swedencentral. Separate from location so the account and model can deploy in another region while the gateway stays supported.') +@description('Region for the standalone AI Gateway, release-gated to eastus2 / swedencentral. Separate from location so the account and model can deploy in another region while the gateway stays supported.') param gatewayLocation string = 'eastus2' // --------------------------------------------------------------------------- @@ -81,15 +81,10 @@ param modelCapacity int = 1 // --------------------------------------------------------------------------- // AI Gateway tier // --------------------------------------------------------------------------- -@description('Globally unique AI Gateway name. Resolves to .azure-api.net. Leave empty to auto-generate.') +@description('AI Gateway resource name: 3-24 lowercase letters, digits or hyphens, starting and ending with a letter or digit. The service returns its runtime hostname. Leave empty to auto-generate.') +@maxLength(24) param gatewayName string = '' -@description('Publisher email required by the API Management service at create time.') -param publisherEmail string = 'noreply@example.com' - -@description('Publisher organization name required by the API Management service at create time.') -param publisherName string = 'AI Gateway tier hub' - @description('Tokens-per-minute budget enforced on the model by the gateway token-limit policy. Low by default so a burst test visibly throttles (HTTP 429).') param tokensPerMinute int = 100 @@ -165,28 +160,24 @@ resource project 'Microsoft.CognitiveServices/accounts/projects@2025-04-01-previ } // =========================================================================== -// AI Gateway tier instance — Microsoft.ApiManagement/service with the AIGateway -// SKU. Release-gated preview: deploys only where the SKU is enabled (East US 2, +// Standalone AI Gateway instance — Microsoft.ApiManagement/aigateways. +// Release-gated preview: deploys only where AI Gateway is enabled (East US 2, // Sweden Central). // =========================================================================== var effectiveGatewayName = empty(gatewayName) ? 'aigw${uniqueSuffix}' : gatewayName var foundryUserRoleId = '53ca6127-db72-4b80-b1b0-d745d6d5456d' var foundryEndpoint = endsWith(account.properties.endpoint, '/') ? account.properties.endpoint : '${account.properties.endpoint}/' -resource aiGateway 'Microsoft.ApiManagement/service@2025-09-01-preview' = { +resource aiGateway 'Microsoft.ApiManagement/aigateways@2026-11-01' = { name: effectiveGatewayName location: gatewayLocation identity: { type: 'SystemAssigned' } sku: { - name: 'AIGateway' - capacity: 1 - } - properties: { - publisherEmail: publisherEmail - publisherName: publisherName + name: 'Standard' } + properties: {} } // Grant the gateway's managed identity Foundry User on the account (keyless backend). @@ -200,14 +191,15 @@ resource foundryUserRole 'Microsoft.Authorization/roleAssignments@2022-04-01' = } } -// The AIGateway SKU auto-creates a 'default' workspace that holds providers/models. -resource defaultWorkspace 'Microsoft.ApiManagement/service/workspaces@2025-09-01-preview' existing = { +// Child operations use the preview API independently of the parent gateway API. +// AI Gateway auto-creates a 'default' workspace that holds providers/models. +resource defaultWorkspace 'Microsoft.ApiManagement/aigateways/workspaces@2026-09-01-preview' existing = { parent: aiGateway name: 'default' } // Foundry model provider — imports from the account using managed identity. -resource foundryProvider 'Microsoft.ApiManagement/service/workspaces/modelProviders@2025-09-01-preview' = { +resource foundryProvider 'Microsoft.ApiManagement/aigateways/workspaces/modelProviders@2026-09-01-preview' = { parent: defaultWorkspace name: 'foundry' properties: { @@ -233,12 +225,11 @@ resource foundryProvider 'Microsoft.ApiManagement/service/workspaces/modelProvid } // Register the deployment as a gateway model, with a token-limit policy. -resource gatewayModel 'Microsoft.ApiManagement/service/workspaces/modelProviders/models@2025-09-01-preview' = { +resource gatewayModel 'Microsoft.ApiManagement/aigateways/workspaces/modelProviders/models@2026-09-01-preview' = { parent: foundryProvider name: modelName properties: { displayName: modelName - apiFormat: 'OpenAIChatCompletions' supportedEndpoints: [ '/openai/v1/chat/completions' '/openai/v1/responses' @@ -253,14 +244,33 @@ resource gatewayModel 'Microsoft.ApiManagement/service/workspaces/modelProviders type: 'tokenLimit' period: 'minute' count: tokensPerMinute - counterKey: 'Identity' + counterKey: [ + 'identity' + ] } ] } } +resource gatewayAlias 'Microsoft.ApiManagement/aigateways/workspaces/aliases@2026-09-01-preview' = { + parent: defaultWorkspace + name: replace(modelName, '.', '-') + properties: { + aliasName: modelName + backendModels: [ + { + resourceId: gatewayModel.id + priority: 0 + } + ] + selectionOrder: [ + 'priority' + ] + } +} + // A runtime access key that spoke connections send in the api-key header. -resource runtimeKey 'Microsoft.ApiManagement/service/apiKeys@2025-09-01-preview' = { +resource runtimeKey 'Microsoft.ApiManagement/aigateways/apiKeys@2026-09-01-preview' = { parent: aiGateway name: 'default' properties: { @@ -277,7 +287,8 @@ output accountName string = account.name output accountId string = account.id output projectName string = project.name output gatewayName string = aiGateway.name -output gatewayModelsBaseUrl string = '${aiGateway.properties.gatewayUrl}/default/models/openai/v1' +output gatewayId string = aiGateway.id +output gatewayModelsBaseUrl string = 'https://${aiGateway.properties.frontend.defaultHostname}/default/models/openai/v1' output modelName string = modelName output modelVersion string = modelVersion output modelFormat string = modelFormat diff --git a/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/metadata.json b/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/metadata.json index 5ccc01b96..8961c606c 100644 --- a/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/metadata.json +++ b/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/metadata.json @@ -2,10 +2,10 @@ "$schema": "https://aka.ms/azure-quickstart-templates-metadata-schema#", "type": "QuickStart", "itemDisplayName": "AI Gateway tier (preview) ai-gateway-tier: hub + spoke connection for the Foundry golden path", - "description": "AI Gateway tier (preview) hub-and-spoke module for the Microsoft Foundry golden path. main.bicep deploys a centralized AI Gateway tier (Azure API Management AIGateway SKU, 2025-09-01-preview) that fronts a Foundry model over managed identity, with a token-limit policy and a runtime key; connection.bicep attaches a consumer (spoke) project to it with an 'Admin-connected models' ModelGateway + ApiKey connection, in the same subscription or cross-subscription. An agent then references the model as / via the Responses API. Release-gated preview: deploys where the AIGateway SKU is enabled (East US 2, Sweden Central).", - "summary": "Hub-and-spoke AI Gateway tier module for the golden path. main.bicep = Foundry account + hub project + gpt-5.4 model + AIGateway service + managed-identity provider + token-limit policy + runtime key. connection.bicep = ModelGateway ApiKey connection on an existing spoke project pointed at the hub gateway; an agent consumes it as / via the Responses API.", + "description": "AI Gateway tier (preview) hub-and-spoke module for the Microsoft Foundry golden path. main.bicep deploys a standalone Microsoft.ApiManagement/aigateways resource (Standard SKU, 2026-11-01 parent; 2026-09-01-preview child operations) that fronts a Foundry model over managed identity, with a callable model alias, token-limit policy, and runtime key; connection.bicep attaches a consumer (spoke) project to it with an 'Admin-connected models' ModelGateway + ApiKey connection, in the same subscription or cross-subscription. An agent then references the model as / via the Responses API. Release-gated preview: deploys where AI Gateway is enabled (East US 2, Sweden Central).", + "summary": "Hub-and-spoke AI Gateway tier module for the golden path. main.bicep = Foundry account + hub project + gpt-5.4 model + standalone AI Gateway + managed-identity provider + callable model alias + token-limit policy + runtime key. connection.bicep = ModelGateway ApiKey connection on an existing spoke project pointed at the hub gateway; an agent consumes it as / via the Responses API.", "githubUsername": "mohammadsheraj", - "dateUpdated": "2026-09-01", + "dateUpdated": "2026-11-11", "environments": [ "AzureCloud" ] diff --git a/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/samples/connection.parameters.json b/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/samples/connection.parameters.json index 575a573a6..7e69ff79f 100644 --- a/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/samples/connection.parameters.json +++ b/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/samples/connection.parameters.json @@ -6,7 +6,7 @@ "value": "/subscriptions//resourceGroups//providers/Microsoft.CognitiveServices/accounts//projects/" }, "gatewayModelsBaseUrl": { - "value": "https://.azure-api.net/default/models/openai/v1" + "value": "https:///default/models/openai/v1" }, "connectionName": { "value": "ai-gateway" }, "modelName": { "value": "gpt-5.4" }, diff --git a/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/samples/parameters.json b/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/samples/parameters.json index 96f4415d4..52a8a3b47 100644 --- a/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/samples/parameters.json +++ b/infrastructure/infrastructure-setup-bicep/01-connections/ai-gateway-tier/samples/parameters.json @@ -15,8 +15,6 @@ "modelCapacity": { "value": 1 }, "gatewayName": { "value": "" }, - "publisherEmail": { "value": "noreply@example.com" }, - "publisherName": { "value": "AI Gateway tier hub" }, "tokensPerMinute": { "value": 100 } } }