Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions evals/azure-skills/azure-diagnostics/response-quality.eval.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -512,3 +512,24 @@ stimuli:
- Verification confirms checkpoint blobs advance, restarts resume near the last processed offset, authorization errors stop, and replayed events do not create duplicate business side effects.
tags: *quality-tags
graders: *quality-graders

- name: "Resource details do not establish Azure Resource Health"
prompt: |
Evaluate this synthetic, offline diagnostic claim. Do not query or modify Azure.

Evidence:
- `az resource show --ids <resource-id>` returned the resource document
- The document contains `provisioningState: Succeeded`
- No Azure Resource Health response, workload probe, metric, or application
request result was collected

Can this evidence establish that Azure Resource Health is Available or that
the workload is healthy? Explain the distinction and give the supported
read-only Resource Health lookup that an authorized operator could run.
rubric:
- The response states that `az resource show` provides resource configuration and provisioning metadata and cannot establish Azure Resource Health or workload health.
- The response identifies the current `Microsoft.ResourceHealth/availabilityStatuses/current` lookup, such as through `az rest`, and does not run it in this evidence-only scenario.
- The response notes that the lookup requires `Microsoft.ResourceHealth/AvailabilityStatuses/current/read` and applies only to supported resource types.
- 'The response distinguishes Azure platform availability states from application, guest, dependency, and end-to-end health; it does not infer `Available` or healthy from `provisioningState: Succeeded`.'
tags: *quality-tags
graders: *quality-graders
14 changes: 11 additions & 3 deletions plugins/azure-skills/skills/azure-diagnostics/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ Activate this skill when user wants to:
### Common Diagnostic Commands

```bash
# Check resource health
# Inspect resource configuration and provisioning metadata (not Resource Health)
az resource show --ids RESOURCE_ID
# View activity log
az monitor activity-log list -g RG --max-events 20
Expand Down Expand Up @@ -141,13 +141,21 @@ mcp_azure_mcp_resourcehealth
### Using CLI

```bash
# Check specific resource health
az resource show --ids RESOURCE_ID
# Query the current Azure Resource Health availability status
RESOURCE_ID="/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/<provider>/<resource-type>/<resource-name>"
az rest --method get \
--url "${RESOURCE_ID}/providers/Microsoft.ResourceHealth/availabilityStatuses/current?api-version=2025-05-01"

# Check recent activity
az monitor activity-log list -g RG --max-events 20
```

`az resource show` returns resource configuration and provisioning metadata; it does not query Azure Resource Health and cannot by itself establish that a resource is healthy or `Available`.

The Resource Health request requires Azure authentication and the `Microsoft.ResourceHealth/AvailabilityStatuses/current/read` operation at the applicable scope. It applies only to resource types supported by Azure Resource Health. Interpret `availabilityState` as Azure platform availability evidence (`Available`, `Unavailable`, `Degraded`, or `Unknown`), not as application or workload health. `Available` does not prove the guest, application, dependencies, or user path are healthy, and `Unknown` must not be treated as healthy.

If the resource type is unsupported, the status is unavailable, permission is denied, or the health query fails, report that limitation and leave Resource Health availability unestablished; do not substitute provisioning success or invent a healthy result.

---

## References
Expand Down