diff --git a/CHANGELOG.md b/CHANGELOG.md index b1126c08..b331d665 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,6 +70,28 @@ Both `Agent365.Observability.OtelWrite` (Delegated) and `Agent365.Observability. `IExporterTokenCache`. ### Added +- **Microsoft.Agents.A365.Tooling** - Microsoft Defender for AI real-time protection client + - `DefenderRtpClient.EvaluateHookContextAsync` sends an agent-hooks/0.1 context to the Defender + prevention endpoint (`POST .../v1/protection/evaluate`) at the four points Defender evaluates + (`input`, `pre_tool_call`, `post_tool_call`, `output`) and returns its verdict (`deny` and + `transform` block). A copy of the context is fitted to Defender's request validation; the host's context is + not modified. One deadline (`A365_DEFENDER_RTP_TIMEOUT_MILLISECONDS`) covers token acquisition and the + request, and the endpoint and token authority must be absolute HTTPS URLs. + - Calls carry the agent identity's own app-only token for the Defender API + (`api://86a21212-634e-4553-b3d6-e477e4c9d9ec`, role `RealtimeProtection.Evaluate.All`), resolved by a + `DefenderRtpTokenResolver`, cached per agent and tenant and refreshed in the background before it expires; + `DefenderRtpTokenResolvers.FromAgenticConnection` uses the agent's connection (`IAgenticTokenProvider`), the + same authority as Observability S2S export. + - Every call sends a unique `x-ms-correlation-id`. Failures follow `A365_DEFENDER_RTP_FAIL_MODE` (`open` or + `closed`; any other value is rejected), and a `400` reports the failed validation rule. Each string is clamped + to `A365_DEFENDER_RTP_MAX_CONTENT_CHARACTERS`, and the copy carries at most four times that in all: the content + under decision first, the oldest history dropped first. When the content under decision, or at a tool point the + called tool's declaration, had to be cut, Defender's deny stands and its allow follows the fail mode + (`DefenderRtpEvaluationResult.Truncated`). Lone surrogates are sent as U+FFFD. +- **Microsoft.Agents.A365.Tooling.Extensions.AgentHooks** (new, preview) - `A365DefenderInterceptor`, an + agent-hooks interceptor (`ResponsibleAI.AgentHooks` 0.1.0-beta.1) for Defender, and + `A365AgentHooks.CreateProtectionEmitter` (`parallel/strictest`). When no agent identity is resolved, or resolving + it fails, the interceptor follows the fail mode. - **Microsoft.Agents.A365.Tooling** - V1/V2 per-audience token support for MCP servers - `MCPServerConfig` extended with `audience`, `scope`, `publisher`, and `Headers` fields - `IMcpTokenProvider` interface for pluggable OAuth token acquisition diff --git a/docs/design.md b/docs/design.md index d36000c3..75060c6d 100644 --- a/docs/design.md +++ b/docs/design.md @@ -35,7 +35,8 @@ src/ │ └── Extensions/ │ ├── SemanticKernel/ # SK tool registration │ ├── AgentFramework/ # AF tool registration -│ └── AzureAIFoundry/ # Azure AI Foundry integration +│ ├── AzureAIFoundry/ # Azure AI Foundry integration +│ └── AgentHooks/ # agent-hooks Defender interceptor (preview) └── Tests/ # Test projects docs/ └── design.md # This file @@ -52,10 +53,11 @@ docs/ | `Microsoft.Agents.A365.Observability.Extensions.SemanticKernel` | Semantic Kernel telemetry integration | [design.md](../src/Observability/Extensions/SemanticKernel/docs/design.md) | | `Microsoft.Agents.A365.Observability.Extensions.OpenAI` | OpenAI SDK telemetry integration | [design.md](../src/Observability/Extensions/OpenAI/docs/design.md) | | `Microsoft.Agents.A365.Notifications` | Microsoft 365 notification handling | [design.md](../src/Notification/Microsoft.Agents.A365.Notifications/docs/design.md) | -| `Microsoft.Agents.A365.Tooling` | MCP server discovery and configuration | [design.md](../src/Tooling/Core/docs/design.md) | +| `Microsoft.Agents.A365.Tooling` | MCP server discovery and configuration; Microsoft Defender for AI real-time protection client | [design.md](../src/Tooling/Core/docs/design.md) | | `Microsoft.Agents.A365.Tooling.Extensions.SemanticKernel` | Semantic Kernel MCP tool registration | [design.md](../src/Tooling/Extensions/SemanticKernel/docs/design.md) | | `Microsoft.Agents.A365.Tooling.Extensions.AgentFramework` | Agent Framework MCP tool registration | [design.md](../src/Tooling/Extensions/AgentFramework/docs/design.md) | | `Microsoft.Agents.A365.Tooling.Extensions.AzureAIFoundry` | Azure AI Foundry MCP tool registration | [design.md](../src/Tooling/Extensions/AzureAIFoundry/docs/design.md) | +| `Microsoft.Agents.A365.Tooling.Extensions.AgentHooks` | Microsoft Defender for AI interceptor for agent-hooks hosts (preview) | [README.md](../src/Tooling/Extensions/AgentHooks/README.md) | ## Core Package Documentation @@ -335,6 +337,9 @@ else if (EnvironmentUtils.IsDevelopmentEnvironment()) | `EnableAgent365Exporter` | Enable Agent365 telemetry exporter | `false` | | `ASPNETCORE_ENVIRONMENT` | Environment name (Development, Production) | - | | `SuppressInvokeAgentInput` | Suppress input messages in invoke_agent spans | `false` | +| `ENABLE_A365_DEFENDER_RTP` | Enable Microsoft Defender for AI real-time protection (`DefenderRtpOptions.FromEnvironment`); a value that is not true or false is rejected | `false` | +| `A365_DEFENDER_RTP_ENDPOINT` | Defender prevention endpoint, `https:///v1/protection/evaluate` | - | +| `A365_DEFENDER_RTP_FAIL_MODE` | `open` or `closed` (blocks when Defender returns no verdict); any other value is rejected | open | ## Testing @@ -349,9 +354,11 @@ src/Tests/ ├── Microsoft.Agents.A365.Observability.Extension.Tests/ ├── Microsoft.Agents.A365.Notifications.Tests/ ├── Microsoft.Agents.A365.Tooling.Tests/ +├── Microsoft.Agents.A365.Tooling.Core.Tests/ ├── Microsoft.Agents.A365.Tooling.Extensions.SemanticKernel.Tests/ ├── Microsoft.Agents.A365.Tooling.Extensions.AgentFramework.Tests/ -└── Microsoft.Agents.A365.Tooling.Extensions.AzureAIFoundry.Tests/ +├── Microsoft.Agents.A365.Tooling.Extensions.AzureAIFoundry.Tests/ +└── Microsoft.Agents.A365.Tooling.Extensions.AgentHooks.Tests/ ``` Run tests with: diff --git a/src/Directory.Packages.props b/src/Directory.Packages.props index 40e71f11..f151be67 100644 --- a/src/Directory.Packages.props +++ b/src/Directory.Packages.props @@ -40,6 +40,8 @@ + + + Microsoft.Agents.A365.Tooling.Extensions.AgentHooks + Microsoft Agent 365 agent-hooks protection + Microsoft Agent 365 real-time protection on the agent-hooks control contract (AGENT-HOOKS-0.1): a Microsoft Defender for AI interceptor for any agent-hooks host. + Microsoft Agent 365 agent-hooks protection - Microsoft Defender for AI interceptor + Microsoft Agent 365 Tooling SDK + Microsoft;Agent365;A365;AgentHooks;Defender;Security;AI;Agents + + $(NoWarn);NU5104 + + + + + + + + + + + + diff --git a/src/Tooling/Extensions/AgentHooks/README.md b/src/Tooling/Extensions/AgentHooks/README.md new file mode 100644 index 00000000..b9c593c0 --- /dev/null +++ b/src/Tooling/Extensions/AgentHooks/README.md @@ -0,0 +1,144 @@ +# Microsoft.Agents.A365.Tooling.Extensions.AgentHooks + +Microsoft Agent 365 real-time protection on the [agent-hooks](https://github.com/responsibleai/agent-hooks) +control contract (AGENT-HOOKS-0.1), using the .NET package +[`ResponsibleAI.AgentHooks`](https://www.nuget.org/packages/ResponsibleAI.AgentHooks). + +`A365DefenderInterceptor` is an agent-hooks interceptor for Microsoft Defender for AI. Each context the host +emits at the four points Defender evaluates is sent to the prevention endpoint +(`POST .../v1/protection/evaluate`) as a fitted copy: the client builds a copy of the context that meets +Defender's request validation (spec version, UTC timestamp, `target`, envelope and tool objects with only their +spec members, repaired optional fields and clamped content), keeping its session, sequence and tool call ids. The +host's context is not modified. +Defender's verdict decides: + +| agent-hooks point | When | On `deny` | +|---|---|---| +| `input` | the user's message, before the agent runs | the agent does not run | +| `pre_tool_call` | a tool call, before it runs | the tool does not run | +| `post_tool_call` | a tool result, before the agent uses it | the result is withheld | +| `output` | the reply, before it is sent | the reply is replaced | + +Other points (`agent_startup`, model calls, `agent_shutdown`) are allowed without a call. Defender's warnings and +result labels are passed through to the agent-hooks verdict; a warning reason in the `host_error:` namespace, which +agent-hooks reserves for host failures, is reported as `defender:warning`. + +## Authentication + +Calls carry the **agent identity's own app-only token** in the agent's tenant, for the Defender API +(`api://86a21212-634e-4553-b3d6-e477e4c9d9ec`, role `RealtimeProtection.Evaluate.All`). This is the same +authority as Observability S2S export: `DefenderRtpTokenResolvers.FromAgenticConnection` asks the agent's +connection (`IAgenticTokenProvider`) for the agent identity's assertion and exchanges it. + +Defender accepts only callers whose app-only token carries the application permission +`RealtimeProtection.Evaluate.All` on the Defender API (`86a21212-634e-4553-b3d6-e477e4c9d9ec`). +[microsoft/Agent365-devTools#485](https://github.com/microsoft/Agent365-devTools/pull/485) adds this to +`a365 setup`. Until it ships, a tenant administrator grants it once per agent blueprint, and every agent identity +created from the blueprint inherits it: + +1. If the tenant has no service principal for the Defender API yet, create one: + `az ad sp create --id 86a21212-634e-4553-b3d6-e477e4c9d9ec`. +2. Assign the app role to the blueprint's service principal: + `POST https://graph.microsoft.com/v1.0/servicePrincipals/{blueprint-sp-object-id}/appRoleAssignments` with + `principalId` (the blueprint service principal), `resourceId` (the Defender API service principal) and + `appRoleId` (the id of `RealtimeProtection.Evaluate.All` in that service principal's `appRoles`). Requires + Global Administrator or Privileged Role Administrator. +3. Make it inheritable: + `POST https://graph.microsoft.com/beta/applications/microsoft.graph.agentIdentityBlueprint/{blueprint-object-id}/inheritablePermissions` + with + `{"resourceAppId":"86a21212-634e-4553-b3d6-e477e4c9d9ec","inheritableScopes":{"@odata.type":"#microsoft.graph.allAllowedScopes","kind":"allAllowed"},"inheritableRoles":{"@odata.type":"#microsoft.graph.allAllowedRoles","kind":"allAllowed"}}`. + Requires Agent ID Administrator or Global Administrator. + +The tenant must also be onboarded to Microsoft Defender for AI; otherwise Defender returns `403`, which follows +the fail mode. + +## Usage + +```csharp +using AgentHooks; +using Microsoft.Agents.A365.Tooling.Extensions.AgentHooks; +using Microsoft.Agents.A365.Tooling.Protection.Defender; +using Microsoft.Agents.Authentication; + +var defender = new DefenderRtpClient(DefenderRtpOptions.FromEnvironment(), httpClient); +var tokens = DefenderRtpTokenResolvers.FromAgenticConnection( + (IAgenticTokenProvider)connections.GetDefaultConnection(), httpClient); + +var emitter = A365AgentHooks.CreateProtectionEmitter(defender: defender.Options) + .AddA365Defender(new A365DefenderInterceptor( + defender, + context => new A365DefenderCall( + new DefenderRtpAgentContext + { + AgentId = turnContext.Activity.Recipient.AgenticAppId, // the agent identity + TenantId = turnContext.Activity.Recipient.TenantId, // the agent's tenant + RequestId = turnContext.Activity.Id, + UserId = turnContext.Activity.From.AadObjectId, + }, + tokens), + result => logger.LogInformation( + "Defender {Point} allowed={Allowed} evaluated={Evaluated} cid={CorrelationId}", + result.InterceptionPoint, result.Allowed, result.Evaluated, result.CorrelationId))); + +var builder = new AgentContextBuilder(agentId, "agent-framework", sessionId, agentName); +var record = await emitter.EmitUncheckedAsync(builder.Input(userMessage), cancellationToken); +if (!record.Proceeds) { /* blocked: record.Verdict.Message */ } +``` + +`httpClient` is optional for both the client and the token resolver; without it they share a client that does not +follow redirects. A client you pass must not follow them either (for example `AllowAutoRedirect = false`): a 307 or 308 +would replay the context, or the client assertion, to another host, so a redirected request is treated as a failure. + +The evaluation callback is for logging and telemetry. It runs on the thread pool once the verdict is decided, outside +the emitter's interceptor timeout, so neither what it does nor how long it takes changes the verdict; it may run +after the interceptor returns and alongside later evaluations. An exception it throws is logged (to the optional +`ILogger` passed to `A365DefenderInterceptor`) and ignored. When resolving the identity or evaluating throws, the +result and verdict record only the exception's type; the exception itself goes to the same logger. + +Agents built on Microsoft Agent Framework can register the same interceptor with +`Microsoft.Agents.AI.AgentHooks`, which mediates model and tool calls. + +**Interceptor timeout.** The client evaluates within one deadline, `DefenderRtpOptions.Timeout` (default 10 s), +that covers token acquisition and the request, and applies the fail mode when it passes. The emitter's +per-interceptor timeout must be longer: when it fires first, agent-hooks records a fail-closed host error even +if Defender is configured to fail open. `A365AgentHooks.CreateProtectionEmitter` sets it to +`DefenderRtpOptions.Timeout` plus two seconds. If you register the interceptor on another emitter (for example +`new InterceptionEmitter()`, whose default is 5 s, or through Agent Framework), set its interceptor timeout above +`DefenderRtpOptions.Timeout`, or lower `A365_DEFENDER_RTP_TIMEOUT_MILLISECONDS`. + +## Configuration + +| Variable | Meaning | +|---|---| +| `ENABLE_A365_DEFENDER_RTP` | `true` to call Defender; `false` or unset leaves it off (`1`/`0`, `yes`/`no` and `on`/`off` also work); any other value is rejected | +| `A365_DEFENDER_RTP_ENDPOINT` | the prevention endpoint, an absolute HTTPS URL: `https:///v1/protection/evaluate` | +| `A365_DEFENDER_RTP_FAIL_MODE` | `open` (default) or `closed`, which blocks when no verdict is obtained; any other value is rejected | +| `A365_DEFENDER_RTP_TIMEOUT_MILLISECONDS` | the deadline for one evaluation, including token acquisition (default 10000) | +| `A365_DEFENDER_RTP_AUTHENTICATION_SCOPE` | overrides the Defender API scope | +| `A365_DEFENDER_RTP_MAX_CONTENT_CHARACTERS` | clamps each content string sent; ids, names and roles are not truncated (default 20000). See **Content size** | + +**Content size.** Each content string longer than `A365_DEFENDER_RTP_MAX_CONTENT_CHARACTERS` (default 20000) is +truncated before it is sent, and the request as a whole carries at most four times that much content, in this order: +the content under decision at the point (the input, the tool call's arguments, the tool result or the reply); at a +tool point the called tool's declaration; the tool call's arguments at `post_tool_call`; the other tool declarations +in the agent's order; the message history, newest first; then extensions and other members. Names, keys and nulls +count toward that too, so a context padded with many empty or null items cannot inflate the request, and only as +many tool declarations as the budget can hold are read. Content nested more than 32 levels deep is cut the same way. + +The called tool's declaration is found with a name-only scan of the first 10,000 declarations; its name is sent whole, +and its description and schema count toward the limit. Without any declarations, the `a365` extension's description +of the tool is used. A tool missing from a list scanned to the end is declared from its name. + +The agent's own context is not modified. When the content under decision or the called tool's declaration was cut, +or the list is longer than the scan and the called tool was not among the declarations scanned, Defender has not +seen all of it: its deny still blocks, but its allow does not cover the rest, so the result follows the fail mode and +`Error` says which (`TruncatedContentError`, `TruncatedToolDeclarationError` or `UnseenToolDeclarationError`). Fail open +allows with a `defender:unverified` warning; fail closed denies with `runtime_error:defender_unverified`. Raise the +limit for agents that handle long content. Evaluating long content in chunks is a planned follow-up. Lone UTF-16 +surrogates in any string are replaced with U+FFFD, since Defender cannot parse them. + +Every call sends a unique `x-ms-correlation-id`, returned as `DefenderRtpEvaluationResult.CorrelationId`; +Defender logs each evaluation under it. A `400` reports the failed validation rule in `Error`. + +`ResponsibleAI.AgentHooks` is a prerelease package with a native core (`agent_hooks_ffi`) for linux-x64, +win-x64, osx-x64 and osx-arm64.