diff --git a/.claude/agents/architecture-reviewer.md b/.claude/agents/architecture-reviewer.md index 28f54312..47037967 100644 --- a/.claude/agents/architecture-reviewer.md +++ b/.claude/agents/architecture-reviewer.md @@ -94,7 +94,7 @@ Reviewer: architecture-reviewer ## Files Reviewed - List each file included in the PR with its full path -- Example: `src/Observability/Runtime/Scopes/InvokeAgentScope.cs` +- Example: `src/Tooling/Core/Services/McpToolServerConfigurationService.cs` --- diff --git a/.claude/agents/code-review-manager.md b/.claude/agents/code-review-manager.md index 5dc35e76..708b0df0 100644 --- a/.claude/agents/code-review-manager.md +++ b/.claude/agents/code-review-manager.md @@ -21,7 +21,7 @@ You are a senior engineering manager specializing in code quality and review coo - Balances criticism with recognition of good practices 3. **Enforce Standards**: Ensure all code adheres to project standards: - - .NET 8.0 / netstandard2.0 compatibility as appropriate + - .NET 8.0 compatibility for maintained packages - Copyright headers on all C# files: `// Copyright (c) Microsoft Corporation.\n// Licensed under the MIT License.` - No "Kairo" legacy references - Nullable reference types enabled diff --git a/.claude/agents/prd-task-generator.md b/.claude/agents/prd-task-generator.md index b94e1273..6fc43779 100644 --- a/.claude/agents/prd-task-generator.md +++ b/.claude/agents/prd-task-generator.md @@ -12,12 +12,12 @@ You are a senior software engineer with deep expertise in the Microsoft Agent 36 1. **Analyze PRD Context**: Thoroughly review the provided PRD, understanding both explicit requirements and implicit dependencies. Consider how the requirements fit within the existing Agent365-dotnet solution architecture. 2. **Align with Project Architecture**: Every task you generate must align with the established patterns in this repository: - - The solution structure with four core modules: Runtime, Observability, Notifications, and Tooling + - The solution structure with three core modules: Runtime, Notifications, and Tooling - The Core + Extensions pattern (core packages are framework-agnostic, extensions add framework-specific integration) - Package naming conventions (`Microsoft.Agents.A365.*`) - - OpenTelemetry-based observability patterns + - External observability integration via the Microsoft OpenTelemetry .NET distro; see `docs/observability-migration.md` - MCP (Model Context Protocol) tool integration patterns - - Target frameworks: `net8.0` for most packages, `netstandard2.0` for Runtime/Hosting + - Target framework: `net8.0` for all maintained packages 3. **Consider Design Documentation**: Reference the detailed architecture in `docs/design.md` and per-package design documents in `src///docs/design.md` when breaking down tasks. Ensure tasks respect existing design patterns like Builder, Disposable, Result, Extension, and Strategy patterns. diff --git a/.claude/agents/prd-writer.md b/.claude/agents/prd-writer.md index d9038eb2..191794fa 100644 --- a/.claude/agents/prd-writer.md +++ b/.claude/agents/prd-writer.md @@ -20,10 +20,10 @@ You are a senior software engineering architect with deep expertise in creating - Security and compliance considerations 2. **Contextual Awareness**: You have access to the Agent365 .NET SDK codebase context. When creating PRDs, ensure alignment with: - - The solution structure with four core modules (Runtime, Observability, Notifications, Tooling) + - The solution structure with three core modules (Runtime, Notifications, Tooling) - Existing architectural patterns (Core + Extensions pattern for framework integrations) - - .NET 8.0 and netstandard2.0 target frameworks - - OpenTelemetry-based observability patterns + - .NET 8.0 target framework + - External observability integration via the Microsoft OpenTelemetry .NET distro; see `docs/observability-migration.md` - MCP (Model Context Protocol) integration patterns - Async/await conventions for I/O operations - Required copyright headers and code standards diff --git a/.claude/agents/task-implementer.md b/.claude/agents/task-implementer.md index 0292e918..cc37e27b 100644 --- a/.claude/agents/task-implementer.md +++ b/.claude/agents/task-implementer.md @@ -112,15 +112,13 @@ Before submitting code, verify: ## Package-Specific Guidelines ### Runtime (`Microsoft.Agents.A365.Runtime`) -- Target: `netstandard2.0` for broad compatibility +- Target: `net8.0` - Use `OperationResult` pattern for method returns - Follow `TenantContextHelper` patterns for context extraction -### Observability (`Microsoft.Agents.A365.Observability.*`) -- Use `IDisposable` pattern for scope classes -- Follow `InvokeAgentScope`, `InferenceScope`, `ExecuteToolScope` patterns -- Integrate with OpenTelemetry's `Activity` and `ActivitySource` -- Use `BaggageBuilder` for context propagation +### External Observability +- Observability source and releases are maintained in the Microsoft OpenTelemetry .NET distro. +- Do not reintroduce retired projects or packages; use `docs/observability-migration.md` for application integrations. ### Notifications (`Microsoft.Agents.A365.Notifications`) - Target: `net8.0` diff --git a/.claude/agents/test-coverage-reviewer.md b/.claude/agents/test-coverage-reviewer.md index 0813464b..9074925e 100644 --- a/.claude/agents/test-coverage-reviewer.md +++ b/.claude/agents/test-coverage-reviewer.md @@ -55,11 +55,9 @@ The Microsoft Agent 365 SDK for .NET uses: ``` src/Tests/ ├── Runtime.Tests/ -├── Microsoft.Agents.A365.Observability.Runtime.Tests/ -├── Microsoft.Agents.A365.Observability.Runtime.IntegrationTests/ -├── Microsoft.Agents.A365.Observability.Hosting.Tests/ ├── Microsoft.Agents.A365.Notifications.Tests/ ├── Microsoft.Agents.A365.Tooling.Tests/ +├── Microsoft.Agents.A365.Tooling.Core.Tests/ └── Microsoft.Agents.A365.Tooling.Extensions.*.Tests/ ``` diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 688bd777..ea106598 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -1,6 +1,8 @@ # GitHub Copilot Instructions for Agent365-dotnet -The Microsoft Agent 365 SDK (C#/.NET) extends the Microsoft 365 Agents SDK with enterprise capabilities across four modules: **Observability**, **Notifications**, **Runtime**, and **Tooling**. Packages publish to NuGet under the `Microsoft.Agents.A365.*` prefix. +The Microsoft Agent 365 SDK (C#/.NET) extends the Microsoft 365 Agents SDK with enterprise capabilities across three modules: **Notifications**, **Runtime**, and **Tooling**. Packages publish to NuGet under the `Microsoft.Agents.A365.*` prefix. + +Observability is maintained in the [Microsoft OpenTelemetry .NET distro](https://github.com/microsoft/opentelemetry-distro-dotnet/tree/main). Do not reintroduce retired Observability projects or packages here; see [migration guidance](../docs/observability-migration.md). ## Build, Test, and Lint @@ -17,7 +19,7 @@ dotnet test src/Microsoft.Agents.A365.Sdk.sln ./build/build.ps1 -Pack # produce NuGet packages # Run all tests in ONE project -dotnet test src/Tests/Microsoft.Agents.A365.Runtime.Tests/Microsoft.Agents.A365.Runtime.Tests.csproj +dotnet test src/Tests/Runtime.Tests/Microsoft.Agents.A365.Runtime.Tests.csproj # Run a SINGLE test or test class (xUnit primary; some MSTest) dotnet test src/Microsoft.Agents.A365.Sdk.sln --filter "FullyQualifiedName~TenantContextHelperTests" @@ -27,16 +29,15 @@ dotnet test src/Microsoft.Agents.A365.Sdk.sln --filter "Name=Extract_Returns_Ten dotnet format src/Microsoft.Agents.A365.Sdk.sln --verify-no-changes ``` -CI (`.github/workflows/ci.yml`) builds and tests in **Release** with `--no-restore`/`--no-build`, then packs. Some Observability/Tooling tests read `AZURE_OPENAI_ENDPOINT`, `AZURE_OPENAI_API_KEY`, and `AZURE_OPENAI_DEPLOYMENT` from the environment. +CI (`.github/workflows/ci.yml`) builds and tests in **Release** with `--no-restore`/`--no-build`, then packs into a fresh per-run directory and rejects retired Observability filenames before uploading. It does not publish to nuget.org. Pre-commit hooks (`.pre-commit-config.yaml`, install with `pip install pre-commit && pre-commit install`) run gitleaks (secret scanning), whitespace/EOL fixers, and `dotnet format`. ## Architecture -The SDK follows a consistent **Core + Extensions** pattern. Each module has a `Core`/`Runtime` package with base functionality and per-framework extension packages (SemanticKernel, AgentFramework, AzureAIFoundry, OpenAI). Source lives under `src//`: +The SDK follows a **Core + Extensions** pattern for framework integrations, with Tooling extension packages for SemanticKernel, AgentFramework, and AzureAIFoundry. Source lives under `src//`: - **Runtime** (`src/Runtime/`) — multi-tenant context extraction (`TenantContextHelper` pulls tenant/worker IDs from `HttpContext` claims/headers/items) and the result pattern (`OperationResult` / `OperationError`). -- **Observability** (`src/Observability/`) — OpenTelemetry distributed tracing. Configured via a fluent `Builder` API; tracing uses disposable scope classes (`InvokeAgentScope`, `InferenceScope`, `ExecuteToolScope`) that auto-end spans on dispose. `BaggageMiddleware` seeds tenant/agent context into OTel baggage. A custom `Agent365Exporter` (gated by the `EnableAgent365Exporter` env var) exports spans. - **Notifications** (`src/Notification/`) — event routing for M365 (Teams, email, Office) via `AgentNotification`; sub-channels like `agents:email`, `agents:word`, `agents:excel`, `agents:powerpoint`. - **Tooling** (`src/Tooling/`) — Model Context Protocol (MCP) server discovery and tool registration via `IMcpToolServerConfigurationService`, with framework-specific registration extensions. @@ -48,14 +49,7 @@ Detailed design docs: `docs/design.md`; build internals: `build/BUILD.md`. - **Central package management**: ALL NuGet versions live in `src/Directory.Packages.props`. Add a version there, then reference the package without a `Version` attribute in the `.csproj`. Never put versions in project files. - **Common build props** (`src/Directory.Build.props`): `TreatWarningsAsErrors=true`, `Nullable=enable`, and `GenerateDocumentationFile=true` are global. Warnings — including invalid XML-doc `cref`s (CS1574) — fail the build. Write valid XML doc comments. -- **Target frameworks**: most packages target `net8.0`; some Runtime/Hosting packages target `netstandard2.0` (no implicit usings; `LangVersion` 8.0). Guard framework-specific code accordingly. -- **Observability export config — coordinated change required**: these three must stay in sync. If you change ONE, verify the other two: - | Constant | Location | - |---|---| - | `ProdObservabilityScope` (via `GetObservabilityAuthenticationScope()`) | `src/Observability/Runtime/Common/EnvironmentUtils.cs` | - | `DefaultEndpointHost` | `src/Observability/Runtime/Tracing/Exporters/Agent365ExporterOptions.cs` | - | Export URL path (`BuildEndpointPath()`) | `src/Observability/Runtime/Tracing/Exporters/Agent365ExporterCore.cs` | - Snapshot tests in `src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Exporters/ExportConfigConsistencyTests.cs` catch accidental drift but not intentional-but-incomplete updates — confirm values are correct for the target environment. +- **Target frameworks**: all maintained packages target `net8.0`. - **Tests** mirror source under `src/Tests/` (e.g. `Microsoft.Agents.A365.Runtime.Tests`). xUnit is primary (some MSTest), with Moq for mocking and FluentAssertions for assertions. ## Coding agent rules @@ -178,4 +172,3 @@ When Copilot detects violations: - Project files (`.csproj`, `.sln`), configuration files (`.json`, `.xml`, `.yaml`, `.md`) do not require copyright headers - Build output directories (`bin/`, `obj/`) - AssemblyInfo.cs files that are auto-generated - diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6ec50665..6450c26b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1,4 +1,4 @@ -name: CI - Build, Test, and Publish +name: CI - Build, Test, and Pack permissions: contents: read @@ -48,14 +48,23 @@ jobs: AZURE_OPENAI_DEPLOYMENT: ${{ vars.AZURE_OPENAI_DEPLOYMENT }} - name: Pack - run: dotnet pack Microsoft.Agents.A365.Sdk.sln --no-build --configuration Release --output ../NuGetPackages - - - name: Copy NuGet packages + shell: pwsh run: | - mkdir -p ~/drop/dotnet/NuGetPackages && cp -v ../NuGetPackages/*.nupkg $_ + $outputPath = Join-Path $env:RUNNER_TEMP "agent365-packages-$([guid]::NewGuid().ToString('N'))" + New-Item -ItemType Directory -Path $outputPath | Out-Null + dotnet pack Microsoft.Agents.A365.Sdk.sln --no-build --configuration Release --output $outputPath + if ($LASTEXITCODE -ne 0) { throw "Pack failed" } + if (Get-ChildItem -LiteralPath $outputPath -Filter 'Microsoft.Agents.A365.Observability.*' -File) { + throw "Retired Observability packages must not be released." + } + if (-not (Get-ChildItem -LiteralPath $outputPath -Filter '*.nupkg' -File)) { + throw "Pack produced no NuGet packages." + } + "PACKAGE_OUTPUT=$outputPath" >> $env:GITHUB_ENV - name: Upload NuGet packages as artifact uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: dotnet-packages-${{ github.run_number }} - path: ~/drop/dotnet/NuGetPackages/* + path: ${{ env.PACKAGE_OUTPUT }}/* + if-no-files-found: error diff --git a/.specify/memory/constitution.md b/.specify/memory/constitution.md index 81cb65db..38940481 100644 --- a/.specify/memory/constitution.md +++ b/.specify/memory/constitution.md @@ -28,10 +28,10 @@ Follow-up TODOs: None ## Core Principles ### I. Core + Extensions Architecture -Every capability MUST follow the Core + Extensions pattern. Core packages provide framework-agnostic base functionality. Extension packages integrate with specific frameworks (Semantic Kernel, Agent Framework, Azure AI Foundry, OpenAI). This separation ensures loose coupling, independent versioning, and allows consumers to adopt only what they need. New features MUST NOT add framework-specific code to core packages. +Every capability MUST follow the Core + Extensions pattern. Core packages provide framework-agnostic base functionality. Extension packages integrate with specific frameworks (Semantic Kernel, Agent Framework, Azure AI Foundry). This separation ensures loose coupling, independent versioning, and allows consumers to adopt only what they need. New features MUST NOT add framework-specific code to core packages. ### II. Multi-Tenant First -All components MUST support multi-tenant scenarios from the ground up. Tenant context extraction via `TenantContextHelper` MUST be used consistently for all tenant/worker ID retrieval. Hard-coded tenant identifiers are forbidden. Tenant context MUST propagate through OpenTelemetry baggage in distributed tracing scenarios. +All components MUST support multi-tenant scenarios from the ground up. Tenant context extraction via `TenantContextHelper` MUST be used consistently for all tenant/worker ID retrieval. Hard-coded tenant identifiers are forbidden. Applications integrating external tracing MUST preserve tenant isolation during context propagation. ### III. Strict Code Quality (NON-NEGOTIABLE) - All compiler warnings MUST be treated as errors (`TreatWarningsAsErrors=true`) @@ -44,7 +44,7 @@ All components MUST support multi-tenant scenarios from the ground up. Tenant co All service operations that can fail MUST use the `OperationResult` pattern. Return `OperationResult.Success` for success cases. Return `OperationResult.Failed(OperationError)` with appropriate HTTP status codes for failures. Exceptions MUST NOT be thrown for expected business logic failures. ### V. Disposable Scope Pattern -All tracing scopes (`InvokeAgentScope`, `InferenceScope`, `ExecuteToolScope`) MUST implement `IDisposable`. Consumers MUST use `using` statements or blocks to ensure proper span lifecycle management. Spans are automatically ended on dispose. New scope types MUST follow this pattern. +Consumers of disposable resources MUST use `using` statements or blocks to ensure proper lifecycle management. Observability scope implementations are maintained outside this repository in the Microsoft OpenTelemetry .NET distro, not in new SDK projects. See `docs/observability-migration.md` for the supported application integration. ### VI. Test Coverage Required Every package MUST have a corresponding test project in `src/Tests/`. Tests MUST cover happy path scenarios, error conditions, edge cases, and framework-specific extension behavior. Pre-commit validation MUST pass: solution builds without warnings and all tests pass. @@ -52,8 +52,7 @@ Every package MUST have a corresponding test project in `src/Tests/`. Tests MUST ## Architecture Constraints ### Target Frameworks -- **Primary packages**: `net8.0` -- **Runtime/Hosting packages**: `netstandard2.0` (for broader ASP.NET Core compatibility) +- **Maintained packages**: `net8.0` - **Test projects**: `net8.0` ### Package Management @@ -81,8 +80,7 @@ Before committing any changes, developers MUST verify: ### Design Patterns | Pattern | When to Use | Example | |---------|-------------|---------| -| Builder | Complex configuration | `Builder.WithAgentFramework().Build()` | -| Disposable | Resource/span lifecycle | `using var scope = InvokeAgentScope.Start(...)` | +| Disposable | Resource lifecycle | Dispose owned clients and streams using `using` | | Result | Operation outcomes | `OperationResult.Success`, `OperationResult.Failed(...)` | | Extension Methods | Framework integration | `app.OnAgenticEmailNotification(...)` | diff --git a/CHANGELOG.md b/CHANGELOG.md index f0416b23..68af2e42 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,7 +5,23 @@ All notable changes to the Microsoft Agent 365 SDK will be documented in this fi The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [Unreleased] +## [3.0.0] - Unreleased + +### Breaking Changes + +- **Observability packages retired** - This repository no longer builds or releases + `Microsoft.Agents.A365.Observability.Runtime`, + `Microsoft.Agents.A365.Observability.Hosting`, + `Microsoft.Agents.A365.Observability.Extensions.AgentFramework`, + `Microsoft.Agents.A365.Observability.Extensions.SemanticKernel`, or + `Microsoft.Agents.A365.Observability.Extensions.OpenAI`. + Notifications, Runtime, Tooling, and the Tooling extensions remain maintained. +- **Application migration required** - Use + [`Microsoft.OpenTelemetry`](https://www.nuget.org/packages/Microsoft.OpenTelemetry) + from the [Microsoft OpenTelemetry .NET distro](https://github.com/microsoft/opentelemetry-distro-dotnet/tree/main). + This is not a drop-in replacement or an automatic dependency redirect; follow the + [Observability migration guide](docs/observability-migration.md) for package replacement, + configuration, authentication, context propagation, and framework instrumentation. ## [2.0.0] - 2026-10-20 diff --git a/CLAUDE.md b/CLAUDE.md index 76cda2dd..a8d243f0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -59,17 +59,9 @@ Exceptions: ### Legacy References - **Never** use the keyword "Kairo" in any code - this is legacy terminology that must be replaced -### Observability Export Configuration — Coordinated Review Required +### Retired Observability Packages -The following three constants must stay in sync. If a PR changes **any one** of them, the reviewer (human or Copilot) **must** ask the author to confirm the other two are still correct: - -| Constant | Location | -|---|---| -| `ProdObservabilityScope` | `src/Observability/Runtime/Common/EnvironmentUtils.cs` (accessed via `GetObservabilityAuthenticationScope()`) | -| `DefaultEndpointHost` | `src/Observability/Runtime/Tracing/Exporters/Agent365ExporterOptions.cs` | -| Export URL path pattern | `BuildEndpointPath()` in `src/Observability/Runtime/Tracing/Exporters/Agent365ExporterCore.cs` | - -Snapshot tests in `src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Exporters/ExportConfigConsistencyTests.cs` will fail if any value drifts, but the developer must also verify the values are correct for the target environment — the tests only catch accidental drift, not intentional-but-incomplete updates. +Observability is maintained in the [Microsoft OpenTelemetry .NET distro](https://github.com/microsoft/opentelemetry-distro-dotnet/tree/main), not this repository. Do not reintroduce `Microsoft.Agents.A365.Observability.*` projects or package output. See [migration guidance](docs/observability-migration.md) for application onboarding with `Microsoft.OpenTelemetry` and package-owner release actions. ### Pre-Commit Requirements Before committing changes, ensure: @@ -79,29 +71,22 @@ Before committing changes, ensure: ## Architecture Overview -### Four Core Modules +### Three Core Modules -This SDK extends the Microsoft 365 Agents SDK with four main areas: +This SDK extends the Microsoft 365 Agents SDK with three main areas: **1. Runtime** (`Microsoft.Agents.A365.Runtime`) - Multi-tenant context extraction via `TenantContextHelper` (claims, headers, items) - Standardized result handling with `OperationResult` and `OperationError` - Authorization services for agentic operations -**2. Observability** (6 packages under `Microsoft.Agents.A365.Observability.*`) -- OpenTelemetry-based distributed tracing with custom span processors -- Scope classes: `InvokeAgentScope`, `InferenceScope`, `ExecuteToolScope` -- Framework extensions for Agent Framework, Semantic Kernel, and OpenAI SDK -- ETW (Event Tracing for Windows) support via Hosting package -- Configuration via fluent `Builder` API - -**3. Notifications** (`Microsoft.Agents.A365.Notifications`) +**2. Notifications** (`Microsoft.Agents.A365.Notifications`) - Event-driven notification handling for M365 (Teams, email, Office apps) - Support for `agents:email`, `agents:word`, `agents:excel`, `agents:powerpoint` sub-channels - Type-safe models: `EmailReference`, `EmailResponse`, `WpxComment` - Lifecycle notifications (user identity created, auto sign-in) -**4. Tooling** (4 packages under `Microsoft.Agents.A365.Tooling.*`) +**3. Tooling** (4 packages under `Microsoft.Agents.A365.Tooling.*`) - Model Context Protocol (MCP) server discovery and integration - `IMcpToolServerConfigurationService` for server discovery and tool registration - Framework-specific registration for Semantic Kernel, Agent Framework, Azure AI Foundry @@ -110,26 +95,11 @@ This SDK extends the Microsoft 365 Agents SDK with four main areas: ### Extension Pattern The SDK uses a consistent **Core + Extensions** pattern: - Core packages provide base functionality -- Extension packages integrate with specific frameworks (SemanticKernel, AgentFramework, AzureAIFoundry, OpenAI) -- Example: `Observability/Runtime` (core) → `Observability/Extensions/SemanticKernel` (framework-specific) +- Extension packages integrate with specific frameworks (SemanticKernel, AgentFramework, AzureAIFoundry) +- Example: `Tooling/Core` (core) → `Tooling/Extensions/SemanticKernel` (framework-specific) ### Key Design Patterns -**Builder Pattern**: Fluent API for observability configuration -```csharp -new Builder(services, configuration) - .WithAgentFramework() - .WithSemanticKernel() - .Build(); -``` - -**Disposable Pattern**: Automatic span lifecycle management -```csharp -using var scope = InvokeAgentScope.Start(details, tenantDetails, request); -scope.RecordResponse(response); -// Span automatically ends on dispose -``` - **Result Pattern**: Consistent success/failure handling ```csharp return OperationResult.Success; @@ -139,13 +109,12 @@ return OperationResult.Failed(new OperationError("Message", HttpStatusCode.NotFo ## Project Organization ### Target Frameworks -- Most packages: `net8.0` -- Runtime/Hosting packages: `netstandard2.0` (broader compatibility) +- All maintained packages: `net8.0` ### Build System - **SDK Version**: .NET 8.0.100 (defined in `src/global.json`) - **Build Traversal**: Microsoft.Build.Traversal (v3.4.0) via `src/dirs.proj` -- **Automatic Discovery**: Projects are auto-discovered via wildcards; no manual solution maintenance needed +- **Automatic Discovery**: Traversal projects use wildcards; the solution file must still be maintained explicitly - **Versioning**: Nerdbank.GitVersioning (nbgv) - semantic versioning from git history ### Configuration Files @@ -167,8 +136,7 @@ return OperationResult.Failed(new OperationError("Message", HttpStatusCode.NotFo |---------|---------|---------| | `Microsoft.Agents.Builder` | 1.3.163-beta | Core Agents SDK | | `Microsoft.Agents.AI` | 1.0.0-preview.251016.1 | Agent Framework | -| `OpenTelemetry` | 1.11.2 | Distributed tracing foundation | -| `Microsoft.SemanticKernel` | 1.65.0 | Semantic Kernel orchestration | +| `Microsoft.SemanticKernel.Agents.Core` | See central versions | Semantic Kernel orchestration | | `Azure.AI.OpenAI` | 2.3.0-beta.2 | Azure OpenAI SDK | | `ModelContextProtocol.Core` | 0.2.0-preview.3 | MCP client SDK | | `Azure.AI.Agents.Persistent` | 1.2.0-beta.4 | Azure AI Foundry | @@ -179,11 +147,6 @@ return OperationResult.Failed(new OperationError("Message", HttpStatusCode.NotFo - `TenantContextHelper` - Extract tenant/worker ID from HttpContext - `AgenticAuthorizationService` - Authorization for agent operations -### Observability (`src/Observability/`) -- `Builder` (Runtime) - Fluent OpenTelemetry configuration API -- `InvokeAgentScope`, `InferenceScope`, `ExecuteToolScope` - Tracing scopes -- `BaggageBuilder` - OpenTelemetry baggage context management - ### Notifications (`src/Notification/`) - `AgentNotification` - AgentExtension for routing notifications - `AgentNotificationActivity` - Activity wrapper for notification data @@ -193,19 +156,10 @@ return OperationResult.Failed(new OperationError("Message", HttpStatusCode.NotFo - `McpToolServerConfigurationService` - Implementation - `MCPServerConfig` - Server configuration model -## Environment Variables - -| Variable | Purpose | Default | -|----------|---------|---------| -| `EnableAgent365Exporter` | Enable Agent365 telemetry exporter | `false` | -| `ASPNETCORE_ENVIRONMENT` | Environment (Development, Production) | - | -| `SuppressInvokeAgentInput` | Suppress input in invoke_agent spans | `false` | - ## Testing Structure Tests mirror the source structure in `src/Tests/`: - Unit tests for each package (e.g., `Microsoft.Agents.A365.Runtime.Tests`) -- Integration tests (e.g., `Microsoft.Agents.A365.Observability.Runtime.IntegrationTests`) - Framework-specific extension tests Testing frameworks: xUnit (primary), MSTest, with Moq for mocking and FluentAssertions for assertions. @@ -219,19 +173,11 @@ Microsoft.Agents.A365.[.Extensions.] Examples: - `Microsoft.Agents.A365.Runtime` -- `Microsoft.Agents.A365.Observability.Extensions.SemanticKernel` +- `Microsoft.Agents.A365.Tooling.Extensions.SemanticKernel` - `Microsoft.Agents.A365.Tooling.Extensions.AzureAIFoundry` ## Data Flow Examples -### Agent Invocation Tracing -1. HTTP request enters via middleware -2. `BaggageMiddleware` sets tenant/agent context in OpenTelemetry baggage -3. `InvokeAgentScope` creates root span for agent invocation -4. `InferenceScope` creates child span for LLM inference -5. `ExecuteToolScope` creates child spans for tool executions -6. Span processors add tags and export to Agent365 Exporter or Console - ### MCP Tool Discovery 1. Agent receives user message requiring tools 2. Call `ListToolServersAsync` to query Tooling Gateway diff --git a/DEPENDENCIES.md b/DEPENDENCIES.md index 261bb9aa..c538786a 100644 --- a/DEPENDENCIES.md +++ b/DEPENDENCIES.md @@ -5,36 +5,23 @@ This diagram shows the internal dependencies between Microsoft Agent 365 SDK .NE ```mermaid graph LR Notifications["Microsoft.Agents.A365.Notifications"] - ObservabilityExtensionsAgentFramework["Microsoft.Agents.A365.Observability.Extensions.AgentFramework"] - ObservabilityExtensionsOpenAI["Microsoft.Agents.A365.Observability.Extensions.OpenAI"] - ObservabilityExtensionsSemanticKernel["Microsoft.Agents.A365.Observability.Extensions.SemanticKernel"] - ObservabilityHosting["Microsoft.Agents.A365.Observability.Hosting"] - ObservabilityRuntime["Microsoft.Agents.A365.Observability.Runtime"] Runtime["Microsoft.Agents.A365.Runtime"] Tooling["Microsoft.Agents.A365.Tooling"] ToolingExtensionsAgentFramework["Microsoft.Agents.A365.Tooling.Extensions.AgentFramework"] ToolingExtensionsAzureAIFoundry["Microsoft.Agents.A365.Tooling.Extensions.AzureAIFoundry"] ToolingExtensionsSemanticKernel["Microsoft.Agents.A365.Tooling.Extensions.SemanticKernel"] - ObservabilityExtensionsAgentFramework --> ObservabilityRuntime - ObservabilityExtensionsOpenAI --> ObservabilityRuntime - ObservabilityExtensionsSemanticKernel --> ObservabilityRuntime - ObservabilityHosting --> ObservabilityRuntime Tooling --> Runtime ToolingExtensionsAgentFramework --> Tooling ToolingExtensionsAzureAIFoundry --> Tooling ToolingExtensionsSemanticKernel --> Tooling classDef Notifications fill:#ffcdd2,stroke:#c62828,color:#280505 - classDef Observability fill:#c8e6c9,stroke:#2e7d32,color:#142a14 - classDef ObservabilityExtensions fill:#e8f5e9,stroke:#66bb6a,color:#1f3d1f classDef Runtime fill:#bbdefb,stroke:#1565c0,color:#0d1a26 classDef Tooling fill:#ffe0b2,stroke:#e65100,color:#331a00 classDef ToolingExtensions fill:#fff3e0,stroke:#fb8c00,color:#4d2600 class Notifications Notifications - class ObservabilityRuntime Observability - class ObservabilityExtensionsAgentFramework,ObservabilityExtensionsSemanticKernel,ObservabilityHosting,ObservabilityExtensionsOpenAI ObservabilityExtensions class Runtime Runtime class Tooling Tooling class ToolingExtensionsAgentFramework,ToolingExtensionsAzureAIFoundry,ToolingExtensionsSemanticKernel ToolingExtensions @@ -44,7 +31,5 @@ graph LR - **Notifications** (Red): Notification and messaging extensions - **Runtime** (Blue): Core runtime components -- **Observability** (Green): Telemetry and monitoring core -- **Observability Extensions** (Light Green): Framework-specific observability integrations - **Tooling** (Orange): Agent tooling SDK core - **Tooling Extensions** (Light Orange): Framework-specific tooling integrations diff --git a/README.md b/README.md index 2edff467..876f1f6e 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ # Microsoft Agent 365 SDK - C# /.NET -[![NuGet](https://img.shields.io/nuget/v/Microsoft.Agents.A365.Observability.svg?label=NuGet&logo=nuget)](https://www.nuget.org/packages?q=Microsoft.Agents.A365) -[![NuGet Downloads](https://img.shields.io/nuget/dt/Microsoft.Agents.A365.Observability.svg?label=Downloads&logo=nuget)](https://www.nuget.org/packages?q=Microsoft.Agents.A365) +[![NuGet](https://img.shields.io/nuget/v/Microsoft.Agents.A365.Runtime.svg?label=NuGet&logo=nuget)](https://www.nuget.org/packages?q=Microsoft.Agents.A365) +[![NuGet Downloads](https://img.shields.io/nuget/dt/Microsoft.Agents.A365.Runtime.svg?label=Downloads&logo=nuget)](https://www.nuget.org/packages?q=Microsoft.Agents.A365) [![Build Status](https://img.shields.io/github/actions/workflow/status/microsoft/Agent365-dotnet/.github/workflows/ci.yml?branch=main&label=Build&logo=github)](https://github.com/microsoft/Agent365-dotnet/actions) [![License](https://img.shields.io/github/license/microsoft/Agent365-dotnet?label=License)](LICENSE.md) [![.NET Version](https://img.shields.io/badge/.NET-8.0-512BD4?logo=dotnet)](https://dotnet.microsoft.com/) @@ -10,11 +10,12 @@ > #### Note: > Use the information in this README to contribute to this open-source project. To learn about using this SDK in your projects, refer to the [Microsoft Agent 365 Developer documentation](https://learn.microsoft.com/en-us/microsoft-agent-365/developer/). -The Microsoft Agent 365 SDK extends the Microsoft 365 Agents SDK with enterprise-grade capabilities for building sophisticated agents. This SDK provides comprehensive tooling for observability, notifications, runtime utilities, and development tools that help developers create production-ready agents for platforms including M365, Teams, Copilot Studio, and Webchat. +> **Observability has moved:** This repository no longer builds or releases `Microsoft.Agents.A365.Observability.*` packages. Use [Microsoft.OpenTelemetry on NuGet](https://www.nuget.org/packages/Microsoft.OpenTelemetry) from the [Microsoft OpenTelemetry .NET distro](https://github.com/microsoft/opentelemetry-distro-dotnet/tree/main). See the [Observability migration guide](docs/observability-migration.md) for package and code changes. -The Microsoft Agent 365 SDK focuses on four core areas: +The Microsoft Agent 365 SDK extends the Microsoft 365 Agents SDK with enterprise-grade capabilities for building sophisticated agents. This SDK provides notifications, runtime utilities, and tooling that help developers create production-ready agents for platforms including M365, Teams, Copilot Studio, and Webchat. + +The Microsoft Agent 365 SDK focuses on three core areas: -- **Observability**: Comprehensive tracing, caching, and monitoring capabilities for agent applications - **Notifications**: Agent notification services and models for handling user notifications - **Runtime**: Core utilities and extensions for agent runtime operations - **Tooling**: Developer tools and utilities for building sophisticated agent applications @@ -66,11 +67,6 @@ For more detailed build instructions, see the [build documentation](build/BUILD. - **src/DevTools**: Microsoft Agent 365 Developer Tools - Development tools and code analyzers - **src/Notification**: Microsoft Agent 365 Notifications - Agent notification services and models -- **src/Observability**: Microsoft Agent 365 Observability - Tracing, caching, and monitoring capabilities - - Core: Core observability functionality - - Extensions: Framework-specific extensions for various AI platforms - - Hosting: ETW and hosting support - - Runtime: Runtime observability services - **src/Runtime**: Microsoft Agent 365 Runtime - Core runtime utilities and extensions - Core: Core runtime functionality - Extensions: Runtime extensions for various AI frameworks @@ -106,10 +102,10 @@ The core SDK for building conversational AI agents for Microsoft 365 platforms. ### Microsoft Agent 365 SDK -Enterprise-grade extensions for observability, notifications, runtime utilities, and developer tools. +Enterprise-grade extensions for notifications, runtime utilities, and developer tools. Observability is maintained in the separate Microsoft OpenTelemetry distro linked above. - [Microsoft Agent 365 SDK - C# /.NET repository](https://github.com/microsoft/Agent365-dotnet) - You are here -- [Microsoft Agent 365 SDK - Python repository](https://github.com/microsoft/Agent365-python) +- [Microsoft Agent 365 SDK - Python repository](https://github.com/microsoft/Agent365-python) - [Microsoft Agent 365 SDK - Node.js/TypeScript repository](https://github.com/microsoft/Agent365-nodejs) - [Microsoft Agent 365 SDK Samples repository](https://github.com/microsoft/Agent365-Samples) - [Microsoft Agent 365 developer documentation](https://learn.microsoft.com/en-us/microsoft-agent-365/developer/) @@ -127,5 +123,3 @@ Enterprise-grade extensions for observability, notifications, runtime utilities, Copyright (c) Microsoft Corporation. All rights reserved. Licensed under the MIT License - see the [LICENSE](LICENSE.md) file for details. - - diff --git a/build/BUILD.md b/build/BUILD.md index 62482a80..b4ed1f6f 100644 --- a/build/BUILD.md +++ b/build/BUILD.md @@ -1,163 +1,73 @@ # Centralized Build and Package Management -This solution has been configured with centralized package management and build traversal for efficient building and dependency management. +The SDK uses central dependency versions and traversal builds for Notifications, Runtime, and Tooling. Observability is no longer built here; see the [migration guide](../docs/observability-migration.md). -## Overview +## Build files -The solution uses: -- **global.json**: Defines the .NET SDK version and MSBuild SDK versions -- **Directory.Packages.props**: Centralized NuGet package version management -- **Directory.Build.props**: Common build properties for all projects -- **dirs.proj**: Build traversal projects for organized building +| File | Purpose | +|---|---| +| `src/global.json` | .NET SDK selection and traversal SDK version | +| `src/Directory.Packages.props` | Central NuGet dependency versions and transitive pins | +| `src/Directory.Build.props` | Shared compiler and package settings | +| `src/dirs.proj` | Source traversal through the three maintained modules | +| `src/tests.proj` | Test traversal | +| `src/Microsoft.Agents.A365.Sdk.sln` | Explicit solution project list | +| `build/build.ps1` | PowerShell build/test/pack entry point | +| `build/build.cmd` | Windows wrapper for the PowerShell script | -## File Structure +## Build commands -``` -├── global.json # SDK version management -├── Directory.Packages.props # Centralized package versions -├── Directory.Build.props # Common build properties -├── dirs.proj # Main build traversal -├── tests.proj # Test projects traversal -├── build.ps1 # Main build script -├── build.cmd # Windows build wrapper -└── [folders]/ - └── dirs.proj # Folder-specific build traversal -``` - -## Building the Solution - -### Using PowerShell (Recommended) +Run from the repository root: ```powershell -# Basic build -./build.ps1 - -# Build with tests -./build.ps1 -Test - -# Build with packages -./build.ps1 -Pack - -# Full build (clean, restore, build, test, pack) -./build.ps1 -Clean -Restore -Test -Pack - -# Debug build with tests -./build.ps1 -Configuration Debug -Test +.\build\build.ps1 -Restore +.\build\build.ps1 -Test +.\build\build.ps1 -Pack +.\build\build.ps1 -Clean -Restore -Test -Pack +.\build\build.ps1 -Configuration Debug -Test ``` -### Using Command Line +The script switches to `src` to use the traversal projects and SDK configuration. `-Restore` explicitly restores dependencies; otherwise the build uses existing assets. Configuration defaults to Release. Any failed build, test, or pack command fails the script. -```cmd -# Basic build -build.cmd +Direct solution commands are also supported: -# Build with parameters -build.cmd -Test -Pack +```powershell +dotnet restore src\Microsoft.Agents.A365.Sdk.sln +dotnet build src\Microsoft.Agents.A365.Sdk.sln --configuration Release --no-restore +dotnet test src\Microsoft.Agents.A365.Sdk.sln --configuration Release --no-build +dotnet format src\Microsoft.Agents.A365.Sdk.sln --verify-no-changes ``` -### Using dotnet CLI directly +For direct traversal operations, use `dotnet build src\dirs.proj` and `dotnet test src\tests.proj`. Individual module traversal files discover projects with wildcards while excluding build-output directories. -```bash -# Build all projects -dotnet build dirs.proj +## Central package management -# Run all tests -dotnet test tests.proj +Add dependency versions to `src/Directory.Packages.props`: -# Create packages -dotnet pack dirs.proj --output ../NuGetPackages +```xml + ``` -## Centralized Package Management - -All package versions are managed in `Directory.Packages.props`. To add a new package: - -1. Add the package version to `Directory.Packages.props`: - ```xml - - ``` - -2. Reference it in your project without version: - ```xml - - ``` - -### Version Conflict Resolution - -The centralized package management automatically resolves version conflicts by using the highest version specified in `Directory.Packages.props`. - -## Build Traversal - -The `dirs.proj` files use wildcards to automatically discover and build projects: +Reference the package without a version in the project: ```xml - + ``` -This means: -- ✅ Automatically finds all `.csproj` files in subdirectories -- ✅ Excludes test projects (built separately) -- ✅ Excludes bin/obj folders -- ✅ No need to manually maintain project lists - -## Project Structure Benefits - -### For Developers -- **Single command builds**: `./build.ps1` builds everything -- **Automatic project discovery**: No need to manually add projects to solution -- **Consistent dependencies**: All projects use the same package versions -- **Fast builds**: Only builds what's needed +Central versions must satisfy all projects' requirements; dependency resolution can still fail on incompatible constraints. Preserve transitive/security pins used by remaining modules. Package release versions are derived by Nerdbank.GitVersioning, not dependency-version declarations. -### For CI/CD -- **Reliable builds**: Centralized versions prevent conflicts -- **Easy scripting**: Simple build commands -- **Test separation**: Tests can be run independently -- **Package creation**: Automatic NuGet package generation +## Adding or removing projects -## Common Tasks +Update the relevant traversal inputs and **also** update `src/Microsoft.Agents.A365.Sdk.sln`. Automatic traversal discovery does not maintain the solution file. Add corresponding tests under `src/Tests/`, and regenerate `DEPENDENCIES.md` using `.\generate-dependencies.ps1` when package relationships change. -### Adding a New Project -1. Create your `.csproj` file -2. It will automatically be discovered by the build system -3. No need to modify solution files or build scripts +## Package isolation and release controls -### Adding Dependencies -1. Add package version to `Directory.Packages.props` -2. Reference without version in your project -3. The build system handles the rest +`-Pack` creates a new `NuGetPackages\` directory for every invocation and prints that exact path. It rejects retired `Microsoft.Agents.A365.Observability.*` filenames. Old package files are not included or silently deleted. -### Running Tests -```powershell -# Run all tests -./build.ps1 -Test - -# Run tests for specific configuration -./build.ps1 -Configuration Debug -Test -``` +CI similarly packs into a new temporary directory and uploads only that run's output, failing if no packages were produced. See [package setup](../src/PACKAGE-SETUP.md) for package IDs and content checks. -### Creating Packages -```powershell -# Create packages for all projects -./build.ps1 -Pack - -# Packages will be created in ../NuGetPackages/ -``` +External signing/publishing jobs must consume only this verified run-specific directory and allow only maintained IDs. These repository safeguards do not disable old-branch pipelines or prevent publishing previously cached artifacts; release owners must update those systems separately. No checked-in workflow pushes to nuget.org. ## Troubleshooting -### Build Errors -1. Ensure you have the correct .NET SDK version (specified in `global.json`) -2. Run with `-Clean -Restore` to reset state -3. Check `Directory.Packages.props` for version conflicts - -### Missing Projects -The build system automatically discovers projects. If a project isn't building: -1. Ensure it has a `.csproj` extension -2. Check it's not in `bin/` or `obj/` folders -3. Verify the `dirs.proj` patterns match your structure - -### Package Conflicts -All package versions are centralized. If you need a different version: -1. Update `Directory.Packages.props` -2. The new version will be used across all projects -3. Test thoroughly to ensure compatibility +If a project is missing, verify both the solution entry and module traversal patterns. For dependency failures, inspect restore errors and central versions; do not disable audit or warnings-as-errors in repository configuration to hide errors. For compiler failures, address nullable and XML documentation diagnostics, which are warnings-as-errors. diff --git a/build/build.ps1 b/build/build.ps1 index 668e4462..3fd0ed25 100644 --- a/build/build.ps1 +++ b/build/build.ps1 @@ -57,8 +57,8 @@ param( $ErrorActionPreference = 'Stop' -# Set working directory to script location -Set-Location $PSScriptRoot +# Traversal projects and SDK configuration live under src. +Set-Location (Join-Path $PSScriptRoot '..\src') Write-Host "🔧 Building Microsoft Agents A365 SDK..." -ForegroundColor Green Write-Host "Configuration: $Configuration" -ForegroundColor Cyan @@ -89,17 +89,23 @@ try { if ($Pack) { Write-Host "📦 Creating packages..." -ForegroundColor Yellow - # Ensure output directory exists - $outputPath = "../NuGetPackages" - if (!(Test-Path $outputPath)) { - New-Item -ItemType Directory -Path $outputPath -Force | Out-Null - } + # Isolate this run from packages left by earlier builds or branches. + $outputPath = Join-Path (Split-Path $PSScriptRoot -Parent) "NuGetPackages\$([guid]::NewGuid().ToString('N'))" + New-Item -ItemType Directory -Path $outputPath | Out-Null dotnet pack dirs.proj --configuration $Configuration --verbosity $Verbosity --no-build --output $outputPath if ($LASTEXITCODE -ne 0) { throw "Pack failed" } + + if (Get-ChildItem -LiteralPath $outputPath -Filter 'Microsoft.Agents.A365.Observability.*' -File) { + throw "Retired Observability packages must not be released." + } + if (-not (Get-ChildItem -LiteralPath $outputPath -Filter '*.nupkg' -File)) { + throw "Pack produced no NuGet packages." + } # List created packages Write-Host "📦 Created packages:" -ForegroundColor Green + Write-Host "Output: $outputPath" -ForegroundColor Cyan Get-ChildItem -Path $outputPath -Filter "*.nupkg" | ForEach-Object { Write-Host " - $($_.Name)" -ForegroundColor Cyan } diff --git a/docs/design.md b/docs/design.md index d36000c3..c4a52e3d 100644 --- a/docs/design.md +++ b/docs/design.md @@ -1,396 +1,110 @@ # Microsoft Agent 365 SDK for .NET - Design Documentation -This document describes the architecture and design of the Microsoft Agent 365 SDK for .NET, helping developers understand the project structure and begin reading and contributing code. - ## Overview -The Microsoft Agent 365 SDK for .NET is an enterprise-grade extension framework built on top of the Microsoft 365 Agents SDK. It provides four main pillars of functionality: +The SDK extends the Microsoft 365 Agents SDK with three maintained modules: -- **Observability**: OpenTelemetry-based distributed tracing with custom exporters -- **Notifications**: Event-driven notification handling for Microsoft 365 applications -- **Runtime**: Core utilities for multi-tenant agent applications -- **Tooling**: MCP (Model Context Protocol) server integration for tool discovery and registration +- **Notifications**: Event-driven notification handling for Microsoft 365 applications. +- **Runtime**: Core utilities for multi-tenant agent applications. +- **Tooling**: MCP (Model Context Protocol) server discovery and framework-specific tool registration. -**Target Frameworks**: -- Most packages: `net8.0` -- Observability Runtime and Hosting packages: `netstandard2.0` (for broader compatibility) +All remaining packages target `net8.0`. Observability has moved to the [Microsoft OpenTelemetry .NET distro](https://github.com/microsoft/opentelemetry-distro-dotnet/tree/main); see the [migration guide](observability-migration.md). This repository no longer builds or publishes Observability packages. -## Repository Structure +## Repository structure -``` +```text src/ -├── Runtime/ -│ └── Core/ # Core runtime utilities -├── Observability/ -│ ├── Runtime/ # Observability runtime (OpenTelemetry) -│ ├── Hosting/ # ASP.NET Core hosting middleware -│ └── Extensions/ -│ ├── AgentFramework/ # Agent Framework tracing -│ ├── SemanticKernel/ # Semantic Kernel tracing -│ └── OpenAI/ # OpenAI SDK tracing -├── Notification/ -│ └── Microsoft.Agents.A365.Notifications/ # Notification handling -├── Tooling/ -│ ├── Core/ # MCP server configuration -│ └── Extensions/ -│ ├── SemanticKernel/ # SK tool registration -│ ├── AgentFramework/ # AF tool registration -│ └── AzureAIFoundry/ # Azure AI Foundry integration -└── Tests/ # Test projects -docs/ -└── design.md # This file + Runtime/Core/ + Notification/Microsoft.Agents.A365.Notifications/ + Tooling/ + Core/ + Extensions/ + AgentFramework/ + SemanticKernel/ + AzureAIFoundry/ + Tests/ ``` ## Packages -| Package | Description | Design Document | -|---------|-------------|-----------------| -| `Microsoft.Agents.A365.Runtime` | Core runtime utilities including tenant context, operation results, and authorization | [design.md](../src/Runtime/Core/docs/design.md) | -| `Microsoft.Agents.A365.Observability.Runtime` | OpenTelemetry configuration, span processors, and scope classes | [design.md](../src/Observability/Runtime/docs/design.md) | -| `Microsoft.Agents.A365.Observability.Hosting` | ASP.NET Core middleware and hosting extensions | [design.md](../src/Observability/Hosting/docs/design.md) | -| `Microsoft.Agents.A365.Observability.Extensions.AgentFramework` | Agent Framework telemetry integration | [design.md](../src/Observability/Extensions/AgentFramework/docs/design.md) | -| `Microsoft.Agents.A365.Observability.Extensions.SemanticKernel` | Semantic Kernel telemetry integration | [design.md](../src/Observability/Extensions/SemanticKernel/docs/design.md) | -| `Microsoft.Agents.A365.Observability.Extensions.OpenAI` | OpenAI SDK telemetry integration | [design.md](../src/Observability/Extensions/OpenAI/docs/design.md) | -| `Microsoft.Agents.A365.Notifications` | Microsoft 365 notification handling | [design.md](../src/Notification/Microsoft.Agents.A365.Notifications/docs/design.md) | -| `Microsoft.Agents.A365.Tooling` | MCP server discovery and configuration | [design.md](../src/Tooling/Core/docs/design.md) | -| `Microsoft.Agents.A365.Tooling.Extensions.SemanticKernel` | Semantic Kernel MCP tool registration | [design.md](../src/Tooling/Extensions/SemanticKernel/docs/design.md) | -| `Microsoft.Agents.A365.Tooling.Extensions.AgentFramework` | Agent Framework MCP tool registration | [design.md](../src/Tooling/Extensions/AgentFramework/docs/design.md) | -| `Microsoft.Agents.A365.Tooling.Extensions.AzureAIFoundry` | Azure AI Foundry MCP tool registration | [design.md](../src/Tooling/Extensions/AzureAIFoundry/docs/design.md) | - -## Core Package Documentation - -### Runtime (`Microsoft.Agents.A365.Runtime`) - -Foundational utilities shared across the SDK. - -| Class | Purpose | -|-------|---------| -| `TenantContextHelper` | Extracts tenant and worker IDs from HttpContext (claims, headers, items) | -| `OperationResult` | Standardized success/failure result pattern with error collection | -| `OperationError` | Error details with HTTP status codes | -| `AgenticAuthorizationService` | Authorization handling for agentic operations | - -```csharp -// Extract tenant ID with fallback precedence -var tenantId = TenantContextHelper.GetTenantId(httpContext); - -// Return operation results -return OperationResult.Success; -return OperationResult.Failed(new OperationError("Not found", HttpStatusCode.NotFound)); -``` - -### Observability Runtime (`Microsoft.Agents.A365.Observability.Runtime`) - -OpenTelemetry-based distributed tracing infrastructure. - -| Class | Purpose | -|-------|---------| -| `Builder` | Fluent API for configuring OpenTelemetry tracing | -| `InvokeAgentScope` | Tracing scope for agent invocation operations | -| `InferenceScope` | Tracing scope for LLM inference operations | -| `ExecuteToolScope` | Tracing scope for tool execution | -| `BaggageBuilder` | OpenTelemetry baggage context management | -| `ActivityExtensions` | Extension methods for System.Diagnostics.Activity | - -```csharp -// Configure observability -services.AddOpenTelemetry() - .WithTracing(builder => new Builder(services, configuration) - .WithAgentFramework() - .WithSemanticKernel() - .Build()); - -// Create tracing scopes -using var scope = InvokeAgentScope.Start(invokeAgentDetails, tenantDetails, request); -scope.RecordResponse(response); -``` - -### Notifications (`Microsoft.Agents.A365.Notifications`) - -Event-driven notification handling for Microsoft 365 applications. +| Package | Purpose | Detailed design | +|---|---|---| +| `Microsoft.Agents.A365.Runtime` | Tenant context, operation results, authorization | [Runtime](../src/Runtime/Core/docs/design.md) | +| `Microsoft.Agents.A365.Notifications` | Microsoft 365 notification handling | [Notifications](../src/Notification/Microsoft.Agents.A365.Notifications/docs/design.md) | +| `Microsoft.Agents.A365.Tooling` | MCP server discovery and configuration | [Tooling Core](../src/Tooling/Core/docs/design.md) | +| `Microsoft.Agents.A365.Tooling.Extensions.AgentFramework` | Agent Framework MCP tool registration | [Agent Framework](../src/Tooling/Extensions/AgentFramework/docs/design.md) | +| `Microsoft.Agents.A365.Tooling.Extensions.SemanticKernel` | Semantic Kernel MCP tool registration | [Semantic Kernel](../src/Tooling/Extensions/SemanticKernel/docs/design.md) | +| `Microsoft.Agents.A365.Tooling.Extensions.AzureAIFoundry` | Azure AI Foundry MCP tool registration | [Azure AI Foundry](../src/Tooling/Extensions/AzureAIFoundry/docs/design.md) | -| Class | Purpose | -|-------|---------| -| `AgentNotification` | AgentExtension class for routing notifications | -| `AgentNotificationActivity` | Activity wrapper for notification data | -| `EmailReference` | Email notification entity model | -| `EmailResponse` | Email response entity for replies | -| `WpxComment` | Word/PowerPoint/Excel comment model | +## Architecture -**Supported Sub-channels:** -- `agents:email` - Email notifications -- `agents:word` - Word document notifications -- `agents:excel` - Excel workbook notifications -- `agents:powerpoint` - PowerPoint presentation notifications -- Federated Knowledge Service notifications +Framework-agnostic services live in core packages. Tooling extensions adapt those services to individual AI frameworks without adding framework-specific behavior to the core. -```csharp -// Register notification handlers -app.OnAgenticEmailNotification(async (turnContext, turnState, notification, ct) => -{ - var emailRef = notification.GetEntity(); - var reply = turnContext.Activity.CreateEmailResponseActivity("..."); - await turnContext.SendActivityAsync(reply, ct); -}); - -// Lifecycle notifications -app.OnAgenticUserIdentityCreatedNotification(async (context, state, notification, ct) => -{ - // Handle user identity creation -}); +```mermaid +flowchart TD + App[Agent Application] --> Runtime[Runtime] + App --> Notifications[Notifications] + App --> Tooling[Tooling Core] + Runtime --> Tenant[Tenant context and authorization] + Runtime --> Results[Operation results] + Notifications --> Channels[Email, Word, Excel, PowerPoint] + Tooling --> Discovery[MCP server discovery] + Tooling --> AF[Agent Framework extension] + Tooling --> SK[Semantic Kernel extension] + Tooling --> Foundry[Azure AI Foundry extension] ``` -### Tooling Core (`Microsoft.Agents.A365.Tooling`) - -MCP (Model Context Protocol) server discovery and configuration. +### Runtime -| Class | Purpose | -|-------|---------| -| `IMcpToolServerConfigurationService` | Interface for MCP server discovery | -| `McpToolServerConfigurationService` | Implementation for discovering and configuring MCP servers | -| `MCPServerConfig` | Server configuration model (name, URL, scope, audience) | -| `ToolOptions` | Configuration options for tooling operations | -| `ChatHistoryMessage` | Chat message model for threat protection | +`TenantContextHelper` extracts tenant and worker IDs from HTTP context claims, headers, and items. `AgenticAuthorizationService` handles authorization for agentic operations. `OperationResult` and `OperationError` provide consistent success/failure results with error details and HTTP status codes. ```csharp -// Discover available MCP servers -var servers = await configService.ListToolServersAsync(agentInstanceId, authToken); - -// Get tools from a specific server -var tools = await configService.GetMcpClientToolsAsync(turnContext, serverConfig, authToken, options); - -// Send chat history for threat protection -await configService.SendChatHistoryAsync(turnContext, chatHistory, cancellationToken); -``` - -## Architecture Diagram - -``` - ┌─────────────────────────────────────────────────────┐ - │ Agent Application │ - └──────────┬──────────────────┬───────────────────────┘ - │ │ - ┌────────────────────────────────┼──────────────────┼────────────────────────────────┐ - │ │ │ │ - ▼ ▼ ▼ ▼ - ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ - │ Runtime │ │ Observability │ │ Notifications │ │ Tooling │ - │ │ │ │ │ │ │ │ - │ TenantContext │◄──────────►│ Builder │ │ AgentNotification│ │ McpToolServer │ - │ OperationResult │ │ Scopes │ │ EmailReference │ │ ConfigService │ - │ Authorization │ │ SpanProcessors │ │ WpxComment │ │ MCPServerConfig │ - └─────────────────┘ └────────┬────────┘ └─────────────────┘ └────────┬────────┘ - │ │ - ┌───────────────────────┼───────────────────────┐ │ - │ │ │ │ - ▼ ▼ ▼ │ - ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ - │ AgentFramework │ │ SemanticKernel │ │ OpenAI │ │ - │ Extension │ │ Extension │ │ Extension │ │ - │ │ │ │ │ │ │ - │ SpanProcessor │ │ SpanProcessor │ │ SpanProcessor │ │ - │ BuilderExt │ │ FunctionFilter │ │ ChatToolCallExt │ │ - └─────────────────┘ └─────────────────┘ └─────────────────┘ │ - │ - ┌───────────────────────┬────────────────────────┐ │ - │ │ │ │ - ▼ ▼ ▼ │ - ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ - │ SemanticKernel │ │ AgentFramework │ │ AzureAIFoundry │◄┘ - │ Tooling Ext │ │ Tooling Ext │ │ Tooling Ext │ - │ │ │ │ │ │ - │ McpToolReg │ │ McpToolReg │ │ McpToolReg │ - │ Service │ │ Service │ │ Service │ - └─────────────────┘ └─────────────────┘ └─────────────────┘ -``` - -## Data Flow: Agent Invocation Tracing - -``` -┌──────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ -│ Request │────►│ BaggageMiddleware│────►│ InvokeAgentScope│────►│ InferenceScope │ -│ │ │ │ │ │ │ │ -│ HTTP Request │ │ Set tenant/agent│ │ Agent invocation│ │ LLM inference │ -│ with headers │ │ baggage context │ │ tracing span │ │ tracing span │ -└──────────────┘ └─────────────────┘ └────────┬────────┘ └────────┬────────┘ - │ │ - │ ▼ - │ ┌─────────────────┐ - │ │ ExecuteToolScope│ - │ │ │ - │ │ Tool execution │ - │ │ tracing span │ - │ └────────┬────────┘ - │ │ - ▼ ▼ - ┌─────────────────────────────────────────┐ - │ SpanProcessor │ - │ │ - │ Process spans, add tags, export to │ - │ Agent365 Exporter or Console │ - └─────────────────────────────────────────┘ -``` - -## Data Flow: MCP Tool Discovery - -``` -┌──────────────────┐ ┌─────────────────────┐ ┌─────────────────────┐ -│ Agent receives │────►│ ListToolServersAsync│────►│ For each server: │ -│ user message │ │ │ │ GetMcpClientTools │ -└──────────────────┘ │ Query Tooling │ │ │ - │ Gateway for servers │ │ Connect via MCP SDK │ - └─────────────────────┘ └──────────┬──────────┘ - │ - ▼ - ┌─────────────────────┐ - │ Register tools with │ - │ orchestrator: │ - │ - Semantic Kernel │ - │ - Agent Framework │ - │ - Azure AI Foundry │ - └─────────────────────┘ -``` - -## Design Patterns - -### Builder Pattern - -The observability `Builder` class uses a fluent API for configuration: - -```csharp -new Builder(services, configuration) - .WithAgentFramework() - .WithSemanticKernel() - .WithOpenAI() - .Build(); -``` - -### Disposable Pattern - -Scope classes implement `IDisposable` for automatic span lifecycle management: - -```csharp -using var scope = InvokeAgentScope.Start(details, tenantDetails, request); -// Span is automatically ended when scope is disposed -scope.RecordResponse(response); -``` - -### Result Pattern - -`OperationResult` provides a consistent way to handle success/failure: - -```csharp -public async Task DoWorkAsync() -{ - try - { - // Work... - return OperationResult.Success; - } - catch (Exception ex) - { - return OperationResult.Failed(new OperationError(ex.Message, HttpStatusCode.InternalServerError)); - } -} - -var result = await DoWorkAsync(); -if (!result.Succeeded) -{ - foreach (var error in result.Errors) - Console.WriteLine($"{error.StatusCode}: {error.Message}"); -} -``` - -### Extension Pattern - -Framework integrations use extension methods: - -```csharp -// Observability extensions -public static Builder WithAgentFramework(this Builder builder) { ... } -public static Builder WithSemanticKernel(this Builder builder) { ... } - -// Notification extensions -public static void OnAgenticEmailNotification(this AgentApplication app, ...) { ... } - -// Scope extensions -public static InvokeAgentScope FromTurnContext(this InvokeAgentScope scope, ITurnContext ctx) { ... } -``` - -### Strategy Pattern - -Environment-conditional behavior for exporters: - -```csharp -if (IsAgent365ExporterEnabled()) - tracerProviderBuilder.AddAgent365Exporter(exporterType); -else if (EnvironmentUtils.IsDevelopmentEnvironment()) - tracerProviderBuilder.AddConsoleExporter(); +return OperationResult.Success; +return OperationResult.Failed(new OperationError("Not found", HttpStatusCode.NotFound)); ``` -## Environment Variables +### Notifications -| Variable | Description | Default | -|----------|-------------|---------| -| `EnableAgent365Exporter` | Enable Agent365 telemetry exporter | `false` | -| `ASPNETCORE_ENVIRONMENT` | Environment name (Development, Production) | - | -| `SuppressInvokeAgentInput` | Suppress input messages in invoke_agent spans | `false` | +`AgentNotification` routes notification events; `AgentNotificationActivity` wraps the activity and associated data. Models such as `EmailReference`, `EmailResponse`, and `WpxComment` provide typed access to notification contents. -## Testing +Supported sub-channels include `agents:email`, `agents:word`, `agents:excel`, and `agents:powerpoint`. Lifecycle events include user identity creation and automatic sign-in; Federated Knowledge Service notifications are also supported. -Tests are organized by package in the `src/Tests/` directory: +Applications register handlers through extension methods such as `OnAgenticEmailNotification()` and `OnAgenticUserIdentityCreatedNotification()`. -``` -src/Tests/ -├── Runtime.Tests/ -├── Microsoft.Agents.A365.Observability.Runtime.Tests/ -├── Microsoft.Agents.A365.Observability.Runtime.IntegrationTests/ -├── Microsoft.Agents.A365.Observability.Hosting.Tests/ -├── Microsoft.Agents.A365.Observability.Extension.Tests/ -├── Microsoft.Agents.A365.Notifications.Tests/ -├── Microsoft.Agents.A365.Tooling.Tests/ -├── Microsoft.Agents.A365.Tooling.Extensions.SemanticKernel.Tests/ -├── Microsoft.Agents.A365.Tooling.Extensions.AgentFramework.Tests/ -└── Microsoft.Agents.A365.Tooling.Extensions.AzureAIFoundry.Tests/ -``` +### Tooling -Run tests with: +`IMcpToolServerConfigurationService` abstracts discovery and configuration. `McpToolServerConfigurationService` implements it, using `MCPServerConfig` to describe server endpoints and authentication details. Framework-specific extensions register discovered MCP tools with their orchestrator. -```bash -dotnet test +```mermaid +sequenceDiagram + participant Agent + participant Service as Tooling service + participant Gateway as Tooling Gateway + participant MCP as MCP server + participant Framework + Agent->>Service: ListToolServersAsync + Service->>Gateway: Discover available servers + Gateway-->>Service: Server configurations + Agent->>Service: GetMcpClientToolsAsync + Service->>MCP: Connect and list tools + MCP-->>Service: Tools + Service-->>Framework: Register framework-compatible tools ``` -## Development +Tooling also supports sending chat history for threat protection. HTTP authentication and tenant/context headers remain part of the Tooling integration; applications can supply context from their chosen instrumentation without referencing the retired SDK packages. -### Prerequisites +## Build and testing -- .NET 8.0 SDK -- Visual Studio 2022 or VS Code with C# extension +`src/dirs.proj` traverses Notifications, Runtime, and Tooling. Test projects live under `src/Tests/`; the solution explicitly includes six source and seven test projects. Keep solution entries synchronized when adding, renaming, or deleting projects even where traversal uses automatic discovery. -### Building +From the repository root: -```bash -dotnet build src/Microsoft.Agents.A365.Sdk.sln +```powershell +dotnet build src\Microsoft.Agents.A365.Sdk.sln +dotnet test src\Microsoft.Agents.A365.Sdk.sln +dotnet format src\Microsoft.Agents.A365.Sdk.sln --verify-no-changes ``` -### Package Management - -The solution uses centralized package management via `Directory.Packages.props`. All package versions are defined there. - -## Key Dependencies - -| Package | Version | Purpose | -|---------|---------|---------| -| `Microsoft.Agents.Builder` | 1.3.163-beta | Core Agents SDK | -| `OpenTelemetry` | 1.11.2 | Distributed tracing | -| `Microsoft.SemanticKernel` | 1.65.0 | Semantic Kernel orchestration | -| `Microsoft.Agents.AI` | 1.0.0-preview.251016.1 | Agent Framework | -| `ModelContextProtocol.Core` | 0.2.0-preview.3 | MCP client SDK | -| `Azure.AI.Agents.Persistent` | 1.2.0-beta.4 | Azure AI Foundry | - -## External Resources +NuGet versions are centralized in `src/Directory.Packages.props`; package versions derive from git history through Nerdbank.GitVersioning. Common build properties enable nullable analysis, XML documentation, and warnings as errors. -- [Microsoft Agent 365 Documentation](https://learn.microsoft.com/en-us/dotnet/api/agent365-sdk-dotnet/agent365-overview) -- [OpenTelemetry .NET](https://opentelemetry.io/docs/languages/net/) -- [Model Context Protocol](https://modelcontextprotocol.io/) -- [Microsoft 365 Agents SDK](https://github.com/microsoft/agents) +See [build documentation](../build/BUILD.md), [package setup](../src/PACKAGE-SETUP.md), and the generated [dependency diagram](../DEPENDENCIES.md). Historical changelog entries describe earlier releases, not the current package set. diff --git a/docs/observability-migration.md b/docs/observability-migration.md new file mode 100644 index 00000000..5f9bcc33 --- /dev/null +++ b/docs/observability-migration.md @@ -0,0 +1,130 @@ +# Migrating Observability to Microsoft.OpenTelemetry + +The `Microsoft.Agents.A365.Observability.*` source packages and their dedicated tests have been removed from this repository. Builds of this revision no longer produce those packages. Notifications, Runtime, and Tooling remain here; this change does not add a telemetry dependency to them. + +Observability development and releases are maintained in the [Microsoft OpenTelemetry .NET distro](https://github.com/microsoft/opentelemetry-distro-dotnet/tree/main). + +## Authoritative documentation and packages + +- [Microsoft.OpenTelemetry on NuGet](https://www.nuget.org/packages/Microsoft.OpenTelemetry): the application onboarding package. +- [Official Agent365 migration guide](https://github.com/microsoft/opentelemetry-distro-dotnet/blob/main/docs/agent365-migration.md): API differences, middleware, framework instrumentation, and troubleshooting. +- [Agent365 getting started](https://github.com/microsoft/opentelemetry-distro-dotnet/blob/main/docs/agent365-getting-started.md): exporter configuration, credentials, permissions, and deployment requirements. +- [Migration validation checklist](https://github.com/microsoft/opentelemetry-distro-dotnet/blob/main/skills/microsoft-opentelemetry-migration/references/validation-checklist.md). +- [Customization guide](https://github.com/microsoft/opentelemetry-distro-dotnet/blob/main/docs/customization.md). + +[`Microsoft.OpenTelemetry.A365.Contracts`](https://www.nuget.org/packages/Microsoft.OpenTelemetry.A365.Contracts) is a shared transitive dependency brought in by the distro. **Do not install it as a standalone replacement.** Some compatible types retain `Microsoft.Agents.A365.Observability.*` namespaces even though their assemblies now come from the distro. Namespace names are not package identities. + +Use the documentation matching the distro version you select. This is a package and configuration migration, not an automatic redirect or a guarantee of binary or behavioral compatibility. + +## 1. Replace packages + +Remove the following references wherever present, including central version declarations if no other projects need them: + +| Retired package | +|---| +| `Microsoft.Agents.A365.Observability.Runtime` | +| `Microsoft.Agents.A365.Observability.Hosting` | +| `Microsoft.Agents.A365.Observability.Extensions.AgentFramework` | +| `Microsoft.Agents.A365.Observability.Extensions.SemanticKernel` | +| `Microsoft.Agents.A365.Observability.Extensions.OpenAI` | + +For example, from a consuming application's project directory: + +```powershell +dotnet remove package Microsoft.Agents.A365.Observability.Runtime +dotnet add package Microsoft.OpenTelemetry +``` + +Repeat removal for each installed retired package, and select and pin a supported distro version using your application's package-management policy. With central package management, put the version in `Directory.Packages.props` and reference the package without a version in the project. + +Keep Notifications, Runtime, Tooling, and the AI framework/client packages your application uses. Remove redundant direct OpenTelemetry exporter/instrumentation references only after checking whether your custom configuration still needs them. Do not retain both old and distro telemetry registrations: duplicate providers/processors can duplicate spans or conflict. Also remove project references to copies of the old source. + +## 2. Replace initialization and review API differences + +Replace the old `Builder`, `AddA365Tracing()`, framework `WithAgentFramework()`/`WithSemanticKernel()`/`WithOpenAI()` setup, and standalone exporter registration with the distro's `UseMicrosoftOpenTelemetry()`. + +For example, this helper configures Agent365 export using an application-supplied app-only token resolver: + +```csharp +using Microsoft.AspNetCore.Builder; +using Microsoft.Extensions.DependencyInjection; +using Microsoft.OpenTelemetry; +using OpenTelemetry.Resources; + +static void ConfigureAgent365Telemetry( + WebApplicationBuilder builder, + Func> resolveAppOnlyToken) +{ + builder.Services.AddOpenTelemetry() + .ConfigureResource(resource => resource.AddService("MyAgent")) + .UseMicrosoftOpenTelemetry(options => + { + options.Exporters = ExportTarget.Agent365; + options.Agent365.UseS2SEndpoint = true; + options.Agent365.TokenResolver = (agentId, tenantId) => + resolveAppOnlyToken(agentId, tenantId); + }); +} +``` + +Call the helper before building the host and supply your existing approved token acquisition implementation. Preserve your service name, version, resource attributes, sampling, filtering, and privacy settings. For non-hosted applications, follow the distro's standalone initialization guidance and keep the telemetry provider alive until application shutdown so pending telemetry can flush. + +The old OpenAI `ChatToolCallExtensions.Trace()` convenience extension is not included in the distro. Replace it with direct `ExecuteToolScope.Start()` and response recording as described in the official migration guide. Recompile manual scope, DTO, middleware, and hosting integrations against the selected distro version; do not assume all old extension methods are available. For internal ETW scenarios, consult the distro's ETW documentation and availability rather than treating Contracts as an ETW replacement. + +## 3. Preserve authentication and routing intentionally + +This repository's latest standalone exporter uses an S2S-only endpoint and app-only tokens. The distro documentation also describes a DI/agentic token-cache onboarding path and a configurable `UseS2SEndpoint` option. **These paths are not interchangeable.** The example above selects S2S explicitly; do not accidentally switch to a delegated flow or rely on a different default while migrating. + +Review the selected distro release's authentication behavior with your deployment owners. Keep tenant/agent-aware, app-only token acquisition and cache/refresh per identity. Validate every token before use: accept `idtyp=app`, or, only when `idtyp` is absent, a non-empty `roles` array or a non-empty `oid` equal to `sub`; reject any other `idtyp` and any token containing `scp`. Also verify audience `api://9b975845-388f-4429-889e-eab1ef63949c` and lifetime, and ensure the application's identity and tenant match the exporting agent and tenant. An `Agent365.Observability.OtelWrite` grant is not a universal prerequisite; eligible Agent365-registered instances may use roleless app tokens when service policy permits. Follow deployment-specific permission/admin-consent requirements rather than assuming every app token is authorized. Never hardcode tokens, reuse incoming delegated request tokens, or fall back to OBO. Use the distro's documented `/.default` authentication-scope helper rather than copying old scope strings. + +The distro's DI agentic token cache and `RegisterObservability()` lifecycle support on-behalf-of (OBO) authentication, **not S2S**. S2S migrations must use the manual token resolver and `UseS2SEndpoint = true`; see the [official S2S example](https://github.com/microsoft/opentelemetry-distro-dotnet/blob/main/examples/Microsoft.OpenTelemetry.Agent365.S2S.Demo/README.md). Do not remove the application's credential registration or token acquisition merely because initialization has changed. + +Agent365 export requires its own identity, authentication, and endpoint routing. Installing generic OpenTelemetry packages or selecting `ExportTarget.Otlp` does not preserve that behavior. Use `ExportTarget.Agent365` for that destination; configure OTLP or Azure Monitor separately when those are intentional destinations. + +## 4. Preserve context and framework instrumentation + +The distro subscribes to supported Agent Framework, Semantic Kernel, and OpenAI activity sources, but the frameworks must still emit their activities. Preserve any framework-specific telemetry enablement. The upstream guide documents the OpenAI `OpenAI.Experimental.EnableOpenTelemetry` switch and explicit `.AddSource()` registration for custom Agent Framework/application source names. + +Identity context must be set **before** creating spans. The distro does not automatically register all baggage middleware: + +- Register `BaggageTurnMiddleware` in the agent adapter pipeline when using turn-based context. +- Register `UseObservabilityRequestContext()` before the agent HTTP pipeline when resolving HTTP-level tenant/agent identity. +- For manual processing, scope a `BaggageBuilder` with the correct tenant and agent IDs around the operation and dispose it afterward to restore context. + +See the official migration guide for signatures and middleware ordering. Spans without required tenant/agent identity may be dropped by the Agent365 exporter. Test concurrent tenants to ensure context and tokens cannot leak between requests. + +Review exporter differences as well: Agent365 exports traces, not all metrics/logs; Agent365-only mode can suppress infrastructure instrumentation. For offline buffering on Unix, the distro requires a customer-provisioned private storage directory or an explicit decision to disable offline storage. Review permissions, sensitive-data retention, and replay/backlog handling before rollout. + +## 5. Validate the consuming application + +1. Restore and build without references to retired packages, including transitive dependencies. +2. Exercise an agent invocation, inference, tool call, and output. Compare parent/child relationships, resource attributes, tenant/agent identity, and expected payloads with the previous deployment. +3. Confirm exporter authentication, endpoint routing, and receipt in the intended backend; locally visible spans alone do not prove Agent365 delivery. +4. Verify custom source subscriptions, privacy/sensitive-content controls, error reporting, graceful shutdown, and any offline replay behavior. +5. Test at least two tenant/agent identities and token refresh/failure cases before production rollout. + +Use Console export only for controlled development with non-sensitive data, not production. Follow the distro's [migration validation checklist](https://github.com/microsoft/opentelemetry-distro-dotnet/blob/main/skills/microsoft-opentelemetry-migration/references/validation-checklist.md) where applicable to your selected authentication flow; its `RegisterObservability()` checks apply to OBO, not S2S. + +## Maintainer checklist: existing NuGet packages + +Repository removal does **not** deprecate, unlist, delete, or redirect already-published packages. An authorized package owner must perform the following on [nuget.org](https://www.nuget.org/), following [NuGet's deprecation instructions](https://learn.microsoft.com/en-us/nuget/nuget-org/deprecate-packages): + +1. After this guide is available on the target branch, open **Manage package > Deprecation** for each of the five retired IDs above. +2. Select **all versions**, including stable and preview releases, and choose **Legacy**. +3. Set the alternative package to **Microsoft.OpenTelemetry**. If selecting an alternative version, verify that it is published and suitable for consumers. +4. Use this deprecation message: + + > This package is no longer maintained in Agent365-dotnet. Migrate to Microsoft.OpenTelemetry: https://www.nuget.org/packages/Microsoft.OpenTelemetry. Package and API changes are required; see https://github.com/microsoft/Agent365-dotnet/blob/main/docs/observability-migration.md and https://github.com/microsoft/opentelemetry-distro-dotnet/tree/main. This is not an automatic or drop-in replacement. + +5. Verify the notice and alternative recommendation on each package page. NuGet clients can show the alternative; the custom message is displayed on nuget.org, not all clients. +6. Optionally **unlist** retired versions to hide them from search. Unlisting and deprecation are separate actions; neither prevents restores by exact version. Preserve existing published versions for reproducible restores. + +There is **no automatic dependency redirect**. Consumers must change package references and migrate their applications. Do not publish empty replacement packages under the retired IDs. + +## Release controls: repository versus external systems + +**In this revision:** the solution and traversal build exclude all Observability projects; dedicated tests and exclusive dependencies are removed. CI and `build\build.ps1 -Pack` use new per-run output directories and reject retired Observability filenames before advertising/uploading the output. CI uploads only that run's directory and fails if it contains no packages. The dependency diagram no longer includes retired packages. + +**Required external actions:** update any private publishing/signing/release pipelines to consume only the current run's verified output, not a recursive or root-level wildcard over historical `NuGetPackages` directories. Retire queued releases and cached packages from previous runs. Exclude all retired IDs in the publishing allowlist, and disable old-branch/tag release jobs or permissions that could still publish them. Existing commits, branches, artifacts, and external pipelines are not rewritten by this change. + +The checked-in CI produces artifacts; it does not push packages to nuget.org. Package owners must separately apply the NuGet deprecations above and enforce external release controls. No registry settings are changed by this repository update. diff --git a/generate-dependencies.ps1 b/generate-dependencies.ps1 index 220055da..d1ff19be 100644 --- a/generate-dependencies.ps1 +++ b/generate-dependencies.ps1 @@ -14,11 +14,6 @@ # Define package types and their styling $packageTypes = @{ 'Microsoft.Agents.A365.Notifications' = @{ Type = 'Notifications'; Fill = '#ffcdd2'; Stroke = '#c62828'; Color = '#280505' } - 'Microsoft.Agents.A365.Observability.Runtime' = @{ Type = 'Observability'; Fill = '#c8e6c9'; Stroke = '#2e7d32'; Color = '#142a14' } - 'Microsoft.Agents.A365.Observability.Hosting' = @{ Type = 'Observability Extensions'; Fill = '#e8f5e9'; Stroke = '#66bb6a'; Color = '#1f3d1f' } - 'Microsoft.Agents.A365.Observability.Extensions.AgentFramework' = @{ Type = 'Observability Extensions'; Fill = '#e8f5e9'; Stroke = '#66bb6a'; Color = '#1f3d1f' } - 'Microsoft.Agents.A365.Observability.Extensions.OpenAI' = @{ Type = 'Observability Extensions'; Fill = '#e8f5e9'; Stroke = '#66bb6a'; Color = '#1f3d1f' } - 'Microsoft.Agents.A365.Observability.Extensions.SemanticKernel' = @{ Type = 'Observability Extensions'; Fill = '#e8f5e9'; Stroke = '#66bb6a'; Color = '#1f3d1f' } 'Microsoft.Agents.A365.Runtime' = @{ Type = 'Runtime'; Fill = '#bbdefb'; Stroke = '#1565c0'; Color = '#0d1a26' } 'Microsoft.Agents.A365.Tooling' = @{ Type = 'Tooling'; Fill = '#ffe0b2'; Stroke = '#e65100'; Color = '#331a00' } 'Microsoft.Agents.A365.Tooling.Extensions.AgentFramework' = @{ Type = 'Tooling Extensions'; Fill = '#fff3e0'; Stroke = '#fb8c00'; Color = '#4d2600' } @@ -88,11 +83,6 @@ function Get-AllProjects { # Define the specific projects we want to include $projectPaths = @( 'src\Notification\Microsoft.Agents.A365.Notifications\Microsoft.Agents.A365.Notifications.csproj', - 'src\Observability\Runtime\Microsoft.Agents.A365.Observability.Runtime.csproj', - 'src\Observability\Hosting\Microsoft.Agents.A365.Observability.Hosting.csproj', - 'src\Observability\Extensions\AgentFramework\Microsoft.Agents.A365.Observability.Extensions.AgentFramework.csproj', - 'src\Observability\Extensions\OpenAI\Microsoft.Agents.A365.Observability.Extensions.OpenAI.csproj', - 'src\Observability\Extensions\SemanticKernel\Microsoft.Agents.A365.Observability.Extensions.SemanticKernel.csproj', 'src\Runtime\Core\Microsoft.Agents.A365.Runtime.csproj', 'src\Tooling\Core\Microsoft.Agents.A365.Tooling.csproj', 'src\Tooling\Extensions\AgentFramework\Microsoft.Agents.A365.Tooling.Extensions.AgentFramework.csproj', @@ -205,8 +195,6 @@ $mermaidContent += "`r`n``````" $mermaidContent += "`r`n`r`n## Package Types`r`n" $mermaidContent += "`r`n- **Notifications** (Red): Notification and messaging extensions" $mermaidContent += "`r`n- **Runtime** (Blue): Core runtime components" -$mermaidContent += "`r`n- **Observability** (Green): Telemetry and monitoring core" -$mermaidContent += "`r`n- **Observability Extensions** (Light Green): Framework-specific observability integrations" $mermaidContent += "`r`n- **Tooling** (Orange): Agent tooling SDK core" $mermaidContent += "`r`n- **Tooling Extensions** (Light Orange): Framework-specific tooling integrations" diff --git a/src/Directory.Packages.props b/src/Directory.Packages.props index 40e71f11..556b79ea 100644 --- a/src/Directory.Packages.props +++ b/src/Directory.Packages.props @@ -7,28 +7,16 @@ - - - - - - - - - - - + - - @@ -85,6 +73,5 @@ - \ No newline at end of file diff --git a/src/Microsoft.Agents.A365.Sdk.sln b/src/Microsoft.Agents.A365.Sdk.sln index 2e1bb99c..c07b5124 100644 --- a/src/Microsoft.Agents.A365.Sdk.sln +++ b/src/Microsoft.Agents.A365.Sdk.sln @@ -3,8 +3,6 @@ Microsoft Visual Studio Solution File, Format Version 12.00 # Visual Studio Version 17 VisualStudioVersion = 17.14.36603.0 MinimumVisualStudioVersion = 10.0.40219.1 -Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Microsoft.Agents.A365.Observability.Extensions.SemanticKernel", "Observability\Extensions\SemanticKernel\Microsoft.Agents.A365.Observability.Extensions.SemanticKernel.csproj", "{5F7CC289-B41F-48A3-A433-B1034CD68C2B}" -EndProject Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Microsoft.Agents.A365.Tooling.Extensions.SemanticKernel", "Tooling\Extensions\SemanticKernel\Microsoft.Agents.A365.Tooling.Extensions.SemanticKernel.csproj", "{A5161F55-DA53-46AD-A355-D502184B8754}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Tooling", "Tooling", "{02EA681E-C7D8-13C7-8484-4AC65E1B71E8}" @@ -13,28 +11,16 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Microsoft.Agents.A365.Tooli EndProject Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Microsoft.Agents.A365.Notifications", "Notification\Microsoft.Agents.A365.Notifications\Microsoft.Agents.A365.Notifications.csproj", "{2340D2E1-4860-9469-0000-B8A67A5DF2F5}" EndProject -Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Microsoft.Agents.A365.Observability.Extensions.OpenAI", "Observability\Extensions\OpenAI\Microsoft.Agents.A365.Observability.Extensions.OpenAI.csproj", "{E6979AD1-7171-A4FF-6E26-EF2B933E091F}" -EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Runtime", "Runtime", "{81CCAD4E-143F-45B2-9F12-EBD23F57BA8E}" EndProject -Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Observability", "Observability", "{8675CB1F-2D41-4AAA-A40C-8EE2A37B64E4}" -EndProject -Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "SemanticKernel", "SemanticKernel", "{57BB42BE-33AA-41BB-8B6D-5587BBF6F3DC}" -EndProject -Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "OpenAI", "OpenAI", "{9AF47BAC-31DF-4B49-9EB7-99AD1A7F6716}" -EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Notification", "Notification", "{788C59A5-FB9C-4D9B-B488-6139861EC9F1}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Tests", "Tests", "{2150E333-8FDC-42A3-9474-1A3956D46DE8}" EndProject -Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.Agents.A365.Observability.Runtime.Tests", "Tests\Microsoft.Agents.A365.Observability.Runtime.Tests\Microsoft.Agents.A365.Observability.Runtime.Tests.csproj", "{B3C4D5E6-F7A8-4B9C-0D1E-2F3041526374}" -EndProject Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.Agents.A365.Runtime.Tests", "Tests\Runtime.Tests\Microsoft.Agents.A365.Runtime.Tests.csproj", "{7A8B9C0D-1E2F-4030-5142-6789ABCDEF01}" EndProject Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Microsoft.Agents.A365.Tooling.Extensions.AzureAIFoundry", "Tooling\Extensions\AzureAIFoundry\Microsoft.Agents.A365.Tooling.Extensions.AzureAIFoundry.csproj", "{C5A0294B-784B-456F-8C1A-A3FF4BB4C678}" EndProject -Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Extensions", "Extensions", "{715DDBDC-9E00-40A7-AC63-654F609903C2}" -EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Extensions", "Extensions", "{89AD1AA2-300A-483A-832E-9047201DD2BC}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "AzureAIFoundry", "AzureAIFoundry", "{516CB904-1318-4C78-8EAB-7A56AB1A5EDF}" @@ -43,26 +29,10 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "SemanticKernel", "SemanticK EndProject Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Microsoft.Agents.A365.Runtime", "Runtime\Core\Microsoft.Agents.A365.Runtime.csproj", "{CAD1967A-4732-6CEA-73A3-2FCA58BA3F93}" EndProject -Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Microsoft.Agents.A365.Observability.Runtime", "Observability\Runtime\Microsoft.Agents.A365.Observability.Runtime.csproj", "{23E78D75-FC06-443F-B34E-EA8BC567ED64}" -EndProject -Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Microsoft.Agents.A365.Observability.Hosting", "Observability\Hosting\Microsoft.Agents.A365.Observability.Hosting.csproj", "{3DA88459-22AC-4257-73ED-A0C042353C7C}" -EndProject -Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.Agents.A365.Observability.Hosting.Tests", "Tests\Microsoft.Agents.A365.Observability.Hosting.Tests\Microsoft.Agents.A365.Observability.Hosting.Tests.csproj", "{29B073D4-742B-7FC7-9C25-46CEDC132021}" -EndProject -Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Hosting", "Hosting", "{55BC8A20-8A50-4A6C-9649-884375774673}" -EndProject -Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Runtime", "Runtime", "{7ABD3B28-FB9A-41D6-8D21-4CBFC18B3451}" -EndProject -Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "AgentFramework", "AgentFramework", "{78E49660-FC50-4CC6-8391-7F857F5CCB6F}" -EndProject -Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Microsoft.Agents.A365.Observability.Extensions.AgentFramework", "Observability\Extensions\AgentFramework\Microsoft.Agents.A365.Observability.Extensions.AgentFramework.csproj", "{05AD60B6-0E5A-F382-A03E-8805B36ED6B8}" -EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "AgentFramework", "AgentFramework", "{9D91710A-8176-98AC-53E1-B6BCC2C3DF3D}" EndProject Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Microsoft.Agents.A365.Tooling.Extensions.AgentFramework", "Tooling\Extensions\AgentFramework\Microsoft.Agents.A365.Tooling.Extensions.AgentFramework.csproj", "{1BFA6B30-FC88-4C14-8F0A-9345611989C6}" EndProject -Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Microsoft.Agents.A365.Observability.Runtime.IntegrationTests", "Tests\Microsoft.Agents.A365.Observability.Runtime.IntegrationTests\Microsoft.Agents.A365.Observability.Runtime.IntegrationTests.csproj", "{B4B82E05-8C5C-A573-B04F-17517CAA6D1C}" -EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution Items", "{9C12EEA2-F35C-42D2-914C-3DA53B58683B}" ProjectSection(SolutionItems) = preProject Directory.Build.props = Directory.Build.props @@ -74,8 +44,6 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution SECURITY.md = SECURITY.md EndProjectSection EndProject -Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Microsoft.Agents.A365.Observability.Extension.Tests", "Tests\Microsoft.Agents.A365.Observability.Extension.Tests\Microsoft.Agents.A365.Observability.Extension.Tests.csproj", "{C992C8C8-106C-4DF5-A063-ECBFF74750F9}" -EndProject Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.Agents.A365.Tooling.Extensions.AgentFramework.Tests", "Tests\Microsoft.Agents.A365.Tooling.Extensions.AgentFramework.Tests\Microsoft.Agents.A365.Tooling.Extensions.AgentFramework.Tests.csproj", "{D1E2F3A4-B5C6-4D7E-8F9A-0B1C2D3E4F5A}" EndProject Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.Agents.A365.Tooling.Extensions.SemanticKernel.Tests", "Tests\Microsoft.Agents.A365.Tooling.Extensions.SemanticKernel.Tests\Microsoft.Agents.A365.Tooling.Extensions.SemanticKernel.Tests.csproj", "{E2F3A4B5-C6D7-4E8F-9A0B-1C2D3E4F5A6B}" @@ -86,12 +54,8 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.Agents.A365.Notif EndProject Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.Agents.A365.Tooling.Tests", "Tests\Microsoft.Agents.A365.Tooling.Tests\Microsoft.Agents.A365.Tooling.Tests.csproj", "{A1B2C3D4-E5F6-4A7B-8C9D-0E1F2A3B4C5D}" EndProject -Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Core", "Core", "{0CBC637E-EC70-66CD-FE99-EB46F5448A6D}" -EndProject Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Microsoft.Agents.A365.Tooling.Core.Tests", "Tests\Microsoft.Agents.A365.Tooling.Core.Tests\Microsoft.Agents.A365.Tooling.Core.Tests.csproj", "{5322E121-710E-4CBD-A5C4-0AC9E85F7164}" EndProject -Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Microsoft.Agents.A365.Observability.Extensions.IntegrationTests", "Tests\Microsoft.Agents.A365.Observability.Extensions.IntegrationTests\Microsoft.Agents.A365.Observability.Extensions.IntegrationTests.csproj", "{260338F1-ABA4-4617-81F3-B84AAEA43D52}" -EndProject Global GlobalSection(SolutionConfigurationPlatforms) = preSolution Debug|Any CPU = Debug|Any CPU @@ -102,18 +66,6 @@ Global Release|x86 = Release|x86 EndGlobalSection GlobalSection(ProjectConfigurationPlatforms) = postSolution - {5F7CC289-B41F-48A3-A433-B1034CD68C2B}.Debug|Any CPU.ActiveCfg = Debug|Any CPU - {5F7CC289-B41F-48A3-A433-B1034CD68C2B}.Debug|Any CPU.Build.0 = Debug|Any CPU - {5F7CC289-B41F-48A3-A433-B1034CD68C2B}.Debug|x64.ActiveCfg = Debug|Any CPU - {5F7CC289-B41F-48A3-A433-B1034CD68C2B}.Debug|x64.Build.0 = Debug|Any CPU - {5F7CC289-B41F-48A3-A433-B1034CD68C2B}.Debug|x86.ActiveCfg = Debug|Any CPU - {5F7CC289-B41F-48A3-A433-B1034CD68C2B}.Debug|x86.Build.0 = Debug|Any CPU - {5F7CC289-B41F-48A3-A433-B1034CD68C2B}.Release|Any CPU.ActiveCfg = Release|Any CPU - {5F7CC289-B41F-48A3-A433-B1034CD68C2B}.Release|Any CPU.Build.0 = Release|Any CPU - {5F7CC289-B41F-48A3-A433-B1034CD68C2B}.Release|x64.ActiveCfg = Release|Any CPU - {5F7CC289-B41F-48A3-A433-B1034CD68C2B}.Release|x64.Build.0 = Release|Any CPU - {5F7CC289-B41F-48A3-A433-B1034CD68C2B}.Release|x86.ActiveCfg = Release|Any CPU - {5F7CC289-B41F-48A3-A433-B1034CD68C2B}.Release|x86.Build.0 = Release|Any CPU {A5161F55-DA53-46AD-A355-D502184B8754}.Debug|Any CPU.ActiveCfg = Debug|Any CPU {A5161F55-DA53-46AD-A355-D502184B8754}.Debug|Any CPU.Build.0 = Debug|Any CPU {A5161F55-DA53-46AD-A355-D502184B8754}.Debug|x64.ActiveCfg = Debug|Any CPU @@ -150,30 +102,6 @@ Global {2340D2E1-4860-9469-0000-B8A67A5DF2F5}.Release|x64.Build.0 = Release|Any CPU {2340D2E1-4860-9469-0000-B8A67A5DF2F5}.Release|x86.ActiveCfg = Release|Any CPU {2340D2E1-4860-9469-0000-B8A67A5DF2F5}.Release|x86.Build.0 = Release|Any CPU - {E6979AD1-7171-A4FF-6E26-EF2B933E091F}.Debug|Any CPU.ActiveCfg = Debug|Any CPU - {E6979AD1-7171-A4FF-6E26-EF2B933E091F}.Debug|Any CPU.Build.0 = Debug|Any CPU - {E6979AD1-7171-A4FF-6E26-EF2B933E091F}.Debug|x64.ActiveCfg = Debug|Any CPU - {E6979AD1-7171-A4FF-6E26-EF2B933E091F}.Debug|x64.Build.0 = Debug|Any CPU - {E6979AD1-7171-A4FF-6E26-EF2B933E091F}.Debug|x86.ActiveCfg = Debug|Any CPU - {E6979AD1-7171-A4FF-6E26-EF2B933E091F}.Debug|x86.Build.0 = Debug|Any CPU - {E6979AD1-7171-A4FF-6E26-EF2B933E091F}.Release|Any CPU.ActiveCfg = Release|Any CPU - {E6979AD1-7171-A4FF-6E26-EF2B933E091F}.Release|Any CPU.Build.0 = Release|Any CPU - {E6979AD1-7171-A4FF-6E26-EF2B933E091F}.Release|x64.ActiveCfg = Release|Any CPU - {E6979AD1-7171-A4FF-6E26-EF2B933E091F}.Release|x64.Build.0 = Release|Any CPU - {E6979AD1-7171-A4FF-6E26-EF2B933E091F}.Release|x86.ActiveCfg = Release|Any CPU - {E6979AD1-7171-A4FF-6E26-EF2B933E091F}.Release|x86.Build.0 = Release|Any CPU - {B3C4D5E6-F7A8-4B9C-0D1E-2F3041526374}.Debug|Any CPU.ActiveCfg = Debug|Any CPU - {B3C4D5E6-F7A8-4B9C-0D1E-2F3041526374}.Debug|Any CPU.Build.0 = Debug|Any CPU - {B3C4D5E6-F7A8-4B9C-0D1E-2F3041526374}.Debug|x64.ActiveCfg = Debug|Any CPU - {B3C4D5E6-F7A8-4B9C-0D1E-2F3041526374}.Debug|x64.Build.0 = Debug|Any CPU - {B3C4D5E6-F7A8-4B9C-0D1E-2F3041526374}.Debug|x86.ActiveCfg = Debug|Any CPU - {B3C4D5E6-F7A8-4B9C-0D1E-2F3041526374}.Debug|x86.Build.0 = Debug|Any CPU - {B3C4D5E6-F7A8-4B9C-0D1E-2F3041526374}.Release|Any CPU.ActiveCfg = Release|Any CPU - {B3C4D5E6-F7A8-4B9C-0D1E-2F3041526374}.Release|Any CPU.Build.0 = Release|Any CPU - {B3C4D5E6-F7A8-4B9C-0D1E-2F3041526374}.Release|x64.ActiveCfg = Release|Any CPU - {B3C4D5E6-F7A8-4B9C-0D1E-2F3041526374}.Release|x64.Build.0 = Release|Any CPU - {B3C4D5E6-F7A8-4B9C-0D1E-2F3041526374}.Release|x86.ActiveCfg = Release|Any CPU - {B3C4D5E6-F7A8-4B9C-0D1E-2F3041526374}.Release|x86.Build.0 = Release|Any CPU {7A8B9C0D-1E2F-4030-5142-6789ABCDEF01}.Debug|Any CPU.ActiveCfg = Debug|Any CPU {7A8B9C0D-1E2F-4030-5142-6789ABCDEF01}.Debug|Any CPU.Build.0 = Debug|Any CPU {7A8B9C0D-1E2F-4030-5142-6789ABCDEF01}.Debug|x64.ActiveCfg = Debug|Any CPU @@ -210,54 +138,6 @@ Global {CAD1967A-4732-6CEA-73A3-2FCA58BA3F93}.Release|x64.Build.0 = Release|Any CPU {CAD1967A-4732-6CEA-73A3-2FCA58BA3F93}.Release|x86.ActiveCfg = Release|Any CPU {CAD1967A-4732-6CEA-73A3-2FCA58BA3F93}.Release|x86.Build.0 = Release|Any CPU - {23E78D75-FC06-443F-B34E-EA8BC567ED64}.Debug|Any CPU.ActiveCfg = Debug|Any CPU - {23E78D75-FC06-443F-B34E-EA8BC567ED64}.Debug|Any CPU.Build.0 = Debug|Any CPU - {23E78D75-FC06-443F-B34E-EA8BC567ED64}.Debug|x64.ActiveCfg = Debug|Any CPU - {23E78D75-FC06-443F-B34E-EA8BC567ED64}.Debug|x64.Build.0 = Debug|Any CPU - {23E78D75-FC06-443F-B34E-EA8BC567ED64}.Debug|x86.ActiveCfg = Debug|Any CPU - {23E78D75-FC06-443F-B34E-EA8BC567ED64}.Debug|x86.Build.0 = Debug|Any CPU - {23E78D75-FC06-443F-B34E-EA8BC567ED64}.Release|Any CPU.ActiveCfg = Release|Any CPU - {23E78D75-FC06-443F-B34E-EA8BC567ED64}.Release|Any CPU.Build.0 = Release|Any CPU - {23E78D75-FC06-443F-B34E-EA8BC567ED64}.Release|x64.ActiveCfg = Release|Any CPU - {23E78D75-FC06-443F-B34E-EA8BC567ED64}.Release|x64.Build.0 = Release|Any CPU - {23E78D75-FC06-443F-B34E-EA8BC567ED64}.Release|x86.ActiveCfg = Release|Any CPU - {23E78D75-FC06-443F-B34E-EA8BC567ED64}.Release|x86.Build.0 = Release|Any CPU - {3DA88459-22AC-4257-73ED-A0C042353C7C}.Debug|Any CPU.ActiveCfg = Debug|Any CPU - {3DA88459-22AC-4257-73ED-A0C042353C7C}.Debug|Any CPU.Build.0 = Debug|Any CPU - {3DA88459-22AC-4257-73ED-A0C042353C7C}.Debug|x64.ActiveCfg = Debug|Any CPU - {3DA88459-22AC-4257-73ED-A0C042353C7C}.Debug|x64.Build.0 = Debug|Any CPU - {3DA88459-22AC-4257-73ED-A0C042353C7C}.Debug|x86.ActiveCfg = Debug|Any CPU - {3DA88459-22AC-4257-73ED-A0C042353C7C}.Debug|x86.Build.0 = Debug|Any CPU - {3DA88459-22AC-4257-73ED-A0C042353C7C}.Release|Any CPU.ActiveCfg = Release|Any CPU - {3DA88459-22AC-4257-73ED-A0C042353C7C}.Release|Any CPU.Build.0 = Release|Any CPU - {3DA88459-22AC-4257-73ED-A0C042353C7C}.Release|x64.ActiveCfg = Release|Any CPU - {3DA88459-22AC-4257-73ED-A0C042353C7C}.Release|x64.Build.0 = Release|Any CPU - {3DA88459-22AC-4257-73ED-A0C042353C7C}.Release|x86.ActiveCfg = Release|Any CPU - {3DA88459-22AC-4257-73ED-A0C042353C7C}.Release|x86.Build.0 = Release|Any CPU - {29B073D4-742B-7FC7-9C25-46CEDC132021}.Debug|Any CPU.ActiveCfg = Debug|Any CPU - {29B073D4-742B-7FC7-9C25-46CEDC132021}.Debug|Any CPU.Build.0 = Debug|Any CPU - {29B073D4-742B-7FC7-9C25-46CEDC132021}.Debug|x64.ActiveCfg = Debug|Any CPU - {29B073D4-742B-7FC7-9C25-46CEDC132021}.Debug|x64.Build.0 = Debug|Any CPU - {29B073D4-742B-7FC7-9C25-46CEDC132021}.Debug|x86.ActiveCfg = Debug|Any CPU - {29B073D4-742B-7FC7-9C25-46CEDC132021}.Debug|x86.Build.0 = Debug|Any CPU - {29B073D4-742B-7FC7-9C25-46CEDC132021}.Release|Any CPU.ActiveCfg = Release|Any CPU - {29B073D4-742B-7FC7-9C25-46CEDC132021}.Release|Any CPU.Build.0 = Release|Any CPU - {29B073D4-742B-7FC7-9C25-46CEDC132021}.Release|x64.ActiveCfg = Release|Any CPU - {29B073D4-742B-7FC7-9C25-46CEDC132021}.Release|x64.Build.0 = Release|Any CPU - {29B073D4-742B-7FC7-9C25-46CEDC132021}.Release|x86.ActiveCfg = Release|Any CPU - {29B073D4-742B-7FC7-9C25-46CEDC132021}.Release|x86.Build.0 = Release|Any CPU - {05AD60B6-0E5A-F382-A03E-8805B36ED6B8}.Debug|Any CPU.ActiveCfg = Debug|Any CPU - {05AD60B6-0E5A-F382-A03E-8805B36ED6B8}.Debug|Any CPU.Build.0 = Debug|Any CPU - {05AD60B6-0E5A-F382-A03E-8805B36ED6B8}.Debug|x64.ActiveCfg = Debug|Any CPU - {05AD60B6-0E5A-F382-A03E-8805B36ED6B8}.Debug|x64.Build.0 = Debug|Any CPU - {05AD60B6-0E5A-F382-A03E-8805B36ED6B8}.Debug|x86.ActiveCfg = Debug|Any CPU - {05AD60B6-0E5A-F382-A03E-8805B36ED6B8}.Debug|x86.Build.0 = Debug|Any CPU - {05AD60B6-0E5A-F382-A03E-8805B36ED6B8}.Release|Any CPU.ActiveCfg = Release|Any CPU - {05AD60B6-0E5A-F382-A03E-8805B36ED6B8}.Release|Any CPU.Build.0 = Release|Any CPU - {05AD60B6-0E5A-F382-A03E-8805B36ED6B8}.Release|x64.ActiveCfg = Release|Any CPU - {05AD60B6-0E5A-F382-A03E-8805B36ED6B8}.Release|x64.Build.0 = Release|Any CPU - {05AD60B6-0E5A-F382-A03E-8805B36ED6B8}.Release|x86.ActiveCfg = Release|Any CPU - {05AD60B6-0E5A-F382-A03E-8805B36ED6B8}.Release|x86.Build.0 = Release|Any CPU {1BFA6B30-FC88-4C14-8F0A-9345611989C6}.Debug|Any CPU.ActiveCfg = Debug|Any CPU {1BFA6B30-FC88-4C14-8F0A-9345611989C6}.Debug|Any CPU.Build.0 = Debug|Any CPU {1BFA6B30-FC88-4C14-8F0A-9345611989C6}.Debug|x64.ActiveCfg = Debug|Any CPU @@ -270,30 +150,6 @@ Global {1BFA6B30-FC88-4C14-8F0A-9345611989C6}.Release|x64.Build.0 = Release|Any CPU {1BFA6B30-FC88-4C14-8F0A-9345611989C6}.Release|x86.ActiveCfg = Release|Any CPU {1BFA6B30-FC88-4C14-8F0A-9345611989C6}.Release|x86.Build.0 = Release|Any CPU - {B4B82E05-8C5C-A573-B04F-17517CAA6D1C}.Debug|Any CPU.ActiveCfg = Debug|Any CPU - {B4B82E05-8C5C-A573-B04F-17517CAA6D1C}.Debug|Any CPU.Build.0 = Debug|Any CPU - {B4B82E05-8C5C-A573-B04F-17517CAA6D1C}.Debug|x64.ActiveCfg = Debug|Any CPU - {B4B82E05-8C5C-A573-B04F-17517CAA6D1C}.Debug|x64.Build.0 = Debug|Any CPU - {B4B82E05-8C5C-A573-B04F-17517CAA6D1C}.Debug|x86.ActiveCfg = Debug|Any CPU - {B4B82E05-8C5C-A573-B04F-17517CAA6D1C}.Debug|x86.Build.0 = Debug|Any CPU - {B4B82E05-8C5C-A573-B04F-17517CAA6D1C}.Release|Any CPU.ActiveCfg = Release|Any CPU - {B4B82E05-8C5C-A573-B04F-17517CAA6D1C}.Release|Any CPU.Build.0 = Release|Any CPU - {B4B82E05-8C5C-A573-B04F-17517CAA6D1C}.Release|x64.ActiveCfg = Release|Any CPU - {B4B82E05-8C5C-A573-B04F-17517CAA6D1C}.Release|x64.Build.0 = Release|Any CPU - {B4B82E05-8C5C-A573-B04F-17517CAA6D1C}.Release|x86.ActiveCfg = Release|Any CPU - {B4B82E05-8C5C-A573-B04F-17517CAA6D1C}.Release|x86.Build.0 = Release|Any CPU - {C992C8C8-106C-4DF5-A063-ECBFF74750F9}.Debug|Any CPU.ActiveCfg = Debug|Any CPU - {C992C8C8-106C-4DF5-A063-ECBFF74750F9}.Debug|Any CPU.Build.0 = Debug|Any CPU - {C992C8C8-106C-4DF5-A063-ECBFF74750F9}.Debug|x64.ActiveCfg = Debug|Any CPU - {C992C8C8-106C-4DF5-A063-ECBFF74750F9}.Debug|x64.Build.0 = Debug|Any CPU - {C992C8C8-106C-4DF5-A063-ECBFF74750F9}.Debug|x86.ActiveCfg = Debug|Any CPU - {C992C8C8-106C-4DF5-A063-ECBFF74750F9}.Debug|x86.Build.0 = Debug|Any CPU - {C992C8C8-106C-4DF5-A063-ECBFF74750F9}.Release|Any CPU.ActiveCfg = Release|Any CPU - {C992C8C8-106C-4DF5-A063-ECBFF74750F9}.Release|Any CPU.Build.0 = Release|Any CPU - {C992C8C8-106C-4DF5-A063-ECBFF74750F9}.Release|x64.ActiveCfg = Release|Any CPU - {C992C8C8-106C-4DF5-A063-ECBFF74750F9}.Release|x64.Build.0 = Release|Any CPU - {C992C8C8-106C-4DF5-A063-ECBFF74750F9}.Release|x86.ActiveCfg = Release|Any CPU - {C992C8C8-106C-4DF5-A063-ECBFF74750F9}.Release|x86.Build.0 = Release|Any CPU {D1E2F3A4-B5C6-4D7E-8F9A-0B1C2D3E4F5A}.Debug|Any CPU.ActiveCfg = Debug|Any CPU {D1E2F3A4-B5C6-4D7E-8F9A-0B1C2D3E4F5A}.Debug|Any CPU.Build.0 = Debug|Any CPU {D1E2F3A4-B5C6-4D7E-8F9A-0B1C2D3E4F5A}.Debug|x64.ActiveCfg = Debug|Any CPU @@ -366,57 +222,28 @@ Global {5322E121-710E-4CBD-A5C4-0AC9E85F7164}.Release|x64.Build.0 = Release|Any CPU {5322E121-710E-4CBD-A5C4-0AC9E85F7164}.Release|x86.ActiveCfg = Release|Any CPU {5322E121-710E-4CBD-A5C4-0AC9E85F7164}.Release|x86.Build.0 = Release|Any CPU - {260338F1-ABA4-4617-81F3-B84AAEA43D52}.Debug|Any CPU.ActiveCfg = Debug|Any CPU - {260338F1-ABA4-4617-81F3-B84AAEA43D52}.Debug|Any CPU.Build.0 = Debug|Any CPU - {260338F1-ABA4-4617-81F3-B84AAEA43D52}.Debug|x64.ActiveCfg = Debug|Any CPU - {260338F1-ABA4-4617-81F3-B84AAEA43D52}.Debug|x64.Build.0 = Debug|Any CPU - {260338F1-ABA4-4617-81F3-B84AAEA43D52}.Debug|x86.ActiveCfg = Debug|Any CPU - {260338F1-ABA4-4617-81F3-B84AAEA43D52}.Debug|x86.Build.0 = Debug|Any CPU - {260338F1-ABA4-4617-81F3-B84AAEA43D52}.Release|Any CPU.ActiveCfg = Release|Any CPU - {260338F1-ABA4-4617-81F3-B84AAEA43D52}.Release|Any CPU.Build.0 = Release|Any CPU - {260338F1-ABA4-4617-81F3-B84AAEA43D52}.Release|x64.ActiveCfg = Release|Any CPU - {260338F1-ABA4-4617-81F3-B84AAEA43D52}.Release|x64.Build.0 = Release|Any CPU - {260338F1-ABA4-4617-81F3-B84AAEA43D52}.Release|x86.ActiveCfg = Release|Any CPU - {260338F1-ABA4-4617-81F3-B84AAEA43D52}.Release|x86.Build.0 = Release|Any CPU EndGlobalSection GlobalSection(SolutionProperties) = preSolution HideSolutionNode = FALSE EndGlobalSection GlobalSection(NestedProjects) = preSolution - {5F7CC289-B41F-48A3-A433-B1034CD68C2B} = {57BB42BE-33AA-41BB-8B6D-5587BBF6F3DC} {A5161F55-DA53-46AD-A355-D502184B8754} = {407C231E-438A-462B-8559-65A71551BD8B} {0EDBEB54-6955-8703-E310-2C98177ED550} = {02EA681E-C7D8-13C7-8484-4AC65E1B71E8} {2340D2E1-4860-9469-0000-B8A67A5DF2F5} = {788C59A5-FB9C-4D9B-B488-6139861EC9F1} - {E6979AD1-7171-A4FF-6E26-EF2B933E091F} = {9AF47BAC-31DF-4B49-9EB7-99AD1A7F6716} - {57BB42BE-33AA-41BB-8B6D-5587BBF6F3DC} = {715DDBDC-9E00-40A7-AC63-654F609903C2} - {9AF47BAC-31DF-4B49-9EB7-99AD1A7F6716} = {715DDBDC-9E00-40A7-AC63-654F609903C2} - {B3C4D5E6-F7A8-4B9C-0D1E-2F3041526374} = {2150E333-8FDC-42A3-9474-1A3956D46DE8} {7A8B9C0D-1E2F-4030-5142-6789ABCDEF01} = {2150E333-8FDC-42A3-9474-1A3956D46DE8} {C5A0294B-784B-456F-8C1A-A3FF4BB4C678} = {516CB904-1318-4C78-8EAB-7A56AB1A5EDF} - {715DDBDC-9E00-40A7-AC63-654F609903C2} = {8675CB1F-2D41-4AAA-A40C-8EE2A37B64E4} {89AD1AA2-300A-483A-832E-9047201DD2BC} = {02EA681E-C7D8-13C7-8484-4AC65E1B71E8} {516CB904-1318-4C78-8EAB-7A56AB1A5EDF} = {89AD1AA2-300A-483A-832E-9047201DD2BC} {407C231E-438A-462B-8559-65A71551BD8B} = {89AD1AA2-300A-483A-832E-9047201DD2BC} {CAD1967A-4732-6CEA-73A3-2FCA58BA3F93} = {81CCAD4E-143F-45B2-9F12-EBD23F57BA8E} - {23E78D75-FC06-443F-B34E-EA8BC567ED64} = {7ABD3B28-FB9A-41D6-8D21-4CBFC18B3451} - {3DA88459-22AC-4257-73ED-A0C042353C7C} = {55BC8A20-8A50-4A6C-9649-884375774673} - {29B073D4-742B-7FC7-9C25-46CEDC132021} = {2150E333-8FDC-42A3-9474-1A3956D46DE8} - {55BC8A20-8A50-4A6C-9649-884375774673} = {8675CB1F-2D41-4AAA-A40C-8EE2A37B64E4} - {7ABD3B28-FB9A-41D6-8D21-4CBFC18B3451} = {8675CB1F-2D41-4AAA-A40C-8EE2A37B64E4} - {78E49660-FC50-4CC6-8391-7F857F5CCB6F} = {715DDBDC-9E00-40A7-AC63-654F609903C2} - {05AD60B6-0E5A-F382-A03E-8805B36ED6B8} = {78E49660-FC50-4CC6-8391-7F857F5CCB6F} {9D91710A-8176-98AC-53E1-B6BCC2C3DF3D} = {89AD1AA2-300A-483A-832E-9047201DD2BC} {1BFA6B30-FC88-4C14-8F0A-9345611989C6} = {9D91710A-8176-98AC-53E1-B6BCC2C3DF3D} - {B4B82E05-8C5C-A573-B04F-17517CAA6D1C} = {2150E333-8FDC-42A3-9474-1A3956D46DE8} - {C992C8C8-106C-4DF5-A063-ECBFF74750F9} = {2150E333-8FDC-42A3-9474-1A3956D46DE8} {D1E2F3A4-B5C6-4D7E-8F9A-0B1C2D3E4F5A} = {2150E333-8FDC-42A3-9474-1A3956D46DE8} {E2F3A4B5-C6D7-4E8F-9A0B-1C2D3E4F5A6B} = {2150E333-8FDC-42A3-9474-1A3956D46DE8} {F3A4B5C6-D7E8-4F9A-0B1C-2D3E4F5A6B7C} = {2150E333-8FDC-42A3-9474-1A3956D46DE8} {D8E3F4A2-5B6C-4D7E-8F9A-0B1C2D3E4F5A} = {2150E333-8FDC-42A3-9474-1A3956D46DE8} {A1B2C3D4-E5F6-4A7B-8C9D-0E1F2A3B4C5D} = {2150E333-8FDC-42A3-9474-1A3956D46DE8} - {0CBC637E-EC70-66CD-FE99-EB46F5448A6D} = {02EA681E-C7D8-13C7-8484-4AC65E1B71E8} {5322E121-710E-4CBD-A5C4-0AC9E85F7164} = {2150E333-8FDC-42A3-9474-1A3956D46DE8} - {260338F1-ABA4-4617-81F3-B84AAEA43D52} = {2150E333-8FDC-42A3-9474-1A3956D46DE8} EndGlobalSection GlobalSection(ExtensibilityGlobals) = postSolution SolutionGuid = {0B20062E-2453-4E32-B8A2-85A6013F83A0} diff --git a/src/Observability/Extensions/AgentFramework/AgentFrameworkSpanProcessor.cs b/src/Observability/Extensions/AgentFramework/AgentFrameworkSpanProcessor.cs deleted file mode 100644 index a490dc9e..00000000 --- a/src/Observability/Extensions/AgentFramework/AgentFrameworkSpanProcessor.cs +++ /dev/null @@ -1,82 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using Microsoft.Agents.A365.Observability.Extensions.AgentFramework.Utils; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using OpenTelemetry; -using System.Diagnostics; - -namespace Microsoft.Agents.A365.Observability.Extensions.AgentFramework -{ - internal class AgentFrameworkSpanProcessor : BaseProcessor - { - private const string InvokeAgentOperation = "invoke_agent"; - private const string ChatOperation = "chat"; - private const string ExecuteToolOperation = "execute_tool"; - private const string ToolCallResultTag = "gen_ai.tool.call.result"; - private const string EventContentTag = "gen_ai.event.content"; - private readonly string[] _additionalSources; - - public AgentFrameworkSpanProcessor(params string[] additionalSources) - { - _additionalSources = additionalSources ?? []; - } - - public override void OnStart(Activity activity) - { - } - - public override void OnEnd(Activity activity) - { - if (activity == null) - return; - - if (IsTrackedSource(activity.Source.Name)) - { - var operationName = activity.GetTagItem(OpenTelemetryConstants.GenAiOperationNameKey); - if (operationName is string opName) - { - switch (opName) - { - case InvokeAgentOperation: - case ChatOperation: - var inputMessages = AgentFrameworkMessageMapper.MapInputMessages(activity); - if (inputMessages != null) - { - activity.SetTag(OpenTelemetryConstants.GenAiInputMessagesKey, inputMessages); - } - - var outputMessages = AgentFrameworkMessageMapper.MapOutputMessages(activity); - if (outputMessages != null) - { - activity.SetTag(OpenTelemetryConstants.GenAiOutputMessagesKey, outputMessages); - } - break; - - case ExecuteToolOperation: - var toolCallResult = activity.GetTagItem(ToolCallResultTag); - activity.SetTag(EventContentTag, toolCallResult); - break; - } - } - } - } - - private bool IsTrackedSource(string sourceName) - { - if (sourceName.StartsWith(BuilderExtensions.AgentFrameworkSource)) - { - return true; - } - - foreach (var source in _additionalSources) - { - if (!string.IsNullOrWhiteSpace(source) && sourceName.StartsWith(source)) - { - return true; - } - } - - return false; - } - } -} diff --git a/src/Observability/Extensions/AgentFramework/BuilderExtensions.cs b/src/Observability/Extensions/AgentFramework/BuilderExtensions.cs deleted file mode 100644 index de75521b..00000000 --- a/src/Observability/Extensions/AgentFramework/BuilderExtensions.cs +++ /dev/null @@ -1,72 +0,0 @@ -// ------------------------------------------------------------------------------ -// Copyright (c) Microsoft Corporation. All rights reserved. -// ------------------------------------------------------------------------------ - -namespace Microsoft.Agents.A365.Observability.Extensions.AgentFramework; - -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Agents.A365.Observability.Runtime; -using OpenTelemetry.Trace; -using OpenTelemetry; - -/// -/// Extension methods for configuring Builder with Agent Framework integration. -/// -public static class BuilderExtensions -{ - /// - /// The activity source name for Agent Framework tracing. - /// - public const string AgentFrameworkSource = "Experimental.Microsoft.Agents.AI"; - - /// - /// The activity source name for Agent Framework agent tracing. - /// - public const string AgentFrameworkAgentSource = "Experimental.Microsoft.Agents.AI.Agent"; - - /// - /// The activity source name for Agent Framework chat client tracing. - /// - public const string AgentFrameworkChatClientSource = "Experimental.Microsoft.Agents.AI.ChatClient"; - - /// - /// Adds Agent Framework integration to the builder. - /// - /// The builder to configure. - /// If true, enables Agent Framework activity source tracing for OpenTelemetry. - /// Optional additional activity source names to include in tracing. - /// The configured builder for method chaining. - public static Builder WithAgentFramework(this Builder builder, bool enableRelatedSources = true, params string[] additionalSources) - { - if (enableRelatedSources) - { - var telmConfig = builder.Services.AddOpenTelemetry() - .WithTracing(tracing => - { - tracing - .AddSource(AgentFrameworkSource) - .AddSource(AgentFrameworkAgentSource) - .AddSource(AgentFrameworkChatClientSource) - .AddProcessor(new AgentFrameworkSpanProcessor(additionalSources)); - - // Add any custom sources provided by the caller - foreach (var source in additionalSources) - { - if (!string.IsNullOrWhiteSpace(source)) - { - tracing.AddSource(source); - } - } - }); - - if (builder.Configuration != null - && !string.IsNullOrEmpty(builder.Configuration["EnableOtlpExporter"]) - && bool.TryParse(builder.Configuration["EnableOtlpExporter"], out bool enabled) && enabled) - { - telmConfig.UseOtlpExporter(); - } - } - - return builder; - } -} \ No newline at end of file diff --git a/src/Observability/Extensions/AgentFramework/Microsoft.Agents.A365.Observability.Extensions.AgentFramework.csproj b/src/Observability/Extensions/AgentFramework/Microsoft.Agents.A365.Observability.Extensions.AgentFramework.csproj deleted file mode 100644 index 7f3e177a..00000000 --- a/src/Observability/Extensions/AgentFramework/Microsoft.Agents.A365.Observability.Extensions.AgentFramework.csproj +++ /dev/null @@ -1,28 +0,0 @@ - - - Microsoft.Agents.A365.Observability.Extensions.AgentFramework - Microsoft.Agents.A365.Observability.Extensions.AgentFramework - net8.0 - - Microsoft.Agents.A365.Observability.Extensions.AgentFramework - Microsoft Agent 365 Agent Framework Observability SDK - Agent Framework observability integration for Microsoft Agent 365 SDK, providing tracing and monitoring for Agent Framework operations. - Microsoft Agent 365 Agent Framework Observability Integration - Tracing and monitoring for Agent Framework agents - Microsoft Agent 365 Agent Framework Observability SDK - Microsoft;Agents;A365;Observability;AgentFramework;OpenTelemetry;AI;Agents;Tracing;Monitoring - - - - - - - - - - - - - - - - diff --git a/src/Observability/Extensions/AgentFramework/README.md b/src/Observability/Extensions/AgentFramework/README.md deleted file mode 100644 index 8a0b7486..00000000 --- a/src/Observability/Extensions/AgentFramework/README.md +++ /dev/null @@ -1,30 +0,0 @@ -# Microsoft.Agents.A365.Observability.Extensions.AgentFramework - -Microsoft Agent Framework integration extensions for the Microsoft Agent 365 Observability SDK. This package enables integration between the Agent Framework and the Observability module, processing OpenTelemetry traces from the Agent Framework to align with the Observability schema. - -## Installation - -```bash -dotnet add package Microsoft.Agents.A365.Observability.Extensions.AgentFramework -``` - -## Documentation - -For detailed usage information, configuration examples, and best practices, see the [Microsoft Agents 365 Observability documentation](https://learn.microsoft.com/en-us/microsoft-agent-365/developer/observability?tabs=dotnet). - -## Support - -For issues, questions, or feedback: - -- File issues in the [GitHub Issues](https://github.com/microsoft/Agent365-dotnet/issues) section -- See the [main documentation](../../../../README.md) for more information - -## Trademarks - -*Microsoft, Windows, Microsoft Azure and/or other Microsoft products and services referenced in the documentation may be either trademarks or registered trademarks of Microsoft in the United States and/or other countries. The licenses for this project do not grant you rights to use any Microsoft names, logos, or trademarks. Microsoft's general trademark guidelines can be found at http://go.microsoft.com/fwlink/?LinkID=254653.* - -## License - -Copyright (c) Microsoft Corporation. All rights reserved. - -Licensed under the MIT License - see the [LICENSE](../../../../LICENSE.md) file for details. \ No newline at end of file diff --git a/src/Observability/Extensions/AgentFramework/Utils/AgentFrameworkMessageMapper.cs b/src/Observability/Extensions/AgentFramework/Utils/AgentFrameworkMessageMapper.cs deleted file mode 100644 index ee3c5426..00000000 --- a/src/Observability/Extensions/AgentFramework/Utils/AgentFrameworkMessageMapper.cs +++ /dev/null @@ -1,301 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Extensions.AgentFramework.Utils; - -using Microsoft.Agents.A365.Observability.Runtime.Tracing; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts.Messages; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using System; -using System.Collections.Generic; -using System.Diagnostics; -using System.Text.Json; - -/// -/// Maps Agent Framework span tag messages to A365 versioned message format. -/// Agent Framework sets gen_ai.input.messages / gen_ai.output.messages as span tags -/// containing JSON arrays of {role, parts[{type, content}], finish_reason?}. -/// This mapper converts them to / . -/// -internal static class AgentFrameworkMessageMapper -{ - - /// - /// Maps the gen_ai.input.messages tag value to a serialized A365 JSON string. - /// - public static string? MapInputMessages(Activity activity) - { - var json = GetTagValue(activity, OpenTelemetryConstants.GenAiInputMessagesKey); - if (string.IsNullOrEmpty(json)) - return null; - - try - { - using var doc = JsonDocument.Parse(json); - if (doc.RootElement.ValueKind != JsonValueKind.Array) - return null; - - var chatMessages = new List(); - - foreach (var msgElement in doc.RootElement.EnumerateArray()) - { - var role = GetStringProperty(msgElement, "role"); - var mappedRole = MapRole(role, MessageRole.User); - var parts = MapParts(msgElement); - var name = GetStringProperty(msgElement, "name"); - - if (parts.Count > 0) - { - chatMessages.Add(new ChatMessage(mappedRole, parts, name)); - } - } - - if (chatMessages.Count == 0) - return null; - - return MessageUtils.Serialize(new InputMessages(chatMessages)); - } - catch (JsonException) - { - return null; - } - } - - /// - /// Maps the gen_ai.output.messages tag value to a serialized A365 JSON string. - /// - public static string? MapOutputMessages(Activity activity) - { - var json = GetTagValue(activity, OpenTelemetryConstants.GenAiOutputMessagesKey); - if (string.IsNullOrEmpty(json)) - return null; - - try - { - using var doc = JsonDocument.Parse(json); - if (doc.RootElement.ValueKind != JsonValueKind.Array) - return null; - - var outputMessages = new List(); - - foreach (var msgElement in doc.RootElement.EnumerateArray()) - { - var role = MapRole(GetStringProperty(msgElement, "role"), MessageRole.Assistant); - var parts = MapParts(msgElement); - var finishReason = GetStringProperty(msgElement, "finish_reason"); - - if (parts.Count > 0) - { - outputMessages.Add(new OutputMessage(role, parts, finishReason: finishReason)); - } - } - - if (outputMessages.Count == 0) - return null; - - return MessageUtils.Serialize(new OutputMessages(outputMessages)); - } - catch (JsonException) - { - return null; - } - } - - private static List MapParts(JsonElement msgElement) - { - var parts = new List(); - - if (!msgElement.TryGetProperty("parts", out var partsElement) || - partsElement.ValueKind != JsonValueKind.Array) - { - return parts; - } - - foreach (var partElement in partsElement.EnumerateArray()) - { - var partType = GetStringProperty(partElement, "type"); - if (string.IsNullOrEmpty(partType)) - continue; - - var part = MapSinglePart(partType, partElement); - if (part != null) - { - parts.Add(part); - } - } - - return parts; - } - - private static IMessagePart? MapSinglePart(string partType, JsonElement partElement) - { - switch (partType.ToLowerInvariant()) - { - case "text": - var textContent = GetStringProperty(partElement, "content"); - return !string.IsNullOrEmpty(textContent) ? new TextPart(textContent) : null; - - case "reasoning": - var reasoningContent = GetStringProperty(partElement, "content"); - return !string.IsNullOrEmpty(reasoningContent) ? new ReasoningPart(reasoningContent) : null; - - case "tool_call": - return MapToolCallPart(partElement); - - case "tool_call_response": - return MapToolCallResponsePart(partElement); - - case "blob": - return MapBlobPart(partElement); - - case "file": - return MapFilePart(partElement); - - case "uri": - return MapUriPart(partElement); - - case "server_tool_call": - return MapServerToolCallPart(partElement); - - case "server_tool_call_response": - return MapServerToolCallResponsePart(partElement); - - default: - return MapGenericPart(partType, partElement); - } - } - - private static IMessagePart? MapToolCallPart(JsonElement el) - { - var name = GetStringProperty(el, "name"); - if (string.IsNullOrEmpty(name)) - return null; - - var id = GetStringProperty(el, "id"); - object? arguments = null; - if (el.TryGetProperty("arguments", out var argsEl)) - { - arguments = argsEl.ValueKind == JsonValueKind.String - ? argsEl.GetString() - : argsEl.ToString(); - } - - return new ToolCallRequestPart(name, id, arguments); - } - - private static IMessagePart? MapToolCallResponsePart(JsonElement el) - { - var id = GetStringProperty(el, "id"); - object? response = null; - if (el.TryGetProperty("response", out var respEl)) - { - response = respEl.ValueKind == JsonValueKind.String - ? respEl.GetString() - : respEl.ToString(); - } - - return new ToolCallResponsePart(id, response); - } - - private static IMessagePart? MapBlobPart(JsonElement el) - { - var modality = GetStringProperty(el, "modality"); - var content = GetStringProperty(el, "content"); - if (string.IsNullOrEmpty(modality) || string.IsNullOrEmpty(content)) - return null; - - var mimeType = GetStringProperty(el, "mime_type"); - return new BlobPart(modality, content, mimeType); - } - - private static IMessagePart? MapFilePart(JsonElement el) - { - var modality = GetStringProperty(el, "modality"); - var fileId = GetStringProperty(el, "file_id"); - if (string.IsNullOrEmpty(modality) || string.IsNullOrEmpty(fileId)) - return null; - - var mimeType = GetStringProperty(el, "mime_type"); - return new FilePart(modality, fileId, mimeType); - } - - private static IMessagePart? MapUriPart(JsonElement el) - { - var modality = GetStringProperty(el, "modality"); - var uri = GetStringProperty(el, "uri"); - if (string.IsNullOrEmpty(modality) || string.IsNullOrEmpty(uri)) - return null; - - var mimeType = GetStringProperty(el, "mime_type"); - return new UriPart(modality, uri, mimeType); - } - - private static IMessagePart? MapServerToolCallPart(JsonElement el) - { - var name = GetStringProperty(el, "name"); - if (string.IsNullOrEmpty(name)) - return null; - - var id = GetStringProperty(el, "id"); - var payload = new Dictionary(); - if (el.TryGetProperty("server_tool_call", out var stcEl)) - { - payload["server_tool_call"] = stcEl.ToString(); - } - - return new ServerToolCallPart(name, payload, id); - } - - private static IMessagePart? MapServerToolCallResponsePart(JsonElement el) - { - var id = GetStringProperty(el, "id"); - var payload = new Dictionary(); - if (el.TryGetProperty("server_tool_call_response", out var strEl)) - { - payload["server_tool_call_response"] = strEl.ToString(); - } - - return new ServerToolCallResponsePart(payload, id); - } - - private static IMessagePart? MapGenericPart(string type, JsonElement el) - { - var data = new Dictionary(); - foreach (var prop in el.EnumerateObject()) - { - if (prop.Name != "type") - { - data[prop.Name] = prop.Value.ValueKind == JsonValueKind.String - ? prop.Value.GetString()! - : prop.Value.ToString(); - } - } - - return new GenericPart(type, data); - } - - private static MessageRole MapRole(string? role, MessageRole defaultRole) - { - if (string.IsNullOrEmpty(role)) - return defaultRole; - - if (role.Equals("system", StringComparison.OrdinalIgnoreCase)) return MessageRole.System; - if (role.Equals("user", StringComparison.OrdinalIgnoreCase)) return MessageRole.User; - if (role.Equals("assistant", StringComparison.OrdinalIgnoreCase)) return MessageRole.Assistant; - if (role.Equals("tool", StringComparison.OrdinalIgnoreCase)) return MessageRole.Tool; - - return defaultRole; - } - - private static string? GetTagValue(Activity activity, string key) - { - return activity.GetTagItem(key) as string; - } - - private static string? GetStringProperty(JsonElement element, string propertyName) - { - if (element.TryGetProperty(propertyName, out var prop) && prop.ValueKind == JsonValueKind.String) - return prop.GetString(); - return null; - } -} diff --git a/src/Observability/Extensions/AgentFramework/docs/design.md b/src/Observability/Extensions/AgentFramework/docs/design.md deleted file mode 100644 index 84e4770e..00000000 --- a/src/Observability/Extensions/AgentFramework/docs/design.md +++ /dev/null @@ -1,386 +0,0 @@ -# Microsoft.Agents.A365.Observability.Extensions.AgentFramework - Design Documentation - -## Overview - -The `Microsoft.Agents.A365.Observability.Extensions.AgentFramework` package provides OpenTelemetry tracing integration for the Microsoft Agent Framework. It includes a span processor that intercepts and enriches telemetry from Agent Framework activity sources. - -## Architecture - -``` -Microsoft.Agents.A365.Observability.Extensions.AgentFramework -├── Public API -│ └── BuilderExtensions # Extension method for Builder -├── Internal -│ ├── AgentFrameworkSpanProcessor # Span processing logic -│ └── Utils/ -│ └── AgentFrameworkSpanProcessorHelper # Helper methods -└── Models/ - └── AgentFrameworkMessageContent # Message content wrapper -``` - -## Key Components - -### BuilderExtensions - -**Source**: [BuilderExtensions.cs](../BuilderExtensions.cs) - -Provides the `WithAgentFramework()` extension method for the observability `Builder`. - -```csharp -// Configure observability with Agent Framework support -new Builder(services, configuration) - .WithAgentFramework() - .Build(); - -// Without related sources (manual configuration) -new Builder(services, configuration) - .WithAgentFramework(enableRelatedSources: false) - .Build(); - -// With additional activity sources -new Builder(services, configuration) - .WithAgentFramework(true, "Custom.Source.Name", "Another.Source") - .Build(); -``` - -**Implementation:** - -```csharp -public static Builder WithAgentFramework(this Builder builder, bool enableRelatedSources = true, params string[] additionalSources) -{ - if (enableRelatedSources) - { - var telmConfig = builder.Services.AddOpenTelemetry() - .WithTracing(tracing => - { - tracing - .AddSource(AgentFrameworkSource) - .AddSource(AgentFrameworkAgentSource) - .AddSource(AgentFrameworkChatClientSource) - .AddProcessor(new AgentFrameworkSpanProcessor(additionalSources)); - - // Add any custom sources provided by the caller - foreach (var source in additionalSources) - { - if (!string.IsNullOrWhiteSpace(source)) - { - tracing.AddSource(source); - } - } - }); - - if (builder.Configuration != null - && !string.IsNullOrEmpty(builder.Configuration["EnableOtlpExporter"]) - && bool.TryParse(builder.Configuration["EnableOtlpExporter"], out bool enabled) && enabled) - { - telmConfig.UseOtlpExporter(); - } - } - - return builder; -} -``` - -**Parameters:** - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `enableRelatedSources` | `bool` | `true` | Enable OpenTelemetry for Agent Framework and add related sources | -| `additionalSources` | `string[]` | `[]` | Additional activity source names to track | - -**Configuration:** - -| Key | Description | -|-----|-------------| -| `EnableOtlpExporter` | Set to `true` to enable OTLP exporter | - -**Tracked Activity Sources:** -- `Experimental.Microsoft.Agents.AI` -- `Experimental.Microsoft.Agents.AI.Agent` -- `Experimental.Microsoft.Agents.AI.ChatClient` -- Additional custom sources (optional) - -### AgentFrameworkSpanProcessor - -**Source**: [AgentFrameworkSpanProcessor.cs](../AgentFrameworkSpanProcessor.cs) - -A `BaseProcessor` that processes spans from Agent Framework activity sources. - -**Processed Operations:** - -| Operation | Processing | -|-----------|------------| -| `invoke_agent` | Processes input/output messages | -| `chat` | Processes input/output messages | -| `execute_tool` | Extracts tool call result to event content | - -```csharp -internal class AgentFrameworkSpanProcessor : BaseProcessor -{ - private const string InvokeAgentOperation = "invoke_agent"; - private const string ChatOperation = "chat"; - private const string ExecuteToolOperation = "execute_tool"; - - public override void OnEnd(Activity activity) - { - if (IsTrackedSource(activity.Source.Name)) - { - var operationName = activity.GetTagItem(GenAiOperationNameKey); - switch (operationName) - { - case InvokeAgentOperation: - case ChatOperation: - AgentFrameworkSpanProcessorHelper.ProcessInputOutputMessages(activity); - break; - - case ExecuteToolOperation: - var result = activity.GetTagItem(ToolCallResultTag); - activity.SetTag(EventContentTag, result); - break; - } - } - } -} -``` - -### AgentFrameworkSpanProcessorHelper - -**Source**: [AgentFrameworkSpanProcessorHelper.cs](../Utils/AgentFrameworkSpanProcessorHelper.cs) - -Helper class for processing span attributes and message content. - -```csharp -internal static class AgentFrameworkSpanProcessorHelper -{ - public static void ProcessInputOutputMessages(Activity activity) - { - // Extract and process gen_ai.content.prompt events - // Extract and process gen_ai.content.completion events - // Set gen_ai.input_messages and gen_ai.output_messages tags - } -} -``` - -### AgentFrameworkMessageContent - -Model class for wrapping message content from Agent Framework spans. - -```csharp -internal class AgentFrameworkMessageContent -{ - public string? Role { get; set; } - public string? Content { get; set; } -} -``` - -## Design Patterns - -### Extension Pattern - -The package extends the core `Builder` through extension methods: - -```csharp -public static class BuilderExtensions -{ - public const string AgentFrameworkSource = "Experimental.Microsoft.Agents.AI"; - public const string AgentFrameworkAgentSource = "Experimental.Microsoft.Agents.AI.Agent"; - public const string AgentFrameworkChatClientSource = "Experimental.Microsoft.Agents.AI.ChatClient"; - - public static Builder WithAgentFramework( - this Builder builder, - bool enableRelatedSources = true, - params string[] additionalSources) - { - if (enableRelatedSources) - { - var telmConfig = builder.Services.AddOpenTelemetry() - .WithTracing(tracing => - { - tracing - .AddSource(AgentFrameworkSource) - .AddSource(AgentFrameworkAgentSource) - .AddSource(AgentFrameworkChatClientSource) - .AddProcessor(new AgentFrameworkSpanProcessor(additionalSources)); - - foreach (var source in additionalSources) - { - if (!string.IsNullOrWhiteSpace(source)) - tracing.AddSource(source); - } - }); - - // Optional OTLP exporter - if (builder.Configuration?["EnableOtlpExporter"] == "true") - telmConfig.UseOtlpExporter(); - } - - return builder; - } -} -``` - -### Span Processor Pattern - -The processor inherits from OpenTelemetry's `BaseProcessor`: - -```csharp -internal class AgentFrameworkSpanProcessor : BaseProcessor -{ - private readonly string[] _additionalSources; - - public AgentFrameworkSpanProcessor(params string[] additionalSources) - { - _additionalSources = additionalSources ?? []; - } - - public override void OnStart(Activity activity) - { - // Pre-processing (not used for Agent Framework) - } - - public override void OnEnd(Activity activity) - { - // Post-processing - enrich spans before export - } - - private bool IsTrackedSource(string sourceName) - { - if (sourceName.StartsWith(AgentFrameworkSource)) - return true; - - return _additionalSources.Any(s => - !string.IsNullOrWhiteSpace(s) && sourceName.StartsWith(s)); - } -} -``` - -## Data Flow - -``` -┌─────────────────────────────┐ -│ Agent Framework │ -│ │ -│ AIAgent.InvokeAsync() │ -│ ChatClient.CompleteAsync() │ -│ Tool execution │ -└──────────────┬──────────────┘ - │ - ▼ Activity Source Events -┌─────────────────────────────┐ -│ Experimental.Microsoft. │ -│ Agents.AI.* │ -│ │ -│ Creates Activity/Span │ -│ with gen_ai.* tags │ -└──────────────┬──────────────┘ - │ - ▼ OnEnd callback -┌─────────────────────────────┐ -│ AgentFrameworkSpanProcessor │ -│ │ -│ 1. Check if tracked source │ -│ 2. Get operation name │ -│ 3. Process based on type: │ -│ - invoke_agent: messages │ -│ - chat: messages │ -│ - execute_tool: result │ -└──────────────┬──────────────┘ - │ - ▼ Enriched Activity -┌─────────────────────────────┐ -│ OpenTelemetry Exporter │ -│ │ -│ Export to: │ -│ - Agent365 Exporter │ -│ - Console │ -│ - OTLP │ -└─────────────────────────────┘ -``` - -## Span Attribute Mapping - -### invoke_agent / chat Operations - -| Source Tag | Target Tag | -|------------|------------| -| `gen_ai.content.prompt` events | `gen_ai.input_messages` | -| `gen_ai.content.completion` events | `gen_ai.output_messages` | - -### execute_tool Operation - -| Source Tag | Target Tag | -|------------|------------| -| `gen_ai.tool.call.result` | `gen_ai.event.content` | - -## File Structure - -``` -src/Observability/Extensions/AgentFramework/ -├── BuilderExtensions.cs # Builder extension method -├── AgentFrameworkSpanProcessor.cs # Span processor -├── Utils/ -│ └── AgentFrameworkSpanProcessorHelper.cs # Helper methods -├── Models/ -│ └── AgentFrameworkMessageContent.cs # Message model -├── Microsoft.Agents.A365.Observability.Extensions.AgentFramework.csproj -└── docs/ - └── design.md # This file -``` - -## Dependencies - -| Package | Purpose | -|---------|---------| -| `Microsoft.Agents.A365.Observability.Runtime` | Core observability | -| `OpenTelemetry.Instrumentation.AspNetCore` | ASP.NET Core instrumentation | -| `OpenTelemetry.Instrumentation.Http` | HTTP client instrumentation | - -## Usage Examples - -### Basic Setup - -```csharp -// Program.cs -builder.Services.AddOpenTelemetry() - .WithTracing(tracerBuilder => - { - new Builder(builder.Services, builder.Configuration) - .WithAgentFramework() - .Build(); - }); -``` - -### With Custom Sources - -```csharp -// Add custom activity sources to track -new Builder(services, configuration) - .WithAgentFramework( - "MyApp.CustomAgent", - "MyApp.CustomTools" - ) - .Build(); -``` - -### Agent Framework Usage - -```csharp -public class MyAgentService -{ - private readonly AIAgent _agent; - - public async Task ProcessAsync(string message) - { - // Agent Framework creates spans automatically - // AgentFrameworkSpanProcessor enriches them - var response = await _agent.InvokeAsync(message); - return response.Content; - } -} -``` - -## External Resources - -- [Microsoft Agent Framework](https://github.com/microsoft/agents) -- [OpenTelemetry Span Processors](https://opentelemetry.io/docs/specs/otel/trace/sdk/#span-processor) -- [OpenTelemetry GenAI Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) diff --git a/src/Observability/Extensions/OpenAI/BuilderExtensions.cs b/src/Observability/Extensions/OpenAI/BuilderExtensions.cs deleted file mode 100644 index a0105ebc..00000000 --- a/src/Observability/Extensions/OpenAI/BuilderExtensions.cs +++ /dev/null @@ -1,34 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Extensions.OpenAI; - -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Agents.A365.Observability.Runtime; -using OpenTelemetry.Trace; - -/// -/// Extension methods for configuring Builder with OpenAI integration. -/// -public static class BuilderExtensions -{ - /// - /// Adds OpenAI integration to the builder. - /// - /// The builder to configure. - /// Whether to enable related tracing sources for OpenTelemetry. - /// The configured builder for method chaining. - public static Builder WithOpenAI(this Builder builder, bool enableRelatedSources = true) - { - if (enableRelatedSources) - { - AppContext.SetSwitch("OpenAI.Experimental.EnableOpenTelemetry", true); - builder.Services.AddOpenTelemetry() - .WithTracing(tracing => tracing - .AddSource(OpenAITelemetryConstants.OpenAISourceWildcard) - .AddProcessor(new OpenAISpanProcessor())); - } - - return builder; - } -} \ No newline at end of file diff --git a/src/Observability/Extensions/OpenAI/ChatToolCallExtensions.cs b/src/Observability/Extensions/OpenAI/ChatToolCallExtensions.cs deleted file mode 100644 index 440d062f..00000000 --- a/src/Observability/Extensions/OpenAI/ChatToolCallExtensions.cs +++ /dev/null @@ -1,35 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using OpenAI.Chat; - -namespace Microsoft.Agents.A365.Observability.Extensions.OpenAI; - -/// -/// Extension methods for ChatToolCall. -/// -public static class ChatToolCallExtensions -{ - /// - /// Starts an ExecuteToolScope for the given ChatToolCall for OpenTelemetry tracing. - /// - /// The ChatToolCall instance. - /// The agent identifier. - /// The tenant identifier. - /// An ExecuteToolScope. - public static ExecuteToolScope? Trace(this ChatToolCall chatToolCall, string agentId, string? tenantId = null) - { - var details = new ToolCallDetails( - chatToolCall.FunctionName, - chatToolCall.FunctionArguments?.ToString(), - chatToolCall.Id, - null, - chatToolCall.Kind.ToString() - ); - - var agentDetails = new AgentDetails(agentId: agentId, tenantId: tenantId); - return ExecuteToolScope.Start(new Request(), details, agentDetails); - } -} diff --git a/src/Observability/Extensions/OpenAI/Microsoft.Agents.A365.Observability.Extensions.OpenAI.csproj b/src/Observability/Extensions/OpenAI/Microsoft.Agents.A365.Observability.Extensions.OpenAI.csproj deleted file mode 100644 index b35e1b3f..00000000 --- a/src/Observability/Extensions/OpenAI/Microsoft.Agents.A365.Observability.Extensions.OpenAI.csproj +++ /dev/null @@ -1,29 +0,0 @@ - - - Microsoft.Agents.A365.Observability.Extensions.OpenAI - Microsoft.Agents.A365.Observability.Extensions.OpenAI - net8.0 - - Microsoft.Agents.A365.Observability.Extensions.OpenAI - Microsoft Agent 365 OpenAI Observability SDK - OpenAI observability integration for Microsoft Agent 365 SDK, providing tracing and monitoring for OpenAI operations. - Microsoft Agent 365 OpenAI Observability Integration - Tracing and monitoring for OpenAI agents - Microsoft Agent 365 OpenAI Observability SDK - Microsoft;Agents;A365;Observability;OpenAI;OpenTelemetry;AI;Agents;Tracing;Monitoring - - $(NoWarn);NU5104 - - - - - - - - - - - - - - - diff --git a/src/Observability/Extensions/OpenAI/OpenAISpanProcessor.cs b/src/Observability/Extensions/OpenAI/OpenAISpanProcessor.cs deleted file mode 100644 index 13245a46..00000000 --- a/src/Observability/Extensions/OpenAI/OpenAISpanProcessor.cs +++ /dev/null @@ -1,36 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Extensions.OpenAI; - -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using OpenTelemetry; -using System.Diagnostics; -using System.Linq; - -internal class OpenAISpanProcessor : BaseProcessor -{ - private static readonly string TargetSourceName = OpenAITelemetryConstants.OpenAISource; - - public override void OnStart(Activity activity) - { - } - - public override void OnEnd(Activity activity) - { - if (activity.Source.Name.StartsWith(TargetSourceName)) - { - var tags = activity.Tags.ToDictionary(kv => kv.Key, kv => kv.Value); - if (tags.TryGetValue(OpenTelemetryConstants.GenAiOperationNameKey, out var operationName)) - { - switch (operationName) - { - case OpenAITelemetryConstants.ChatOperation: - // Span emitted by OpenAI SDK follows Microsoft Agent 365 schema, so no modification needed. - // Placeholder for any plumbing if needed in the future. - break; - } - } - } - } -} diff --git a/src/Observability/Extensions/OpenAI/OpenAITelemetryConstants.cs b/src/Observability/Extensions/OpenAI/OpenAITelemetryConstants.cs deleted file mode 100644 index bb3ace52..00000000 --- a/src/Observability/Extensions/OpenAI/OpenAITelemetryConstants.cs +++ /dev/null @@ -1,17 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Extensions.OpenAI; - -/// -/// Contains constants for operation names, tag names, and activity source names used in OpenAI tracing. -/// -internal static class OpenAITelemetryConstants -{ - // Operation Names - public const string ChatOperation = "chat"; - - // Activity Source Names - public const string OpenAISource = "OpenAI"; - public const string OpenAISourceWildcard = "OpenAI.*"; -} diff --git a/src/Observability/Extensions/OpenAI/README.md b/src/Observability/Extensions/OpenAI/README.md deleted file mode 100644 index 150dabbb..00000000 --- a/src/Observability/Extensions/OpenAI/README.md +++ /dev/null @@ -1,31 +0,0 @@ -# Microsoft.Agents.A365.Observability.Extensions.OpenAI - -OpenAI-specific tracing and instrumentation extensions for the Microsoft Agent 365 Observability SDK. This package provides specialized observability features for OpenAI-based agent applications. - - -## Installation - -```bash -dotnet add package Microsoft.Agents.A365.Observability.Extensions.OpenAI -``` - -## Documentation - -For detailed usage information, configuration examples, and best practices, see the [Microsoft Agents 365 Observability documentation](https://learn.microsoft.com/en-us/microsoft-agent-365/developer/observability?tabs=dotnet). - -## Support - -For issues, questions, or feedback: - -- File issues in the [GitHub Issues](https://github.com/microsoft/Agent365-dotnet/issues) section -- See the [main documentation](../../../../README.md) for more information - -## Trademarks - -*Microsoft, Windows, Microsoft Azure and/or other Microsoft products and services referenced in the documentation may be either trademarks or registered trademarks of Microsoft in the United States and/or other countries. The licenses for this project do not grant you rights to use any Microsoft names, logos, or trademarks. Microsoft's general trademark guidelines can be found at http://go.microsoft.com/fwlink/?LinkID=254653.* - -## License - -Copyright (c) Microsoft Corporation. All rights reserved. - -Licensed under the MIT License - see the [LICENSE](../../../../LICENSE.md) file for details. \ No newline at end of file diff --git a/src/Observability/Extensions/OpenAI/docs/design.md b/src/Observability/Extensions/OpenAI/docs/design.md deleted file mode 100644 index 1537238b..00000000 --- a/src/Observability/Extensions/OpenAI/docs/design.md +++ /dev/null @@ -1,317 +0,0 @@ -# Microsoft.Agents.A365.Observability.Extensions.OpenAI - Design Documentation - -## Overview - -The `Microsoft.Agents.A365.Observability.Extensions.OpenAI` package provides OpenTelemetry tracing integration for the OpenAI SDK. It includes a span processor that intercepts OpenAI API telemetry and extension methods for working with tool calls. - -## Architecture - -``` -Microsoft.Agents.A365.Observability.Extensions.OpenAI -├── Public API -│ ├── BuilderExtensions # Extension method for Builder -│ └── ChatToolCallExtensions # Tool call helper extensions -├── Internal -│ ├── OpenAISpanProcessor # Span processing logic -│ └── OpenAITelemetryConstants # Telemetry constants -``` - -## Key Components - -### BuilderExtensions - -**Source**: [BuilderExtensions.cs](../BuilderExtensions.cs) - -Provides the `WithOpenAI()` extension method for the observability `Builder`. - -```csharp -// Configure observability with OpenAI support -new Builder(services, configuration) - .WithOpenAI() - .Build(); - -// Without related sources (manual configuration) -new Builder(services, configuration) - .WithOpenAI(enableRelatedSources: false) - .Build(); -``` - -**Implementation:** - -```csharp -public static Builder WithOpenAI(this Builder builder, bool enableRelatedSources = true) -{ - if (enableRelatedSources) - { - AppContext.SetSwitch("OpenAI.Experimental.EnableOpenTelemetry", true); - builder.Services.AddOpenTelemetry() - .WithTracing(tracing => tracing - .AddSource(OpenAITelemetryConstants.OpenAISourceWildcard) - .AddProcessor(new OpenAISpanProcessor())); - } - - return builder; -} -``` - -**Parameters:** - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `enableRelatedSources` | `bool` | `true` | Enable OpenTelemetry for OpenAI SDK and add related sources | - -### OpenAISpanProcessor - -**Source**: [OpenAISpanProcessor.cs](../OpenAISpanProcessor.cs) - -A `BaseProcessor` that processes spans from the OpenAI SDK activity source (`OpenAI.*`). - -**Processed Operations:** -- `chat` - Chat completions - -```csharp -internal class OpenAISpanProcessor : BaseProcessor -{ - private static readonly string TargetSourceName = OpenAITelemetryConstants.OpenAISource; - - public override void OnStart(Activity activity) - { - } - - public override void OnEnd(Activity activity) - { - if (activity.Source.Name.StartsWith(TargetSourceName)) - { - var tags = activity.Tags.ToDictionary(kv => kv.Key, kv => kv.Value); - if (tags.TryGetValue(OpenTelemetryConstants.GenAiOperationNameKey, out var operationName)) - { - switch (operationName) - { - case OpenAITelemetryConstants.ChatOperation: - // Span emitted by OpenAI SDK follows Microsoft Agent 365 schema, - // so no modification needed. - // Placeholder for any plumbing if needed in the future. - break; - } - } - } - } -} -``` - -### ChatToolCallExtensions - -**Source**: [ChatToolCallExtensions.cs](../ChatToolCallExtensions.cs) - -Extension methods for working with OpenAI `ChatToolCall` objects in telemetry contexts. - -```csharp -// Extract telemetry data from tool calls -var toolCall = response.ToolCalls.First(); -var telemetryData = toolCall.ToTelemetryData(); - -// Record tool call in current span -Activity.Current?.RecordToolCall(toolCall); -``` - -**Extension Methods:** - -| Method | Description | -|--------|-------------| -| `ToTelemetryData()` | Converts tool call to telemetry-friendly format | -| `RecordToolCall()` | Records tool call details in Activity | -| `GetToolName()` | Extracts tool name from call | -| `GetToolArguments()` | Extracts tool arguments as dictionary | - -### OpenAITelemetryConstants - -**Source**: [OpenAITelemetryConstants.cs](../OpenAITelemetryConstants.cs) - -Constants for OpenAI telemetry integration. - -```csharp -internal static class OpenAITelemetryConstants -{ - // Operation Names - public const string ChatOperation = "chat"; - - // Activity Source Names - public const string OpenAISource = "OpenAI"; - public const string OpenAISourceWildcard = "OpenAI.*"; -} -``` - -## Design Patterns - -### Extension Pattern - -The package extends the core `Builder` and enables OpenTelemetry for the OpenAI SDK: - -```csharp -public static class BuilderExtensions -{ - public static Builder WithOpenAI(this Builder builder, bool enableRelatedSources = true) - { - if (enableRelatedSources) - { - // Enable OpenAI SDK's experimental OpenTelemetry support - AppContext.SetSwitch("OpenAI.Experimental.EnableOpenTelemetry", true); - - builder.Services.AddOpenTelemetry() - .WithTracing(tracing => tracing - .AddSource(OpenAITelemetryConstants.OpenAISourceWildcard) - .AddProcessor(new OpenAISpanProcessor())); - } - - return builder; - } -} -``` - -### Span Processor Pattern - -The processor monitors OpenAI spans and can enrich them if needed: - -```csharp -internal class OpenAISpanProcessor : BaseProcessor -{ - public override void OnEnd(Activity activity) - { - if (activity.Source.Name.StartsWith("OpenAI")) - { - // OpenAI SDK already follows Agent365 schema - // Processor provides extension point for future modifications - } - } -} -``` - -## Data Flow - -``` -┌─────────────────────────────┐ -│ Azure OpenAI SDK │ -│ │ -│ ChatClient.CompleteAsync() │ -│ EmbeddingClient.Generate() │ -└──────────────┬──────────────┘ - │ - ▼ Activity Source Events -┌─────────────────────────────┐ -│ Azure.AI.OpenAI │ -│ │ -│ Creates Activity/Span │ -│ with OpenAI-specific tags │ -└──────────────┬──────────────┘ - │ - ▼ OnEnd callback -┌─────────────────────────────┐ -│ OpenAISpanProcessor │ -│ │ -│ 1. Check if OpenAI source │ -│ 2. Normalize tag names │ -│ 3. Extract token usage │ -│ 4. Process response data │ -└──────────────┬──────────────┘ - │ - ▼ Enriched Activity -┌─────────────────────────────┐ -│ OpenTelemetry Exporter │ -│ │ -│ Spans follow Agent365 │ -│ schema standards │ -└─────────────────────────────┘ -``` - -## Span Attribute Mapping - -| OpenAI SDK Tag | Agent365 Tag | -|----------------|--------------| -| `prompt_tokens` | `gen_ai.usage.input_tokens` | -| `completion_tokens` | `gen_ai.usage.output_tokens` | -| `model` | `gen_ai.request.model` | -| `finish_reason` | `gen_ai.response.finish_reasons` | - -## File Structure - -``` -src/Observability/Extensions/OpenAI/ -├── BuilderExtensions.cs # Builder extension -├── OpenAISpanProcessor.cs # Span processor -├── OpenAITelemetryConstants.cs # Constants -├── ChatToolCallExtensions.cs # Tool call extensions -├── Microsoft.Agents.A365.Observability.Extensions.OpenAI.csproj -└── docs/ - └── design.md # This file -``` - -## Dependencies - -| Package | Purpose | -|---------|---------| -| `Microsoft.Agents.A365.Observability.Runtime` | Core observability | -| `Azure.AI.OpenAI` | Azure OpenAI SDK | - -## Usage Examples - -### Basic Setup - -```csharp -// Program.cs -builder.Services.AddOpenTelemetry() - .WithTracing(tracerBuilder => - { - new Builder(builder.Services, builder.Configuration) - .WithOpenAI() - .Build(); - }); -``` - -### With Azure OpenAI Client - -```csharp -var client = new AzureOpenAIClient( - new Uri(endpoint), - new AzureKeyCredential(apiKey) -); - -var chatClient = client.GetChatClient(deploymentName); - -// Spans are automatically captured and processed -var response = await chatClient.CompleteChatAsync(messages); -``` - -### Tool Call Telemetry - -```csharp -var response = await chatClient.CompleteChatAsync(messages, options); - -foreach (var toolCall in response.Value.ToolCalls) -{ - // Record tool call in telemetry - Activity.Current?.RecordToolCall(toolCall); - - // Execute tool - var result = await ExecuteToolAsync(toolCall); - - // Add result to messages for next completion - messages.Add(new ToolChatMessage(toolCall.Id, result)); -} -``` - -### Combined with Other Extensions - -```csharp -// Use multiple framework extensions together -new Builder(services, configuration) - .WithAgentFramework() - .WithSemanticKernel() - .WithOpenAI() - .Build(); -``` - -## External Resources - -- [Azure OpenAI SDK for .NET](https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/openai/Azure.AI.OpenAI) -- [OpenTelemetry GenAI Conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) -- [Azure OpenAI Service](https://learn.microsoft.com/azure/ai-services/openai/) diff --git a/src/Observability/Extensions/SemanticKernel/BuilderExtensions.cs b/src/Observability/Extensions/SemanticKernel/BuilderExtensions.cs deleted file mode 100644 index 13041d95..00000000 --- a/src/Observability/Extensions/SemanticKernel/BuilderExtensions.cs +++ /dev/null @@ -1,44 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -namespace Microsoft.Agents.A365.Observability.Extensions.SemanticKernel; - -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Agents.A365.Observability.Runtime; -using Microsoft.SemanticKernel; -using OpenTelemetry.Trace; -using OpenTelemetry; - -/// -/// Extension methods for configuring Builder with SemanticKernel integration. -/// -public static class BuilderExtensions -{ - /// - /// Adds SemanticKernel integration to the builder with function invocation filtering. - /// - /// The builder to configure. - /// Whether to enable related tracing sources for OpenTelemetry. - /// The configured builder for method chaining. - public static Builder WithSemanticKernel(this Builder builder, bool enableRelatedSources = true) - { - builder.Services.AddSingleton(); - if (enableRelatedSources) - { - AppContext.SetSwitch("Microsoft.SemanticKernel.Experimental.GenAI.EnableOTelDiagnosticsSensitive", true); - - var telmConfig = builder.Services.AddOpenTelemetry() - .WithTracing(tracing => tracing - .AddSource(SemanticKernelTelemetryConstants.SemanticKernelSourceWildcard) - .AddProcessor(new SemanticKernelSpanProcessor(builder.Configuration))); - - if (builder.Configuration != null - && !string.IsNullOrEmpty(builder.Configuration["EnableOtlpExporter"]) - && bool.TryParse(builder.Configuration["EnableOtlpExporter"], out bool enabled) && enabled) - { - telmConfig.UseOtlpExporter(); - } - } - - return builder; - } -} \ No newline at end of file diff --git a/src/Observability/Extensions/SemanticKernel/ChatCompletionAgentExtensions.cs b/src/Observability/Extensions/SemanticKernel/ChatCompletionAgentExtensions.cs deleted file mode 100644 index 0c55966b..00000000 --- a/src/Observability/Extensions/SemanticKernel/ChatCompletionAgentExtensions.cs +++ /dev/null @@ -1,29 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -namespace Microsoft.Agents.A365.Observability.Extensions.SemanticKernel; - -using Microsoft.SemanticKernel.Agents; - -/// -/// Extension methods for enabling tracing on SemanticKernel agents. -/// -public static class ChatCompletionAgentExtensions -{ - /// - /// Wraps a ChatCompletionAgent with tracing capabilities. - /// - /// The ChatCompletionAgent to wrap. - /// A new TracingChatCompletionAgent that provides automatic tracing. - public static ChatCompletionAgent WithTracing(this ChatCompletionAgent agent) - { - ArgumentNullException.ThrowIfNull(agent); - - var filters = agent.Kernel.FunctionInvocationFilters; - if (!filters.OfType().Any()) - { - filters.Add(new FunctionInvocationFilter()); - } - - return agent; - } -} \ No newline at end of file diff --git a/src/Observability/Extensions/SemanticKernel/FunctionInvocationFilter.cs b/src/Observability/Extensions/SemanticKernel/FunctionInvocationFilter.cs deleted file mode 100644 index f438e64b..00000000 --- a/src/Observability/Extensions/SemanticKernel/FunctionInvocationFilter.cs +++ /dev/null @@ -1,57 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -namespace Microsoft.Agents.A365.Observability.Extensions.SemanticKernel; - -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.SemanticKernel; -using System.Diagnostics; -using System.Text.Json; -using System.Text.Json.Serialization; - -/// -/// Function invocation filter that adds tracing capabilities to SemanticKernel function calls. -/// -public sealed class FunctionInvocationFilter : IFunctionInvocationFilter -{ - private static readonly JsonSerializerOptions SerializerOptions = new() - { - Converters = { new JsonStringEnumConverter(JsonNamingPolicy.CamelCase) } - }; - - /// - public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func next) - { - var arguments = JsonSerializer.Serialize(context.Arguments, SerializerOptions); - - if (Activity.Current?.OperationName.StartsWith(ExecuteToolScope.OperationName) ?? false) - { - // If we are already in a tool execution scope, we do not need to create a new one - Activity.Current.AddTag(OpenTelemetryConstants.GenAiToolArgumentsKey, arguments); - Activity.Current.AddTag(OpenTelemetryConstants.GenAiToolTypeKey, ToolType.Function); - await InvokeWithErrorHandlingAsync(next, context); - Activity.Current.AddTag(OpenTelemetryConstants.GenAiToolCallResultKey, GetResult(context)); - Activity.Current.AddTag(OpenTelemetryConstants.GenAiToolCallIdKey, context.Function.PluginName); - return; - } - } - - private async Task InvokeWithErrorHandlingAsync(Func next, FunctionInvocationContext context) - { - try - { - await next(context); - } - catch (Exception ex) - { - Activity.Current?.AddTag(OpenTelemetryConstants.ErrorTypeKey, ex.GetType().Name); - Activity.Current?.AddTag(OpenTelemetryConstants.ErrorMessageKey, ex.Message); - throw; - } - } - - private static string GetResult(FunctionInvocationContext context) - { - return JsonSerializer.Serialize(context.Result.GetValue(), SerializerOptions); - } -} \ No newline at end of file diff --git a/src/Observability/Extensions/SemanticKernel/Microsoft.Agents.A365.Observability.Extensions.SemanticKernel.csproj b/src/Observability/Extensions/SemanticKernel/Microsoft.Agents.A365.Observability.Extensions.SemanticKernel.csproj deleted file mode 100644 index 6a47340c..00000000 --- a/src/Observability/Extensions/SemanticKernel/Microsoft.Agents.A365.Observability.Extensions.SemanticKernel.csproj +++ /dev/null @@ -1,29 +0,0 @@ - - - Microsoft.Agents.A365.Observability.Extensions.SemanticKernel - Microsoft.Agents.A365.Observability.Extensions.SemanticKernel - net8.0 - - Microsoft.Agents.A365.Observability.Extensions.SemanticKernel - Microsoft Agent 365 Semantic Kernel Observability SDK - Semantic Kernel observability integration for Microsoft Agent 365 SDK, providing automatic tracing for ChatCompletionAgent operations. - Microsoft Agent 365 Semantic Kernel Observability Integration - Automatic tracing for Semantic Kernel agents - Microsoft Agent 365 Semantic Kernel Observability SDK - Microsoft;A365Observability;SemanticKernel;OpenTelemetry;AI;Agents;Tracing;Monitoring - - $(NoWarn);NU5104 - - - - - - - - - - - - - - - diff --git a/src/Observability/Extensions/SemanticKernel/Models/AiChoice.cs b/src/Observability/Extensions/SemanticKernel/Models/AiChoice.cs deleted file mode 100644 index 57a6df5e..00000000 --- a/src/Observability/Extensions/SemanticKernel/Models/AiChoice.cs +++ /dev/null @@ -1,78 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Extensions.SemanticKernel.Models; - -using System.Text.Json.Serialization; - -/// -/// Represents an AI choice containing a message with role and tool calls information. -/// -public sealed class AiChoice -{ - /// - /// Gets or sets the AI choice message, which includes role and tool calls information. - /// - [JsonPropertyName("message")] - public AiChoiceMessage? Message { get; set; } -} - -/// -/// Represents an AI choice message, including the role and associated tool calls. -/// -public sealed class AiChoiceMessage -{ - /// - /// Gets or sets the role of the AI message. - /// - [JsonPropertyName("role")] - public string? Role { get; set; } - - /// - /// Gets or sets the direct content of the AI message. - /// - [JsonPropertyName("content")] - public string? Content { get; set; } - - /// - /// Gets or sets the collection of tool calls associated with the AI message. - /// - [JsonPropertyName("tool_calls")] - public List? ToolCalls { get; set; } -} - -/// -/// Represents a tool call within an AI choice message. -/// -public sealed class AiChoiceToolCall -{ - /// - /// Gets or sets the function associated with the tool call. - /// - [JsonPropertyName("function")] - public AiChoiceFunction? Function { get; set; } -} - -/// -/// Represents a function within a tool call. -/// -public sealed class AiChoiceFunction -{ - /// - /// Gets or sets the arguments for the function. - /// - [JsonPropertyName("arguments")] - public AiChoiceArguments? Arguments { get; set; } -} - -/// -/// Represents the arguments for an AI choice function. -/// -public sealed class AiChoiceArguments -{ - /// - /// Gets or sets the message body for the function arguments. - /// - [JsonPropertyName("messageBody")] - public string? MessageBody { get; set; } -} \ No newline at end of file diff --git a/src/Observability/Extensions/SemanticKernel/Models/MessageContent.cs b/src/Observability/Extensions/SemanticKernel/Models/MessageContent.cs deleted file mode 100644 index be4cbe9f..00000000 --- a/src/Observability/Extensions/SemanticKernel/Models/MessageContent.cs +++ /dev/null @@ -1,31 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System.Text.Json.Serialization; - -namespace Microsoft.Agents.A365.Observability.Extensions.SemanticKernel.Models; - -/// -/// Represents the structure of a message as found in OpenTelemetry activity events and invocation input tags. -/// Used in the gen_ai.agent.invocation_input tag and gen_ai.event.content attribute. -/// -public class MessageContent -{ - /// - /// The role of the message, such as "user" or "assistant". - /// - [JsonPropertyName("role")] - public string? Role { get; set; } - - /// - /// The content of the event. - /// - [JsonPropertyName("content")] - public string? Content { get; set; } - - /// - /// The name associated with the message. - /// - [JsonPropertyName("name")] - public string? Name { get; set; } -} \ No newline at end of file diff --git a/src/Observability/Extensions/SemanticKernel/Models/NestedContent.cs b/src/Observability/Extensions/SemanticKernel/Models/NestedContent.cs deleted file mode 100644 index a2881f22..00000000 --- a/src/Observability/Extensions/SemanticKernel/Models/NestedContent.cs +++ /dev/null @@ -1,25 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System.Text.Json.Serialization; - -namespace Microsoft.Agents.A365.Observability.Extensions.SemanticKernel.Models; - -/// -/// Represents a nested content structure that may appear within a MessageContent's Content property. -/// Used when Content is a JSON object with contentType and content fields. -/// -public class NestedContent -{ - /// - /// The type of the content (e.g., "Text"). - /// - [JsonPropertyName("contentType")] - public string? ContentType { get; set; } - - /// - /// The actual content value. - /// - [JsonPropertyName("content")] - public string? Content { get; set; } -} diff --git a/src/Observability/Extensions/SemanticKernel/README.md b/src/Observability/Extensions/SemanticKernel/README.md deleted file mode 100644 index 09fb0694..00000000 --- a/src/Observability/Extensions/SemanticKernel/README.md +++ /dev/null @@ -1,30 +0,0 @@ -# Microsoft.Agents.A365.Observability.Extensions.SemanticKernel - -Semantic Kernel integration extensions for the Microsoft Agent 365 Observability SDK. This package provides specialized observability features for Semantic Kernel-based agent applications. - -## Installation - -```bash -dotnet add package Microsoft.Agents.A365.Observability.Extensions.SemanticKernel -``` - -## Documentation - -For detailed usage information, configuration examples, and best practices, see the [Microsoft Agents 365 Observability documentation](https://learn.microsoft.com/en-us/microsoft-agent-365/developer/observability?tabs=dotnet). - -## Support - -For issues, questions, or feedback: - -- File issues in the [GitHub Issues](https://github.com/microsoft/Agent365-dotnet/issues) section -- See the [main documentation](../../../../README.md) for more information - -## Trademarks - -*Microsoft, Windows, Microsoft Azure and/or other Microsoft products and services referenced in the documentation may be either trademarks or registered trademarks of Microsoft in the United States and/or other countries. The licenses for this project do not grant you rights to use any Microsoft names, logos, or trademarks. Microsoft's general trademark guidelines can be found at http://go.microsoft.com/fwlink/?LinkID=254653.* - -## License - -Copyright (c) Microsoft Corporation. All rights reserved. - -Licensed under the MIT License - see the [LICENSE](../../../../LICENSE.md) file for details. \ No newline at end of file diff --git a/src/Observability/Extensions/SemanticKernel/SemanticKernelSpanProcessor.cs b/src/Observability/Extensions/SemanticKernel/SemanticKernelSpanProcessor.cs deleted file mode 100644 index 9533c922..00000000 --- a/src/Observability/Extensions/SemanticKernel/SemanticKernelSpanProcessor.cs +++ /dev/null @@ -1,74 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Extensions.SemanticKernel; - -using Microsoft.Agents.A365.Observability.Extensions.SemanticKernel.Utils; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Processors; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Extensions.Configuration; -using OpenTelemetry; -using System.Diagnostics; - -internal class SemanticKernelSpanProcessor : BaseProcessor -{ - private static readonly string TargetSourceName = SemanticKernelTelemetryConstants.SemanticKernelSource; - private readonly bool _suppressInvokeAgentInput; - - /// - /// Initializes a new instance of the class. - /// - /// The configuration instance for accessing settings. - public SemanticKernelSpanProcessor(IConfiguration? configuration = null) - { - this._suppressInvokeAgentInput = configuration != null && bool.TryParse(configuration[SemanticKernelTelemetryConstants.SuppressInvokeAgentInputConfigKey], out var suppress) && suppress; - } - - public override void OnStart(Activity activity) - { - } - - public override void OnEnd(Activity activity) - { - if (activity.Source.Name.StartsWith(TargetSourceName)) - { - var operationName = activity.GetTagItem(OpenTelemetryConstants.GenAiOperationNameKey) as string; - if (operationName != null) - { - switch (operationName) - { - case SemanticKernelTelemetryConstants.InvokeAgentOperation: - SemanticKernelSpanProcessorHelper.ProcessInvocationInputOutputTag(activity: activity, suppressInvocationInput: this._suppressInvokeAgentInput); - break; - - case SemanticKernelTelemetryConstants.ExecuteToolOperation: - // Span emitted by SK SDK follows Microsoft Agent 365 schema, so no modification needed. - // FunctionInvocationFilter already adds other relevant tags. - // Placeholder for any plumbing if needed in the future. - break; - - case SemanticKernelTelemetryConstants.ChatCompletionsOperation: - case SemanticKernelTelemetryConstants.ChatOperation: - activity.SetTag(OpenTelemetryConstants.GenAiOperationNameKey, InferenceOperationType.Chat.ToString()); - activity.DisplayName = activity.DisplayName - .Replace(SemanticKernelTelemetryConstants.ChatCompletionsOperation, InferenceOperationType.Chat.ToString()) - .Replace(SemanticKernelTelemetryConstants.ChatOperation, InferenceOperationType.Chat.ToString()); - - var inputMessages = SemanticKernelMessageMapper.MapInputMessages(activity); - if (inputMessages != null) - { - activity.SetTag(OpenTelemetryConstants.GenAiInputMessagesKey, inputMessages); - } - - var outputMessages = SemanticKernelMessageMapper.MapOutputMessages(activity); - if (outputMessages != null) - { - activity.SetTag(OpenTelemetryConstants.GenAiOutputMessagesKey, outputMessages); - } - break; - } - } - } - } -} diff --git a/src/Observability/Extensions/SemanticKernel/SemanticKernelTelemetryConstants.cs b/src/Observability/Extensions/SemanticKernel/SemanticKernelTelemetryConstants.cs deleted file mode 100644 index 118c48cd..00000000 --- a/src/Observability/Extensions/SemanticKernel/SemanticKernelTelemetryConstants.cs +++ /dev/null @@ -1,27 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Extensions.SemanticKernel; - -/// -/// Contains constants for operation names, tag names, and activity source names used in SemanticKernel and OpenAI tracing. -/// -internal static class SemanticKernelTelemetryConstants -{ - // Operation Names - public const string InvokeAgentOperation = "invoke_agent"; - public const string ExecuteToolOperation = "execute_tool"; - public const string ChatCompletionsOperation = "chat.completions"; - public const string ChatOperation = "chat"; - - // Activity Source Names - public const string SemanticKernelSource = "Microsoft.SemanticKernel"; - public const string SemanticKernelSourceWildcard = "Microsoft.SemanticKernel*"; - public const string AzureAISourceWildcard = "Azure.AI.*"; - - // Configuration Keys - public const string SuppressInvokeAgentInputConfigKey = "SuppressInvokeAgentInput"; - - // Event Tag Keys - public const string EventContentTag = "gen_ai.event.content"; -} diff --git a/src/Observability/Extensions/SemanticKernel/Utils/SemanticKernelMessageMapper.cs b/src/Observability/Extensions/SemanticKernel/Utils/SemanticKernelMessageMapper.cs deleted file mode 100644 index 696d5b84..00000000 --- a/src/Observability/Extensions/SemanticKernel/Utils/SemanticKernelMessageMapper.cs +++ /dev/null @@ -1,305 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Extensions.SemanticKernel.Utils; - -using Microsoft.Agents.A365.Observability.Runtime.Tracing; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts.Messages; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using System.Collections.Generic; -using System.Diagnostics; -using System.Text.Json; - -/// -/// Maps Semantic Kernel span events to A365 versioned message format. -/// SK emits gen_ai.* events with gen_ai.event.content JSON payloads; -/// this mapper converts them to / . -/// -internal static class SemanticKernelMessageMapper -{ - /// - /// Maps all input-related span events to a serialized A365 JSON string. - /// Input events: gen_ai.system.message, gen_ai.user.message, gen_ai.assistant.message, gen_ai.tool.message. - /// - public static string? MapInputMessages(Activity activity) - { - if (activity.Events == null) - return null; - - var chatMessages = new List(); - - foreach (var ev in activity.Events) - { - var content = GetEventContentTag(ev); - if (string.IsNullOrEmpty(content)) - continue; - - switch (ev.Name) - { - case "gen_ai.system.message": - case "gen_ai.tool.developer": - MapSystemMessage(content, chatMessages); - break; - - case "gen_ai.user.message": - MapUserMessage(content, chatMessages); - break; - - case "gen_ai.assistant.message": - MapAssistantMessage(content, chatMessages); - break; - - case "gen_ai.tool.message": - MapToolMessage(content, chatMessages); - break; - } - } - - if (chatMessages.Count == 0) - return null; - - return MessageUtils.Serialize(new InputMessages(chatMessages)); - } - - /// - /// Maps gen_ai.choice span events to a serialized A365 JSON string. - /// - public static string? MapOutputMessages(Activity activity) - { - if (activity.Events == null) - return null; - - var outputMessages = new List(); - - foreach (var ev in activity.Events) - { - if (ev.Name != OpenTelemetryConstants.GenAiChoiceEventName) - continue; - - var content = GetEventContentTag(ev); - if (string.IsNullOrEmpty(content)) - continue; - - MapChoiceMessage(content, outputMessages); - } - - if (outputMessages.Count == 0) - return null; - - return MessageUtils.Serialize(new OutputMessages(outputMessages)); - } - - private static void MapSystemMessage(string json, List messages) - { - try - { - using var doc = JsonDocument.Parse(json); - var root = doc.RootElement; - var textContent = GetStringProperty(root, "content"); - if (string.IsNullOrEmpty(textContent)) - return; - - messages.Add(new ChatMessage( - MessageRole.System, - new IMessagePart[] { new TextPart(textContent) }, - GetStringProperty(root, "name"))); - } - catch (JsonException) { } - } - - private static void MapUserMessage(string json, List messages) - { - try - { - using var doc = JsonDocument.Parse(json); - var root = doc.RootElement; - var textContent = GetStringProperty(root, "content"); - if (string.IsNullOrEmpty(textContent)) - return; - - messages.Add(new ChatMessage( - MessageRole.User, - new IMessagePart[] { new TextPart(textContent) }, - GetStringProperty(root, "name"))); - } - catch (JsonException) { } - } - - /// - /// Maps an assistant message with optional tool calls. - /// - private static void MapAssistantMessage(string json, List messages) - { - try - { - using var doc = JsonDocument.Parse(json); - var root = doc.RootElement; - var parts = new List(); - - var textContent = GetStringProperty(root, "content"); - if (!string.IsNullOrEmpty(textContent)) - { - parts.Add(new TextPart(textContent)); - } - - if (root.TryGetProperty("tool_calls", out var toolCallsElement) - && toolCallsElement.ValueKind == JsonValueKind.Array) - { - foreach (var tc in toolCallsElement.EnumerateArray()) - { - var toolCallId = GetStringProperty(tc, "id"); - string? functionName = null; - object? arguments = null; - - if (tc.TryGetProperty("function", out var funcElement)) - { - functionName = GetStringProperty(funcElement, "name"); - if (funcElement.TryGetProperty("arguments", out var argsElement)) - { - arguments = argsElement.ValueKind == JsonValueKind.String - ? argsElement.GetString() - : argsElement.ToString(); - } - } - - if (!string.IsNullOrEmpty(functionName)) - { - parts.Add(new ToolCallRequestPart(functionName, toolCallId, arguments)); - } - } - } - - if (parts.Count > 0) - { - messages.Add(new ChatMessage( - MessageRole.Assistant, - parts, - GetStringProperty(root, "name"))); - } - } - catch (JsonException) { } - } - - private static void MapToolMessage(string json, List messages) - { - try - { - using var doc = JsonDocument.Parse(json); - var root = doc.RootElement; - var textContent = GetStringProperty(root, "content"); - if (string.IsNullOrEmpty(textContent)) - return; - - // Only set ID if explicitly present in the tool message JSON - var id = GetStringProperty(root, "id") ?? GetStringProperty(root, "tool_call_id"); - - messages.Add(new ChatMessage( - MessageRole.Tool, - new IMessagePart[] { new ToolCallResponsePart(id, textContent) })); - } - catch (JsonException) { } - } - - private static void MapChoiceMessage(string json, List messages) - { - try - { - using var doc = JsonDocument.Parse(json); - var root = doc.RootElement; - - if (!root.TryGetProperty("message", out var msgElement)) - return; - - var parts = new List(); - - var textContent = GetStringProperty(msgElement, "content"); - if (!string.IsNullOrEmpty(textContent)) - { - parts.Add(new TextPart(textContent)); - } - - if (msgElement.TryGetProperty("tool_calls", out var toolCallsElement) - && toolCallsElement.ValueKind == JsonValueKind.Array) - { - foreach (var tc in toolCallsElement.EnumerateArray()) - { - var toolCallId = GetStringProperty(tc, "id"); - string? functionName = null; - object? arguments = null; - - if (tc.TryGetProperty("function", out var funcElement)) - { - functionName = GetStringProperty(funcElement, "name"); - if (funcElement.TryGetProperty("arguments", out var argsElement)) - { - arguments = argsElement.ValueKind == JsonValueKind.String - ? argsElement.GetString() - : argsElement.ToString(); - } - } - - if (!string.IsNullOrEmpty(functionName)) - { - parts.Add(new ToolCallRequestPart(functionName, toolCallId, arguments)); - } - } - } - - if (parts.Count == 0) - return; - - var role = MapRole(GetStringProperty(msgElement, "role"), MessageRole.Assistant); - var finishReason = MapFinishReason(GetStringProperty(root, "finish_reason")); - - messages.Add(new OutputMessage(role, parts, finishReason: finishReason)); - } - catch (JsonException) { } - } - - internal static MessageRole MapRole(string? role, MessageRole defaultRole) - { - if (string.IsNullOrEmpty(role)) - return defaultRole; - - if (role.Equals("system", System.StringComparison.OrdinalIgnoreCase)) return MessageRole.System; - if (role.Equals("user", System.StringComparison.OrdinalIgnoreCase)) return MessageRole.User; - if (role.Equals("assistant", System.StringComparison.OrdinalIgnoreCase)) return MessageRole.Assistant; - if (role.Equals("tool", System.StringComparison.OrdinalIgnoreCase)) return MessageRole.Tool; - - return defaultRole; - } - - private static string? MapFinishReason(string? skFinishReason) - { - if (string.IsNullOrEmpty(skFinishReason)) - return null; - - if (skFinishReason.Equals("Stop", System.StringComparison.OrdinalIgnoreCase)) return "stop"; - if (skFinishReason.Equals("Length", System.StringComparison.OrdinalIgnoreCase)) return "length"; - if (skFinishReason.Equals("ContentFilter", System.StringComparison.OrdinalIgnoreCase)) return "content_filter"; - if (skFinishReason.Equals("ToolCalls", System.StringComparison.OrdinalIgnoreCase)) return "tool_calls"; - - return skFinishReason.ToLowerInvariant(); - } - - private static string? GetEventContentTag(ActivityEvent activityEvent) - { - if (activityEvent.Tags == null) - return null; - - foreach (var tag in activityEvent.Tags) - { - if (tag.Key == SemanticKernelTelemetryConstants.EventContentTag) - return tag.Value as string; - } - - return null; - } - - private static string? GetStringProperty(JsonElement element, string propertyName) - { - if (element.TryGetProperty(propertyName, out var prop) && prop.ValueKind == JsonValueKind.String) - return prop.GetString(); - return null; - } -} diff --git a/src/Observability/Extensions/SemanticKernel/Utils/SemanticKernelSpanProcessorHelper.cs b/src/Observability/Extensions/SemanticKernel/Utils/SemanticKernelSpanProcessorHelper.cs deleted file mode 100644 index 046be0d8..00000000 --- a/src/Observability/Extensions/SemanticKernel/Utils/SemanticKernelSpanProcessorHelper.cs +++ /dev/null @@ -1,285 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Extensions.SemanticKernel.Utils; - -using Microsoft.Agents.A365.Observability.Extensions.SemanticKernel.Models; -using Microsoft.Agents.A365.Observability.Runtime.Tracing; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts.Messages; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using System.Diagnostics; -using System.Linq; -using System.Text.Json; -using System.Text.RegularExpressions; - -/// -/// Provides helper methods for processing and filtering Semantic Kernel span tags and events. -/// -public static class SemanticKernelSpanProcessorHelper -{ - private static readonly Regex UnquotedPropertyValueRegex = - new Regex( - @"(""[a-zA-Z0-9_]+"":\s*)([^""\s][^,}\s]*)", - RegexOptions.Compiled); - - private static readonly JsonSerializerOptions JsonOptions = new() - { - PropertyNamingPolicy = JsonNamingPolicy.CamelCase, - WriteIndented = false - }; - - /// - /// Processes and filters the gen_ai.agent.invocation_input and gen_ai.agent.invocation_output tags to remove system role messages. - /// - /// The activity containing the tags to process. - /// Whether to suppress the invocation input messages. - public static void ProcessInvocationInputOutputTag(Activity activity, bool suppressInvocationInput = false) - { - if (suppressInvocationInput) - { - RemoveTagIfExists(activity, OpenTelemetryConstants.GenAiInputMessagesKey); - RemoveTagIfExists(activity, OpenTelemetryConstants.GenAiAgentInvocationInputKey); - } - else - { - MapToStructuredFormat(activity, OpenTelemetryConstants.GenAiAgentInvocationInputKey, OpenTelemetryConstants.GenAiInputMessagesKey, isOutput: false); - MapToStructuredFormat(activity, OpenTelemetryConstants.GenAiInputMessagesKey, OpenTelemetryConstants.GenAiInputMessagesKey, isOutput: false); - } - - MapToStructuredFormat(activity, OpenTelemetryConstants.GenAiAgentInvocationOutputKey, OpenTelemetryConstants.GenAiOutputMessagesKey, isOutput: true); - MapToStructuredFormat(activity, OpenTelemetryConstants.GenAiOutputMessagesKey, OpenTelemetryConstants.GenAiOutputMessagesKey, isOutput: true); - } - - /// - /// Gets the value of a tag from the activity by key. - /// - /// The activity containing the tag. - /// The key of the tag to retrieve. - /// The tag value as a string, or null if not found. - private static string? GetTagValue(Activity activity, string key) - { - return activity.TagObjects - .OfType>() - .FirstOrDefault(k => k.Key == key).Value as string; - } - - /// - /// Removes a tag from the activity if it exists. - /// - /// The activity containing the tag to remove. - /// The key of the tag to remove. - private static void RemoveTagIfExists(Activity activity, string key) - { - if (GetTagValue(activity, key) != null) - { - activity.SetTag(key, null); - } - } - - /// - /// Quotes unquoted property values in the JSON string. - /// - /// The JSON string to process. - /// The JSON string with quoted property values. - private static string QuoteUnquotedPropertyValues(string json) - { - var quoted = UnquotedPropertyValueRegex.Replace(json, "$1\"$2\""); - - if (quoted.Length > 2 && - quoted.StartsWith("\"", StringComparison.Ordinal) && - quoted.EndsWith("\"", StringComparison.Ordinal)) - { - try - { - var unescaped = JsonSerializer.Deserialize(quoted); - if (!string.IsNullOrEmpty(unescaped)) - { - quoted = UnquotedPropertyValueRegex.Replace(unescaped, "$1\"$2\""); - } - } - catch (JsonException) - { - // If not a valid double-encoded string, continue with quoted - } - } - - return quoted; - } - - /// - /// Maps invocation message tags to the A365 structured message format. - /// Reads from , writes structured result to , - /// and removes the source tag when they differ. - /// - private static void MapToStructuredFormat(Activity activity, string sourceTagName, string targetTagName, bool isOutput) - { - var jsonString = GetTagValue(activity, sourceTagName); - if (string.IsNullOrEmpty(jsonString)) - return; - - try - { - List? messageArray = null; - - try - { - messageArray = JsonSerializer.Deserialize>(jsonString, JsonOptions); - } - catch (JsonException) - { - var strList = JsonSerializer.Deserialize>(jsonString, JsonOptions); - if (strList != null) - { - messageArray = new List(); - foreach (var s in strList) - { - if (s == null) continue; - var mc = TryDeserializeMessageContent(s); - messageArray.Add(mc ?? new MessageContent - { - Role = isOutput ? "assistant" : "user", - Content = s - }); - } - } - } - - if (messageArray == null || messageArray.Count == 0) - return; - - string? serialized = null; - - if (isOutput) - { - var outputMessages = new List(); - foreach (var msg in messageArray) - { - FilterMessageContent(msg); - if (string.IsNullOrEmpty(msg.Content)) continue; - var role = SemanticKernelMessageMapper.MapRole(msg.Role, MessageRole.Assistant); - outputMessages.Add(new OutputMessage(role, new IMessagePart[] { new TextPart(msg.Content) })); - } - - if (outputMessages.Count > 0) - { - serialized = MessageUtils.Serialize(new OutputMessages(outputMessages)); - } - } - else - { - var chatMessages = new List(); - foreach (var msg in messageArray) - { - FilterMessageContent(msg); - if (string.IsNullOrEmpty(msg.Content)) continue; - var role = SemanticKernelMessageMapper.MapRole(msg.Role, MessageRole.User); - chatMessages.Add(new ChatMessage(role, new IMessagePart[] { new TextPart(msg.Content) }, msg.Name)); - } - - if (chatMessages.Count > 0) - { - serialized = MessageUtils.Serialize(new InputMessages(chatMessages)); - } - } - - if (serialized != null) - { - activity.SetTag(targetTagName, serialized); - } - - if (sourceTagName != targetTagName) - { - activity.SetTag(sourceTagName, null); - } - } - catch (JsonException) - { - // Leave original tag value - } - } - - /// - /// Attempts to deserialize a string into a MessageContent object. - /// - /// The string to deserialize. - /// The deserialized MessageContent object, or null if deserialization fails. - private static MessageContent? TryDeserializeMessageContent(string s) - { - try - { - return JsonSerializer.Deserialize(s, JsonOptions); - } - catch (JsonException) - { - var fixedString = QuoteUnquotedPropertyValues(s); - try - { - return JsonSerializer.Deserialize(fixedString, JsonOptions); - } - catch (JsonException) - { - return null; - } - } - } - - /// - /// Filters and extracts the message content from messages. - /// For user messages, trims the prefix up to and including 'Message:'. - /// For all messages, if Content is a JSON object with a nested 'content' property, extracts the inner content. - /// - /// The MessageContent to filter. - private static void FilterMessageContent(MessageContent? message) - { - if (message == null || string.IsNullOrEmpty(message.Content)) - { - return; - } - - // Try to extract nested content if Content is a JSON object with a 'content' property - TryExtractNestedContent(message); - - // For user messages, trim the "Message:" prefix - if (string.Equals(message.Role, "user", StringComparison.OrdinalIgnoreCase)) - { - var idx = message.Content.IndexOf("Message:", StringComparison.OrdinalIgnoreCase); - if (idx >= 0) - { - message.Content = message.Content[(idx + "Message:".Length)..].Trim(); - } - } - } - - /// - /// Attempts to extract nested content from a message's Content property. - /// If Content is a JSON object with a 'content' property, replaces Content with the inner content value. - /// - /// The MessageContent to process. - private static void TryExtractNestedContent(MessageContent message) - { - if (string.IsNullOrEmpty(message.Content)) - { - return; - } - - var content = message.Content.Trim(); - if (!content.StartsWith("{", StringComparison.Ordinal) || !content.EndsWith("}", StringComparison.Ordinal)) - { - return; - } - - try - { - var nestedContent = JsonSerializer.Deserialize(content, JsonOptions); - if (nestedContent?.Content != null) - { - message.Content = nestedContent.Content; - } - } - catch (JsonException) - { - // Content is not a valid nested content JSON, keep the original value - } - } - -} \ No newline at end of file diff --git a/src/Observability/Extensions/SemanticKernel/docs/design.md b/src/Observability/Extensions/SemanticKernel/docs/design.md deleted file mode 100644 index 389119da..00000000 --- a/src/Observability/Extensions/SemanticKernel/docs/design.md +++ /dev/null @@ -1,465 +0,0 @@ -# Microsoft.Agents.A365.Observability.Extensions.SemanticKernel - Design Documentation - -## Overview - -The `Microsoft.Agents.A365.Observability.Extensions.SemanticKernel` package provides OpenTelemetry tracing integration for Microsoft Semantic Kernel. It includes a span processor that intercepts and transforms Semantic Kernel telemetry to align with Agent365 observability standards, plus a function invocation filter for enhanced tool tracing. - -## Architecture - -``` -Microsoft.Agents.A365.Observability.Extensions.SemanticKernel -├── Public API -│ ├── BuilderExtensions # Extension method for Builder -│ ├── ChatCompletionAgentExtensions # Agent helper extensions -│ └── FunctionInvocationFilter # SK function filter -├── Internal -│ ├── SemanticKernelSpanProcessor # Span processing logic -│ ├── SemanticKernelTelemetryConstants # Telemetry constants -│ └── Utils/ -│ └── SemanticKernelSpanProcessorHelper # Helper methods -└── Models/ - ├── MessageContent # Chat message model - ├── AiChoice # AI response choice - └── NestedContent # Nested content model -``` - -## Key Components - -### BuilderExtensions - -**Source**: [BuilderExtensions.cs](../BuilderExtensions.cs) - -Provides the `WithSemanticKernel()` extension method for the observability `Builder`. - -```csharp -// Configure observability with Semantic Kernel support -new Builder(services, configuration) - .WithSemanticKernel() - .Build(); - -// Without related sources (manual configuration) -new Builder(services, configuration) - .WithSemanticKernel(enableRelatedSources: false) - .Build(); -``` - -**Implementation:** - -```csharp -public static Builder WithSemanticKernel(this Builder builder, bool enableRelatedSources = true) -{ - builder.Services.AddSingleton(); - - if (enableRelatedSources) - { - AppContext.SetSwitch("Microsoft.SemanticKernel.Experimental.GenAI.EnableOTelDiagnosticsSensitive", true); - - var telmConfig = builder.Services.AddOpenTelemetry() - .WithTracing(tracing => tracing - .AddSource(SemanticKernelTelemetryConstants.SemanticKernelSourceWildcard) - .AddProcessor(new SemanticKernelSpanProcessor(builder.Configuration))); - - if (builder.Configuration != null - && !string.IsNullOrEmpty(builder.Configuration["EnableOtlpExporter"]) - && bool.TryParse(builder.Configuration["EnableOtlpExporter"], out bool enabled) && enabled) - { - telmConfig.UseOtlpExporter(); - } - } - - return builder; -} -``` - -**Parameters:** - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `enableRelatedSources` | `bool` | `true` | Enable OpenTelemetry for Semantic Kernel and add related sources | - -**Configuration:** - -| Key | Description | -|-----|-------------| -| `EnableOtlpExporter` | Set to `true` to enable OTLP exporter | - -### SemanticKernelSpanProcessor - -**Source**: [SemanticKernelSpanProcessor.cs](../SemanticKernelSpanProcessor.cs) - -A `BaseProcessor` that processes spans from the Semantic Kernel activity source (`Microsoft.SemanticKernel`). - -**Processed Operations:** - -| Operation | Processing | -|-----------|------------| -| `invoke_agent` | Processes input/output tags, optionally suppresses input | -| `execute_tool` | No modification (SK follows Agent365 schema) | -| `chat.completions` | Transforms to `chat`, extracts user/choice messages | - -```csharp -internal class SemanticKernelSpanProcessor : BaseProcessor -{ - private readonly bool _suppressInvokeAgentInput; - - public SemanticKernelSpanProcessor(IConfiguration? configuration = null) - { - _suppressInvokeAgentInput = configuration != null - && bool.TryParse(configuration[SuppressInvokeAgentInputConfigKey], out var suppress) - && suppress; - } - - public override void OnEnd(Activity activity) - { - if (activity.Source.Name.StartsWith(SemanticKernelSource)) - { - var operationName = activity.GetTagItem(GenAiOperationNameKey); - switch (operationName) - { - case "invoke_agent": - ProcessInvocationInputOutputTag(activity, _suppressInvokeAgentInput); - break; - - case "chat.completions": - // Transform to standard "chat" operation - activity.SetTag(GenAiOperationNameKey, "Chat"); - activity.DisplayName = activity.DisplayName.Replace("chat.completions", "Chat"); - - // Extract user and choice messages from events - var messages = GetGenAiUserAndChoiceMessageContent(activity); - // Set gen_ai.input_messages and gen_ai.output_messages - break; - } - } - } -} -``` - -### FunctionInvocationFilter - -**Source**: [FunctionInvocationFilter.cs](../FunctionInvocationFilter.cs) - -An `IFunctionInvocationFilter` that adds telemetry tags to function/tool invocation spans. - -```csharp -public class FunctionInvocationFilter : IFunctionInvocationFilter -{ - public async Task OnFunctionInvocationAsync( - FunctionInvocationContext context, - Func next) - { - // Add telemetry tags before invocation - Activity.Current?.SetTag("gen_ai.tool.name", context.Function.Name); - Activity.Current?.SetTag("gen_ai.tool.plugin", context.Function.PluginName); - - await next(context); - - // Add result tags after invocation - Activity.Current?.SetTag("gen_ai.tool.result", context.Result?.ToString()); - } -} - -// Register with Semantic Kernel -kernel.FunctionInvocationFilters.Add(new FunctionInvocationFilter()); -``` - -### ChatCompletionAgentExtensions - -**Source**: [ChatCompletionAgentExtensions.cs](../ChatCompletionAgentExtensions.cs) - -Helper extensions for working with Semantic Kernel's `ChatCompletionAgent`. - -```csharp -// Create agent with observability -var agent = kernel.CreateChatCompletionAgent("MyAgent") - .WithObservability(); -``` - -### SemanticKernelTelemetryConstants - -**Source**: [SemanticKernelTelemetryConstants.cs](../SemanticKernelTelemetryConstants.cs) - -Constants for Semantic Kernel telemetry integration. - -```csharp -internal static class SemanticKernelTelemetryConstants -{ - // Operation Names - public const string InvokeAgentOperation = "invoke_agent"; - public const string ExecuteToolOperation = "execute_tool"; - public const string ChatCompletionsOperation = "chat.completions"; - - // Activity Source Names - public const string SemanticKernelSource = "Microsoft.SemanticKernel"; - public const string SemanticKernelSourceWildcard = "Microsoft.SemanticKernel*"; - public const string AzureAISourceWildcard = "Azure.AI.*"; - - // Configuration Keys - public const string SuppressInvokeAgentInputConfigKey = "SuppressInvokeAgentInput"; -} -``` - -### SemanticKernelSpanProcessorHelper - -**Source**: [SemanticKernelSpanProcessorHelper.cs](../Utils/SemanticKernelSpanProcessorHelper.cs) - -Helper methods for processing Semantic Kernel spans. - -```csharp -internal static class SemanticKernelSpanProcessorHelper -{ - public static void ProcessInvocationInputOutputTag( - Activity activity, - bool suppressInvocationInput) - { - // Process input/output message tags - // Optionally suppress input for privacy - } - - public static Dictionary GetGenAiUserAndChoiceMessageContent( - Activity activity) - { - // Extract gen_ai.user and gen_ai.choice events - // Return as dictionary with user messages and choice messages - } -} -``` - -## Design Patterns - -### Extension Pattern - -The package extends the core `Builder` and registers the function invocation filter: - -```csharp -public static class BuilderExtensions -{ - public static Builder WithSemanticKernel(this Builder builder, bool enableRelatedSources = true) - { - builder.Services.AddSingleton(); - - if (enableRelatedSources) - { - AppContext.SetSwitch("Microsoft.SemanticKernel.Experimental.GenAI.EnableOTelDiagnosticsSensitive", true); - - var telmConfig = builder.Services.AddOpenTelemetry() - .WithTracing(tracing => tracing - .AddSource(SemanticKernelTelemetryConstants.SemanticKernelSourceWildcard) - .AddProcessor(new SemanticKernelSpanProcessor(builder.Configuration))); - - if (builder.Configuration != null - && !string.IsNullOrEmpty(builder.Configuration["EnableOtlpExporter"]) - && bool.TryParse(builder.Configuration["EnableOtlpExporter"], out bool enabled) && enabled) - { - telmConfig.UseOtlpExporter(); - } - } - - return builder; - } -} -``` - -### Filter Pattern - -Uses Semantic Kernel's filter pipeline: - -```csharp -public class FunctionInvocationFilter : IFunctionInvocationFilter -{ - public async Task OnFunctionInvocationAsync( - FunctionInvocationContext context, - Func next) - { - // Before invocation - EnrichSpanWithFunctionInfo(context); - - await next(context); - - // After invocation - EnrichSpanWithResult(context); - } -} -``` - -### Span Transformation Pattern - -Transforms Semantic Kernel spans to Agent365 schema: - -```csharp -// SK emits: gen_ai.operation.name = "chat.completions" -// Transform to: gen_ai.operation.name = "Chat" -activity.SetTag(GenAiOperationNameKey, InferenceOperationType.Chat.ToString()); -activity.DisplayName = activity.DisplayName.Replace("chat.completions", "Chat"); -``` - -## Data Flow - -``` -┌─────────────────────────────┐ -│ Semantic Kernel │ -│ │ -│ ChatCompletionAgent │ -│ Kernel.InvokeAsync() │ -│ Plugin functions │ -└──────────────┬──────────────┘ - │ - ▼ Activity Source Events -┌─────────────────────────────┐ -│ Microsoft.SemanticKernel │ -│ │ -│ Creates Activity/Span │ -│ with SK-specific tags │ -│ - chat.completions │ -│ - invoke_agent │ -│ - execute_tool │ -└──────────────┬──────────────┘ - │ - ▼ OnEnd callback -┌─────────────────────────────┐ -│ SemanticKernelSpanProcessor │ -│ │ -│ 1. Check if SK source │ -│ 2. Get operation name │ -│ 3. Transform: │ -│ - chat.completions→Chat │ -│ - Extract user/choice │ -│ - Process input/output │ -└──────────────┬──────────────┘ - │ - ▼ Transformed Activity -┌─────────────────────────────┐ -│ OpenTelemetry Exporter │ -│ │ -│ Spans follow Agent365 │ -│ schema standards │ -└─────────────────────────────┘ -``` - -## Span Attribute Mapping - -### chat.completions Operation - -| Source | Target | -|--------|--------| -| `gen_ai.operation.name = "chat.completions"` | `gen_ai.operation.name = "Chat"` | -| `gen_ai.user` events | `gen_ai.input_messages` | -| `gen_ai.choice` events | `gen_ai.output_messages` | - -### invoke_agent Operation - -| Source | Target | -|--------|--------| -| Input message content | `gen_ai.input_messages` (unless suppressed) | -| Output message content | `gen_ai.output_messages` | - -### execute_tool Operation - -No transformation needed - Semantic Kernel follows Agent365 schema. - -## Configuration - -| Configuration Key | Description | Default | -|-------------------|-------------|---------| -| `SuppressInvokeAgentInput` | Suppress input messages in invoke_agent spans | `false` | - -```json -{ - "SuppressInvokeAgentInput": "true" -} -``` - -## File Structure - -``` -src/Observability/Extensions/SemanticKernel/ -├── BuilderExtensions.cs # Builder extension -├── SemanticKernelSpanProcessor.cs # Span processor -├── SemanticKernelTelemetryConstants.cs # Constants -├── FunctionInvocationFilter.cs # SK function filter -├── ChatCompletionAgentExtensions.cs # Agent extensions -├── Utils/ -│ └── SemanticKernelSpanProcessorHelper.cs # Helper methods -├── Models/ -│ ├── MessageContent.cs # Message model -│ ├── AiChoice.cs # Choice model -│ └── NestedContent.cs # Nested content -├── Microsoft.Agents.A365.Observability.Extensions.SemanticKernel.csproj -└── docs/ - └── design.md # This file -``` - -## Dependencies - -| Package | Purpose | -|---------|---------| -| `Microsoft.Agents.A365.Observability.Runtime` | Core observability | -| `Microsoft.SemanticKernel` | Semantic Kernel SDK | -| `Microsoft.SemanticKernel.Agents.Core` | SK Agents | - -## Usage Examples - -### Basic Setup - -```csharp -// Program.cs -builder.Services.AddOpenTelemetry() - .WithTracing(tracerBuilder => - { - new Builder(builder.Services, builder.Configuration) - .WithSemanticKernel() - .Build(); - }); -``` - -### With Function Filter - -```csharp -// Add filter to kernel -var kernel = Kernel.CreateBuilder() - .AddAzureOpenAIChatCompletion(deploymentName, endpoint, apiKey) - .Build(); - -kernel.FunctionInvocationFilters.Add(new FunctionInvocationFilter()); - -// Use kernel - spans will include function telemetry -var result = await kernel.InvokeAsync(myFunction, arguments); -``` - -### Suppress Input Messages - -```csharp -// appsettings.json -{ - "SuppressInvokeAgentInput": "true" -} - -// Or via environment variable -Environment.SetEnvironmentVariable("SuppressInvokeAgentInput", "true"); -``` - -### Chat Completion Agent - -```csharp -var agent = new ChatCompletionAgent -{ - Name = "MyAssistant", - Instructions = "You are a helpful assistant.", - Kernel = kernel -}; - -// Spans will be processed and transformed automatically -var history = new ChatHistory(); -history.AddUserMessage("Hello!"); - -await foreach (var message in agent.InvokeAsync(history)) -{ - Console.WriteLine(message.Content); -} -``` - -## External Resources - -- [Microsoft Semantic Kernel](https://github.com/microsoft/semantic-kernel) -- [Semantic Kernel Telemetry](https://learn.microsoft.com/semantic-kernel/concepts/enterprise-readiness/observability) -- [OpenTelemetry GenAI Conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) diff --git a/src/Observability/Hosting/Caching/AgenticTokenCache.cs b/src/Observability/Hosting/Caching/AgenticTokenCache.cs deleted file mode 100644 index c6da07d7..00000000 --- a/src/Observability/Hosting/Caching/AgenticTokenCache.cs +++ /dev/null @@ -1,435 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using Microsoft.Agents.A365.Observability.Runtime.Common; -using System; -using System.Collections.Concurrent; -using System.Collections.Generic; -using System.Diagnostics; -using System.IdentityModel.Tokens.Jwt; -using System.Threading; -using System.Threading.Tasks; - -namespace Microsoft.Agents.A365.Observability.Hosting.Caching -{ - /// - /// Caches app-only observability tokens per (agentId, tenantId) using the provided resolver. - /// Includes automatic periodic cleanup that clears expired token values while keeping resolver registrations. - /// - public class AgenticTokenCache : IExporterTokenCache, IDisposable - { - private sealed class Entry - { - public ObservabilityTokenResolver TokenResolver { get; set; } - public string? Token { get; set; } - public string[] Scopes { get; set; } - public DateTimeOffset? ExpiresAt { get; set; } - public DateTimeOffset? AcquiredAt { get; set; } - public SemaphoreSlim RefreshLock { get; } = new SemaphoreSlim(1, 1); - - public Entry(ObservabilityTokenResolver tokenResolver, string[] scopes) - { - TokenResolver = tokenResolver; - Scopes = scopes; - } - - /// - /// Clears the cached token value for security purposes. - /// - public void ClearToken() - { - Token = null; - ExpiresAt = null; - AcquiredAt = null; - } - } - - private readonly ConcurrentDictionary<(string AgentId, string TenantId), Entry> _map = new ConcurrentDictionary<(string AgentId, string TenantId), Entry>(); - private readonly Func _utcNow; - private readonly Timer? _cleanupTimer; - private int _disposed; // Using int for Interlocked operations - private int _removedDelegatedRegistrationLogged; - private static readonly TimeSpan RefreshSkew = TimeSpan.FromMinutes(5); - private static readonly TimeSpan MaxOpaqueTokenAge = TimeSpan.FromHours(1); - - /// - /// Default interval for automatic cleanup of expired tokens (5 minutes). - /// - public static readonly TimeSpan DefaultCleanupInterval = TimeSpan.FromMinutes(5); - - /// - /// Initializes a new instance of the class. - /// - /// The interval for automatic cleanup of expired tokens. Defaults to 5 minutes if not specified. Set to TimeSpan.Zero to disable automatic cleanup. - public AgenticTokenCache(TimeSpan? cleanupInterval = null) - : this(cleanupInterval, () => DateTimeOffset.UtcNow) - { - } - - internal AgenticTokenCache(TimeSpan? cleanupInterval, Func utcNow) - { - _utcNow = utcNow ?? throw new ArgumentNullException(nameof(utcNow)); - var interval = cleanupInterval ?? DefaultCleanupInterval; - if (interval > TimeSpan.Zero) - { - _cleanupTimer = new Timer( - _ => RemoveExpiredTokens(), - null, - interval, - interval); - } - // When interval <= TimeSpan.Zero, _cleanupTimer remains null (no automatic cleanup) - } - - /// - /// Registers an app-only observability token resolver for the specified agent and tenant. - /// - /// The agent identifier. - /// The tenant identifier. - /// The app-only token resolver. - /// The observability scopes. - /// - /// First registration wins. Repeated calls for the same agent and tenant are idempotent - /// and do not replace the resolver or clear the cached token. Use - /// - /// to replace the resolver used by future refreshes. - /// - public void RegisterObservability(string agentId, string tenantId, ObservabilityTokenResolver tokenGenerator, string[] observabilityScopes) - { - ValidateAgentAndTenant(agentId, tenantId); - if (tokenGenerator == null) - { - throw new ArgumentNullException(nameof(tokenGenerator)); - } - - var scopes = ValidateScopes(observabilityScopes); - var entry = new Entry(tokenGenerator, scopes); - _map.TryAdd(GetKey(agentId, tenantId), entry); - } - - /// - /// Delegated OBS token acquisition was removed. This overload is retained only to produce a compile-time error. - /// - /// The agent identifier. - /// The tenant identifier. - /// The removed delegated token generator. - /// The observability scopes. - [Obsolete("Delegated OBS token acquisition has been removed. Register an ObservabilityTokenResolver app-only callback instead.", error: true)] - public void RegisterObservability(string agentId, string tenantId, AgenticTokenStruct tokenGenerator, string[] observabilityScopes) - { - if (Interlocked.Exchange(ref _removedDelegatedRegistrationLogged, 1) == 0) - { - Trace.TraceError("AgenticTokenCache.RegisterObservability with AgenticTokenStruct is removed. OBS export requires an app-only ObservabilityTokenResolver."); - } - } - - /// - /// Gets the observability token for the specified agent and tenant. - /// - /// The agent identifier. - /// The tenant identifier. - /// - /// The observability token if available; otherwise, null. - /// - public async Task GetObservabilityToken(string agentId, string tenantId) - { - if (string.IsNullOrWhiteSpace(agentId) || string.IsNullOrWhiteSpace(tenantId)) - { - return null; - } - - if (!_map.TryGetValue(GetKey(agentId, tenantId), out var entry)) - { - return null; - } - - if (IsTokenUsable(entry)) - { - return entry.Token; - } - - return await RefreshEntryAsync(agentId, tenantId, entry, replacementResolver: null, replacementScopes: null).ConfigureAwait(false); - } - - /// - /// Refreshes and caches an app-only observability token for the specified agent and tenant. - /// - /// The agent identifier. - /// The tenant identifier. - /// The app-only token resolver. - /// The refreshed token. - public Task RefreshObservabilityToken(string agentId, string tenantId, ObservabilityTokenResolver tokenResolver) - { - return RefreshObservabilityToken(agentId, tenantId, tokenResolver, EnvironmentUtils.GetObservabilityAuthenticationScope()); - } - - /// - /// Refreshes and caches an app-only observability token for the specified agent and tenant. - /// - /// The agent identifier. - /// The tenant identifier. - /// The app-only token resolver. - /// The observability scopes. - /// The refreshed token. - public async Task RefreshObservabilityToken(string agentId, string tenantId, ObservabilityTokenResolver tokenResolver, string[] observabilityScopes) - { - ValidateAgentAndTenant(agentId, tenantId); - if (tokenResolver == null) - { - throw new ArgumentNullException(nameof(tokenResolver)); - } - - var scopes = ValidateScopes(observabilityScopes); - var key = GetKey(agentId, tenantId); - var entry = _map.GetOrAdd(key, _ => new Entry(tokenResolver, scopes)); - - return await RefreshEntryAsync(agentId, tenantId, entry, tokenResolver, scopes).ConfigureAwait(false); - } - - private async Task RefreshEntryAsync( - string agentId, - string tenantId, - Entry entry, - ObservabilityTokenResolver? replacementResolver, - string[]? replacementScopes) - { - // Serialize refreshes per agent and tenant: concurrent callers share one acquisition, and a - // failed refresh cannot clear a token that another caller cached while it was waiting. - await entry.RefreshLock.WaitAsync().ConfigureAwait(false); - try - { - // Only explicit refreshes replace the resolver; cache-driven refreshes use the current one. - if (replacementResolver != null && replacementScopes != null) - { - entry.TokenResolver = replacementResolver; - entry.Scopes = replacementScopes; - } - - if (IsTokenUsable(entry)) - { - return entry.Token!; - } - - try - { - var token = await entry.TokenResolver(agentId, tenantId, entry.Scopes).ConfigureAwait(false); - if (string.IsNullOrWhiteSpace(token)) - { - throw new InvalidOperationException("The observability token resolver returned an empty token."); - } - - entry.Token = token; - entry.ExpiresAt = GetTokenExpiration(token!); - entry.AcquiredAt = _utcNow(); - - return token!; - } - catch - { - entry.ClearToken(); - throw; - } - } - finally - { - entry.RefreshLock.Release(); - } - } - - /// - /// Invalidates (removes) the cached entry for a specific agent and tenant. - /// - /// The agent identifier. - /// The tenant identifier. - /// True if the entry was found and removed; otherwise, false. - public bool InvalidateToken(string agentId, string tenantId) - { - if (string.IsNullOrWhiteSpace(agentId) || string.IsNullOrWhiteSpace(tenantId)) - return false; - - var key = GetKey(agentId, tenantId); - if (_map.TryRemove(key, out var entry)) - { - // Clear the token value for security - entry.ClearToken(); - return true; - } - return false; - } - - /// - /// Invalidates (removes) all cached entries. - /// - public void InvalidateAll() - { - // Clear all token values before removing entries - foreach (var kvp in _map) - { - kvp.Value.ClearToken(); - } - _map.Clear(); - } - - /// - /// Clears all expired tokens from the cache. Resolver registrations are kept, so the next - /// call acquires a new token. - /// - /// The number of expired tokens that were cleared. - public int RemoveExpiredTokens() - { - var now = _utcNow(); - int removedCount = 0; - - foreach (var kvp in _map) - { - var entry = kvp.Value; - - // Skip entries with a refresh in flight; the next cleanup pass re-evaluates them. - if (!entry.RefreshLock.Wait(0)) - { - continue; - } - - try - { - if (IsTokenExpired(entry, now)) - { - // Clear the token value for security - entry.ClearToken(); - removedCount++; - } - } - finally - { - entry.RefreshLock.Release(); - } - } - - return removedCount; - } - - /// - /// Gets the current number of entries in the cache. - /// - /// The number of entries currently cached. - public int Count => _map.Count; - - /// - /// Extracts the expiration date and time from a JWT access token. - /// - /// The returned expiration is based on the 'exp' claim in the token payload. No - /// validation of token signature or claims is performed; callers should ensure the token is trusted before - /// relying on the expiration value. - /// The JWT access token from which to retrieve the expiration information. Cannot be null, empty, or - /// whitespace. - /// A representing the token's expiration date and time, or - /// if the token is null, empty, or whitespace. - private static DateTimeOffset? GetTokenExpiration(string token) - { - if (string.IsNullOrWhiteSpace(token)) - { - return null; - } - - if (token.Split('.').Length < 2) - { - return null; - } - - var handler = new JwtSecurityTokenHandler(); - JwtSecurityToken jwtToken; - try - { - jwtToken = handler.ReadJwtToken(token); - } - catch (ArgumentException) - { - return null; - } - - if (jwtToken.Payload.Expiration == null) - { - return null; - } - - return new DateTimeOffset(jwtToken.ValidTo, TimeSpan.Zero); - } - - // A tuple key keeps identities apart even when an ID contains a separator character. - private static (string AgentId, string TenantId) GetKey(string agentId, string tenantId) => (agentId, tenantId); - - private bool IsTokenUsable(Entry entry) - { - return !string.IsNullOrEmpty(entry.Token) - && !IsTokenExpired(entry, _utcNow()); - } - - private static bool IsTokenExpired(Entry entry, DateTimeOffset now) - { - if (string.IsNullOrEmpty(entry.Token)) - { - return false; - } - - if (entry.ExpiresAt.HasValue) - { - return now >= entry.ExpiresAt.Value.Subtract(RefreshSkew); - } - - if (entry.AcquiredAt.HasValue) - { - return now >= entry.AcquiredAt.Value.Add(MaxOpaqueTokenAge); - } - - return true; - } - - private static void ValidateAgentAndTenant(string agentId, string tenantId) - { - if (string.IsNullOrWhiteSpace(agentId)) - { - throw new ArgumentException("Value cannot be null or whitespace.", nameof(agentId)); - } - - if (string.IsNullOrWhiteSpace(tenantId)) - { - throw new ArgumentException("Value cannot be null or whitespace.", nameof(tenantId)); - } - } - - private static string[] ValidateScopes(string[] observabilityScopes) - { - if (observabilityScopes == null || observabilityScopes.Length == 0) - { - throw new ArgumentException("Observability scopes cannot be null or empty.", nameof(observabilityScopes)); - } - - return (string[])observabilityScopes.Clone(); - } - - /// - /// Disposes the cache and stops the automatic cleanup timer. - /// - public void Dispose() - { - Dispose(true); - GC.SuppressFinalize(this); - } - - /// - /// Disposes the cache resources. - /// - /// True if called from Dispose(), false if called from finalizer. - protected virtual void Dispose(bool disposing) - { - // Thread-safe disposal check using Interlocked - if (Interlocked.Exchange(ref _disposed, 1) == 1) - return; - - if (disposing) - { - _cleanupTimer?.Dispose(); - InvalidateAll(); - } - } - } -} diff --git a/src/Observability/Hosting/Caching/AgenticTokenStruct.cs b/src/Observability/Hosting/Caching/AgenticTokenStruct.cs deleted file mode 100644 index e68b46e5..00000000 --- a/src/Observability/Hosting/Caching/AgenticTokenStruct.cs +++ /dev/null @@ -1,55 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using Microsoft.Agents.Builder; -using Microsoft.Agents.Builder.App.UserAuth; -using System; - -namespace Microsoft.Agents.A365.Observability.Hosting.Caching -{ - /// - /// Struct containing UserAuthorization and TurnContext for token generation. - /// - public class AgenticTokenStruct - { - /// - /// UserAuthorization instance used to acquire tokens. - /// - public UserAuthorization UserAuthorization { get; set; } - - /// - /// ITurnContext instance used to acquire tokens. - /// - public ITurnContext TurnContext { get; set; } - - /// - /// Handler name to use with the UserAuthorization system. - /// - public string AuthHandlerName { get; set; } - - /// - /// Connection name, if applicable, to use for the exchange. - /// - public string? ConnectionName { get; set; } - - /// - /// Initializes a new instance of the class. - /// - /// - /// - /// - /// - /// - [Obsolete("Delegated OBS token acquisition has been removed. Use ObservabilityTokenResolver with app-only tokens instead.", error: true)] - public AgenticTokenStruct( - UserAuthorization userAuthorization, - ITurnContext turnContext, - string authHandlerName, - string? connectionName = null) - { - UserAuthorization = userAuthorization ?? throw new ArgumentNullException(nameof(userAuthorization)); - TurnContext = turnContext ?? throw new ArgumentNullException(nameof(turnContext)); - AuthHandlerName = authHandlerName ?? throw new ArgumentNullException(nameof(authHandlerName)); - ConnectionName = connectionName; - } - } -} \ No newline at end of file diff --git a/src/Observability/Hosting/Caching/IExporterTokenCache.cs b/src/Observability/Hosting/Caching/IExporterTokenCache.cs deleted file mode 100644 index 7e84e099..00000000 --- a/src/Observability/Hosting/Caching/IExporterTokenCache.cs +++ /dev/null @@ -1,26 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using System.Threading.Tasks; - -namespace Microsoft.Agents.A365.Observability.Hosting.Caching -{ - /// - /// Cache only for observability (exporter) scoped tokens per (agentId, tenantId). - /// - public interface IExporterTokenCache where T : class - { - /// - /// Registers a credential or resolver used for app-only observability token acquisition. - /// Whether a repeated registration for the same agent and tenant replaces the existing one is - /// implementation-specific: keeps the first registration and - /// replaces it. - /// - void RegisterObservability(string agentId, string tenantId, T tokenGenerator, string[] observabilityScopes); - - /// - /// Returns an observability token or null when not registered. - /// Implementations that acquire a new token may propagate resolver failures to the caller. - /// - Task GetObservabilityToken(string agentId, string tenantId); - } -} \ No newline at end of file diff --git a/src/Observability/Hosting/Caching/ObservabilityTokenResolver.cs b/src/Observability/Hosting/Caching/ObservabilityTokenResolver.cs deleted file mode 100644 index c24966bf..00000000 --- a/src/Observability/Hosting/Caching/ObservabilityTokenResolver.cs +++ /dev/null @@ -1,17 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System.Threading.Tasks; - -namespace Microsoft.Agents.A365.Observability.Hosting.Caching -{ - /// - /// Resolves an app-only OBS token for the exporting agent and tenant. - /// The resolver must not perform delegated, OBO, or user_fic authentication for OBS export. - /// - /// The exporting agent identifier. - /// The tenant identifier. - /// The OBS token scopes to request. - /// The final OBS access token for the exporting agent. - public delegate Task ObservabilityTokenResolver(string agentId, string tenantId, string[] observabilityScopes); -} diff --git a/src/Observability/Hosting/Caching/README.md b/src/Observability/Hosting/Caching/README.md deleted file mode 100644 index 5374bdff..00000000 --- a/src/Observability/Hosting/Caching/README.md +++ /dev/null @@ -1,304 +0,0 @@ -# OBS Token Caches - Token Expiration and Invalidation - -## Overview - -Agent 365 OBS export is S2S-only. Exporters always send traces to -`/observabilityService/tenants/{tenantId}/otlp/agents/{agentId}/traces?api-version=1` -and require app-only OBS tokens for the exporting agent identity. - -`ServiceTokenCache` is a reference implementation of `IExporterTokenCache` that provides secure token caching with built-in expiration and invalidation features for observability exporters. -`AgenticTokenCache` stores app-only tokens resolved by `ObservabilityTokenResolver`; the -former delegated `AgenticTokenStruct` registration is obsolete and no longer performs OBO -or TurnContext token exchange for OBS. `RegisterObservability` is idempotent: -first registration wins and repeated calls do not replace the resolver or clear a cached -token. Use `RefreshObservabilityToken` to replace the resolver used by future refreshes. - -Resolvers must validate the token they return: accept `idtyp=app`, or, when `idtyp` is -absent, a non-empty `roles` array or a non-empty `oid` equal to `sub`; reject any other -`idtyp` and reject any token containing `scp`. They must also verify the OBS audience -(`api://9b975845-388f-4429-889e-eab1ef63949c`) and token lifetime. - -## Features - -### 🔐 Security Features - -- **Automatic Token Expiration**: Tokens expire after a configurable time period (default: 1 hour) -- **Automatic Cleanup**: Expired tokens are automatically removed on access -- **Manual Invalidation**: Support for explicit token removal (individual or all) -- **Thread-Safe Operations**: All operations are thread-safe using `ConcurrentDictionary` - -### ⚙️ Configuration - -- **Default Expiration**: Configurable default expiration time for all tokens -- **Per-Token Expiration**: Ability to override expiration on a per-token basis -- **Validation**: Comprehensive input validation with descriptive error messages - -## Usage - -### Basic Usage with Default Settings - -```csharp -using Microsoft.Agents.A365.Observability.Hosting.Caching; - -// Create cache with default 1-hour expiration -var cache = new ServiceTokenCache(); - -// Register a token -cache.RegisterObservability( - agentId: "my-agent", - tenantId: "my-tenant", - token: "observability-token-xyz", - observabilityScopes: new[] { "https://example.com/.default" } -); - -// Retrieve the token (returns null if expired or not found) -var token = cache.GetObservabilityToken("my-agent", "my-tenant"); -``` - -### App-only Resolver Cache - -```csharp -var cache = new AgenticTokenCache(); - -ObservabilityTokenResolver resolver = async (agentId, tenantId, scopes) => -{ - var token = await AcquireAppOnlyObsTokenAsync(agentId, tenantId, scopes); - ValidateAppOnlyObsToken(token); - return token; -}; - -await cache.RefreshObservabilityToken("my-agent", "my-tenant", resolver); -var token = await cache.GetObservabilityToken("my-agent", "my-tenant"); -``` - -`RefreshObservabilityToken` returns the cached token without calling the resolver while the -cached token is still usable. It propagates acquisition failures and clears stale cached -token state when the resolver fails or returns an empty token. Call it from the exporter's -`TokenResolver` or catch errors on the request path; the exporter will fail the batch -without attempting a delegated fallback. JWT tokens are refreshed near `exp`; opaque tokens -without an `exp` claim use a one-hour fallback max age from acquisition. The three-argument -`RefreshObservabilityToken` overload passes the default app-only OBS scope -`api://9b975845-388f-4429-889e-eab1ef63949c/.default` to the resolver unless -`A365_OBSERVABILITY_SCOPE_OVERRIDE` is set. - -Concurrent refreshes for the same agent and tenant are serialized, so callers share one -acquisition. The automatic cleanup and `RemoveExpiredTokens` clear expired token values but -keep the resolver registration, so the next `GetObservabilityToken` call acquires a new token. - -### Custom Default Expiration - -```csharp -// Create cache with custom default expiration (30 minutes) -var cache = new ServiceTokenCache(TimeSpan.FromMinutes(30)); - -cache.RegisterObservability( - agentId: "my-agent", - tenantId: "my-tenant", - token: "observability-token-xyz", - observabilityScopes: new[] { "https://example.com/.default" } -); -``` - -### Per-Token Custom Expiration - -```csharp -var cache = new ServiceTokenCache(); - -// Register a token with custom expiration (5 minutes) -cache.RegisterObservability( - agentId: "my-agent", - tenantId: "my-tenant", - token: "short-lived-token", - observabilityScopes: new[] { "https://example.com/.default" }, - expiresIn: TimeSpan.FromMinutes(5) -); -``` - -### Manual Token Invalidation - -```csharp -var cache = new ServiceTokenCache(); - -cache.RegisterObservability("agent1", "tenant1", "token1", scopes); -cache.RegisterObservability("agent2", "tenant2", "token2", scopes); - -// Invalidate a specific token -bool removed = cache.InvalidateToken("agent1", "tenant1"); -// removed = true if token was found and removed - -// Invalidate all tokens -cache.InvalidateAll(); -``` - -### Periodic Cleanup of Expired Tokens - -```csharp -var cache = new ServiceTokenCache(); - -// Register some tokens -cache.RegisterObservability("agent1", "tenant1", "token1", scopes); -cache.RegisterObservability("agent2", "tenant2", "token2", scopes); - -// ... wait for some to expire ... - -// Remove all expired tokens and get count -int expiredCount = cache.RemoveExpiredTokens(); -Console.WriteLine($"Removed {expiredCount} expired tokens"); -``` - -## Dependency Injection - -The recommended way to use `ServiceTokenCache` is through dependency injection: - -```csharp -using Microsoft.Agents.A365.Observability.Hosting; -using Microsoft.Agents.A365.Observability.Hosting.Caching; -using Microsoft.Extensions.DependencyInjection; - -var services = new ServiceCollection(); - -// Add service tracing exporter (automatically registers ServiceTokenCache) -services.AddServiceTracingExporter(clusterCategory: "production"); - -var serviceProvider = services.BuildServiceProvider(); - -// Get the cache instance -var cache = serviceProvider.GetRequiredService>(); -``` - -## Best Practices - -### 1. Choose Appropriate Expiration Times - -```csharp -// For short-lived operations (testing, dev) -var cache = new ServiceTokenCache(TimeSpan.FromMinutes(5)); - -// For production (default) -var cache = new ServiceTokenCache(TimeSpan.FromHours(1)); - -// For long-lived tokens -var cache = new ServiceTokenCache(TimeSpan.FromHours(24)); -``` - -### 2. Handle Token Expiration Gracefully - -```csharp -var token = cache.GetObservabilityToken(agentId, tenantId); -if (token == null) -{ - // Token expired or not found - refresh and re-register - var newToken = await AcquireNewTokenAsync(agentId, tenantId); - cache.RegisterObservability(agentId, tenantId, newToken, scopes); - token = newToken; -} -``` - -### 3. Periodic Cleanup in Background Services - -```csharp -public class TokenCleanupService : BackgroundService -{ - private readonly IExporterTokenCache _cache; - - public TokenCleanupService(IExporterTokenCache cache) - { - _cache = cache; - } - - protected override async Task ExecuteAsync(CancellationToken stoppingToken) - { - while (!stoppingToken.IsCancellationRequested) - { - // Run cleanup every 5 minutes - await Task.Delay(TimeSpan.FromMinutes(5), stoppingToken); - - if (_cache is ServiceTokenCache serviceCache) - { - var removed = serviceCache.RemoveExpiredTokens(); - if (removed > 0) - { - Console.WriteLine($"Cleaned up {removed} expired tokens"); - } - } - } - } -} -``` - -### 4. Security Considerations - -- **Never log tokens**: Avoid logging the actual token values -- **Use appropriate expiration**: Match expiration time with your security requirements -- **Keep workload auth separate**: OBS export tokens are app-only and independent of MCP/Graph/OBO tokens -- **Clear cache on security events**: Use `InvalidateAll()` in response to security events - -```csharp -// On security breach detection -public void OnSecurityBreach() -{ - cache.InvalidateAll(); -} -``` - -## Error Handling - -The cache validates all inputs and throws `ArgumentException` for invalid parameters: - -```csharp -try -{ - cache.RegisterObservability(null, "tenant", "token", scopes); -} -catch (ArgumentException ex) -{ - // "Value cannot be null or whitespace. (Parameter 'agentId')" - Console.WriteLine(ex.Message); -} - -try -{ - cache.RegisterObservability("agent", "tenant", "token", Array.Empty()); -} -catch (ArgumentException ex) -{ - // "Observability scopes cannot be null or empty. (Parameter 'observabilityScopes')" - Console.WriteLine(ex.Message); -} -``` - -## Thread Safety - -All operations are thread-safe. You can safely use the same cache instance across multiple threads: - -```csharp -var cache = new ServiceTokenCache(); - -// Safe to call from multiple threads concurrently -Parallel.For(0, 100, i => -{ - cache.RegisterObservability( - $"agent-{i}", - $"tenant-{i}", - $"token-{i}", - scopes - ); -}); -``` - -## Migration from Previous Version - -If you're upgrading from a delegated OBS token flow, replace `AgenticTokenStruct` and -`RegisterObservability(..., AgenticTokenStruct, ...)` with an app-only -`ObservabilityTokenResolver`: - -```csharp -await cache.RefreshObservabilityToken("agent", "tenant", resolver); -var token = await cache.GetObservabilityToken("agent", "tenant"); -``` - -## See Also - -- [IExporterTokenCache Interface](IExporterTokenCache.cs) -- [AgenticTokenCache](AgenticTokenCache.cs) - App-only resolver cache -- [Observability SDK Documentation](../README.md) diff --git a/src/Observability/Hosting/Caching/ServiceTokenCache.cs b/src/Observability/Hosting/Caching/ServiceTokenCache.cs deleted file mode 100644 index fe94a0bc..00000000 --- a/src/Observability/Hosting/Caching/ServiceTokenCache.cs +++ /dev/null @@ -1,249 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using System; -using System.Collections.Concurrent; -using System.Collections.Generic; -using System.Threading; -using System.Threading.Tasks; - -namespace Microsoft.Agents.A365.Observability.Hosting.Caching -{ - /// - /// Caches observability tokens per (agentId, tenantId) with expiration and invalidation support. - /// Includes automatic periodic cleanup of expired tokens for improved memory management. - /// - public class ServiceTokenCache : IExporterTokenCache, IDisposable - { - private sealed class Entry - { - public string? Token { get; set; } - public string[] Scopes { get; } - public DateTimeOffset ExpiresAt { get; } - - public Entry(string token, string[] scopes, DateTimeOffset expiresAt) - { - Token = token; - Scopes = scopes; - ExpiresAt = expiresAt; - } - - /// - /// Clears the cached token value for security purposes. - /// - public void ClearToken() - { - Token = null; - } - } - - private readonly ConcurrentDictionary _map = new ConcurrentDictionary(); - private readonly TimeSpan _defaultExpiration; - private readonly Timer? _cleanupTimer; - private int _disposed; // Using int for Interlocked operations - - /// - /// Default interval for automatic cleanup of expired tokens (5 minutes). - /// - public static readonly TimeSpan DefaultCleanupInterval = TimeSpan.FromMinutes(5); - - /// - /// Initializes a new instance of the class. - /// - /// The default expiration time for tokens. Defaults to 1 hour if not specified. - /// The interval for automatic cleanup of expired tokens. Defaults to 5 minutes if not specified. Set to TimeSpan.Zero to disable automatic cleanup. - public ServiceTokenCache(TimeSpan? defaultExpiration = null, TimeSpan? cleanupInterval = null) - { - _defaultExpiration = defaultExpiration ?? TimeSpan.FromHours(1); - - if (_defaultExpiration <= TimeSpan.Zero) - throw new ArgumentException("Default expiration must be greater than zero.", nameof(defaultExpiration)); - - var interval = cleanupInterval ?? DefaultCleanupInterval; - if (interval > TimeSpan.Zero) - { - _cleanupTimer = new Timer( - _ => RemoveExpiredTokens(), - null, - interval, - interval); - } - // When interval <= TimeSpan.Zero, _cleanupTimer remains null (no automatic cleanup) - } - - /// - /// Registers an observability token for a specific agent and tenant with default expiration. - /// - /// The agent identifier. - /// The tenant identifier. - /// The observability token. - /// The observability scopes. - public void RegisterObservability(string agentId, string tenantId, string token, string[] observabilityScopes) - { - RegisterObservability(agentId, tenantId, token, observabilityScopes, null); - } - - /// - /// Registers an observability token for a specific agent and tenant with custom expiration. - /// - /// The agent identifier. - /// The tenant identifier. - /// The observability token. - /// The observability scopes. - /// Optional custom expiration time. Uses default if not specified. - public void RegisterObservability(string agentId, string tenantId, string token, string[] observabilityScopes, TimeSpan? expiresIn) - { - if (string.IsNullOrWhiteSpace(agentId)) - throw new ArgumentException("Value cannot be null or whitespace.", nameof(agentId)); - - if (string.IsNullOrWhiteSpace(tenantId)) - throw new ArgumentException("Value cannot be null or whitespace.", nameof(tenantId)); - - if (string.IsNullOrWhiteSpace(token)) - throw new ArgumentException("Value cannot be null or whitespace.", nameof(token)); - - if (observabilityScopes == null || observabilityScopes.Length == 0) - throw new ArgumentException("Observability scopes cannot be null or empty.", nameof(observabilityScopes)); - - var expiration = expiresIn ?? _defaultExpiration; - if (expiration <= TimeSpan.Zero) - throw new ArgumentException("Expiration time must be greater than zero.", nameof(expiresIn)); - - var expiresAt = DateTimeOffset.UtcNow.Add(expiration); - var entry = new Entry(token, observabilityScopes, expiresAt); - - var key = GetKey(agentId, tenantId); - _map.AddOrUpdate(key, entry, (k, old) => entry); - } - - /// - /// Retrieves the observability token for a specific agent and tenant. - /// Returns null if the token is not found or has expired. - /// - /// The agent identifier. - /// The tenant identifier. - /// The observability token if valid; otherwise, null. - public async Task GetObservabilityToken(string agentId, string tenantId) - { - if (string.IsNullOrWhiteSpace(agentId) || string.IsNullOrWhiteSpace(tenantId)) - return null; - - var key = GetKey(agentId, tenantId); - - if (!_map.TryGetValue(key, out var entry)) - return null; - - // Check if token has expired - if (DateTimeOffset.UtcNow >= entry.ExpiresAt) - { - // Clear token before removal for security - if (_map.TryRemove(key, out var removedEntry)) - { - removedEntry.ClearToken(); - } - return null; - } - - return await Task.FromResult(entry.Token).ConfigureAwait(false); - } - - /// - /// Invalidates (removes) the cached token for a specific agent and tenant. - /// - /// The agent identifier. - /// The tenant identifier. - /// True if the token was found and removed; otherwise, false. - public bool InvalidateToken(string agentId, string tenantId) - { - if (string.IsNullOrWhiteSpace(agentId) || string.IsNullOrWhiteSpace(tenantId)) - return false; - - var key = GetKey(agentId, tenantId); - if (_map.TryRemove(key, out var entry)) - { - // Clear the token value for security - entry.ClearToken(); - return true; - } - return false; - } - - /// - /// Invalidates (removes) all cached tokens. - /// - public void InvalidateAll() - { - // Clear all token values before removing entries - foreach (var kvp in _map) - { - kvp.Value.ClearToken(); - } - _map.Clear(); - } - - /// - /// Removes all expired tokens from the cache. - /// - /// The number of expired tokens that were removed. - public int RemoveExpiredTokens() - { - var now = DateTimeOffset.UtcNow; - var expiredKeys = new List(); - - // Find expired keys without using LINQ - foreach (var kvp in _map) - { - if (now >= kvp.Value.ExpiresAt) - { - expiredKeys.Add(kvp.Key); - } - } - - int removedCount = 0; - foreach (var key in expiredKeys) - { - if (_map.TryRemove(key, out var entry)) - { - // Clear the token value for security - entry.ClearToken(); - removedCount++; - } - } - - return removedCount; - } - - /// - /// Gets the current number of tokens in the cache. - /// - /// The number of tokens currently cached. - public int Count => _map.Count; - - /// - /// Disposes the cache and stops the automatic cleanup timer. - /// - public void Dispose() - { - Dispose(true); - GC.SuppressFinalize(this); - } - - /// - /// Disposes the cache resources. - /// - /// True if called from Dispose(), false if called from finalizer. - protected virtual void Dispose(bool disposing) - { - // Thread-safe disposal check using Interlocked - if (Interlocked.Exchange(ref _disposed, 1) == 1) - return; - - if (disposing) - { - _cleanupTimer?.Dispose(); - InvalidateAll(); - } - } - - private static string GetKey(string agentId, string tenantId) => $"{agentId}:{tenantId}"; - } -} diff --git a/src/Observability/Hosting/Extensions/BaggageBuilderExtensions.cs b/src/Observability/Hosting/Extensions/BaggageBuilderExtensions.cs deleted file mode 100644 index ca9c8e26..00000000 --- a/src/Observability/Hosting/Extensions/BaggageBuilderExtensions.cs +++ /dev/null @@ -1,94 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using System; -using Microsoft.Agents.A365.Observability.Runtime.Common; -using Microsoft.Agents.Builder; - -namespace Microsoft.Agents.A365.Observability.Hosting.Extensions -{ - /// - /// Utility class for BaggageBuilder extensions. - /// - public static class BaggageBuilderExtensions - { - /// - /// Sets the baggage values from TurnContext. - /// - public static BaggageBuilder FromTurnContext(this BaggageBuilder baggageBuilder, ITurnContext turnContext) - { - if (turnContext is null) - { - throw new ArgumentNullException(nameof(turnContext)); - } - - baggageBuilder - .SetCallerBaggage(turnContext) - .SetTargetAgentBaggage(turnContext) - .SetTenantIdBaggage(turnContext) - .SetChannelBaggage(turnContext) - .SetConversationIdBaggage(turnContext); - - return baggageBuilder; - } - - /// - /// Sets the caller-related baggage values from the TurnContext. - /// - /// The BaggageBuilder instance. - /// The turn context containing activity information. - /// The updated BaggageBuilder instance. - public static BaggageBuilder SetCallerBaggage(this BaggageBuilder baggageBuilder, ITurnContext turnContext) - { - baggageBuilder.SetRange(turnContext.GetCallerBaggagePairs()); - return baggageBuilder; - } - - /// - /// Sets the target agent-related baggage values from the TurnContext. - /// - /// The BaggageBuilder instance. - /// The turn context containing activity information. - /// The updated BaggageBuilder instance. - public static BaggageBuilder SetTargetAgentBaggage(this BaggageBuilder baggageBuilder, ITurnContext turnContext) - { - baggageBuilder.SetRange(turnContext.GetTargetAgentBaggagePairs()); - return baggageBuilder; - } - - /// - /// Sets the tenant ID baggage value, extracting from ChannelData if necessary. - /// - /// The BaggageBuilder instance. - /// The turn context containing activity information. - /// The updated BaggageBuilder instance. - public static BaggageBuilder SetTenantIdBaggage(this BaggageBuilder baggageBuilder, ITurnContext turnContext) - { - baggageBuilder.SetRange(turnContext.GetTenantIdPair()); - return baggageBuilder; - } - - /// - /// Sets the channel baggage values from the TurnContext. - /// - /// The BaggageBuilder instance. - /// The turn context containing activity information. - /// The updated BaggageBuilder instance. - public static BaggageBuilder SetChannelBaggage(this BaggageBuilder baggageBuilder, ITurnContext turnContext) - { - baggageBuilder.SetRange(turnContext.GetChannelBaggagePairs()); - return baggageBuilder; - } - - /// - /// Sets the conversation ID and item link baggage values from the TurnContext. - /// - /// The BaggageBuilder instance. - /// The turn context containing activity information. - /// The updated BaggageBuilder instance. - public static BaggageBuilder SetConversationIdBaggage(this BaggageBuilder baggageBuilder, ITurnContext turnContext) - { - baggageBuilder.SetRange(turnContext.GetConversationIdAndItemLinkPairs()); - return baggageBuilder; - } - } -} \ No newline at end of file diff --git a/src/Observability/Hosting/Extensions/InvokeAgentScopeExtensions.cs b/src/Observability/Hosting/Extensions/InvokeAgentScopeExtensions.cs deleted file mode 100644 index ebd5d9aa..00000000 --- a/src/Observability/Hosting/Extensions/InvokeAgentScopeExtensions.cs +++ /dev/null @@ -1,97 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Agents.Builder; - -namespace Microsoft.Agents.A365.Observability.Hosting.Extensions -{ - /// - /// Extension methods for InvokeAgentScope. - /// - public static class InvokeAgentScopeExtensions - { - /// - /// Sets tag values from TurnContext. - /// - public static InvokeAgentScope FromTurnContext(this InvokeAgentScope invokeAgentScope, ITurnContext turnContext) - { - if (turnContext is null) - { - throw new ArgumentNullException(nameof(turnContext)); - } - - invokeAgentScope - .SetCallerTags(turnContext) - .SetTargetAgentTags(turnContext) - .SetTenantIdTags(turnContext) - .SetChannelTags(turnContext) - .SetConversationIdTags(turnContext); - - return invokeAgentScope; - } - - /// - /// Sets the caller-related attribute values from the TurnContext. - /// - /// The InvokeAgentScope instance. - /// The turn context containing activity information. - /// The updated InvokeAgentScope instance. - public static InvokeAgentScope SetCallerTags(this InvokeAgentScope invokeAgentScope, ITurnContext turnContext) - { - invokeAgentScope.RecordAttributes(turnContext.GetCallerBaggagePairs()); - return invokeAgentScope; - } - - /// - /// Sets the target agent-related tags from the TurnContext. - /// - public static InvokeAgentScope SetTargetAgentTags(this InvokeAgentScope invokeAgentScope, ITurnContext turnContext) - { - invokeAgentScope.RecordAttributes(turnContext.GetTargetAgentBaggagePairs()); - return invokeAgentScope; - } - - /// - /// Sets the tenant ID tag, extracting from ChannelData if necessary. - /// - public static InvokeAgentScope SetTenantIdTags(this InvokeAgentScope invokeAgentScope, ITurnContext turnContext) - { - invokeAgentScope.RecordAttributes(turnContext.GetTenantIdPair()); - return invokeAgentScope; - } - - /// - /// Sets the channel tags from the TurnContext. - /// - public static InvokeAgentScope SetChannelTags(this InvokeAgentScope invokeAgentScope, ITurnContext turnContext) - { - invokeAgentScope.RecordAttributes(turnContext.GetChannelBaggagePairs()); - return invokeAgentScope; - } - - /// - /// Sets the conversation ID and item link tags from the TurnContext. - /// - public static InvokeAgentScope SetConversationIdTags(this InvokeAgentScope invokeAgentScope, ITurnContext turnContext) - { - invokeAgentScope.RecordAttributes(turnContext.GetConversationIdAndItemLinkPairs()); - return invokeAgentScope; - } - - /// - /// Sets the input message tag from the TurnContext using the structured OTEL message format. - /// - public static InvokeAgentScope SetInputMessageTags(this InvokeAgentScope invokeAgentScope, ITurnContext turnContext) - { - var text = turnContext?.Activity?.Text; - if (text != null) - { - invokeAgentScope.RecordInputMessages(new[] { text }); - } - - return invokeAgentScope; - } - } -} diff --git a/src/Observability/Hosting/Extensions/TurnContextExtensions.cs b/src/Observability/Hosting/Extensions/TurnContextExtensions.cs deleted file mode 100644 index 832911a5..00000000 --- a/src/Observability/Hosting/Extensions/TurnContextExtensions.cs +++ /dev/null @@ -1,120 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System.Collections.Generic; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Agents.Builder; - -namespace Microsoft.Agents.A365.Observability.Hosting.Extensions -{ - /// - /// Extension methods for extracting values from ITurnContext. - /// - public static class TurnContextExtensions - { - private const string O11ySpanIdKey = "O11ySpanId"; - private const string O11yTraceIdKey = "O11yTraceId"; - - /// - /// Extracts caller-related baggage key-value pairs from the provided turn context. - /// - public static IEnumerable> GetCallerBaggagePairs(this ITurnContext turnContext) - { - yield return new KeyValuePair(OpenTelemetryConstants.UserIdKey, turnContext.Activity?.From?.AadObjectId); - yield return new KeyValuePair(OpenTelemetryConstants.UserNameKey, turnContext.Activity?.From?.Name); - } - - /// - /// Extracts target agent-related baggage key-value pairs from the provided turn context. - /// - public static IEnumerable> GetTargetAgentBaggagePairs(this ITurnContext turnContext) - { - yield return new KeyValuePair(OpenTelemetryConstants.GenAiAgentIdKey, turnContext.Activity?.Recipient?.AgenticAppId ?? turnContext.Activity?.Recipient?.Id); - yield return new KeyValuePair(OpenTelemetryConstants.GenAiAgentNameKey, turnContext.Activity?.Recipient?.Name); - yield return new KeyValuePair(OpenTelemetryConstants.AgentAUIDKey, turnContext.Activity?.Recipient?.AgenticUserId ?? turnContext.Activity?.Recipient?.AadObjectId); - yield return new KeyValuePair(OpenTelemetryConstants.GenAiAgentDescriptionKey, turnContext.Activity?.Recipient?.Role); - } - - /// - /// Extracts the tenant ID baggage key-value pair, attempting to retrieve from ChannelData if necessary. - /// - public static IEnumerable> GetTenantIdPair(this ITurnContext turnContext) - { - var tenantId = turnContext.Activity?.Recipient?.TenantId; - if (string.IsNullOrWhiteSpace(tenantId) && turnContext.Activity?.ChannelData != null) - { - try - { - var channelDataJson = turnContext.Activity.ChannelData.ToString(); - if (!string.IsNullOrWhiteSpace(channelDataJson)) - { - using var doc = System.Text.Json.JsonDocument.Parse(channelDataJson); - if (doc.RootElement.TryGetProperty("tenant", out var tenantElem) && - tenantElem.TryGetProperty("id", out var idElem) && - idElem.ValueKind == System.Text.Json.JsonValueKind.String) - { - tenantId = idElem.GetString(); - } - } - } - catch - { - } - } - yield return new KeyValuePair(OpenTelemetryConstants.TenantIdKey, tenantId); - } - - /// - /// Extracts channel baggage key-value pairs from the provided turn context. - /// - public static IEnumerable> GetChannelBaggagePairs(this ITurnContext turnContext) - { - yield return new KeyValuePair(OpenTelemetryConstants.ChannelNameKey, turnContext.Activity?.ChannelId?.Channel); - - // Try to get SubChannel from ChannelId first, then fall back to productContext in ChannelData - var subChannel = turnContext.Activity?.ChannelId?.SubChannel; - if (string.IsNullOrWhiteSpace(subChannel) && turnContext.Activity?.ChannelData != null) - { - try - { - var channelDataJson = turnContext.Activity.ChannelData.ToString(); - if (!string.IsNullOrWhiteSpace(channelDataJson)) - { - using var doc = System.Text.Json.JsonDocument.Parse(channelDataJson); - if (doc.RootElement.TryGetProperty("productContext", out var productContextElem) && - productContextElem.ValueKind == System.Text.Json.JsonValueKind.String) - { - subChannel = productContextElem.GetString(); - } - } - } - catch - { - // Ignore ChannelData parsing failures and keep subChannel fallback behavior. - } - } - - yield return new KeyValuePair(OpenTelemetryConstants.ChannelLinkKey, subChannel); - } - - /// - /// Extracts conversation ID and item link baggage key-value pairs from the provided turn context. - /// - public static IEnumerable> GetConversationIdAndItemLinkPairs(this ITurnContext turnContext) - { - string? conversationId = turnContext?.Activity?.Conversation?.Id; - string? itemLink = turnContext?.Activity?.ServiceUrl; - yield return new KeyValuePair(OpenTelemetryConstants.GenAiConversationIdKey, conversationId); - yield return new KeyValuePair(OpenTelemetryConstants.GenAiConversationItemLinkKey, itemLink); - } - - /// - /// Injects observability context into the turn context. - /// - public static void InjectObservabilityContext(this ITurnContext turnContext, OpenTelemetryScope observabilityScope) - { - turnContext.StackState[O11ySpanIdKey] = observabilityScope.Id; - turnContext.StackState[O11yTraceIdKey] = observabilityScope.TraceId; - } - } -} diff --git a/src/Observability/Hosting/Microsoft.Agents.A365.Observability.Hosting.csproj b/src/Observability/Hosting/Microsoft.Agents.A365.Observability.Hosting.csproj deleted file mode 100644 index a000e655..00000000 --- a/src/Observability/Hosting/Microsoft.Agents.A365.Observability.Hosting.csproj +++ /dev/null @@ -1,27 +0,0 @@ - - - Microsoft.Agents.A365.Observability.Hosting - Microsoft.Agents.A365.Observability.Hosting - netstandard2.0 - 8.0 - disable - - Microsoft.Agents.A365.Observability.Hosting - Microsoft Agent 365 Observability Hosting SDK - Observability hosting components for Microsoft Agent 365 SDK - Microsoft Agent 365 Observability Hosting SDK - Provides observability hosting components - Microsoft Agent 365 Observability Hosting SDK - Microsoft;Agents;A365;Observability;OpenTelemetry;AI;Agents;Tracing;Monitoring;Telemetry - - - - - - - - - - - - - diff --git a/src/Observability/Hosting/Middleware/BaggageTurnMiddleware.cs b/src/Observability/Hosting/Middleware/BaggageTurnMiddleware.cs deleted file mode 100644 index 9d5111a9..00000000 --- a/src/Observability/Hosting/Middleware/BaggageTurnMiddleware.cs +++ /dev/null @@ -1,46 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System.Threading; -using System.Threading.Tasks; -using Microsoft.Agents.A365.Observability.Hosting.Extensions; -using Microsoft.Agents.A365.Observability.Runtime.Common; -using Microsoft.Agents.Builder; -using Microsoft.Agents.Core.Models; - -namespace Microsoft.Agents.A365.Observability.Hosting.Middleware -{ - /// - /// Bot Framework middleware that propagates OpenTelemetry baggage context - /// derived from . - /// - /// - /// Async replies (ContinueConversation events) are passed through without - /// baggage setup because their context is established by the originating turn. - /// - public sealed class BaggageTurnMiddleware : IMiddleware - { - /// - public async Task OnTurnAsync(ITurnContext turnContext, NextDelegate next, CancellationToken cancellationToken = default) - { - var activity = turnContext.Activity; - bool isAsyncReply = activity != null - && activity.Type == ActivityTypes.Event - && activity.Name == ActivityEventNames.ContinueConversation; - - if (isAsyncReply) - { - await next(cancellationToken).ConfigureAwait(false); - return; - } - - var builder = new BaggageBuilder(); - builder.FromTurnContext(turnContext); - - using (builder.Build()) - { - await next(cancellationToken).ConfigureAwait(false); - } - } - } -} diff --git a/src/Observability/Hosting/Middleware/ObservabilityBaggageMiddleware.cs b/src/Observability/Hosting/Middleware/ObservabilityBaggageMiddleware.cs deleted file mode 100644 index 50c067cc..00000000 --- a/src/Observability/Hosting/Middleware/ObservabilityBaggageMiddleware.cs +++ /dev/null @@ -1,72 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using Microsoft.Agents.A365.Observability.Runtime.Common; -using Microsoft.AspNetCore.Builder; -using Microsoft.AspNetCore.Http; -using System; -using System.Threading.Tasks; - -namespace Microsoft.Agents.A365.Observability.Hosting.Middleware -{ - /// - /// ASP.NET Core middleware for setting per-request observability context (baggage). - /// - public sealed class ObservabilityBaggageMiddleware - { - private readonly RequestDelegate _next; - private readonly Func _resolver; - - /// - /// Initializes a new instance of the class. - /// - /// The next middleware in the pipeline. - /// Function to resolve tenant and agent IDs from the HTTP context. - /// Whether to tag the current activity (currently unused). - public ObservabilityBaggageMiddleware( - RequestDelegate next, - Func? resolver = null, - bool tagActivity = true) - { - _next = next; - // require explicit resolver to avoid accidental usage without tenant/agent resolution - _resolver = resolver ?? throw new ArgumentNullException(nameof(resolver)); - } - - /// - /// Invokes the middleware to set observability context for the current request. - /// - /// The HTTP context. - /// A task representing the asynchronous operation. - public async Task InvokeAsync(HttpContext ctx) - { - var (tenant, agent) = _resolver(ctx); - - using (new BaggageBuilder() - .TenantId(tenant) - .AgentId(agent) - .Build()) - { - await _next(ctx); - } - } - - } - - /// - /// A middleware to set per-request observability context (baggage). - /// Middleware to be used in ASP.NET scenarios. - /// - public static class ObservabilityBaggageMiddlewareExtensions - { - /// - /// Adds the observability baggage middleware to the application pipeline. - /// - /// The application builder. - /// Function to resolve tenant and agent IDs from the HTTP context. - /// The application builder for method chaining. - public static IApplicationBuilder UseObservabilityRequestContext( - this IApplicationBuilder app, - Func? resolver) - => app.UseMiddleware(resolver); - } -} diff --git a/src/Observability/Hosting/Middleware/OutputLoggingMiddleware.cs b/src/Observability/Hosting/Middleware/OutputLoggingMiddleware.cs deleted file mode 100644 index 8dafa08f..00000000 --- a/src/Observability/Hosting/Middleware/OutputLoggingMiddleware.cs +++ /dev/null @@ -1,185 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Collections.Generic; -using System.Diagnostics; -using System.Threading; -using System.Threading.Tasks; -using Microsoft.Agents.A365.Observability.Hosting.Extensions; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Agents.Builder; -using Microsoft.Agents.Core.Models; - -namespace Microsoft.Agents.A365.Observability.Hosting.Middleware -{ - /// - /// Bot Framework middleware that creates spans - /// for outgoing messages. - /// - /// - /// - /// Links to a parent span when is set in - /// . - /// - /// - /// Privacy note: Outgoing message content is captured verbatim as - /// span attributes and exported to the configured telemetry backend. - /// - /// - public sealed class OutputLoggingMiddleware : IMiddleware - { - /// - /// The key used to store the parent - /// trace context reference. Set this value to a W3C traceparent string - /// (e.g. "00-{trace_id}-{span_id}-{trace_flags}") to link - /// spans as children of an - /// . - /// - public const string A365ParentTraceparentKey = "A365ParentTraceparent"; - - /// - public async Task OnTurnAsync(ITurnContext turnContext, NextDelegate next, CancellationToken cancellationToken = default) - { - var agentDetails = DeriveAgentDetails(turnContext); - - if (agentDetails == null) - { - await next(cancellationToken).ConfigureAwait(false); - return; - } - - var userDetails = DeriveUserDetails(turnContext); - var conversationId = turnContext.Activity?.Conversation?.Id; - var channel = DeriveChannel(turnContext); - - turnContext.OnSendActivities(CreateSendHandler( - turnContext, - agentDetails, - userDetails, - conversationId, - channel)); - - await next(cancellationToken).ConfigureAwait(false); - } - - private static AgentDetails? DeriveAgentDetails(ITurnContext turnContext) - { - var recipient = turnContext.Activity?.Recipient; - if (recipient == null) - { - return null; - } - - // Gate on the recipient having an agentic identity - var agentId = recipient.AgenticAppId ?? recipient.Id; - if (string.IsNullOrEmpty(agentId)) - { - return null; - } - - return new AgentDetails( - agentId: agentId, - agentName: recipient.Name, - agenticUserId: recipient.AadObjectId, - agenticUserEmail: recipient.AgenticUserId, - agentDescription: recipient.Role, - tenantId: recipient.TenantId); - } - - private static UserDetails? DeriveUserDetails(ITurnContext turnContext) - { - var from = turnContext.Activity?.From; - if (from == null) - { - return null; - } - - return new UserDetails( - userId: from.Id ?? string.Empty, - userName: from.Name ?? string.Empty, - userEmail: from.AgenticUserId ?? string.Empty); - } - - private static Channel? DeriveChannel(ITurnContext turnContext) - { - var channelId = turnContext.Activity?.ChannelId; - if (channelId == null) - { - return null; - } - - return new Channel( - name: channelId.Channel, - link: channelId.SubChannel); - } - - private static SendActivitiesHandler CreateSendHandler( - ITurnContext turnContext, - AgentDetails agentDetails, - UserDetails? userDetails, - string? conversationId, - Channel? channel) - { - return async (ctx, activities, nextSend) => - { - var messages = new List(); - foreach (var a in activities) - { - if (string.Equals(a.Type, ActivityTypes.Message, StringComparison.OrdinalIgnoreCase) - && !string.IsNullOrEmpty(a.Text)) - { - messages.Add(a.Text); - } - } - - if (messages.Count == 0) - { - return await nextSend().ConfigureAwait(false); - } - - // Read parent traceparent lazily so the agent handler can set it during logic() - ActivityContext? parentContext = null; - if (turnContext.StackState.TryGetValue(A365ParentTraceparentKey, out var traceparentValue) && traceparentValue != null) - { - var traceparent = traceparentValue.ToString(); - if (!string.IsNullOrEmpty(traceparent)) - { - parentContext = TraceContextHelper.ExtractContextFromHeaders( - new Dictionary { { "traceparent", traceparent } }); - } - } - - var request = new Request( - conversationId: conversationId, - channel: channel); - - SpanDetails? spanDetails = parentContext.HasValue - ? new SpanDetails(parentContext: parentContext) - : null; - - var outputScope = OutputScope.Start( - request: request, - response: new Response(messages), - agentDetails: agentDetails, - userDetails: userDetails, - spanDetails: spanDetails); - - try - { - return await nextSend().ConfigureAwait(false); - } - catch (Exception ex) - { - outputScope.RecordError(ex); - throw; - } - finally - { - outputScope.Dispose(); - } - }; - } - } -} diff --git a/src/Observability/Hosting/ObservabilityBuilderExtensions.cs b/src/Observability/Hosting/ObservabilityBuilderExtensions.cs deleted file mode 100644 index 17b6fbd5..00000000 --- a/src/Observability/Hosting/ObservabilityBuilderExtensions.cs +++ /dev/null @@ -1,70 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -namespace Microsoft.Agents.A365.Observability.Hosting -{ - using System; - using Microsoft.Agents.A365.Observability.Runtime; - using Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters; - using Microsoft.AspNetCore.Hosting; - using Microsoft.Extensions.Hosting; - - /// - /// Provides extension methods for configuring Microsoft Agent 365 SDK with OpenTelemetry tracing. - /// - public static class ObservabilityBuilderExtensions - { - /// - /// Adds the Microsoft Agent 365 SDK with OpenTelemetry tracing for AI agents and tools. - /// - /// The web host builder to which tracing services will be added. - /// An optional delegate to further configure the tracing builder. - /// Specifies whether to use the OpenTelemetry builder for configuration. Defaults to true. - /// The type of Agent 365 exporter to use for tracing. Defaults to . - /// The original instance with tracing configured. - public static IWebHostBuilder AddA365Tracing( - this IWebHostBuilder webHostBuilder, - Action? configure = null, - bool useOpenTelemetryBuilder = true, - Agent365ExporterType agent365ExporterType = Agent365ExporterType.Agent365Exporter) - { - webHostBuilder.ConfigureServices((context, services) => - { - var localBuilder = new Builder( - services: services, - useOpenTelemetryBuilder: useOpenTelemetryBuilder, - agent365ExporterType: agent365ExporterType, - configuration: context.Configuration); - configure?.Invoke(localBuilder); - localBuilder.Build(); - }); - return webHostBuilder; - } - - /// - /// Adds the Microsoft Agent 365 SDK with OpenTelemetry tracing for AI agents and tools. - /// - /// The generic host builder to which tracing services will be added. - /// An optional delegate to further configure the tracing builder. - /// Specifies whether to use the OpenTelemetry builder for configuration. Defaults to true. - /// The type of Agent 365 exporter to use for tracing. Defaults to . - /// The original instance with tracing configured. - public static IHostBuilder AddA365Tracing( - this IHostBuilder builder, - Action? configure = null, - bool useOpenTelemetryBuilder = true, - Agent365ExporterType agent365ExporterType = Agent365ExporterType.Agent365Exporter) - { - builder.ConfigureServices((context, services) => - { - var localBuilder = new Builder( - services: services, - useOpenTelemetryBuilder: useOpenTelemetryBuilder, - agent365ExporterType: agent365ExporterType, - configuration: context.Configuration); - configure?.Invoke(localBuilder); - localBuilder.Build(); - }); - return builder; - } - } -} diff --git a/src/Observability/Hosting/ObservabilityServiceCollectionExtensions.cs b/src/Observability/Hosting/ObservabilityServiceCollectionExtensions.cs deleted file mode 100644 index 7996a808..00000000 --- a/src/Observability/Hosting/ObservabilityServiceCollectionExtensions.cs +++ /dev/null @@ -1,61 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -namespace Microsoft.Agents.A365.Observability.Hosting -{ - using Microsoft.Extensions.DependencyInjection; - using Microsoft.Agents.A365.Observability.Hosting.Caching; - using Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters; - - /// - /// Provides extension methods for configuring Microsoft Agent 365 SDK with OpenTelemetry tracing. - /// - public static class ObservabilityServiceCollectionExtensions - { - /// - /// Adds agentic token handling to the service collection. - /// - /// The service collection to add to. - /// The cluster category for the Microsoft Agent 365 exporter options. Defaults to production. - /// The updated service collection. - public static IServiceCollection AddAgenticTracingExporter(this IServiceCollection services, string? clusterCategory = "production") - { - services.AddSingleton, AgenticTokenCache>(); - - services.AddSingleton(sp => - { - var cache = sp.GetRequiredService>(); - return new Agent365ExporterOptions - { - ClusterCategory = clusterCategory ?? "production", - TokenResolver = async (agentId, tenantId) => await cache.GetObservabilityToken(agentId, tenantId) - }; - }); - - return services; - } - - /// - /// Adds a service tracing exporter to the service collection. - /// Uses the service-to-service (S2S) endpoint for trace exports. - /// - /// The service collection to add to. - /// The cluster category for the Microsoft Agent 365 exporter options. Defaults to production. - /// The updated service collection. - public static IServiceCollection AddServiceTracingExporter(this IServiceCollection services, string? clusterCategory = "production") - { - services.AddSingleton, ServiceTokenCache>(); - - services.AddSingleton(sp => - { - var cache = sp.GetRequiredService>(); - return new Agent365ExporterOptions - { - ClusterCategory = clusterCategory ?? "production", - TokenResolver = async (agentId, tenantId) => await cache.GetObservabilityToken(agentId, tenantId).ConfigureAwait(false) - }; - }); - - return services; - } - } -} diff --git a/src/Observability/Hosting/README.md b/src/Observability/Hosting/README.md deleted file mode 100644 index bbdf35de..00000000 --- a/src/Observability/Hosting/README.md +++ /dev/null @@ -1,44 +0,0 @@ -# Microsoft.Agents.A365.Observability.Hosting - -The Hosting package provides ETW (Event Tracing for Windows) integration for the Microsoft Agent 365 Observability SDK. This package enables high-performance event tracing on Windows platforms for production monitoring scenarios. - -## OBS Token Caching - -OBS export is S2S-only. Hosting helpers must provide app-only tokens for the exporting -agent identity; delegated TurnContext/UserAuthorization exchange is no longer used for OBS. -`AgenticTokenCache.RegisterObservability(..., AgenticTokenStruct, ...)` and -`AgenticTokenStruct` construction are obsolete compile-time errors. Register or refresh an -`ObservabilityTokenResolver` instead, then wire `Agent365ExporterOptions.TokenResolver` to -`GetObservabilityToken`. - -The resolver is responsible for acquiring and validating the final OBS token. It must reject -delegated `scp` tokens, accept app-only tokens (`idtyp=app`, or no `idtyp` with non-empty -`roles` or `oid == sub`), verify the OBS audience, and ensure the token is not expired. -Workload MCP/Graph/OBO authentication remains separate and unchanged. - -## Installation - -```bash -dotnet add package Microsoft.Agents.A365.Observability.Hosting -``` - -## Documentation - -For detailed usage information, configuration examples, and best practices, see the [Microsoft Agents 365 Observability documentation](https://learn.microsoft.com/en-us/microsoft-agent-365/developer/observability?tabs=dotnet). - -## Support - -For issues, questions, or feedback: - -- File issues in the [GitHub Issues](https://github.com/microsoft/Agent365-dotnet/issues) section -- See the [main documentation](../../../README.md) for more information - -## Trademarks - -*Microsoft, Windows, Microsoft Azure and/or other Microsoft products and services referenced in the documentation may be either trademarks or registered trademarks of Microsoft in the United States and/or other countries. The licenses for this project do not grant you rights to use any Microsoft names, logos, or trademarks. Microsoft's general trademark guidelines can be found at http://go.microsoft.com/fwlink/?LinkID=254653.* - -## License - -Copyright (c) Microsoft Corporation. All rights reserved. - -Licensed under the MIT License - see the [LICENSE](../../../LICENSE.md) file for details. \ No newline at end of file diff --git a/src/Observability/Hosting/docs/design.md b/src/Observability/Hosting/docs/design.md deleted file mode 100644 index 8a33ce5d..00000000 --- a/src/Observability/Hosting/docs/design.md +++ /dev/null @@ -1,413 +0,0 @@ -# Microsoft.Agents.A365.Observability.Hosting - Design Documentation - -## Overview - -The `Microsoft.Agents.A365.Observability.Hosting` package provides ASP.NET Core hosting integration for the Agent365 observability infrastructure. It includes middleware for baggage propagation, token caching for exporters, and extension methods for working with the Microsoft 365 Agents SDK `TurnContext`. - -## Architecture - -``` -Microsoft.Agents.A365.Observability.Hosting -├── Middleware/ -│ └── ObservabilityBaggageMiddleware # Per-request baggage context -├── Caching/ -│ ├── IExporterTokenCache # Token cache interface -│ ├── AgenticTokenCache # Agentic token caching -│ ├── ServiceTokenCache # Service token caching -│ └── AgenticTokenStruct # Token data structure -├── Extensions/ -│ ├── BaggageBuilderExtensions # Baggage context helpers -│ ├── InvokeAgentScopeExtensions # Scope enrichment from TurnContext -│ ├── TurnContextExtensions # TurnContext telemetry extraction -│ ├── ObservabilityBuilderExtensions # Builder extensions -│ └── ObservabilityServiceCollectionExtensions # DI setup -└── Internal/ - └── AttributeKeys # Internal attribute key constants -``` - -## Key Components - -### ObservabilityBaggageMiddleware - -**Source**: [ObservabilityBaggageMiddleware.cs](../Middleware/ObservabilityBaggageMiddleware.cs) - -ASP.NET Core middleware that sets per-request observability context (baggage) for distributed tracing. - -```csharp -// Configure middleware in Program.cs -app.UseObservabilityRequestContext(ctx => -{ - var tenantId = ctx.User?.FindFirst("tenant_id")?.Value; - var agentId = ctx.Request.Headers["X-Agent-Id"].FirstOrDefault(); - return (tenantId, agentId); -}); -``` - -**Parameters:** - -| Parameter | Type | Description | -|-----------|------|-------------| -| `next` | `RequestDelegate` | Next middleware in pipeline | -| `resolver` | `Func` | Function to resolve tenant and agent IDs | - -**Behavior:** -1. Calls the resolver function with the current `HttpContext` -2. Sets baggage context using `BaggageBuilder.SetRequestContext()` -3. Context is automatically disposed after request completes - -### InvokeAgentScopeExtensions - -**Source**: [InvokeAgentScopeExtensions.cs](../Extensions/InvokeAgentScopeExtensions.cs) - -Extension methods for enriching `InvokeAgentScope` with data from `ITurnContext`. - -```csharp -using var scope = InvokeAgentScope.Start(invokeAgentDetails, tenantDetails); - -// Enrich scope with all available TurnContext data -scope.FromTurnContext(turnContext); - -// Or selectively enrich specific attributes -scope.SetCallerTags(turnContext); -scope.SetExecutionTypeTags(turnContext); -scope.SetTargetAgentTags(turnContext); -scope.SetTenantIdTags(turnContext); -scope.SetSourceMetadataTags(turnContext); -scope.SetConversationIdTags(turnContext); -scope.SetInputMessageTags(turnContext); -``` - -**Extension Methods:** - -| Method | Purpose | -|--------|---------| -| `FromTurnContext()` | Sets all available tags from TurnContext | -| `SetCallerTags()` | Sets caller-related attributes | -| `SetExecutionTypeTags()` | Sets execution type based on caller/recipient status | -| `SetTargetAgentTags()` | Sets target agent attributes | -| `SetTenantIdTags()` | Sets tenant ID from ChannelData | -| `SetSourceMetadataTags()` | Sets source metadata attributes | -| `SetConversationIdTags()` | Sets conversation ID and item link | -| `SetInputMessageTags()` | Sets input message from Activity.Text | - -### TurnContextExtensions - -**Source**: [TurnContextExtensions.cs](../Extensions/TurnContextExtensions.cs) - -Extension methods for extracting telemetry data from `ITurnContext`. - -```csharp -// Get baggage key-value pairs -var callerPairs = turnContext.GetCallerBaggagePairs(); -var executionTypePair = turnContext.GetExecutionTypePair(); -var targetAgentPairs = turnContext.GetTargetAgentBaggagePairs(); -var tenantIdPair = turnContext.GetTenantIdPair(); -var conversationPairs = turnContext.GetConversationIdAndItemLinkPairs(); -var sourceMetadataPairs = turnContext.GetSourceMetadataBaggagePairs(); -``` - -**Extracted Attributes:** - -| Category | Attributes | -|----------|------------| -| Caller | `gen_ai.caller.id`, `gen_ai.caller.upn`, `gen_ai.caller.name`, `gen_ai.caller.client_ip` | -| Execution | `gen_ai.execution.type` (User, Agent, AgentToAgent) | -| Target Agent | `gen_ai.agent.name`, `gen_ai.agent.id`, `gen_ai.agent.upn` | -| Tenant | `tenant_id` | -| Conversation | `conversation_id`, `item_link` | -| Source | Source metadata from activity | - -### Token Caching - -Token caches for managing app-only authentication tokens used by telemetry exporters. -OBS export always uses the S2S OTLP route and never uses TurnContext or delegated -authorization to acquire OBS tokens. - -**IExporterTokenCache Interface:** - -```csharp -public interface IExporterTokenCache where T : class -{ - void RegisterObservability(string agentId, string tenantId, T tokenGenerator, string[] observabilityScopes); - Task GetObservabilityToken(string agentId, string tenantId); -} -``` - -**AgenticTokenCache:** - -Caches app-only OBS tokens per `(agentId, tenantId)` using an `ObservabilityTokenResolver`. -The previous `AgenticTokenStruct`/`UserAuthorization` registration is obsolete with -`error: true`; typed callers must migrate, and dynamic callers do not trigger a delegated -exchange. Use `RefreshObservabilityToken(agentId, tenantId, tokenResolver, scopes)` to -refresh at export time and let acquisition failures propagate to the exporter. - -**ServiceTokenCache:** - -Caches already-acquired app-only tokens for service-to-service operations. - -Resolvers must validate the final OBS token before caching it: accept `idtyp=app`, or, -when `idtyp` is absent, a non-empty `roles` array or a non-empty `oid` equal to `sub`; -reject any other `idtyp` and any `scp` claim. Resolvers must also verify the OBS audience -(`api://9b975845-388f-4429-889e-eab1ef63949c`) and token lifetime. Workload MCP/Graph/OBO -authorization is separate and unchanged. - -### BaggageBuilderExtensions - -**Source**: [BaggageBuilderExtensions.cs](../Extensions/BaggageBuilderExtensions.cs) - -Extensions for the `BaggageBuilder` class from the Runtime package. - -```csharp -// Set baggage from TurnContext -BaggageBuilder.SetFromTurnContext(turnContext); - -// Set baggage from HttpContext -BaggageBuilder.SetFromHttpContext(httpContext); -``` - -### ObservabilityServiceCollectionExtensions - -**Source**: [ObservabilityServiceCollectionExtensions.cs](../Extensions/ObservabilityServiceCollectionExtensions.cs) - -Extension methods for configuring observability services in the DI container. - -```csharp -// Add observability services -services.AddAgent365Observability(configuration); - -// Add with custom options -services.AddAgent365Observability(configuration, options => -{ - options.EnableAgent365Exporter = true; - options.ServiceName = "MyAgent"; -}); -``` - -## Design Patterns - -### Middleware Pattern - -The baggage middleware follows ASP.NET Core middleware conventions: - -```csharp -public sealed class ObservabilityBaggageMiddleware -{ - private readonly RequestDelegate _next; - private readonly Func _resolver; - - public ObservabilityBaggageMiddleware( - RequestDelegate next, - Func? resolver) - { - _next = next; - _resolver = resolver ?? throw new ArgumentNullException(nameof(resolver)); - } - - public async Task InvokeAsync(HttpContext ctx) - { - var (tenant, agent) = _resolver(ctx); - - using (BaggageBuilder.SetRequestContext(tenant, agent)) - { - await _next(ctx); - } - } -} -``` - -### Extension Methods Pattern - -TurnContext extensions use a consistent pattern for extracting key-value pairs: - -```csharp -public static class TurnContextExtensions -{ - public static IEnumerable> GetCallerBaggagePairs( - this ITurnContext turnContext) - { - var activity = turnContext?.Activity; - if (activity == null) yield break; - - yield return new KeyValuePair( - OpenTelemetryConstants.GenAiCallerIdKey, - activity.From?.Id); - - yield return new KeyValuePair( - OpenTelemetryConstants.GenAiCallerUpnKey, - activity.From?.Properties?.GetValueOrDefault("upn")?.ToString()); - - // ... more attributes - } -} -``` - -### Fluent Extension Pattern - -InvokeAgentScope extensions support method chaining: - -```csharp -public static InvokeAgentScope SetCallerTags( - this InvokeAgentScope invokeAgentScope, - ITurnContext turnContext) -{ - invokeAgentScope.RecordAttributes(turnContext.GetCallerBaggagePairs()); - return invokeAgentScope; // Enable chaining -} -``` - -## Data Flow - -``` -┌─────────────────┐ ┌─────────────────────────┐ ┌─────────────────┐ -│ HTTP Request │────►│ ObservabilityBaggage │────►│ BaggageBuilder │ -│ │ │ Middleware │ │ │ -│ Headers, Claims │ │ │ │ Set tenant_id │ -│ │ │ resolver(ctx) => │ │ Set agent_id │ -│ │ │ (tenant, agent) │ │ │ -└─────────────────┘ └─────────────────────────┘ └────────┬────────┘ - │ - ▼ -┌─────────────────┐ ┌─────────────────────────┐ ┌─────────────────┐ -│ TurnContext │────►│ InvokeAgentScope │────►│ OpenTelemetry │ -│ │ │ Extensions │ │ Span │ -│ Activity │ │ │ │ │ -│ From, Recipient │ │ .FromTurnContext() │ │ With all │ -│ ChannelData │ │ .SetCallerTags() │ │ enriched tags │ -└─────────────────┘ └─────────────────────────┘ └─────────────────┘ -``` - -## File Structure - -``` -src/Observability/Hosting/ -├── Middleware/ -│ └── ObservabilityBaggageMiddleware.cs # Per-request baggage -├── Caching/ -│ ├── IExporterTokenCache.cs # Token cache interface -│ ├── AgenticTokenCache.cs # Agentic token caching -│ ├── ServiceTokenCache.cs # Service token caching -│ └── AgenticTokenStruct.cs # Token data structure -├── Extensions/ -│ ├── BaggageBuilderExtensions.cs # Baggage extensions -│ ├── InvokeAgentScopeExtensions.cs # Scope enrichment -│ ├── TurnContextExtensions.cs # TurnContext data extraction -│ ├── ObservabilityBuilderExtensions.cs # Builder extensions -│ └── ObservabilityServiceCollectionExtensions.cs # DI setup -├── Microsoft.Agents.A365.Observability.Hosting.csproj -└── docs/ - └── design.md # This file -``` - -## Dependencies - -| Package | Purpose | -|---------|---------| -| `Microsoft.Agents.A365.Observability.Runtime` | Core observability | -| `Microsoft.Agents.Builder` | TurnContext, Activity | -| `Microsoft.AspNetCore.Hosting` | ASP.NET Core hosting | -| `Microsoft.AspNetCore.Http.Abstractions` | HttpContext | - -## Usage Examples - -### Full Pipeline Setup - -```csharp -// Program.cs -var builder = WebApplication.CreateBuilder(args); - -// Add observability services -builder.Services.AddAgent365Observability(builder.Configuration); - -var app = builder.Build(); - -// Add baggage middleware early in pipeline -app.UseObservabilityRequestContext(ctx => -{ - // Extract from JWT claims - var tenantId = ctx.User?.FindFirst("tid")?.Value - ?? ctx.User?.FindFirst("tenant_id")?.Value; - - // Extract from custom header - var agentId = ctx.Request.Headers["X-Agent-Id"].FirstOrDefault(); - - return (tenantId, agentId); -}); - -app.UseAuthentication(); -app.UseAuthorization(); -app.MapControllers(); - -app.Run(); -``` - -### Agent Message Handler - -```csharp -public class AgentHandler : IActivityHandler -{ - public async Task OnMessageActivityAsync(ITurnContext turnContext, CancellationToken ct) - { - var tenantDetails = new TenantDetails - { - TenantId = turnContext.Activity.ChannelData? - .GetProperty("tenant")?.GetProperty("id")?.GetString() - }; - - using var scope = InvokeAgentScope.Start( - invokeAgentDetails: CreateInvokeDetails(turnContext), - tenantDetails: tenantDetails - ); - - // Enrich with all available context - scope.FromTurnContext(turnContext); - - try - { - var response = await ProcessMessageAsync(turnContext.Activity.Text); - scope.RecordResponse(response); - - await turnContext.SendActivityAsync(response, cancellationToken: ct); - } - catch (Exception ex) - { - scope.SetStatus(ActivityStatusCode.Error, ex.Message); - throw; - } - } -} -``` - -### Custom Token Cache Implementation - -```csharp -public class RedisTokenCache : IExporterTokenCache -{ - private readonly IDistributedCache _cache; - - public async Task GetTokenAsync(string resource, CancellationToken ct) - { - var key = $"exporter_token:{resource}"; - return await _cache.GetStringAsync(key, ct); - } - - public async Task SetTokenAsync( - string resource, string token, DateTimeOffset expiry, CancellationToken ct) - { - var key = $"exporter_token:{resource}"; - var options = new DistributedCacheEntryOptions - { - AbsoluteExpiration = expiry - }; - await _cache.SetStringAsync(key, token, options, ct); - } -} - -// Register in DI -services.AddSingleton(); -``` - -## External Resources - -- [ASP.NET Core Middleware](https://learn.microsoft.com/en-us/aspnet/core/fundamentals/middleware/) -- [Microsoft 365 Agents SDK](https://github.com/microsoft/agents) -- [OpenTelemetry Context Propagation](https://opentelemetry.io/docs/concepts/context-propagation/) diff --git a/src/Observability/README.md b/src/Observability/README.md deleted file mode 100644 index ed84b307..00000000 --- a/src/Observability/README.md +++ /dev/null @@ -1,76 +0,0 @@ -# Microsoft Agent 365 Observability SDK - -[![NuGet](https://img.shields.io/nuget/v/Microsoft.Agents.A365.Observability.svg?label=Core)](https://www.nuget.org/packages/Microsoft.Agents.A365.Observability/) -[![NuGet](https://img.shields.io/nuget/v/Microsoft.Agents.A365.Observability.Runtime.svg?label=Runtime)](https://www.nuget.org/packages/Microsoft.Agents.A365.Observability.Runtime/) -[![NuGet](https://img.shields.io/nuget/v/Microsoft.Agents.A365.Observability.Hosting.svg?label=Hosting)](https://www.nuget.org/packages/Microsoft.Agents.A365.Observability.Hosting/) -[![NuGet](https://img.shields.io/nuget/v/Microsoft.Agents.A365.Observability.Extensions.OpenAI.svg?label=OpenAI)](https://www.nuget.org/packages/Microsoft.Agents.A365.Observability.Extensions.OpenAI/) -[![NuGet](https://img.shields.io/nuget/v/Microsoft.Agents.A365.Observability.Extensions.SemanticKernel.svg?label=SemanticKernel)](https://www.nuget.org/packages/Microsoft.Agents.A365.Observability.Extensions.SemanticKernel/) -[![NuGet](https://img.shields.io/nuget/v/Microsoft.Agents.A365.Observability.Extensions.AgentFramework.svg?label=AgentFramework)](https://www.nuget.org/packages/Microsoft.Agents.A365.Observability.Extensions.AgentFramework/) -[![Downloads](https://img.shields.io/nuget/dt/Microsoft.Agents.A365.Observability.svg)](https://www.nuget.org/packages/Microsoft.Agents.A365.Observability/) - -The Microsoft Agent 365 Observability SDK provides comprehensive monitoring, tracing, and diagnostics capabilities for AI agent applications. This module enables developers to gain deep insights into agent behavior, performance, and execution patterns through industry-standard observability tools. - -Agent 365 OBS export uses the S2S OTLP endpoint only and requires an app-only OBS token for -the exporting agent identity. The SDK does not acquire delegated OBS tokens or fall back to -the delegated `/observability` route. - -## Overview - -Building production-ready AI agents requires robust observability to understand agent behavior, diagnose issues, and optimize performance. This module provides: - -- Distributed tracing for agent invocations and tool executions -- Integration with OpenTelemetry and Azure Monitor -- Specialized telemetry for AI agent operations -- Performance metrics and diagnostics -- Caching instrumentation and monitoring - -## Installation - -```bash -# Core packages -dotnet add package Microsoft.Agents.A365.Observability -dotnet add package Microsoft.Agents.A365.Observability.Runtime -dotnet add package Microsoft.Agents.A365.Observability.Hosting - -# Framework-specific extensions -dotnet add package Microsoft.Agents.A365.Observability.Extensions.OpenAI -dotnet add package Microsoft.Agents.A365.Observability.Extensions.SemanticKernel -dotnet add package Microsoft.Agents.A365.Observability.Extensions.AgentFramework -``` - -## Package Structure - -The Observability module is organized into several packages, each with detailed documentation: - -### Core Packages - -- **[Microsoft.Agents.A365.Observability](Core/README.md)** - Core observability functionality including token caching and scope abstractions -- **[Microsoft.Agents.A365.Observability.Runtime](Runtime/README.md)** - Runtime services with full tracing setup, exporters, and OpenTelemetry integration -- **[Microsoft.Agents.A365.Observability.Hosting](Hosting/README.md)** - ETW (Event Tracing for Windows) support for production monitoring - -### Framework Extensions - -- **[Microsoft.Agents.A365.Observability.Extensions.OpenAI](Extensions/OpenAI/README.md)** - OpenAI SDK tracing and ChatToolCall extensions -- **[Microsoft.Agents.A365.Observability.Extensions.SemanticKernel](Extensions/SemanticKernel/README.md)** - Semantic Kernel integration with function invocation filtering -- **[Microsoft.Agents.A365.Observability.Extensions.AgentFramework](Extensions/AgentFramework/README.md)** - Microsoft Agent Framework integration with multi-source activity tracking - -## Support - -For issues, questions, or feedback: - -- File issues in the [GitHub Issues](https://github.com/microsoft/Agent365-dotnet/issues) section -- See the [main documentation](../../README.md) for more information - -## Contributing - -This project welcomes contributions and suggestions. See the [Contributing Guide](../../README.md#contributing) for details. - -## Trademarks - -*Microsoft, Windows, Microsoft Azure and/or other Microsoft products and services referenced in the documentation may be either trademarks or registered trademarks of Microsoft in the United States and/or other countries. The licenses for this project do not grant you rights to use any Microsoft names, logos, or trademarks. Microsoft's general trademark guidelines can be found at http://go.microsoft.com/fwlink/?LinkID=254653.* - -## License - -Copyright (c) Microsoft Corporation. All rights reserved. - -Licensed under the MIT License - see the [LICENSE](../../LICENSE.md) file for details. \ No newline at end of file diff --git a/src/Observability/Runtime/Builder.cs b/src/Observability/Runtime/Builder.cs deleted file mode 100644 index 23de71a1..00000000 --- a/src/Observability/Runtime/Builder.cs +++ /dev/null @@ -1,128 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -namespace Microsoft.Agents.A365.Observability.Runtime -{ - using Microsoft.Agents.A365.Observability.Runtime.Common; - using Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters; - using Microsoft.Agents.A365.Observability.Runtime.Tracing.Processors; - using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; - using Microsoft.Extensions.Configuration; - using Microsoft.Extensions.DependencyInjection; - using OpenTelemetry; - using OpenTelemetry.Trace; - using System; - - /// - /// Builder for configuring SDK with OpenTelemetry tracing. - /// - public sealed class Builder - { - private readonly IServiceCollection _services; - private readonly bool _useOpenTelemetryBuilder; - private readonly Agent365ExporterType _agent365ExporterType; - private bool _isBuilt = false; - - /// - /// Gets the configuration instance used for feature toggles and runtime settings. - /// - public IConfiguration? Configuration { get; } - - /// - /// Initializes a new instance of the class. - /// - /// The service collection to configure. - /// The configuration instance. - /// Whether to use the OpenTelemetryBuilder to add OpenTelemetry services to the supplied service collection. - /// The type of Agent365 exporter to use. - public Builder(IServiceCollection services, IConfiguration? configuration, bool useOpenTelemetryBuilder = true, Agent365ExporterType agent365ExporterType = Agent365ExporterType.Agent365Exporter) - { - this._services = services; - this._useOpenTelemetryBuilder = useOpenTelemetryBuilder; - this._agent365ExporterType = agent365ExporterType; - Configuration = configuration; - } - - /// - /// Gets the services collection for continued configuration. - /// - public IServiceCollection Services => _services; - - /// - /// Builds the AI configuration and returns the service collection. - /// - /// The configured service collection. - public IServiceCollection Build() - { - EnsureBuilt(); - return _services; - } - - private bool IsAgent365ExporterEnabled() - { - if (Configuration != null && Configuration["EnableAgent365Exporter"] != null) - { - string enabledEnv = Configuration["EnableAgent365Exporter"]!; - return enabledEnv.Equals(bool.TrueString, StringComparison.OrdinalIgnoreCase); - } - return false; - } - - private void EnsureBuilt() - { - if (_isBuilt) - return; - - AppContext.SetSwitch("Azure.Experimental.EnableActivitySource", true); - - // Configure OpenTelemetry with all processors in a single call - // NOTE: _useOpenTelemetryBuilder = true does two things. - // 1. Uses the provided service collection to create the tracer provider (using IDeferredTracerProviderBuilder in ObservabilityTracerProviderBuilderExtensions.AddAgent365Exporter()) - // 2. Adds open telemetry and tracing services to the service collecion. - // _useOpenTelemetryBuilder = false just uses the provided service collection to create the tracer provider (using ObservabilityTracerProviderBuilderExtensions.AddAgent365Exporter(IServiceCollection)) - if (this._useOpenTelemetryBuilder) - { - _services - .AddOpenTelemetry() - .WithTracing(tracerProviderBuilder => - { - this.Configure(tracerProviderBuilder: tracerProviderBuilder); - }); - } - else - { - var tracerProviderBuilder = Sdk.CreateTracerProviderBuilder(); - this.Configure(tracerProviderBuilder: tracerProviderBuilder); - tracerProviderBuilder.Build(); - } - - _isBuilt = true; - } - - private void Configure(TracerProviderBuilder tracerProviderBuilder) - { - tracerProviderBuilder - .SetSampler(new ParentBasedSampler( - rootSampler: new AlwaysOnSampler(), - localParentNotSampled: new AlwaysOnSampler(), - remoteParentNotSampled: new AlwaysOnSampler())) - .AddSource(OpenTelemetryConstants.SourceName) - .AddProcessor(new ActivityProcessor()); - - if (IsAgent365ExporterEnabled()) - { - if (this._useOpenTelemetryBuilder) - { - tracerProviderBuilder.AddAgent365Exporter(exporterType: this._agent365ExporterType); - } - else - { - tracerProviderBuilder.AddAgent365Exporter(serviceCollection: this._services, exporterType: this._agent365ExporterType); - } - } - else if (EnvironmentUtils.IsDevelopmentEnvironment()) - { - tracerProviderBuilder.AddConsoleExporter(); - } - } - } -} \ No newline at end of file diff --git a/src/Observability/Runtime/Common/ActivityExtensions.cs b/src/Observability/Runtime/Common/ActivityExtensions.cs deleted file mode 100644 index 96e86752..00000000 --- a/src/Observability/Runtime/Common/ActivityExtensions.cs +++ /dev/null @@ -1,47 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System.Diagnostics; - -namespace Microsoft.Agents.A365.Observability.Runtime.Common -{ - /// - /// Extension methods for Activity. - /// - public static class ActivityExtensions - { - /// - /// Gets the value of a tag (takes precedence) or baggage item from the activity. - /// - public static string? GetAttributeOrBaggage(this Activity activity, string key) - { - var tagValue = activity.GetTagItem(key); - if (tagValue != null) - { - return tagValue.ToString(); - } - - var baggageValue = activity.GetBaggageItem(key); - return string.IsNullOrEmpty(baggageValue) ? null : baggageValue; - } - - /// - /// Sets a tag on the activity only if it does not already exist. - /// - public static void CoalesceTag(this Activity activity, string key, params string?[] values) - { - var tagValue = activity.GetTagItem(key); - if (tagValue == null) - { - foreach (var value in values) - { - if (!string.IsNullOrEmpty(value)) - { - activity.SetTag(key, value); - break; - } - } - } - } - } -} diff --git a/src/Observability/Runtime/Common/BaggageBuilder.cs b/src/Observability/Runtime/Common/BaggageBuilder.cs deleted file mode 100644 index cca4c610..00000000 --- a/src/Observability/Runtime/Common/BaggageBuilder.cs +++ /dev/null @@ -1,335 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using OpenTelemetry; -using System; -using System.Collections.Generic; -using System.Net; - -namespace Microsoft.Agents.A365.Observability.Runtime.Common -{ - /// - /// Per request baggage builder - /// - /// - /// - /// Learn more about baggage attributes - /// - /// - /// Certification Requirements: To ensure the agent passes certification, the following properties must be set using their respective methods: - /// - /// - /// - /// - /// - /// - /// - /// Scope-Specific Recommendations: The following properties are typically set at individual scopes and should be applied as appropriate: - /// - /// - /// - /// - /// - /// - /// - /// - /// - /// - /// - /// - /// - /// Learn more about certification requirements - /// - /// - public class BaggageBuilder - { - private sealed class Scope : IDisposable - { - private readonly Baggage _previous; - private bool _disposed; - public Scope(Baggage prev) => _previous = prev; - public void Dispose() - { - if (_disposed) return; - Baggage.Current = _previous; - _disposed = true; - } - } - - private readonly Dictionary _pairs = new Dictionary(); - - /// - /// Sets the tenant ID baggage value. - /// - /// - /// This property must be set for the agent to pass certification requirements. - /// - public BaggageBuilder TenantId(string? v) - { - Set(OpenTelemetryConstants.TenantIdKey, v); - return this; - } - - /// - /// Sets the agent ID baggage value. - /// - /// - /// This property should be set to pass certification, but is typically set at individual scopes. - /// - public BaggageBuilder AgentId(string? v) - { - Set(OpenTelemetryConstants.GenAiAgentIdKey, v); - return this; - } - - /// - /// Sets the agent name baggage value. - /// - /// - /// This property should be set to pass certification, but is typically set at individual scopes. - /// - public BaggageBuilder AgentName(string? v) - { - Set(OpenTelemetryConstants.GenAiAgentNameKey, v); - return this; - } - - /// - /// Sets the agent description baggage value. - /// - /// - /// This property should be set to pass certification, but is typically set at individual scopes. - /// - public BaggageBuilder AgentDescription(string? v) - { - Set(OpenTelemetryConstants.GenAiAgentDescriptionKey, v); - return this; - } - - /// - /// Sets the agent version baggage value. - /// - public BaggageBuilder AgentVersion(string? v) - { - Set(OpenTelemetryConstants.GenAiAgentVersionKey, v); - return this; - } - - /// - /// Sets the agentic user ID baggage value. - /// - /// - /// This property should be set to pass certification, but is typically set at individual scopes. - /// - public BaggageBuilder AgenticUserId(string? v) - { - Set(OpenTelemetryConstants.AgentAUIDKey, v); - return this; - } - - /// - /// Sets the agentic user email baggage value. - /// - /// - /// This property should be set to pass certification, but is typically set at individual scopes. - /// - public BaggageBuilder AgenticUserEmail(string? v) - { - Set(OpenTelemetryConstants.AgentEmailKey, v); - return this; - } - - /// - /// Sets the agent blueprint ID baggage value. - /// - /// - /// This property should be set to pass certification, but is typically set at individual scopes. - /// - public BaggageBuilder AgentBlueprintId(string? v) - { - Set(OpenTelemetryConstants.AgentBlueprintIdKey, v); - return this; - } - - /// - /// Sets the agent platform ID baggage value. - /// - public BaggageBuilder AgentPlatformId(string? v) - { - Set(OpenTelemetryConstants.AgentPlatformIdKey, v); - return this; - } - - /// - /// Sets the user ID baggage value. - /// - /// - /// This property should be set to pass certification, but is typically set at individual scopes. - /// - public BaggageBuilder UserId(string? v) - { - Set(OpenTelemetryConstants.UserIdKey, v); - return this; - } - - /// - /// Sets the user email baggage value. - /// - /// - /// This property should be set to pass certification, but is typically set at individual scopes. - /// - public BaggageBuilder UserEmail(string? v) - { - Set(OpenTelemetryConstants.UserEmailKey, v); - return this; - } - - /// - /// Sets the user name baggage value. - /// - /// - /// This property should be set to pass certification, but is typically set at individual scopes. - /// - public BaggageBuilder UserName(string? v) - { - Set(OpenTelemetryConstants.UserNameKey, v); - return this; - } - - /// - /// Sets the user client IP baggage value. - /// - public BaggageBuilder UserClientIp(IPAddress v) - { - Set(OpenTelemetryConstants.CallerClientIpKey, v.ToString()); - return this; - } - - /// - /// Sets the invoke agent server address and port baggage values. - /// - /// The server address (hostname) of the target agent service. - /// Optional server port. Only recorded when different from 443. - /// The current builder instance for method chaining. - public BaggageBuilder InvokeAgentServer(string? address, int? port = null) - { - Set(OpenTelemetryConstants.ServerAddressKey, address); - if (port.HasValue && port.Value != 443) - { - Set(OpenTelemetryConstants.ServerPortKey, port.Value.ToString()); - } - return this; - } - - /// - /// Sets the conversation ID baggage value. - /// - /// - /// This property must be set using this method in order for the agent to pass certification requirements. - /// - public BaggageBuilder ConversationId(string? v) - { - Set(OpenTelemetryConstants.GenAiConversationIdKey, v); - return this; - } - - /// - /// Sets the conversation item link baggage value. - /// - public BaggageBuilder ConversationItemLink(string? v) - { - Set(OpenTelemetryConstants.GenAiConversationItemLinkKey, v); - return this; - } - - /// - /// Sets the channel name baggage value. - /// - /// - /// This property must be set for the agent to pass certification requirements. - /// - public BaggageBuilder ChannelName(string? v) - { - Set(OpenTelemetryConstants.ChannelNameKey, v); - return this; - } - - /// - /// Sets the channel link baggage value. - /// - public BaggageBuilder ChannelLink(string? v) - { - Set(OpenTelemetryConstants.ChannelLinkKey, v); - return this; - } - - /// - /// Sets the session ID baggage value. - /// - public BaggageBuilder SessionId(string? v) - { - Set(OpenTelemetryConstants.SessionIdKey, v); - return this; - } - - /// - /// Sets the session description baggage value. - /// - public BaggageBuilder SessionDescription(string? v) - { - Set(OpenTelemetryConstants.SessionDescriptionKey, v); - return this; - } - - /// - /// Sets the operation source baggage value. - /// To be used with server spans to identify the source of the operation e.g(ACF, ATG) - /// - /// - /// This property must be set for the agent to pass certification requirements. - /// - /// The operation source identifying where the operation originated. - /// The current builder instance for method chaining. - public BaggageBuilder OperationSource(string source) - { - Set(OpenTelemetryConstants.ServiceNameKey, source); - return this; - } - - /// - /// Applies the collected baggage to the current context. - /// - public IDisposable Build() - { - var previous = Baggage.Current; - // Iterate through all key/value pairs and set them in _pairs - foreach (var kvp in _pairs) - { - if (!string.IsNullOrWhiteSpace(kvp.Key)) - { - Baggage.Current = Baggage.Current.SetBaggage(kvp.Key, kvp.Value); - } - } - return new Scope(previous); - } - - /// - /// Adds a baggage key/value if the value is not null or whitespace. - /// - public void Set(string k, string? v) { if (!string.IsNullOrWhiteSpace(v)) _pairs[k] = v; } - - /// - /// Sets multiple baggage values from an enumerable of key-value pairs. - /// - public BaggageBuilder SetRange(IEnumerable> pairs) - { - if (pairs == null) return this; - foreach (var kvp in pairs) - { - Set(kvp.Key, kvp.Value?.ToString()); - } - return this; - } - } -} \ No newline at end of file diff --git a/src/Observability/Runtime/Common/EnvironmentUtils.cs b/src/Observability/Runtime/Common/EnvironmentUtils.cs deleted file mode 100644 index 0cffb654..00000000 --- a/src/Observability/Runtime/Common/EnvironmentUtils.cs +++ /dev/null @@ -1,79 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using System; - -namespace Microsoft.Agents.A365.Observability.Runtime.Common -{ - /// - /// Utility logic for environment-related operations. - /// - public class EnvironmentUtils - { - private const string ProdObservabilityScope = "api://9b975845-388f-4429-889e-eab1ef63949c/.default"; - private const string ProdObservabilityClusterCategory = "prod"; - private const string DevelopmentEnvironmentName = "development"; - - /// - /// Returns the app-only OBS resource scope for authenticating to the observability service. - /// - /// The authentication scope. - public static string[] GetObservabilityAuthenticationScope() - { - var overrideScope = Environment.GetEnvironmentVariable("A365_OBSERVABILITY_SCOPE_OVERRIDE"); - return new[] { !string.IsNullOrEmpty(overrideScope) ? overrideScope : ProdObservabilityScope }; - } - - /// - /// [Deprecated] Returns the app-only OBS resource scope for authenticating to the observability service. - /// - /// Cluster category (deprecated, defaults to production). - /// The authentication scope. - [Obsolete("Cluster category argument is deprecated and will be removed in future versions. Defaults to production.")] - public static string[] GetObservabilityAuthenticationScope(string clusterCategory = ProdObservabilityClusterCategory) - { - // clusterCategory is ignored; always returns production scope - return new[] { ProdObservabilityScope }; - } - - /// - /// Returns the cluster category for the observability service based on the current environment. - /// - /// - public static string GetObservabilityClusterCategory() - { - return ProdObservabilityClusterCategory; - } - - /// - /// [Deprecated] Returns the cluster category for the observability service based on the cluster category. - /// - /// Cluster category (deprecated, defaults to production). - /// - public static string GetObservabilityClusterCategory(string clusterCategory = ProdObservabilityClusterCategory) - { - // clusterCategory is ignored; always returns production category - return ProdObservabilityClusterCategory; - } - - /// - /// Returns true if the current environment is a development environment. - /// - /// - public static bool IsDevelopmentEnvironment() - { - var environment = GetCurrentEnvironment(); - return string.Equals(environment, DevelopmentEnvironmentName, StringComparison.OrdinalIgnoreCase); - } - - /// - /// Gets the current environment name. - /// - /// The current environment name. - private static string GetCurrentEnvironment() - { - return Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT") ?? - Environment.GetEnvironmentVariable("DOTNET_ENVIRONMENT") ?? - DevelopmentEnvironmentName; - } - } -} diff --git a/src/Observability/Runtime/Common/ExportFormatter.cs b/src/Observability/Runtime/Common/ExportFormatter.cs deleted file mode 100644 index a69fdd66..00000000 --- a/src/Observability/Runtime/Common/ExportFormatter.cs +++ /dev/null @@ -1,487 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using Microsoft.Agents.A365.Observability.Runtime.DTOs; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Extensions.Logging; -using Microsoft.Extensions.Logging.Abstractions; -using OpenTelemetry.Resources; -using System; -using System.Collections.Generic; -using System.Diagnostics; -using System.Linq; -using System.Text; -using System.Text.Json; -using System.Text.Json.Serialization; - -namespace Microsoft.Agents.A365.Observability.Runtime.Common -{ - /// - /// Provides functionality to format Activity spans into OTLP JSON payloads. - /// - - public class ExportFormatter - { - private const int MaxSpanSizeBytes = 250 * 1024; - private static readonly string[] LargePayloadAttributeKeys = new[] - { - OpenTelemetryConstants.GenAiToolArgumentsKey, - OpenTelemetryConstants.GenAiToolCallResultKey, - OpenTelemetryConstants.GenAiInputMessagesKey, - OpenTelemetryConstants.GenAiOutputMessagesKey, - AutoInstrumentationConstants.GenAiInvocationInputKey, - AutoInstrumentationConstants.GenAiInvocationOutputKey - }; - - private readonly ILogger _logger; - - /// - /// Initializes a new instance of the class. - /// - /// The logger instance used to log messages during the export formatting process. - public ExportFormatter(ILogger logger) - { - _logger = logger ?? NullLogger.Instance; - } - - /// - /// Formats a collection of Activity spans into an OTLP JSON payload compatible with the Agent 365 Observability ingestion service. - /// - /// The collection of Activity spans to be formatted into the OTLP payload. - /// The OpenTelemetry resource associated with the spans, containing resource attributes. - /// A JSON string representing the OTLP payload for the provided activities and resource. - public string FormatMany(IEnumerable activities, Resource resource) - { - var resourceAttributes = GetResourceAttributes(resource); - var serviceName = GetServiceName(resource); - var serviceVersion = GetServiceVersion(resource); - - var scopeMap = new Dictionary<(string Name, string? Version), List>(); - - foreach (var activity in activities) - { - var key = (activity.Source.Name, activity.Source.Version); - if (!scopeMap.TryGetValue(key, out var spans)) - { - spans = new List(); - scopeMap[key] = spans; - } - spans.Add(BuildOtlpSpanWithTruncation(activity)); - } - - var scopeSpans = new List(scopeMap.Count); - foreach (var kv in scopeMap) - { - scopeSpans.Add(new ScopeSpans - { - Scope = new InstrumentationScope - { - Name = kv.Key.Name, - Version = kv.Key.Version - }, - Spans = kv.Value - }); - } - - var resourceAttrs = MapResourceAttributes(resourceAttributes, serviceName, serviceVersion); - - var payload = new ExportTraceServicePayload - { - ResourceSpans = new List - { - new ResourceSpans - { - Resource = new OtlpResource { Attributes = resourceAttrs }, - ScopeSpans = scopeSpans - } - } - }; - - return SerializePayload(payload); - } - - /// - /// Formats a single Activity span into an OTLP JSON payload compatible with the Agent 365 Observability ingestion service. - /// - /// The Activity span to be formatted into the OTLP payload. - /// The OpenTelemetry resource associated with the span, containing resource attributes. - /// A JSON string representing the OTLP payload for the provided activity and resource. - public string FormatSingle(Activity activity, Resource resource) - { - var resourceAttributes = GetResourceAttributes(resource); - var serviceName = GetServiceName(resource); - var serviceVersion = GetServiceVersion(resource); - - var resourceAttrs = MapResourceAttributes(resourceAttributes, serviceName, serviceVersion); - - var payload = new ExportTraceEtwPayload - { - ResourceSpan = new EtwResourceSpan() - { - Resource = new OtlpResource { Attributes = resourceAttrs }, - ScopeSpan = new EtwScopeSpan() - { - Scope = new InstrumentationScope - { - Name = activity.Source.Name, - Version = activity.Source.Version - }, - Span = BuildOtlpSpan(activity) - } - } - }; - - return SerializePayload(payload); - } - - /// - /// Formats the log data for the OTLP payload. - /// - /// The operation data containing the log information. - /// A JSON string representing the OTLP payload for the log data. - public string FormatLogData(IDictionary data) - { - var payload = new - { - Name = data["Name"], - Attributes = data["Attributes"], - StartTimeUnixNano = data.TryGetValue("StartTime", out var startTimeObj) && startTimeObj != null ? ToUnixNanos(((DateTimeOffset)startTimeObj).UtcDateTime) : 0, - EndTimeUnixNano = data.TryGetValue("EndTime", out var endTimeObj) && endTimeObj != null ? ToUnixNanos(((DateTimeOffset)endTimeObj).UtcDateTime) : 0, - SpanId = data["SpanId"], - ParentSpanId = data["ParentSpanId"], - TraceId = data.TryGetValue("TraceId", out var traceIdObj) ? traceIdObj : null, - Kind = data.TryGetValue("SpanKind", out var spanKindObj) && spanKindObj != null ? spanKindObj : SpanKindConstants.Client, - Status = data.TryGetValue("Status", out var statusObj) && statusObj != null - ? statusObj - : new Dictionary { { "code", 0 }, { "message", "" } } - }; - - return SerializePayload(payload); - } - - private static Dictionary GetResourceAttributes(Resource resource) - { - var resourceAttributes = new Dictionary(); - foreach (var kvp in resource.Attributes) - { - resourceAttributes[kvp.Key] = kvp.Value; - } - return resourceAttributes; - } - - private static string? GetServiceName(Resource resource) - { - return resource.Attributes.FirstOrDefault(a => a.Key == "service.name").Value?.ToString(); - } - - private static string? GetServiceVersion(Resource resource) - { - return resource.Attributes.FirstOrDefault(a => a.Key == "service.version").Value?.ToString(); - } - - private OtlpSpan BuildOtlpSpanWithTruncation(Activity activity) - { - var span = BuildOtlpSpan(activity); - - if (span.Attributes == null) - return span; - - // Check initial size - if (Encoding.UTF8.GetByteCount(SerializePayload(span)) <= MaxSpanSizeBytes) - return span; - - // Gather key sizes - var keySizes = new List<(string Key, int Size, string? Value)>(); - foreach (var key in LargePayloadAttributeKeys) - { - if (span.Attributes.TryGetValue(key, out var valueObj)) - { - var value = valueObj as string; - int size = !string.IsNullOrEmpty(value) ? Encoding.UTF8.GetByteCount(value) : 0; - keySizes.Add((key, size, value)); - _logger?.LogInformation($"Activity '{activity.DisplayName}': Key '{key}' size = {size / 1024} KB."); - } - } - - // Sort keys by size descending - var sorted = keySizes - .Where(k => !string.IsNullOrEmpty(k.Value) && k.Size > 0) - .OrderByDescending(k => k.Size) - .ToList(); - - foreach (var (key, size, _) in sorted) - { - span.Attributes[key] = "TRUNCATED"; - _logger?.LogInformation($"Truncated '{key}' in activity '{activity.DisplayName}' to reduce size. Previous size: {size / 1024} KB."); - - // Re-check size after each truncation - var json = SerializePayload(span); - if (Encoding.UTF8.GetByteCount(json) <= MaxSpanSizeBytes) - { - break; - } - } - - return span; - } - - private OtlpSpan BuildOtlpSpan(Activity activity) - { - return new OtlpSpan - { - TraceId = ToHex(activity.TraceId), - SpanId = ToHex(activity.SpanId), - ParentSpanId = activity.ParentSpanId != default ? ToHex(activity.ParentSpanId) : null, - Name = activity.DisplayName, - Kind = activity.Kind, - StartTimeUnixNano = ToUnixNanos(activity.StartTimeUtc), - EndTimeUnixNano = ToUnixNanos(activity.StartTimeUtc + activity.Duration), - Attributes = MapAttributes(activity), - Events = MapEvents(activity), - Links = MapLinks(activity), - Status = new Dictionary - { - { "code", activity.Status }, - { "message", activity.StatusDescription ?? "" } - } - }; - } - - private static string SerializePayload(T payload) - { - return JsonSerializer.Serialize(payload, new JsonSerializerOptions - { - DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull, - PropertyNamingPolicy = null, - WriteIndented = false - }); - } - - private static string ToHex(ActivityTraceId id) - { - return id.ToHexString().ToLowerInvariant(); - } - - private static string ToHex(ActivitySpanId id) - { - return id.ToHexString().ToLowerInvariant(); - } - - private static ulong ToUnixNanos(DateTime utc) - { - var dt = utc.Kind == DateTimeKind.Utc ? utc : utc.ToUniversalTime(); - var unixEpoch = new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc); - var ns = (dt - unixEpoch).Ticks * 100; - return (ulong)ns; - } - - private static Dictionary MapAttributes(Activity activity) - { - var dict = new Dictionary(); - - foreach (var tag in activity.TagObjects) - { - dict.Add(tag.Key, tag.Value ?? ""); - } - - return dict; - } - - private static List? MapEvents(Activity activity) - { - if (activity.Events is null) return null; - var events = new List(); - foreach (var ev in activity.Events) - { - var attrs = new Dictionary(); - foreach (var tag in ev.Tags) - { - attrs.Add(tag.Key, tag.Value ?? ""); - } - - events.Add(new OtlpEvent - { - TimeUnixNano = ToUnixNanos(ev.Timestamp.UtcDateTime), - Name = ev.Name, - Attributes = attrs.Count > 0 ? attrs : null - }); - } - return events.Count > 0 ? events : null; - } - - private static List? MapLinks(Activity activity) - { - if (activity.Links is null) return null; - var links = new List(); - foreach (var link in activity.Links) - { - var attrs = new Dictionary(); - if (link.Tags != null) - { - foreach (var tag in link.Tags) - { - attrs.Add(tag.Key, tag.Value ?? ""); - } - } - - links.Add(new OtlpLink - { - TraceId = ToHex(link.Context.TraceId), - SpanId = ToHex(link.Context.SpanId), - Attributes = attrs.Count > 0 ? attrs : null - }); - } - return links.Count > 0 ? links : null; - } - - private static Dictionary MapResourceAttributes( - IReadOnlyDictionary attrs, - string? serviceName, - string? serviceVersion) - { - var dict = new Dictionary(); - foreach (var kvp in attrs) - { - dict.Add(kvp.Key, kvp.Value ?? ""); - } - - return dict; - } - } - - #region OTLP JSON DTOs (snake_case with JsonPropertyName) - - // Root request - internal sealed class ExportTraceServicePayload - { - [JsonPropertyName("resourceSpans")] - public List ResourceSpans { get; set; } = new List(); - } - - internal sealed class ExportTraceEtwPayload - { - [JsonPropertyName("resourceSpan")] - public EtwResourceSpan ResourceSpan { get; set; } = new EtwResourceSpan(); - } - - internal sealed class EtwResourceSpan - { - [JsonPropertyName("resource")] - public OtlpResource? Resource { get; set; } - - [JsonPropertyName("scopeSpan")] - public EtwScopeSpan ScopeSpan { get; set; } = new EtwScopeSpan(); - } - - internal sealed class EtwScopeSpan - { - [JsonPropertyName("scope")] - public InstrumentationScope? Scope { get; set; } - - [JsonPropertyName("span")] - public OtlpSpan Span { get; set; } = new OtlpSpan(); - } - - internal sealed class ResourceSpans - { - [JsonPropertyName("resource")] - public OtlpResource? Resource { get; set; } - - [JsonPropertyName("scopeSpans")] - public List ScopeSpans { get; set; } = new List(); - } - - internal sealed class OtlpResource - { - [JsonPropertyName("attributes")] - public Dictionary? Attributes { get; set; } - } - - internal sealed class ScopeSpans - { - [JsonPropertyName("scope")] - public InstrumentationScope? Scope { get; set; } - - [JsonPropertyName("spans")] - public List Spans { get; set; } = new List(); - } - - internal sealed class InstrumentationScope - { - [JsonPropertyName("name")] - public string? Name { get; set; } - - [JsonPropertyName("version")] - public string? Version { get; set; } - } - - internal sealed class OtlpSpan - { - [JsonPropertyName("traceId")] - public string TraceId { get; set; } = default!; // 32-char hex - - [JsonPropertyName("spanId")] - public string SpanId { get; set; } = default!; // 16-char hex - - [JsonPropertyName("parentSpanId")] - public string? ParentSpanId { get; set; } - - [JsonPropertyName("name")] - public string? Name { get; set; } - - [JsonPropertyName("kind")] - public ActivityKind? Kind { get; set; } - - [JsonPropertyName("startTimeUnixNano")] - public ulong StartTimeUnixNano { get; set; } - - [JsonPropertyName("endTimeUnixNano")] - public ulong EndTimeUnixNano { get; set; } - - [JsonPropertyName("attributes")] - public Dictionary? Attributes { get; set; } - - [JsonPropertyName("events")] - public List? Events { get; set; } - - [JsonPropertyName("links")] - public List? Links { get; set; } - - [JsonPropertyName("status")] - public Dictionary? Status { get; set; } - } - - internal sealed class OtlpEvent - { - [JsonPropertyName("timeUnixNano")] - public ulong TimeUnixNano { get; set; } - - [JsonPropertyName("name")] - public string? Name { get; set; } - - [JsonPropertyName("attributes")] - public Dictionary? Attributes { get; set; } - } - - internal sealed class OtlpLink - { - [JsonPropertyName("traceId")] - public string TraceId { get; set; } = default!; - - [JsonPropertyName("spanId")] - public string SpanId { get; set; } = default!; - - [JsonPropertyName("attributes")] - public Dictionary? Attributes { get; set; } - } - - internal sealed class OtlpStatus - { - // STATUS_CODE_UNSET | STATUS_CODE_OK | STATUS_CODE_ERROR - [JsonPropertyName("code")] - public string? Code { get; set; } - - [JsonPropertyName("message")] - public string? Message { get; set; } - } - #endregion -} diff --git a/src/Observability/Runtime/Common/PowerPlatformApiDiscovery.cs b/src/Observability/Runtime/Common/PowerPlatformApiDiscovery.cs deleted file mode 100644 index 9f530506..00000000 --- a/src/Observability/Runtime/Common/PowerPlatformApiDiscovery.cs +++ /dev/null @@ -1,124 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Text.RegularExpressions; - -namespace Microsoft.Agents.A365.Observability.Runtime.Common -{ - /// - /// Provides discovery and endpoint generation for Power Platform APIs across different cluster categories. - /// - public class PowerPlatformApiDiscovery - { - private readonly string clusterCategory; - - /// - /// Initializes a new instance of the class. - /// - /// The cluster category. - public PowerPlatformApiDiscovery(string clusterCategory) - { - this.clusterCategory = clusterCategory ?? throw new ArgumentNullException(nameof(clusterCategory)); - } - - /// - /// Gets the token audience URL for authentication. - /// - /// The token audience URL. - public string GetTokenAudience() - { - return $"https://{GetEnvironmentApiHostNameSuffix()}"; - } - - /// - /// Gets the token endpoint host name. - /// - /// The token endpoint host name. - public string GetTokenEndpointHost() - { - return GetEnvironmentApiHostNameSuffix(); - } - - /// - /// Gets the tenant-specific API endpoint. - /// - /// The tenant identifier. - /// The tenant API endpoint URL. - public string GetTenantEndpoint(string tenantId) - { - return GeneratePowerPlatformApiDomain(tenantId); - } - - /// - /// Gets the tenant island cluster API endpoint. - /// - /// The tenant identifier. - /// The tenant island cluster API endpoint URL. - public string GetTenantIslandClusterEndpoint(string tenantId) - { - return GeneratePowerPlatformApiDomain(tenantId, "il-"); - } - - private string GeneratePowerPlatformApiDomain(string hostNameIdentifier, string hostNamePrefix = "") - { - if (!Regex.IsMatch(hostNameIdentifier, "^[a-zA-Z0-9-]+$")) - { - throw new ArgumentException($"Cannot generate Power Platform API endpoint because the tenant identifier contains invalid host name characters, only alphanumeric and dash characters are expected: {hostNameIdentifier}"); - } - - const string hostNameInfix = "tenant"; - int hexNameSuffixLength = GetHexApiSuffixLength(); - string hexName = hostNameIdentifier.ToLowerInvariant().Replace("-", ""); - - if (hexNameSuffixLength >= hexName.Length) - { - throw new ArgumentException($"Cannot generate Power Platform API endpoint because the normalized tenant identifier must be at least {hexNameSuffixLength + 1} characters in length: {hexName}"); - } - - string hexNameSuffix = hexName.Substring(hexName.Length - hexNameSuffixLength); - string hexNamePrefix = hexName.Substring(0, hexName.Length - hexNameSuffixLength); - string hostNameSuffix = GetEnvironmentApiHostNameSuffix(); - - return $"{hostNamePrefix}{hexNamePrefix}.{hexNameSuffix}.{hostNameInfix}.{hostNameSuffix}"; - } - - private int GetHexApiSuffixLength() - { - switch (clusterCategory) - { - case "firstrelease": - case "production": - case "prod": - return 2; - default: - return 1; - } - } - - private string GetEnvironmentApiHostNameSuffix() - { - switch (clusterCategory) - { - case "firstrelease": - case "production": - case "prod": - return "api.powerplatform.com"; - case "gov": - return "api.gov.powerplatform.microsoft.us"; - case "high": - return "api.high.powerplatform.microsoft.us"; - case "dod": - return "api.appsplatform.us"; - case "mooncake": - return "api.powerplatform.partner.microsoftonline.cn"; - case "ex": - return "api.powerplatform.eaglex.ic.gov"; - case "rx": - return "api.powerplatform.microsoft.scloud"; - default: - throw new ArgumentException($"Invalid ClusterCategory value: {clusterCategory}"); - } - } - } -} diff --git a/src/Observability/Runtime/DTOs/ApplyGuardrailData.cs b/src/Observability/Runtime/DTOs/ApplyGuardrailData.cs deleted file mode 100644 index 04ac5db3..00000000 --- a/src/Observability/Runtime/DTOs/ApplyGuardrailData.cs +++ /dev/null @@ -1,41 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using System; -using System.Collections.Generic; - -namespace Microsoft.Agents.A365.Observability.Runtime.DTOs -{ - /// - /// Encapsulates all telemetry data for an apply_guardrail operation. - /// - public class ApplyGuardrailData : BaseData - { - /// - /// Initializes a new instance of the class. - /// - /// The parent span ID for distributed tracing. - /// The telemetry attributes (tags). - /// Optional custom start time for the operation. - /// Optional custom end time for the operation. - /// Optional span ID for the operation. If not provided one will be created. - /// Optional span kind override. Defaults to null (unset). Use as appropriate. - /// Optional trace ID for distributed tracing. - public ApplyGuardrailData( - string parentSpanId, - IDictionary? attributes = null, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - string? spanId = null, - string? spanKind = null, - string? traceId = null) - : base(attributes, startTime, endTime, spanId, parentSpanId, spanKind, traceId) - { } - - /// - /// Gets the name of the operation. - /// - public override string Name => OpenTelemetryConstants.OperationNames.ApplyGuardrail.ToString(); - } -} diff --git a/src/Observability/Runtime/DTOs/BaseData.cs b/src/Observability/Runtime/DTOs/BaseData.cs deleted file mode 100644 index 9c016497..00000000 --- a/src/Observability/Runtime/DTOs/BaseData.cs +++ /dev/null @@ -1,130 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using System; -using System.Collections.Generic; -using System.Diagnostics; - -namespace Microsoft.Agents.A365.Observability.Runtime.DTOs -{ - /// - /// Encapsulates all telemetry data for an operation, including attributes, timing, and span information. - /// - public abstract class BaseData - { - /// - /// Initializes a new instance - /// - /// The telemetry attributes (tags). - /// Optional custom start time for the operation. - /// Optional custom end time for the operation. - /// Optional span ID for the operation. If not provided one will be created. - /// Optional parent span ID for distributed tracing. - /// Optional span kind for the operation. Use values (e.g., , , ). - /// Optional trace ID for distributed tracing. Groups all spans belonging to the same trace. - public BaseData( - IDictionary? attributes = null, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - string? spanId = null, - string? parentSpanId = null, - string? spanKind = null, - string? traceId = null) - { - Attributes = attributes ?? new Dictionary(); - StartTime = startTime; - EndTime = endTime; - // Generate a random span ID if not provided. Use ActivitySpanId for consistency with tracing. - SpanId = spanId ?? ActivitySpanId.CreateRandom().ToString(); - ParentSpanId = parentSpanId; - SpanKind = spanKind; - TraceId = traceId; - } - - /// - /// Gets the name of the operation. - /// - public abstract string Name { get; } - - /// - /// Gets the telemetry attributes (tags) for the invoke_agent operation. - /// - public IDictionary Attributes { get; } - - /// - /// Gets the custom start time for the operation, if provided. - /// - public DateTimeOffset? StartTime { get; set; } - - /// - /// Gets the custom end time for the operation, if provided. - /// - public DateTimeOffset? EndTime { get; set; } - - /// - /// Gets the span ID for the operation, if provided. - /// - public string SpanId { get; } - - /// - /// Gets the parent span ID for distributed tracing, if provided. - /// - public string? ParentSpanId { get; } - - /// - /// Gets the span kind for the operation, if provided. See for valid values. - /// - public string? SpanKind { get; } - - /// - /// Gets the trace ID for distributed tracing. Groups all spans belonging to the same trace. - /// - public string? TraceId { get; } - - /// - /// Gets or sets the OpenTelemetry status code for the operation. Defaults to - /// (0); set to (2) - /// when an error is recorded. - /// - public SpanStatusCode StatusCode { get; set; } = SpanStatusCode.Unset; - - /// - /// Gets or sets the OpenTelemetry status message. Per the OTel spec this is only populated for an error status. - /// - public string? StatusMessage { get; set; } - - /// - /// Gets the duration of the operation if both start and end times are provided. - /// - public TimeSpan Duration => StartTime.HasValue && EndTime.HasValue - ? EndTime.Value - StartTime.Value - : TimeSpan.Zero; - - /// - /// Converts the telemetry data to a dictionary representation. - /// - public Dictionary ToDictionary() - { - var dict = new Dictionary - { - { "Name", Name }, - { "Attributes", Attributes }, - { "StartTime", StartTime }, - { "EndTime", EndTime }, - { "SpanId", SpanId }, - { "ParentSpanId", ParentSpanId }, - { "TraceId", TraceId }, - { "SpanKind", SpanKind }, - { "Duration", Duration }, - { - "Status", new Dictionary - { - { "code", (int)StatusCode }, - { "message", StatusMessage ?? "" } - } - } - }; - - return dict; - } - } -} diff --git a/src/Observability/Runtime/DTOs/Builders/ApplyGuardrailDataBuilder.cs b/src/Observability/Runtime/DTOs/Builders/ApplyGuardrailDataBuilder.cs deleted file mode 100644 index fd234fb2..00000000 --- a/src/Observability/Runtime/DTOs/Builders/ApplyGuardrailDataBuilder.cs +++ /dev/null @@ -1,128 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using System; -using System.Collections.Generic; -using System.Runtime.Serialization; - -namespace Microsoft.Agents.A365.Observability.Runtime.DTOs.Builders -{ - /// - /// Builds an ApplyGuardrailData instance. - /// - public class ApplyGuardrailDataBuilder : BaseDataBuilder - { - /// - /// Builds complete data for an apply_guardrail operation. - /// - /// The details of the guardrail evaluation. - /// The details of the agent (includes tenant ID). - /// The conversation id. - /// The parent span ID for distributed tracing. - /// Optional custom start time for the operation. - /// Optional custom end time for the operation. - /// Optional span ID for the operation. - /// Optional channel information for the operation. - /// Optional details about the caller. - /// Optional dictionary of extra attributes. - /// Optional span kind override. - /// Optional trace ID for distributed tracing. - /// Optional exception describing a failure; sets an OTel error status and the error.type attribute. - /// An ApplyGuardrailData object containing all telemetry data. - public static ApplyGuardrailData Build( - GuardrailDetails guardrailDetails, - AgentDetails agentDetails, - string conversationId, - string parentSpanId, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - string? spanId = null, - Channel? channel = null, - CallerDetails? callerDetails = null, - IDictionary? extraAttributes = null, - string? spanKind = null, - string? traceId = null, - Exception? error = null) - { - var attributes = BuildAttributes(guardrailDetails, agentDetails, conversationId, channel, callerDetails, extraAttributes); - - return ApplyStatus(new ApplyGuardrailData(parentSpanId, attributes, startTime, endTime, spanId, spanKind, traceId), error); - } - - private static Dictionary BuildAttributes( - GuardrailDetails guardrailDetails, - AgentDetails agentDetails, - string conversationId, - Channel? channel, - CallerDetails? callerDetails, - IDictionary? extraAttributes = null) - { - var attributes = new Dictionary(); - - // SDK attributes - AddSdkAttributes(attributes); - - // Operation name - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiOperationNameKey, OpenTelemetryConstants.ApplyGuardrailOperationName); - - AddAgentDetails(attributes, agentDetails); - - // Guardrail details - AddGuardrailDetails(attributes, guardrailDetails); - - // Conversation - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiConversationIdKey, conversationId); - - // Channel - AddChannelAttributes(attributes, channel); - - // Caller details - AddCallerDetails(attributes, callerDetails); - - // Extra attributes - AddExtraAttributes(attributes, extraAttributes); - - return attributes; - } - - private static void AddGuardrailDetails( - Dictionary attributes, - GuardrailDetails details) - { - // Required attributes - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiSecurityDecisionTypeKey, details.DecisionType.ToString().ToLowerInvariant()); - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiSecurityTargetTypeKey, details.TargetType); - - // Guardian attributes - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiGuardianIdKey, details.GuardianId); - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiGuardianNameKey, details.GuardianName); - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiGuardianProviderNameKey, details.GuardianProviderName); - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiGuardianVersionKey, details.GuardianVersion); - - // Target attributes - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiSecurityTargetIdKey, details.TargetId); - - // Decision attributes - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiSecurityDecisionReasonKey, details.DecisionReason); - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiSecurityDecisionCodeKey, details.DecisionCode); - - // Policy attributes - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiSecurityPolicyIdKey, details.PolicyId); - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiSecurityPolicyNameKey, details.PolicyName); - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiSecurityPolicyVersionKey, details.PolicyVersion); - - // Content attributes - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiSecurityContentInputHashKey, details.ContentInputHash); - if (details.ContentModified.HasValue) - { - attributes[OpenTelemetryConstants.GenAiSecurityContentModifiedKey] = details.ContentModified.Value; - } - - // Correlation - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiSecurityExternalEventIdKey, details.ExternalEventId); - } - - } -} diff --git a/src/Observability/Runtime/DTOs/Builders/BaseDataBuilder.cs b/src/Observability/Runtime/DTOs/Builders/BaseDataBuilder.cs deleted file mode 100644 index 7b9bcafb..00000000 --- a/src/Observability/Runtime/DTOs/Builders/BaseDataBuilder.cs +++ /dev/null @@ -1,188 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using Microsoft.Agents.A365.Observability.Runtime.Tracing; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using System; -using System.Collections.Generic; - -namespace Microsoft.Agents.A365.Observability.Runtime.DTOs.Builders -{ - /// - /// Base class for building telemetry data. - /// - public abstract class BaseDataBuilder where T : BaseData - { - - /// - /// Applies an OpenTelemetry status to the built data based on an optional error, and records - /// the error.type attribute when an error is present. When is - /// null the status is left as . - /// - /// The telemetry data to annotate. - /// Optional exception describing a failure for the operation. - /// The same instance, for fluent use. - protected static T ApplyStatus(T data, Exception? error) - { - var status = SpanStatusBuilder.FromError(error, data.Attributes); - data.StatusCode = status.Code; - data.StatusMessage = status.Message; - return data; - } - - /// - /// Adds attributes for input messages. - /// - protected static void AddInputMessagesAttributes(IDictionary attributes, string[]? messages) - { - if (messages != null && messages.Length > 0) - { - var wrapper = MessageUtils.NormalizeInputMessages(messages); - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiInputMessagesKey, MessageUtils.Serialize(wrapper)); - } - } - - /// - /// Adds attributes for output messages. - /// - protected static void AddOutputMessagesAttributes(IDictionary attributes, string[]? messages) - { - if (messages != null && messages.Length > 0) - { - var wrapper = MessageUtils.NormalizeOutputMessages(messages); - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiOutputMessagesKey, MessageUtils.Serialize(wrapper)); - } - } - - /// - /// Adds agent details to the attributes dictionary, including tenant ID. - /// - protected static void AddAgentDetails(IDictionary attributes, AgentDetails agentDetails) - { - if (agentDetails == null) return; - - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiAgentIdKey, agentDetails.AgentId); - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiAgentNameKey, agentDetails.AgentName); - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiAgentDescriptionKey, agentDetails.AgentDescription); - AddIfNotNull(attributes, OpenTelemetryConstants.AgentAUIDKey, agentDetails.AgenticUserId); - AddIfNotNull(attributes, OpenTelemetryConstants.AgentEmailKey, agentDetails.AgenticUserEmail); - AddIfNotNull(attributes, OpenTelemetryConstants.AgentBlueprintIdKey, agentDetails.AgentBlueprintId); - AddIfNotNull(attributes, OpenTelemetryConstants.AgentPlatformIdKey, agentDetails.AgentPlatformId); - AddIfNotNull(attributes, OpenTelemetryConstants.TenantIdKey, agentDetails.TenantId); - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiProviderNameKey, agentDetails.ProviderName); - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiAgentVersionKey, agentDetails.AgentVersion); - } - - /// - /// Adds endpoint details to the attributes dictionary. - /// - protected static void AddEndpointDetails(IDictionary attributes, Uri? endpoint) - { - if (endpoint == null) return; - - AddIfNotNull(attributes, OpenTelemetryConstants.ServerAddressKey, endpoint.Host); - - // Only record port if it is different from 443 - if (endpoint.Port != 443) - { - AddIfNotNull(attributes, OpenTelemetryConstants.ServerPortKey, endpoint.Port.ToString()); - } - } - - /// - /// Adds request details to the attributes dictionary. - /// - protected static void AddRequestDetails(IDictionary attributes, Request? request) - { - if (request == null) return; - - AddChannelAttributes(attributes, request.Channel); - } - - /// - /// Adds caller details to the attributes dictionary. - /// Extracts user details and caller agent details from the composite CallerDetails. - /// - protected static void AddCallerDetails(IDictionary attributes, CallerDetails? callerDetails) - { - if (callerDetails == null) return; - - var userDetails = callerDetails.UserDetails; - if (userDetails != null) - { - AddIfNotNull(attributes, OpenTelemetryConstants.UserIdKey, userDetails.UserId); - AddIfNotNull(attributes, OpenTelemetryConstants.UserEmailKey, userDetails.UserEmail); - AddIfNotNull(attributes, OpenTelemetryConstants.UserNameKey, userDetails.UserName); - AddIfNotNull(attributes, OpenTelemetryConstants.CallerClientIpKey, userDetails.UserClientIP?.ToString()); - } - - AddCallerAgentDetails(attributes, callerDetails.CallerAgentDetails); - } - - /// - /// Adds caller agent details to the attributes dictionary. - /// - protected static void AddCallerAgentDetails(IDictionary attributes, AgentDetails? callerAgentDetails) - { - if (callerAgentDetails == null) return; - - AddIfNotNull(attributes, OpenTelemetryConstants.CallerAgentNameKey, callerAgentDetails.AgentName); - AddIfNotNull(attributes, OpenTelemetryConstants.CallerAgentIdKey, callerAgentDetails.AgentId); - AddIfNotNull(attributes, OpenTelemetryConstants.CallerAgentBlueprintIdKey, callerAgentDetails.AgentBlueprintId); - AddIfNotNull(attributes, OpenTelemetryConstants.CallerAgentAUIDKey, callerAgentDetails.AgenticUserId); - AddIfNotNull(attributes, OpenTelemetryConstants.CallerAgentEmailKey, callerAgentDetails.AgenticUserEmail); - AddIfNotNull(attributes, OpenTelemetryConstants.CallerAgentPlatformIdKey, callerAgentDetails.AgentPlatformId); - AddIfNotNull(attributes, OpenTelemetryConstants.CallerAgentVersionKey, callerAgentDetails.AgentVersion); - } - - /// - /// Adds channel attributes to the attributes dictionary. - /// - protected static void AddChannelAttributes(IDictionary attributes, Channel? channel) - { - if (channel == null) return; - - AddIfNotNull(attributes, OpenTelemetryConstants.ChannelNameKey, channel.Name); - AddIfNotNull(attributes, OpenTelemetryConstants.ChannelLinkKey, channel.Link); - } - - /// - /// Adds telemetry SDK attributes (name, version, language) to the attributes dictionary. - /// - protected static void AddSdkAttributes(IDictionary attributes) - { - attributes[OpenTelemetryConstants.TelemetrySdkNameKey] = OpenTelemetryConstants.TelemetrySdkNameValue; - attributes[OpenTelemetryConstants.TelemetrySdkVersionKey] = OpenTelemetryConstants.TelemetrySdkVersionValue; - attributes[OpenTelemetryConstants.TelemetrySdkLanguageKey] = OpenTelemetryConstants.TelemetrySdkLanguageValue; - } - - /// - /// Adds a key-value pair to the dictionary if the value is not null. - /// - protected static void AddIfNotNull(IDictionary attributes, string key, object? value) - { - if (value != null) - { - attributes[key] = value; - } - } - - /// - /// Adds extra attributes to the attributes dictionary. - /// Extra attributes cannot override keys already set by the builder on this span. - /// - protected static void AddExtraAttributes(IDictionary attributes, IDictionary? extraAttributes) - { - if (extraAttributes == null) return; - - foreach (var kvp in extraAttributes) - { - if (kvp.Value != null && !attributes.ContainsKey(kvp.Key)) - { - attributes[kvp.Key] = kvp.Value; - } - } - } - } -} diff --git a/src/Observability/Runtime/DTOs/Builders/ExecuteInferenceDataBuilder.cs b/src/Observability/Runtime/DTOs/Builders/ExecuteInferenceDataBuilder.cs deleted file mode 100644 index 68027c2f..00000000 --- a/src/Observability/Runtime/DTOs/Builders/ExecuteInferenceDataBuilder.cs +++ /dev/null @@ -1,122 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using System; -using System.Collections.Generic; -using static Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes.OpenTelemetryConstants; - -namespace Microsoft.Agents.A365.Observability.Runtime.DTOs.Builders -{ - /// - /// Builds an ExecuteInferenceData instance. - /// - public class ExecuteInferenceDataBuilder : BaseDataBuilder - { - /// - /// Builds complete data for an inference operation. - /// - /// The details of the inference call. - /// The details of the agent (includes tenant ID). - /// The conversation id. - /// Optional input messages for the inference. - /// Optional output messages from the inference. - /// Optional custom start time for the operation. - /// Optional custom end time for the operation. - /// Optional span ID for the operation. - /// Optional parent span ID for distributed tracing. - /// Optional channel information for the inference call. - /// Optional agent thought process for the inference. - /// Optional details about the caller. - /// Optional dictionary of extra attributes. - /// Optional trace ID for distributed tracing. - /// Optional exception describing a failure; sets an OTel error status and the error.type attribute. - /// An ExecuteInferenceData object containing all telemetry data. - public static ExecuteInferenceData Build( - InferenceCallDetails inferenceCallDetails, - AgentDetails agentDetails, - string conversationId, - string[]? inputMessages = null, - string[]? outputMessages = null, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - string? spanId = null, - string? parentSpanId = null, - Channel? channel = null, - string? thoughtProcess = null, - CallerDetails? callerDetails = null, - IDictionary? extraAttributes = null, - string? traceId = null, - Exception? error = null) - { - var attributes = BuildAttributes( - inferenceCallDetails, - agentDetails, - conversationId, - inputMessages, - outputMessages, - channel, - thoughtProcess, - callerDetails, - extraAttributes); - - return ApplyStatus(new ExecuteInferenceData(attributes, startTime, endTime, spanId, parentSpanId, traceId), error); - } - - private static Dictionary BuildAttributes( - InferenceCallDetails inferenceCallDetails, - AgentDetails agentDetails, - string conversationId, - string[]? inputMessages, - string[]? outputMessages, - Channel? channel, - string? thoughtProcess, - CallerDetails? callerDetails, - IDictionary? extraAttributes = null) - { - var attributes = new Dictionary(); - - // SDK attributes - AddSdkAttributes(attributes); - - // Agent details (includes tenant ID) - AddAgentDetails(attributes, agentDetails); - - // Inference call details - AddInferenceCallDetails(attributes, inferenceCallDetails); - - // Conversation - AddIfNotNull(attributes, GenAiConversationIdKey, conversationId); - - // Input/output messages - AddInputMessagesAttributes(attributes, inputMessages); - AddOutputMessagesAttributes(attributes, outputMessages); - - // Thought process - AddIfNotNull(attributes, GenAiAgentThoughtProcessKey, thoughtProcess); - - // Channel - AddChannelAttributes(attributes, channel); - - // Add caller details - AddCallerDetails(attributes, callerDetails); - - // Add any extra attributes - AddExtraAttributes(attributes, extraAttributes); - - return attributes; - } - - private static void AddInferenceCallDetails( - IDictionary attributes, - InferenceCallDetails inferenceCallDetails) - { - AddIfNotNull(attributes, GenAiOperationNameKey, inferenceCallDetails.OperationName.ToString().ToLowerInvariant()); - AddIfNotNull(attributes, GenAiRequestModelKey, inferenceCallDetails.Model); - AddIfNotNull(attributes, GenAiProviderNameKey, inferenceCallDetails.ProviderName); - AddIfNotNull(attributes, GenAiUsageInputTokensKey, inferenceCallDetails.InputTokens?.ToString()); - AddIfNotNull(attributes, GenAiUsageOutputTokensKey, inferenceCallDetails.OutputTokens?.ToString()); - AddIfNotNull(attributes, GenAiResponseFinishReasonsKey, inferenceCallDetails.FinishReasons != null ? string.Join(",", inferenceCallDetails.FinishReasons) : null); - } - } -} diff --git a/src/Observability/Runtime/DTOs/Builders/ExecuteToolDataBuilder.cs b/src/Observability/Runtime/DTOs/Builders/ExecuteToolDataBuilder.cs deleted file mode 100644 index 5fe87b6e..00000000 --- a/src/Observability/Runtime/DTOs/Builders/ExecuteToolDataBuilder.cs +++ /dev/null @@ -1,149 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using Microsoft.Agents.A365.Observability.Runtime.Tracing; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using System; -using System.Collections.Generic; -using static Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes.OpenTelemetryConstants; - -namespace Microsoft.Agents.A365.Observability.Runtime.DTOs.Builders -{ - /// - /// Builds an ExecuteToolData instance. - /// - public class ExecuteToolDataBuilder : BaseDataBuilder - { - private const string ExecuteToolOperationName = "execute_tool"; - - /// - /// Builds complete data for an execute_tool operation. - /// - /// The details of the tool call. - /// The details of the agent (includes tenant ID). - /// The conversation id. - /// Optional response content from the tool. - /// Optional custom start time for the operation. - /// Optional custom end time for the operation. - /// Optional span ID for the operation. - /// Optional parent span ID for distributed tracing. - /// Optional channel information for the operation. - /// Optional details about the caller. - /// Optional dictionary of extra attributes. - /// Optional span kind override. Use or as appropriate. - /// Optional trace ID for distributed tracing. - /// Optional exception describing a failure; sets an OTel error status and the error.type attribute. - /// An ExecuteToolData object containing all telemetry data. - public static ExecuteToolData Build( - ToolCallDetails toolCallDetails, - AgentDetails agentDetails, - string conversationId, - string? responseContent = null, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - string? spanId = null, - string? parentSpanId = null, - Channel? channel = null, - CallerDetails? callerDetails = null, - IDictionary? extraAttributes = null, - string? spanKind = null, - string? traceId = null, - Exception? error = null) - { - var attributes = BuildAttributes(toolCallDetails, agentDetails, conversationId, responseContent, channel, callerDetails, extraAttributes); - - return ApplyStatus(new ExecuteToolData(attributes, startTime, endTime, spanId, parentSpanId, spanKind, traceId), error); - } - - private static Dictionary BuildAttributes( - ToolCallDetails toolCallDetails, - AgentDetails agentDetails, - string conversationId, - string? responseContent, - Channel? channel, - CallerDetails? callerDetails, - IDictionary? extraAttributes = null) - { - var attributes = new Dictionary(); - - // SDK attributes - AddSdkAttributes(attributes); - - // Operation name - AddIfNotNull(attributes, GenAiOperationNameKey, ExecuteToolDataBuilder.ExecuteToolOperationName); - - AddAgentDetails(attributes, agentDetails); - - // Tool details - AddToolDetails(attributes, toolCallDetails); - - // Conversation - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiConversationIdKey, conversationId); - - // Response content — ensure JSON object per OTEL spec - if (responseContent != null) - { - if (MessageUtils.IsJson(responseContent)) - { - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiToolCallResultKey, responseContent); - } - else - { - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiToolCallResultKey, - MessageUtils.Serialize(new Dictionary { { "result", responseContent } })); - } - } - - // Channel - AddChannelAttributes(attributes, channel); - - // Add caller details - AddCallerDetails(attributes, callerDetails); - - // Add any extra attributes - AddExtraAttributes(attributes, extraAttributes); - - return attributes; - } - - private static void AddToolDetails( - Dictionary attributes, - ToolCallDetails toolCallDetails) - { - var (toolName, arguments, toolCallId, description, toolType, endpoint, toolServerName) = toolCallDetails; - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiToolNameKey, toolName); - - // Arguments — prefer structured dict, then ensure JSON string per OTEL spec - if (toolCallDetails.ArgumentsObject != null) - { - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiToolArgumentsKey, MessageUtils.Serialize(toolCallDetails.ArgumentsObject)); - } - else if (arguments != null) - { - if (MessageUtils.IsJson(arguments)) - { - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiToolArgumentsKey, arguments); - } - else - { - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiToolArgumentsKey, - MessageUtils.Serialize(new Dictionary { { "arguments", arguments } })); - } - } - - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiToolCallIdKey, toolCallId); - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiToolDescriptionKey, description); - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiToolTypeKey, toolType); - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiToolServerNameKey, toolServerName); - if (endpoint != null) - { - AddIfNotNull(attributes, OpenTelemetryConstants.ServerAddressKey, endpoint.Host); - if (endpoint.Port != 443) - { - AddIfNotNull(attributes, OpenTelemetryConstants.ServerPortKey, endpoint.Port.ToString()); - } - } - } - } -} diff --git a/src/Observability/Runtime/DTOs/Builders/InvokeAgentDataBuilder.cs b/src/Observability/Runtime/DTOs/Builders/InvokeAgentDataBuilder.cs deleted file mode 100644 index 2d425e4c..00000000 --- a/src/Observability/Runtime/DTOs/Builders/InvokeAgentDataBuilder.cs +++ /dev/null @@ -1,132 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using System; -using System.Collections.Generic; -using static Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes.OpenTelemetryConstants; - -namespace Microsoft.Agents.A365.Observability.Runtime.DTOs.Builders -{ - /// - /// Builds an InvokeAgentData instance - /// - public class InvokeAgentDataBuilder : BaseDataBuilder - { - private const string InvokeAgentOperationName = "invoke_agent"; - - /// - /// Builds complete data for an invoke_agent operation. - /// - /// The scope-level details of the agent invocation. - /// The details of the agent (includes tenant ID). - /// The required conversation ID for the agent invocation. - /// The request content for the invoked agent. - /// The details of the caller. - /// Optional input messages to include in the telemetry. - /// Optional output messages to include in the telemetry. - /// Optional custom start time for the operation. - /// Optional custom end time for the operation. - /// Optional span ID for the operation. - /// Optional parent span ID for distributed tracing. - /// Optional dictionary of extra attributes. - /// Optional span kind override. Use or as appropriate. - /// Optional trace ID for distributed tracing. - /// Optional exception describing a failure; sets an OTel error status and the error.type attribute. - /// An InvokeAgentData object containing all telemetry data. - public static InvokeAgentData Build( - InvokeAgentScopeDetails invokeAgentScopeDetails, - AgentDetails agentDetails, - string conversationId, - Request? request = null, - CallerDetails? callerDetails = null, - string[]? inputMessages = null, - string[]? outputMessages = null, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - string? spanId = null, - string? parentSpanId = null, - IDictionary? extraAttributes = null, - string? spanKind = null, - string? traceId = null, - Exception? error = null) - { - var attributes = BuildAttributes( - invokeAgentScopeDetails, - agentDetails, - conversationId, - request, - callerDetails, - inputMessages, - outputMessages, - extraAttributes); - - return ApplyStatus(new InvokeAgentData( - attributes, - startTime, - endTime, - spanId, - parentSpanId, - spanKind, - traceId), error); - } - - /// - /// Builds all attributes for an invoke_agent operation. - /// - /// The scope-level details of the agent invocation. - /// The details of the agent (includes tenant ID). - /// The conversation ID for the agent invocation. - /// The request content for the invoked agent. - /// The details of the caller. - /// Optional input messages to include in the attributes. - /// Optional output messages to include in the attributes. - /// Optional dictionary of extra attributes. - /// A dictionary of attribute key-value pairs. - private static Dictionary BuildAttributes( - InvokeAgentScopeDetails invokeAgentScopeDetails, - AgentDetails agentDetails, - string conversationId, - Request? request = null, - CallerDetails? callerDetails = null, - string[]? inputMessages = null, - string[]? outputMessages = null, - IDictionary? extraAttributes = null) - { - var attributes = new Dictionary(); - - // SDK attributes - AddSdkAttributes(attributes); - - // Operation name - AddIfNotNull(attributes, GenAiOperationNameKey, InvokeAgentDataBuilder.InvokeAgentOperationName); - - // Add agent details (includes tenant ID) - AddAgentDetails(attributes, agentDetails); - - // Add endpoint details - AddEndpointDetails(attributes, invokeAgentScopeDetails.Endpoint); - - // Add request details - AddRequestDetails(attributes, request); - - // Add conversation ID - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiConversationIdKey, conversationId); - - // Add caller details (includes user details and caller agent details) - AddCallerDetails(attributes, callerDetails); - - // Add input messages - AddInputMessagesAttributes(attributes, inputMessages); - - // Add output messages - AddOutputMessagesAttributes(attributes, outputMessages); - - // Add any extra attributes - AddExtraAttributes(attributes, extraAttributes); - - return attributes; - } - } -} diff --git a/src/Observability/Runtime/DTOs/Builders/OutputDataBuilder.cs b/src/Observability/Runtime/DTOs/Builders/OutputDataBuilder.cs deleted file mode 100644 index cc39b907..00000000 --- a/src/Observability/Runtime/DTOs/Builders/OutputDataBuilder.cs +++ /dev/null @@ -1,102 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using Microsoft.Agents.A365.Observability.Runtime.Tracing; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using System; -using System.Collections.Generic; - -namespace Microsoft.Agents.A365.Observability.Runtime.DTOs.Builders -{ - /// - /// Builds an OutputData instance. - /// - public class OutputDataBuilder : BaseDataBuilder - { - private const string OutputMessagesOperationName = "output_messages"; - - /// - /// Builds complete data for an output_messages operation. - /// - /// The details of the agent (includes tenant ID). - /// The response containing output messages. - /// Optional conversation ID for the output operation. - /// Optional channel information for the output operation. - /// Optional details about the caller. - /// Optional custom start time for the operation. - /// Optional custom end time for the operation. - /// Optional span ID for the operation. - /// Optional parent span ID for distributed tracing. - /// Optional dictionary of extra attributes. - /// Optional trace ID for distributed tracing. - /// Optional exception describing a failure; sets an OTel error status and the error.type attribute. - /// An OutputData object containing all telemetry data. - public static OutputData Build( - AgentDetails agentDetails, - Response response, - string? conversationId = null, - Channel? channel = null, - CallerDetails? callerDetails = null, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - string? spanId = null, - string? parentSpanId = null, - IDictionary? extraAttributes = null, - string? traceId = null, - Exception? error = null) - { - var attributes = BuildAttributes(agentDetails, response, conversationId, channel, callerDetails, extraAttributes); - - return ApplyStatus(new OutputData(attributes, startTime, endTime, spanId, parentSpanId, traceId), error); - } - - private static Dictionary BuildAttributes( - AgentDetails agentDetails, - Response response, - string? conversationId, - Channel? channel, - CallerDetails? callerDetails, - IDictionary? extraAttributes = null) - { - var attributes = new Dictionary(); - - // SDK attributes - AddSdkAttributes(attributes); - - // Operation name - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiOperationNameKey, OutputMessagesOperationName); - - AddAgentDetails(attributes, agentDetails); - - // Output messages from response — prefer structured content - if (response.ToolResultObject != null) - { - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiOutputMessagesKey, MessageUtils.Serialize(response.ToolResultObject)); - } - else if (response.OutputContent != null) - { - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiOutputMessagesKey, MessageUtils.Serialize(response.OutputContent)); - } - else if (response.Messages.Count > 0) - { - var wrapper = MessageUtils.NormalizeOutputMessages(response.Messages); - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiOutputMessagesKey, MessageUtils.Serialize(wrapper)); - } - - // Conversation ID - AddIfNotNull(attributes, OpenTelemetryConstants.GenAiConversationIdKey, conversationId); - - // Channel - AddChannelAttributes(attributes, channel); - - // Caller details - AddCallerDetails(attributes, callerDetails); - - // Add any extra attributes - AddExtraAttributes(attributes, extraAttributes); - - return attributes; - } - } -} diff --git a/src/Observability/Runtime/DTOs/Builders/SpanStatusBuilder.cs b/src/Observability/Runtime/DTOs/Builders/SpanStatusBuilder.cs deleted file mode 100644 index 36c571c7..00000000 --- a/src/Observability/Runtime/DTOs/Builders/SpanStatusBuilder.cs +++ /dev/null @@ -1,57 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Collections.Generic; -using Azure; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; - -namespace Microsoft.Agents.A365.Observability.Runtime.DTOs.Builders -{ - /// - /// Builds an OpenTelemetry-compliant from an exception and, for error - /// statuses, records the error.type attribute on the span attributes. - /// - /// - /// Centralizes the exception-to-status mapping so that the ETW DTO logging path stays consistent - /// with the Activity-based scope path (). - /// - public static class SpanStatusBuilder - { - /// - /// Creates a from an optional exception. - /// - /// - /// The exception to record. When null, an status is - /// returned and is left untouched. - /// - /// - /// Optional span attributes dictionary. When an error is supplied and this is non-null, the - /// error.type attribute is written following OTel semantic conventions. - /// - /// - /// An status (with the exception message) when - /// is non-null; otherwise an status. - /// - public static SpanStatus FromError(Exception? error, IDictionary? attributes = null) - { - if (error == null) - { - return new SpanStatus(SpanStatusCode.Unset); - } - - // Mirrors OpenTelemetryScope.RecordError: prefer the HTTP status from a RequestFailedException, - // otherwise fall back to the exception's full type name. - var errorType = error is RequestFailedException requestFailed && requestFailed.Status != 0 - ? requestFailed.Status.ToString() - : error.GetType().FullName ?? "error"; - - if (attributes != null) - { - attributes[OpenTelemetryConstants.ErrorTypeKey] = errorType; - } - - return new SpanStatus(SpanStatusCode.Error, error.Message); - } - } -} diff --git a/src/Observability/Runtime/DTOs/ExecuteInferenceData.cs b/src/Observability/Runtime/DTOs/ExecuteInferenceData.cs deleted file mode 100644 index cb73ba4c..00000000 --- a/src/Observability/Runtime/DTOs/ExecuteInferenceData.cs +++ /dev/null @@ -1,39 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using System; -using System.Collections.Generic; - -namespace Microsoft.Agents.A365.Observability.Runtime.DTOs -{ - /// - /// Encapsulates all telemetry data for an inference execution operation. - /// - public class ExecuteInferenceData : BaseData - { - /// - /// Initializes a new instance of the class. - /// - /// The telemetry attributes (tags). - /// Optional custom start time for the operation. - /// Optional custom end time for the operation. - /// Optional span ID for the operation. If not provided one will be created. - /// Optional parent span ID for distributed tracing. - /// Optional trace ID for distributed tracing. - public ExecuteInferenceData( - IDictionary? attributes = null, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - string? spanId = null, - string? parentSpanId = null, - string? traceId = null) - : base(attributes, startTime, endTime, spanId, parentSpanId, traceId: traceId) - { } - - /// - /// Gets the name of the operation. - /// - public override string Name => OpenTelemetryConstants.OperationNames.ExecuteInference.ToString(); - } -} diff --git a/src/Observability/Runtime/DTOs/ExecuteToolData.cs b/src/Observability/Runtime/DTOs/ExecuteToolData.cs deleted file mode 100644 index fd05140a..00000000 --- a/src/Observability/Runtime/DTOs/ExecuteToolData.cs +++ /dev/null @@ -1,41 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using System; -using System.Collections.Generic; - -namespace Microsoft.Agents.A365.Observability.Runtime.DTOs -{ - /// - /// Encapsulates all telemetry data for an execute_tool operation. - /// - public class ExecuteToolData : BaseData - { - /// - /// Initializes a new instance of the class. - /// - /// The telemetry attributes (tags). - /// Optional custom start time for the operation. - /// Optional custom end time for the operation. - /// Optional span ID for the operation. If not provided one will be created. - /// Optional parent span ID for distributed tracing. - /// Optional span kind override. Defaults to null (unset). Use or as appropriate. - /// Optional trace ID for distributed tracing. - public ExecuteToolData( - IDictionary? attributes = null, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - string? spanId = null, - string? parentSpanId = null, - string? spanKind = null, - string? traceId = null) - : base(attributes, startTime, endTime, spanId, parentSpanId, spanKind, traceId) - { } - - /// - /// Gets the name of the operation. - /// - public override string Name => OpenTelemetryConstants.OperationNames.ExecuteTool.ToString(); - } -} diff --git a/src/Observability/Runtime/DTOs/InvokeAgentData.cs b/src/Observability/Runtime/DTOs/InvokeAgentData.cs deleted file mode 100644 index 718f03f5..00000000 --- a/src/Observability/Runtime/DTOs/InvokeAgentData.cs +++ /dev/null @@ -1,42 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using System; -using System.Collections.Generic; - -namespace Microsoft.Agents.A365.Observability.Runtime.DTOs -{ - /// - /// Encapsulates all telemetry data for an invoke_agent operation, including attributes, timing, and span information. - /// - public class InvokeAgentData : BaseData - { - /// - /// Initializes a new instance of the class. - /// - /// The telemetry attributes (tags). - /// Optional custom start time for the operation. - /// Optional custom end time for the operation. - /// Optional span ID for the operation. If not provided one will be created. - /// Optional parent span ID for distributed tracing. - /// Optional span kind override. Defaults to null (unset). Use or as appropriate. - /// Optional trace ID for distributed tracing. - public InvokeAgentData( - IDictionary? attributes = null, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - string? spanId = null, - string? parentSpanId = null, - string? spanKind = null, - string? traceId = null) - : base(attributes, startTime, endTime, spanId, parentSpanId, spanKind, traceId) - { - } - - /// - /// Gets the name of the operation. - /// - public override string Name => OpenTelemetryConstants.OperationNames.InvokeAgent.ToString(); - } -} diff --git a/src/Observability/Runtime/DTOs/OutputData.cs b/src/Observability/Runtime/DTOs/OutputData.cs deleted file mode 100644 index a1b625f9..00000000 --- a/src/Observability/Runtime/DTOs/OutputData.cs +++ /dev/null @@ -1,39 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using System; -using System.Collections.Generic; - -namespace Microsoft.Agents.A365.Observability.Runtime.DTOs -{ - /// - /// Encapsulates all telemetry data for an output_messages operation. - /// - public class OutputData : BaseData - { - /// - /// Initializes a new instance of the class. - /// - /// The telemetry attributes (tags). - /// Optional custom start time for the operation. - /// Optional custom end time for the operation. - /// Optional span ID for the operation. If not provided one will be created. - /// Optional parent span ID for distributed tracing. - /// Optional trace ID for distributed tracing. - public OutputData( - IDictionary? attributes = null, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - string? spanId = null, - string? parentSpanId = null, - string? traceId = null) - : base(attributes, startTime, endTime, spanId, parentSpanId, traceId: traceId) - { } - - /// - /// Gets the name of the operation. - /// - public override string Name => OpenTelemetryConstants.OperationNames.OutputMessages.ToString(); - } -} diff --git a/src/Observability/Runtime/DTOs/SpanKindConstants.cs b/src/Observability/Runtime/DTOs/SpanKindConstants.cs deleted file mode 100644 index 3c18df91..00000000 --- a/src/Observability/Runtime/DTOs/SpanKindConstants.cs +++ /dev/null @@ -1,38 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Runtime.DTOs -{ - /// - /// Provides string constants for OpenTelemetry span kind values. - /// These constants are used in DTOs and Builders to avoid taking a dependency - /// on . - /// - public static class SpanKindConstants - { - /// - /// Indicates that the span covers server-side handling of a synchronous RPC or other remote request. - /// - public const string Server = "Server"; - - /// - /// Indicates that the span describes a request to some remote service. - /// - public const string Client = "Client"; - - /// - /// Indicates that the span describes a producer sending a message to a broker. - /// - public const string Producer = "Producer"; - - /// - /// Indicates that the span describes a consumer receiving a message from a broker. - /// - public const string Consumer = "Consumer"; - - /// - /// Default span kind. Indicates that the span represents an internal operation within an application. - /// - public const string Internal = "Internal"; - } -} diff --git a/src/Observability/Runtime/DTOs/SpanStatus.cs b/src/Observability/Runtime/DTOs/SpanStatus.cs deleted file mode 100644 index a3621afb..00000000 --- a/src/Observability/Runtime/DTOs/SpanStatus.cs +++ /dev/null @@ -1,34 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Runtime.DTOs -{ - /// - /// Represents an OpenTelemetry span status (code and optional message), - /// following the OTel specification where = 0, - /// = 1, and = 2. - /// - public readonly struct SpanStatus - { - /// - /// Initializes a new instance of the struct. - /// - /// The status code. Defaults to . - /// Optional status message. Per the OTel spec this is only meaningful for an error status. - public SpanStatus(SpanStatusCode code = SpanStatusCode.Unset, string? message = null) - { - Code = code; - Message = message; - } - - /// - /// Gets the status code (, , or ). - /// - public SpanStatusCode Code { get; } - - /// - /// Gets the optional status message. Only populated for an error status. - /// - public string? Message { get; } - } -} diff --git a/src/Observability/Runtime/DTOs/SpanStatusCode.cs b/src/Observability/Runtime/DTOs/SpanStatusCode.cs deleted file mode 100644 index d586c5c3..00000000 --- a/src/Observability/Runtime/DTOs/SpanStatusCode.cs +++ /dev/null @@ -1,30 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Runtime.DTOs -{ - /// - /// Represents the OpenTelemetry span status code as defined by the OTLP specification. - /// The numeric values map directly onto the OTLP Status.StatusCode enum and are - /// emitted as-is on the ETW telemetry payload. - /// - public enum SpanStatusCode - { - /// - /// The default status; the span has not been explicitly marked successful or failed. - /// Maps to OTLP STATUS_CODE_UNSET. - /// - Unset = 0, - - /// - /// The operation completed successfully (explicitly set by the application). - /// Maps to OTLP STATUS_CODE_OK. - /// - Ok = 1, - - /// - /// The operation failed. Maps to OTLP STATUS_CODE_ERROR. - /// - Error = 2, - } -} diff --git a/src/Observability/Runtime/Etw/A365EtwLogger.cs b/src/Observability/Runtime/Etw/A365EtwLogger.cs deleted file mode 100644 index 1c0df7c9..00000000 --- a/src/Observability/Runtime/Etw/A365EtwLogger.cs +++ /dev/null @@ -1,233 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using Microsoft.Agents.A365.Observability.Runtime.DTOs.Builders; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Extensions.Logging; -using System; -using System.Collections.Generic; - -namespace Microsoft.Agents.A365.Observability.Runtime.Etw -{ - /// - /// Provides ETW logging functionality for tracing events. - /// - public class A365EtwLogger : IA365EtwLogger - { - private ILogger logger { get; } - private const string InvokeAgentEventName = "InvokeAgent"; - private static readonly EventId InvokeAgentEventId = new EventId(1001, InvokeAgentEventName); - private const string ExecuteInferenceEventName = "ExecuteInference"; - private static readonly EventId ExecuteInferenceEventId = new EventId(1002, ExecuteInferenceEventName); - private const string ExecuteToolEventName = "ExecuteTool"; - private static readonly EventId ExecuteToolEventId = new EventId(1003, ExecuteToolEventName); - private const string OutputMessagesEventName = "OutputMessages"; - private static readonly EventId OutputMessagesEventId = new EventId(1004, OutputMessagesEventName); - private const string ApplyGuardrailEventName = "ApplyGuardrail"; - private static readonly EventId ApplyGuardrailEventId = new EventId(1005, ApplyGuardrailEventName); - - /// - /// Initializes a new instance of the class. - /// - /// The logger factory. - public A365EtwLogger(ILoggerFactory factory) - { - var baseCategory = typeof(T).FullName!; - logger = factory.CreateLogger(Constants.EtwCategoryPrefix + baseCategory); - } - - /// - public void LogInferenceCall( - InferenceCallDetails inferenceCallDetails, - AgentDetails agentDetails, - string conversationId, - string[]? inputMessages, - string[]? outputMessages, - DateTimeOffset? startTime, - DateTimeOffset? endTime, - string? spanId, - string? parentSpanId, - Channel? channel, - CallerDetails? callerDetails, - string? traceId, - Exception? error = null) - { - var data = ExecuteInferenceDataBuilder.Build( - inferenceCallDetails, - agentDetails, - conversationId, - inputMessages, - outputMessages, - startTime, - endTime, - spanId, - parentSpanId, - channel, - callerDetails: callerDetails, - traceId: traceId, - error: error); - - logger.Log( - LogLevel.Information, - ExecuteInferenceEventId, - data.ToDictionary(), - null, - LogFormatter - ); - } - - /// - public void LogInvokeAgent( - InvokeAgentScopeDetails invokeAgentScopeDetails, - AgentDetails agentDetails, - string conversationId, - Request? request, - CallerDetails? callerDetails, - string[]? inputMessages, - string[]? outputMessages, - DateTimeOffset? startTime, - DateTimeOffset? endTime, - string? spanId, - string? parentSpanId, - string? traceId, - Exception? error = null) - { - var data = InvokeAgentDataBuilder.Build( - invokeAgentScopeDetails, - agentDetails, - conversationId, - request, - callerDetails, - inputMessages, - outputMessages, - startTime, - endTime, - spanId, - parentSpanId, - traceId: traceId, - error: error); - - logger.Log( - LogLevel.Information, - InvokeAgentEventId, - data.ToDictionary(), - null, - LogFormatter - ); - } - - /// - public void LogToolCall( - ToolCallDetails toolCallDetails, - AgentDetails agentDetails, - string conversationId, - string? responseContent, - DateTimeOffset? startTime, - DateTimeOffset? endTime, - string? spanId, - string? parentSpanId, - Channel? channel, - CallerDetails? callerDetails, - string? traceId, - Exception? error = null) - { - var data = ExecuteToolDataBuilder.Build( - toolCallDetails, - agentDetails, - conversationId, - responseContent, - startTime, - endTime, - spanId, - parentSpanId, - channel, - callerDetails: callerDetails, - traceId: traceId, - error: error); - - logger.Log( - LogLevel.Information, - ExecuteToolEventId, - data.ToDictionary(), - null, - LogFormatter - ); - } - - /// - public void LogOutput( - AgentDetails agentDetails, - Response response, - string? conversationId = null, - Channel? channel = null, - CallerDetails? callerDetails = null, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - string? spanId = null, - string? parentSpanId = null, - string? traceId = null, - Exception? error = null) - { - var data = OutputDataBuilder.Build( - agentDetails, - response, - conversationId, - channel, - callerDetails, - startTime, - endTime, - spanId, - parentSpanId, - traceId: traceId, - error: error); - - logger.Log( - LogLevel.Information, - OutputMessagesEventId, - data.ToDictionary(), - null, - LogFormatter - ); - } - - /// - public void LogApplyGuardrail( - GuardrailDetails guardrailDetails, - AgentDetails agentDetails, - string conversationId, - string parentSpanId, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - string? spanId = null, - Channel? channel = null, - CallerDetails? callerDetails = null, - string? traceId = null, - Exception? error = null) - { - var data = ApplyGuardrailDataBuilder.Build( - guardrailDetails, - agentDetails, - conversationId, - parentSpanId, - startTime, - endTime, - spanId, - channel, - callerDetails: callerDetails, - traceId: traceId, - error: error); - - logger.Log( - LogLevel.Information, - ApplyGuardrailEventId, - data.ToDictionary(), - null, - LogFormatter - ); - } - - private static string LogFormatter(Dictionary data, Exception? ex) - { - return $"Name: {data["Name"]}, SpanId: {data["SpanId"]}, ParentSpanId: {data["ParentSpanId"]}, TraceId: {data["TraceId"]}"; - } - } -} diff --git a/src/Observability/Runtime/Etw/Constants.cs b/src/Observability/Runtime/Etw/Constants.cs deleted file mode 100644 index a291c36e..00000000 --- a/src/Observability/Runtime/Etw/Constants.cs +++ /dev/null @@ -1,15 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -namespace Microsoft.Agents.A365.Observability.Runtime.Etw -{ - /// - /// Constants used in ETW logging. - /// - public class Constants - { - /// - /// Category prefix for ETW logs. - /// - public const string EtwCategoryPrefix = "A365.Observability.Etw."; - } -} diff --git a/src/Observability/Runtime/Etw/EtwEventSource.cs b/src/Observability/Runtime/Etw/EtwEventSource.cs deleted file mode 100644 index 0cf82079..00000000 --- a/src/Observability/Runtime/Etw/EtwEventSource.cs +++ /dev/null @@ -1,41 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using System.Diagnostics.Tracing; - -namespace Microsoft.Agents.A365.Observability.Runtime.Etw -{ - /// - /// ETW Event Source for Observability. - /// - [EventSource(Name = "A365-O11y-EventSource")] - public class EtwEventSource : EventSource - { - /// - /// Singleton instance of the EtwEventSource. - /// - public static readonly EtwEventSource Log = new EtwEventSource(EventSourceSettings.ThrowOnEventWriteErrors); - - private EtwEventSource(EventSourceSettings settings) : base(settings) { } - - /// - /// Handler for stopping a span. - /// Writes an ETW event with the necessary information from the span. - /// - [Event(1000, - Level = EventLevel.Informational, - Opcode = EventOpcode.Stop, - Message = "A365 Otel span: Name={0} Id={1} Body={4}")] - public void SpanStop(string name, string spanId, string traceId, string parentSpanId, string content) => - WriteEvent(1000, name, spanId, traceId, parentSpanId, content); - - /// - /// Handler for logging JSON messages. - /// Writes an ETW event with the provided JSON message. - /// - [Event(2000, - Level = EventLevel.Informational, - Message = "{0}")] - public void LogJson(string message) => - WriteEvent(2000, message); - } -} diff --git a/src/Observability/Runtime/Etw/EtwLogProcessor.cs b/src/Observability/Runtime/Etw/EtwLogProcessor.cs deleted file mode 100644 index 219593d0..00000000 --- a/src/Observability/Runtime/Etw/EtwLogProcessor.cs +++ /dev/null @@ -1,49 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using Microsoft.Agents.A365.Observability.Runtime.Common; -using Microsoft.Extensions.Logging; -using OpenTelemetry; -using OpenTelemetry.Logs; -using System.Collections.Generic; - -namespace Microsoft.Agents.A365.Observability.Runtime.Etw -{ - /// - /// Processes logs by emitting ETW events. - /// - public class EtwLogProcessor : BaseProcessor - { - private readonly ExportFormatter _formatter; - private readonly ILogger? _logger; - - /// - /// Initializes a new instance of the class. - /// - /// The formatter used to format log data. - /// The logger used to log messages. - public EtwLogProcessor(ExportFormatter formatter, ILogger? logger = null) - { - _formatter = formatter; - _logger = logger; - } - /// - /// Emits an ETW event with log details. - /// - public override void OnEnd(LogRecord data) - { - var attributes = new Dictionary(); - if (data.Attributes != null) { - foreach (var kvp in data.Attributes) - { - attributes[kvp.Key] = kvp.Value; - } - } - - var jsonContent = _formatter.FormatLogData(attributes); - - _logger?.LogInformation($"EtwLogProcessor: Emitting ETW log event. `Name`: {attributes["Name"]} `SpanId`: {attributes["SpanId"]}"); - - EtwEventSource.Log.LogJson(jsonContent); - } - } -} diff --git a/src/Observability/Runtime/Etw/EtwLoggingBuilder.cs b/src/Observability/Runtime/Etw/EtwLoggingBuilder.cs deleted file mode 100644 index 4e510f92..00000000 --- a/src/Observability/Runtime/Etw/EtwLoggingBuilder.cs +++ /dev/null @@ -1,75 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using Microsoft.Agents.A365.Observability.Runtime.Common; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.Logging; -using OpenTelemetry.Logs; -using System; - -namespace Microsoft.Agents.A365.Observability.Runtime.Etw -{ - /// - /// Builds the ETW + OpenTelemetry logging configuration. - /// - public sealed class EtwLoggingBuilder - { - private static readonly Lazy FallbackConsoleLoggerFactory = new Lazy(() => LoggerFactory.Create(b => b.AddConsole())); - private readonly IServiceCollection _services; - private bool _isBuilt = false; - - /// - /// Initializes a new instance of the class. - /// - /// The service collection to configure. - internal EtwLoggingBuilder(IServiceCollection services) - { - _services = services; - } - - /// - /// Builds the ETW logging configuration and returns the service collection. - /// - /// The configured service collection. - public IServiceCollection Build() - { - EnsureBuilt(); - return _services; - } - - private void EnsureBuilt() - { - if (_isBuilt) - return; - - _services - .AddSingleton(typeof(IA365EtwLogger<>), typeof(A365EtwLogger<>)) - .AddSingleton(sp => - { - var logger = sp.GetService>() ?? FallbackConsoleLoggerFactory.Value.CreateLogger(); - return new ExportFormatter(logger); - }) - .AddLogging(logging => - { - logging.AddOpenTelemetry(otelLogging => - { - otelLogging.ParseStateValues = true; - otelLogging.AddProcessor(sp => - { - return new EtwLogProcessor(formatter: sp.GetRequiredService(), logger: sp.GetService>() ?? FallbackConsoleLoggerFactory.Value.CreateLogger()); - }); - if (EnvironmentUtils.IsDevelopmentEnvironment()) - { - otelLogging.AddConsoleExporter(); - } - }); - }) - .Configure(options => - { - options.AddFilter( - (category, level) => category != null && category.StartsWith(Constants.EtwCategoryPrefix, StringComparison.Ordinal)); - }); - - _isBuilt = true; - } - } -} diff --git a/src/Observability/Runtime/Etw/EtwScopeEventProcessor.cs b/src/Observability/Runtime/Etw/EtwScopeEventProcessor.cs deleted file mode 100644 index b5588e2c..00000000 --- a/src/Observability/Runtime/Etw/EtwScopeEventProcessor.cs +++ /dev/null @@ -1,50 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using Microsoft.Agents.A365.Observability.Runtime.Common; -using Microsoft.Extensions.Logging; -using OpenTelemetry; -using OpenTelemetry.Resources; -using System.Diagnostics; - -namespace Microsoft.Agents.A365.Observability.Runtime.Etw -{ - /// - /// Processes spans by emitting ETW events. - /// - public class EtwScopeEventProcessor: BaseProcessor - { - private readonly ExportFormatter _formatter; - private readonly Resource _resource; - private readonly ILogger? _logger; - - /// - /// Initializes a new instance of the class. - /// - /// The formatter instance used to serialize activity data. - /// The OpenTelemetry resource to use for event formatting. - /// The logger instance used for logging. - public EtwScopeEventProcessor(ExportFormatter formatter, Resource? resource = null, ILogger? logger = null) - { - _formatter = formatter; - _resource = resource ?? ResourceBuilder.CreateEmpty().Build(); - _logger = logger; - } - - /// - /// Called when an activity ends. Emits an ETW event with span details. - /// - public override void OnEnd(Activity data) - { - var activityContent = _formatter.FormatSingle(data, _resource); - - _logger?.LogInformation($"EtwScopeEventProcessor: Emitting SpanStop for `Name`: {data.DisplayName}, `SpanId`: {data.SpanId.ToHexString()}."); - - EtwEventSource.Log.SpanStop( - data.DisplayName, - data.SpanId.ToHexString(), - data.TraceId.ToHexString(), - data.ParentSpanId.ToHexString(), - activityContent); - } - } -} diff --git a/src/Observability/Runtime/Etw/EtwServiceCollectionExtensions.cs b/src/Observability/Runtime/Etw/EtwServiceCollectionExtensions.cs deleted file mode 100644 index eed1ee21..00000000 --- a/src/Observability/Runtime/Etw/EtwServiceCollectionExtensions.cs +++ /dev/null @@ -1,35 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using Microsoft.Extensions.DependencyInjection; -using System; - -namespace Microsoft.Agents.A365.Observability.Runtime.Etw -{ - /// - /// Extension methods for configuring ETW in . - /// - public static class EtwServiceCollectionExtensions - { - /// - /// Adds OpenTelemetry tracing with ETW to the service collection. - /// - public static IServiceCollection AddTracingWithEtw(this IServiceCollection services, Action? configure = null) - { - var builder = new EtwTracingBuilder(services); - - configure?.Invoke(builder); - - return builder.Build(); - } - - /// - /// Adds OpenTelemetry logging with ETW to the service collection. - /// - public static IServiceCollection AddLoggingWithEtw(this IServiceCollection services, Action? configure = null) - { - var builder = new EtwLoggingBuilder(services); - configure?.Invoke(builder); - return builder.Build(); - } - } -} diff --git a/src/Observability/Runtime/Etw/EtwTracingBuilder.cs b/src/Observability/Runtime/Etw/EtwTracingBuilder.cs deleted file mode 100644 index bf0e33b9..00000000 --- a/src/Observability/Runtime/Etw/EtwTracingBuilder.cs +++ /dev/null @@ -1,72 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using Microsoft.Agents.A365.Observability.Runtime.Common; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Processors; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.Logging; -using OpenTelemetry.Trace; -using System; - -namespace Microsoft.Agents.A365.Observability.Runtime.Etw -{ - /// - /// Builds the ETW + OpenTelemetry tracing configuration. - /// - public sealed class EtwTracingBuilder - { - private readonly IServiceCollection _services; - private bool _isBuilt = false; - private static readonly Lazy FallbackConsoleLoggerFactory = new Lazy(() => LoggerFactory.Create(b => b.AddConsole())); - - /// - /// Initializes a new instance of the class. - /// - /// The service collection to configure. - internal EtwTracingBuilder(IServiceCollection services) - { - _services = services; - } - - /// - /// Builds the ETW configuration and returns the service collection. - /// - /// The configured service collection. - public IServiceCollection Build() - { - EnsureBuilt(); - return _services; - } - - private void EnsureBuilt() - { - if (_isBuilt) - return; - - _services - .AddSingleton(sp => - { - var logger = sp.GetService>() ?? FallbackConsoleLoggerFactory.Value.CreateLogger(); - return new ExportFormatter(logger); - }) - .AddOpenTelemetry() - .WithTracing(tracing => - { - tracing - .AddSource(OpenTelemetryConstants.SourceName) - .AddProcessor(new ActivityProcessor()) - .AddProcessor(sp => - { - return new EtwScopeEventProcessor(formatter: sp.GetRequiredService(), logger: sp.GetService>() ?? FallbackConsoleLoggerFactory.Value.CreateLogger()); - }); - - if (EnvironmentUtils.IsDevelopmentEnvironment()) - { - tracing.AddConsoleExporter(); - } - }); - - _isBuilt = true; - } - } -} diff --git a/src/Observability/Runtime/Etw/IA365EtwLogger.cs b/src/Observability/Runtime/Etw/IA365EtwLogger.cs deleted file mode 100644 index bb4b5dde..00000000 --- a/src/Observability/Runtime/Etw/IA365EtwLogger.cs +++ /dev/null @@ -1,159 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using System; - -namespace Microsoft.Agents.A365.Observability.Runtime.Etw -{ - /// - /// Interface for ETW Logger - /// - public interface IA365EtwLogger - { - /// - /// Logs an invoke_agent event. - /// - /// The scope-level details of the agent invocation. - /// The details of the agent (includes tenant ID). - /// The required conversation ID. - /// The request content for the invoked agent. - /// The details of the caller. - /// Optional input messages to include in the log. - /// Optional output messages to include in the log. - /// Optional start time of the inference. - /// Optional end time of the inference. - /// Optional span ID for tracing. - /// Optional parent span ID for tracing. - /// Optional trace ID for distributed tracing. - /// Optional exception describing a failure; sets an OTel error status and the error.type attribute. - public void LogInvokeAgent( - InvokeAgentScopeDetails invokeAgentScopeDetails, - AgentDetails agentDetails, - string conversationId, - Request? request = null, - CallerDetails? callerDetails = null, - string[]? inputMessages = null, - string[]? outputMessages = null, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - string? spanId = null, - string? parentSpanId = null, - string? traceId = null, - Exception? error = null); - - /// - /// Logs an inference event. - /// - /// The details of the inference call. - /// The details of the agent (includes tenant ID). - /// The required conversation ID. - /// Optional input messages to include in the log. - /// Optional output messages to include in the log. - /// Optional start time of the inference. - /// Optional end time of the inference. - /// Optional span ID for tracing. - /// Optional parent span ID for tracing. - /// Optional trace ID for distributed tracing. - /// Optional channel information for the inference call. - /// Optional details of the caller. - /// Optional exception describing a failure; sets an OTel error status and the error.type attribute. - public void LogInferenceCall( - InferenceCallDetails inferenceCallDetails, - AgentDetails agentDetails, - string conversationId, - string[]? inputMessages = null, - string[]? outputMessages = null, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - string? spanId = null, - string? parentSpanId = null, - Channel? channel = null, - CallerDetails? callerDetails = null, - string? traceId = null, - Exception? error = null); - - /// - /// Logs an execute_tool event. - /// - /// The details of the tool call. - /// The details of the agent (includes tenant ID). - /// The required conversation ID. - /// Optional response content to include in the log. - /// Optional start time of the tool execution. - /// Optional end time of the tool execution. - /// Optional span ID for tracing. - /// Optional parent span ID for tracing. - /// Optional trace ID for distributed tracing. - /// Optional channel information for the tool call. - /// Optional details of the caller. - /// Optional exception describing a failure; sets an OTel error status and the error.type attribute. - public void LogToolCall( - ToolCallDetails toolCallDetails, - AgentDetails agentDetails, - string conversationId, - string? responseContent = null, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - string? spanId = null, - string? parentSpanId = null, - Channel? channel = null, - CallerDetails? callerDetails = null, - string? traceId = null, - Exception? error = null); - - /// - /// Logs an output_messages event. - /// - /// The details of the agent (includes tenant ID). - /// The response containing output messages. - /// Optional conversation ID for the output. - /// Optional channel information for the output. - /// Optional details of the caller. - /// Optional start time of the output operation. - /// Optional end time of the output operation. - /// Optional span ID for tracing. - /// Optional parent span ID for tracing. - /// Optional trace ID for distributed tracing. - /// Optional exception describing a failure; sets an OTel error status and the error.type attribute. - public void LogOutput( - AgentDetails agentDetails, - Response response, - string? conversationId = null, - Channel? channel = null, - CallerDetails? callerDetails = null, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - string? spanId = null, - string? parentSpanId = null, - string? traceId = null, - Exception? error = null); - - /// - /// Logs an apply_guardrail event. - /// - /// The details of the guardrail evaluation. - /// The details of the agent (includes tenant ID). - /// The required conversation ID. - /// The required parent span ID for tracing. - /// Optional start time of the guardrail evaluation. - /// Optional end time of the guardrail evaluation. - /// Optional span ID for tracing. - /// Optional channel information. - /// Optional details of the caller. - /// Optional trace ID for distributed tracing. - /// Optional exception describing a failure; sets an OTel error status and the error.type attribute. - public void LogApplyGuardrail( - GuardrailDetails guardrailDetails, - AgentDetails agentDetails, - string conversationId, - string parentSpanId, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - string? spanId = null, - Channel? channel = null, - CallerDetails? callerDetails = null, - string? traceId = null, - Exception? error = null); - } -} diff --git a/src/Observability/Runtime/Microsoft.Agents.A365.Observability.Runtime.csproj b/src/Observability/Runtime/Microsoft.Agents.A365.Observability.Runtime.csproj deleted file mode 100644 index c82bf835..00000000 --- a/src/Observability/Runtime/Microsoft.Agents.A365.Observability.Runtime.csproj +++ /dev/null @@ -1,31 +0,0 @@ - - - Microsoft.Agents.A365.Observability.Runtime - Microsoft.Agents.A365.Observability.Runtime - netstandard2.0 - 8.0 - disable - - Microsoft.Agents.A365.Observability.Runtime - Microsoft Agent 365 Observability Runtime SDK - Observability runtime components for Microsoft Agent 365 SDK, providing base OpenTelemetry tracing and monitoring functionality for AI agents and tools. - Microsoft Agent 365 Observability Runtime SDK - Runtime components - Microsoft Agent 365 Observability Runtime SDK - Microsoft;Agents;A365;Observability;OpenTelemetry;AI;Agents;Tracing;Monitoring;Telemetry - - - - - - - - - - - - - - - - - diff --git a/src/Observability/Runtime/ObservabilityBuilderExtensions.cs b/src/Observability/Runtime/ObservabilityBuilderExtensions.cs deleted file mode 100644 index b4d84d4e..00000000 --- a/src/Observability/Runtime/ObservabilityBuilderExtensions.cs +++ /dev/null @@ -1,52 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -namespace Microsoft.Agents.A365.Observability.Runtime -{ - using Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters; - using Microsoft.Extensions.DependencyInjection; - using Microsoft.Extensions.Hosting; - using Microsoft.Extensions.Logging; - using System; - using System.Linq; - - /// - /// Provides extension methods for configuring Microsoft Agent 365 SDK with OpenTelemetry tracing. - /// - public static class ObservabilityBuilderExtensions - { - - /// - /// Adds the Microsoft Agent 365 SDK with OpenTelemetry tracing for AI agents and tools. - /// - /// The type of the application builder implementing . - /// The application builder to which tracing services will be added. - /// An optional delegate to further configure the tracing builder. - /// Specifies whether to use the OpenTelemetry builder for configuration. Defaults to true. - /// The type of Agent 365 exporter to use for tracing. Defaults to . - /// The original instance with tracing configured. - public static TBuilder AddA365Tracing( - this TBuilder builder, - Action? configure = null, - bool useOpenTelemetryBuilder = true, - Agent365ExporterType agent365ExporterType = Agent365ExporterType.Agent365Exporter) where TBuilder : IHostApplicationBuilder - { - if (!builder.Services.Any(s => s.ServiceType == typeof(Builder))) - { - var localbuilder = new Builder( - services: builder.Services!, - useOpenTelemetryBuilder: useOpenTelemetryBuilder, - agent365ExporterType: agent365ExporterType, - configuration: builder.Configuration); - configure?.Invoke(localbuilder); - localbuilder.Build(); - builder.Services.AddSingleton(localbuilder); - } - else - { - Console.WriteLine("A365 tracing has already been configured. Duplicate call to AddA365Tracing() will be ignored."); - } - - return builder; - } - } -} \ No newline at end of file diff --git a/src/Observability/Runtime/README.md b/src/Observability/Runtime/README.md deleted file mode 100644 index 9b249c36..00000000 --- a/src/Observability/Runtime/README.md +++ /dev/null @@ -1,50 +0,0 @@ -# Microsoft.Agents.A365.Observability.Runtime - -The Runtime package provides runtime components for the Microsoft Agent 365 Observability SDK, including exporters, tracing utilities, DTOs, and scope management. - -## Agent 365 Exporter Authentication - -Agent 365 OBS export is S2S-only. The exporter always posts OTLP traces to -`https://{endpoint}/observabilityService/tenants/{tenantId}/otlp/agents/{agentId}/traces?api-version=1`; -the legacy `Agent365ExporterOptions.UseS2SEndpoint` switch is obsolete and ignored. - -Configure either `TokenResolver` or `ContextualTokenResolver` to return the final app-only -OBS token for the exporting agent and tenant. The default app-only OBS scope is -`api://9b975845-388f-4429-889e-eab1ef63949c/.default`. The SDK never reads a delegated -request token, never performs OBO/user_fic authentication for OBS export, and never falls -back to `/observability` on 401, 403, or 404. The resolver is invoked once per tenant/agent -identity group in each export batch, so a batch that contains several identities invokes it -several times. Cache tokens per agent and tenant and refresh only near expiry. - -Resolvers must validate the returned token before handing it to the exporter: accept -`idtyp=app`, or, when `idtyp` is absent, a non-empty `roles` array or a non-empty `oid` -equal to `sub`; reject any other `idtyp` and any token with an `scp` claim. Also verify the -audience is the Agent 365 OBS resource (`api://9b975845-388f-4429-889e-eab1ef63949c`) and -that the token is not expired. - -## Installation - -```bash -dotnet add package Microsoft.Agents.A365.Observability.Runtime -``` - -## Documentation - -For detailed usage information, configuration examples, and best practices, see the [Microsoft Agents 365 Observability documentation](https://learn.microsoft.com/en-us/microsoft-agent-365/developer/observability?tabs=dotnet). - -## Support - -For issues, questions, or feedback: - -- File issues in the [GitHub Issues](https://github.com/microsoft/Agent365-dotnet/issues) section -- See the [main documentation](../../../README.md) for more information - -## Trademarks - -*Microsoft, Windows, Microsoft Azure and/or other Microsoft products and services referenced in the documentation may be either trademarks or registered trademarks of Microsoft in the United States and/or other countries. The licenses for this project do not grant you rights to use any Microsoft names, logos, or trademarks. Microsoft's general trademark guidelines can be found at http://go.microsoft.com/fwlink/?LinkID=254653.* - -## License - -Copyright (c) Microsoft Corporation. All rights reserved. - -Licensed under the MIT License - see the [LICENSE](../../../LICENSE.md) file for details. \ No newline at end of file diff --git a/src/Observability/Runtime/Tracing/Contracts/AgentDetails.cs b/src/Observability/Runtime/Tracing/Contracts/AgentDetails.cs deleted file mode 100644 index a04cfd38..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/AgentDetails.cs +++ /dev/null @@ -1,224 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Net; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts -{ - /// - /// Details about an AI agent in the system. - /// - public class AgentDetails : IEquatable - { - /// - /// Initializes a new instance of the class. - /// - /// The unique identifier for the agent. - /// Optional display name for the agent. - /// Optional description of the agent's purpose. - /// Optional agentic user ID for the agent. - /// Optional email address for the agentic user. - /// Optional Blueprint/Application ID for the agent. - /// Optional Tenant ID for the agent. - /// Optional agent type. - /// Optional client IP address of the agent. - /// Optional platform ID for the agent. - /// Optional provider name (e.g., openai, anthropic). - /// Optional version of the agent (e.g., "1.0.0", "2025-05-01"). - /// - /// - /// Certification Requirements: The following parameters must be set for the agent to pass certification requirements, and these values override any of the same values specified in the class: - /// - /// - /// - /// - /// - /// - /// - /// - /// - /// - /// While many parameters are optional in the API, they must be provided (not null) to meet certification requirements. Learn more about certification requirements - /// - /// - /// Either or should be provided to identify the agent. - /// - /// - public AgentDetails( - string? agentId = null, - string? agentName = null, - string? agentDescription = null, - string? agenticUserId = null, - string? agenticUserEmail = null, - string? agentBlueprintId = null, - string? tenantId = null, - AgentType? agentType = null, - IPAddress? agentClientIP = null, - string? agentPlatformId = null, - string? providerName = null, - string? agentVersion = null) - { - AgentId = agentId; - AgentName = agentName; - AgentDescription = agentDescription; - AgenticUserId = agenticUserId; - AgenticUserEmail = agenticUserEmail; - AgentBlueprintId = agentBlueprintId; - TenantId = tenantId; - AgentType = agentType; - AgentClientIP = agentClientIP; - AgentPlatformId = agentPlatformId; - ProviderName = providerName; - AgentVersion = agentVersion; - } - - /// - /// The unique identifier for the AI agent. - /// - public string? AgentId { get; } - - /// - /// The human-readable name of the AI agent. - /// - public string? AgentName { get; } - - /// - /// Optional agentic user ID for the agent. - /// - public string? AgenticUserId { get; } - - /// - /// Optional email address for the agentic user. - /// - public string? AgenticUserEmail { get; } - - /// - /// Optional Blueprint/Application ID for the agent. - /// - public string? AgentBlueprintId { get; } - - /// - /// A description of the AI agent's purpose or capabilities. - /// - public string? AgentDescription { get; } - - /// - /// The agent type. - /// - public AgentType? AgentType { get; } - - /// - /// Gets the client IP address of the agent. - /// - public IPAddress? AgentClientIP { get; } - - /// - /// The optional platform ID for the agent. - /// - public string? AgentPlatformId { get; } - - /// - /// Optional Tenant ID for the agent. - /// - public string? TenantId { get; } - - /// - /// Optional provider name (e.g., openai, anthropic). - /// - public string? ProviderName { get; } - - /// - /// Optional version of the agent (e.g., "1.0.0", "2025-05-01"). - /// - public string? AgentVersion { get; } - - /// - /// Deconstructs the current instance into discrete values. - /// - /// Receives the agent identifier. - /// Receives the human-readable agent name. - /// Receives the agent description. - /// Receives the agentic user ID. - /// Receives the agentic user email. - /// Receives the agent Blueprint/Application ID. - /// Receives the agent type. - /// Receives the tenant identifier. - /// Receives the client IP address. - /// Receives the platform ID. - /// Receives the agent version. - public void Deconstruct( - out string? agentId, - out string? agentName, - out string? agentDescription, - out string? agenticUserId, - out string? agenticUserEmail, - out string? agentBlueprintId, - out AgentType? agentType, - out string? tenantId, - out IPAddress? agentClientIP, - out string? agentPlatformId, - out string? agentVersion) - { - agentId = AgentId; - agentName = AgentName; - agentDescription = AgentDescription; - agenticUserId = AgenticUserId; - agenticUserEmail = AgenticUserEmail; - agentBlueprintId = AgentBlueprintId; - agentType = AgentType; - tenantId = TenantId; - agentClientIP = AgentClientIP; - agentPlatformId = AgentPlatformId; - agentVersion = AgentVersion; - } - - /// - public bool Equals(AgentDetails? other) - { - if (other is null) - { - return false; - } - - return string.Equals(AgentId, other.AgentId, StringComparison.Ordinal) && - string.Equals(AgentName, other.AgentName, StringComparison.Ordinal) && - string.Equals(AgentDescription, other.AgentDescription, StringComparison.Ordinal) && - string.Equals(AgenticUserId, other.AgenticUserId, StringComparison.Ordinal) && - string.Equals(AgenticUserEmail, other.AgenticUserEmail, StringComparison.Ordinal) && - string.Equals(AgentBlueprintId, other.AgentBlueprintId, StringComparison.Ordinal) && - AgentType == other.AgentType && - string.Equals(TenantId, other.TenantId, StringComparison.Ordinal) && - Equals(AgentClientIP, other.AgentClientIP) && - string.Equals(AgentPlatformId, other.AgentPlatformId, StringComparison.Ordinal) && - string.Equals(AgentVersion, other.AgentVersion, StringComparison.Ordinal); - } - - /// - public override bool Equals(object? obj) - { - return Equals(obj as AgentDetails); - } - - /// - public override int GetHashCode() - { - unchecked - { - int hash = 17; - hash = (hash * 31) + (AgentId != null ? StringComparer.Ordinal.GetHashCode(AgentId) : 0); - hash = (hash * 31) + (AgentName != null ? StringComparer.Ordinal.GetHashCode(AgentName) : 0); - hash = (hash * 31) + (AgentDescription != null ? StringComparer.Ordinal.GetHashCode(AgentDescription) : 0); - hash = (hash * 31) + (AgenticUserId != null ? StringComparer.Ordinal.GetHashCode(AgenticUserId) : 0); - hash = (hash * 31) + (AgenticUserEmail != null ? StringComparer.Ordinal.GetHashCode(AgenticUserEmail) : 0); - hash = (hash * 31) + (AgentBlueprintId != null ? StringComparer.Ordinal.GetHashCode(AgentBlueprintId) : 0); - hash = (hash * 31) + (AgentType?.GetHashCode() ?? 0); - hash = (hash * 31) + (TenantId != null ? StringComparer.Ordinal.GetHashCode(TenantId) : 0); - hash = (hash * 31) + (AgentClientIP?.GetHashCode() ?? 0); - hash = (hash * 31) + (AgentPlatformId != null ? StringComparer.Ordinal.GetHashCode(AgentPlatformId) : 0); - hash = (hash * 31) + (AgentVersion != null ? StringComparer.Ordinal.GetHashCode(AgentVersion) : 0); - return hash; - } - } - } -} \ No newline at end of file diff --git a/src/Observability/Runtime/Tracing/Contracts/AgentType.cs b/src/Observability/Runtime/Tracing/Contracts/AgentType.cs deleted file mode 100644 index f5105819..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/AgentType.cs +++ /dev/null @@ -1,40 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts -{ - /// - /// Supported agent types for generative AI. - /// - public enum AgentType - { - /// - /// Entra embodied agent. - /// - EntraEmbodied, - - /// - /// Entra non-embodied agent. - /// - EntraNonEmbodied, - - /// - /// Microsoft Copilot agent. - /// - MicrosoftCopilot, - - /// - /// Declarative agent. - /// - DeclarativeAgent, - - /// - /// Foundry agent. - /// - Foundry, - - /// - /// Copilot Studio agent. - /// - CopilotStudio - } -} diff --git a/src/Observability/Runtime/Tracing/Contracts/CallerDetails.cs b/src/Observability/Runtime/Tracing/Contracts/CallerDetails.cs deleted file mode 100644 index bb045481..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/CallerDetails.cs +++ /dev/null @@ -1,45 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts -{ - /// - /// Composite caller details for agent-to-agent (A2A) scenarios. - /// Groups the human caller identity and the calling agent identity together. - /// - /// - /// - /// Certification Requirements: The must be provided (not null) with at least - /// , , and - /// set to meet certification requirements. - /// - /// - /// Learn more about certification requirements - /// - /// - public sealed class CallerDetails - { - /// - /// Initializes a new instance of the class. - /// - /// Details about the human user in the call chain. - /// Details about the calling agent in A2A scenarios. - public CallerDetails( - UserDetails? userDetails = null, - AgentDetails? callerAgentDetails = null) - { - UserDetails = userDetails; - CallerAgentDetails = callerAgentDetails; - } - - /// - /// Gets the details about the human user in the call chain. - /// - public UserDetails? UserDetails { get; } - - /// - /// Gets the details about the calling agent in A2A scenarios. - /// - public AgentDetails? CallerAgentDetails { get; } - } -} diff --git a/src/Observability/Runtime/Tracing/Contracts/GuardrailDecisionType.cs b/src/Observability/Runtime/Tracing/Contracts/GuardrailDecisionType.cs deleted file mode 100644 index aca5bf72..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/GuardrailDecisionType.cs +++ /dev/null @@ -1,44 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System.Runtime.Serialization; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts -{ - /// - /// The decision made by a security guardian during guardrail evaluation. - /// - [DataContract] - public enum GuardrailDecisionType - { - /// - /// Content or action is allowed to proceed. - /// - [EnumMember(Value = "allow")] - Allow, - - /// - /// Content or action is logged for review but allowed to proceed. - /// - [EnumMember(Value = "audit")] - Audit, - - /// - /// Content or action is denied/blocked. - /// - [EnumMember(Value = "deny")] - Deny, - - /// - /// Content was modified (e.g., redacted, sanitized, rewritten). - /// - [EnumMember(Value = "modify")] - Modify, - - /// - /// Content or action triggered a warning but is allowed to proceed. - /// - [EnumMember(Value = "warn")] - Warn - } -} diff --git a/src/Observability/Runtime/Tracing/Contracts/GuardrailDetails.cs b/src/Observability/Runtime/Tracing/Contracts/GuardrailDetails.cs deleted file mode 100644 index 6e10be14..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/GuardrailDetails.cs +++ /dev/null @@ -1,197 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts -{ - /// - /// Details of a guardrail evaluation for security operations tracing. - /// - public sealed class GuardrailDetails : IEquatable - { - /// - /// Initializes a new instance of the class. - /// - /// The type of content or action the guardrail is applied to (required). See for well-known values. - /// The decision made by the guardian (required). - /// Human-readable name of the guardian. - /// Unique identifier of the guardian. - /// Provider of the guardian service (e.g., azure.ai.content_safety). - /// Version of the guardian. - /// Identifier of the target being guarded. - /// Human-readable explanation for the decision. - /// Machine-readable decision code. - /// Identifier of the policy that triggered the decision. - /// Human-readable name of the policy. - /// Version of the policy. - /// Hash of the input content for forensic correlation. - /// Whether content was modified by the guardrail. - /// External correlation identifier for SIEM systems. - public GuardrailDetails( - string targetType, - GuardrailDecisionType decisionType, - string? guardianName = null, - string? guardianId = null, - string? guardianProviderName = null, - string? guardianVersion = null, - string? targetId = null, - string? decisionReason = null, - string? decisionCode = null, - string? policyId = null, - string? policyName = null, - string? policyVersion = null, - string? contentInputHash = null, - bool? contentModified = null, - string? externalEventId = null) - { - TargetType = targetType ?? throw new ArgumentNullException(nameof(targetType)); - DecisionType = decisionType; - GuardianName = guardianName; - GuardianId = guardianId; - GuardianProviderName = guardianProviderName; - GuardianVersion = guardianVersion; - TargetId = targetId; - DecisionReason = decisionReason; - DecisionCode = decisionCode; - PolicyId = policyId; - PolicyName = policyName; - PolicyVersion = policyVersion; - ContentInputHash = contentInputHash; - ContentModified = contentModified; - ExternalEventId = externalEventId; - } - - /// - /// Gets the type of content or action the guardrail is applied to. - /// - /// - public string TargetType { get; } - - /// - /// Gets the decision made by the security guardian. - /// - public GuardrailDecisionType DecisionType { get; } - - /// - /// Gets the human-readable name of the guardian. - /// - public string? GuardianName { get; } - - /// - /// Gets the unique identifier of the guardian. - /// - public string? GuardianId { get; } - - /// - /// Gets the provider of the guardian service. - /// - public string? GuardianProviderName { get; } - - /// - /// Gets the version of the guardian. - /// - public string? GuardianVersion { get; } - - /// - /// Gets the identifier of the target being guarded. - /// - public string? TargetId { get; } - - /// - /// Gets the human-readable explanation for the decision. - /// - public string? DecisionReason { get; } - - /// - /// Gets the machine-readable decision code. - /// - public string? DecisionCode { get; } - - /// - /// Gets the identifier of the policy that triggered the decision. - /// - public string? PolicyId { get; } - - /// - /// Gets the human-readable name of the policy. - /// - public string? PolicyName { get; } - - /// - /// Gets the version of the policy. - /// - public string? PolicyVersion { get; } - - /// - /// Gets the hash of the input content for forensic correlation. - /// - public string? ContentInputHash { get; } - - /// - /// Gets whether content was modified by the guardrail. - /// - public bool? ContentModified { get; } - - /// - /// Gets the external correlation identifier for SIEM systems. - /// - public string? ExternalEventId { get; } - - /// - public bool Equals(GuardrailDetails? other) - { - if (other is null) - { - return false; - } - - return string.Equals(TargetType, other.TargetType, StringComparison.Ordinal) && - DecisionType == other.DecisionType && - string.Equals(GuardianName, other.GuardianName, StringComparison.Ordinal) && - string.Equals(GuardianId, other.GuardianId, StringComparison.Ordinal) && - string.Equals(GuardianProviderName, other.GuardianProviderName, StringComparison.Ordinal) && - string.Equals(GuardianVersion, other.GuardianVersion, StringComparison.Ordinal) && - string.Equals(TargetId, other.TargetId, StringComparison.Ordinal) && - string.Equals(DecisionReason, other.DecisionReason, StringComparison.Ordinal) && - string.Equals(DecisionCode, other.DecisionCode, StringComparison.Ordinal) && - string.Equals(PolicyId, other.PolicyId, StringComparison.Ordinal) && - string.Equals(PolicyName, other.PolicyName, StringComparison.Ordinal) && - string.Equals(PolicyVersion, other.PolicyVersion, StringComparison.Ordinal) && - string.Equals(ContentInputHash, other.ContentInputHash, StringComparison.Ordinal) && - ContentModified == other.ContentModified && - string.Equals(ExternalEventId, other.ExternalEventId, StringComparison.Ordinal); - } - - /// - public override bool Equals(object? obj) - { - return Equals(obj as GuardrailDetails); - } - - /// - public override int GetHashCode() - { - unchecked - { - int hash = 17; - hash = (hash * 31) + (TargetType != null ? StringComparer.Ordinal.GetHashCode(TargetType) : 0); - hash = (hash * 31) + DecisionType.GetHashCode(); - hash = (hash * 31) + (GuardianName != null ? StringComparer.Ordinal.GetHashCode(GuardianName) : 0); - hash = (hash * 31) + (GuardianId != null ? StringComparer.Ordinal.GetHashCode(GuardianId) : 0); - hash = (hash * 31) + (GuardianProviderName != null ? StringComparer.Ordinal.GetHashCode(GuardianProviderName) : 0); - hash = (hash * 31) + (GuardianVersion != null ? StringComparer.Ordinal.GetHashCode(GuardianVersion) : 0); - hash = (hash * 31) + (TargetId != null ? StringComparer.Ordinal.GetHashCode(TargetId) : 0); - hash = (hash * 31) + (DecisionReason != null ? StringComparer.Ordinal.GetHashCode(DecisionReason) : 0); - hash = (hash * 31) + (DecisionCode != null ? StringComparer.Ordinal.GetHashCode(DecisionCode) : 0); - hash = (hash * 31) + (PolicyId != null ? StringComparer.Ordinal.GetHashCode(PolicyId) : 0); - hash = (hash * 31) + (PolicyName != null ? StringComparer.Ordinal.GetHashCode(PolicyName) : 0); - hash = (hash * 31) + (PolicyVersion != null ? StringComparer.Ordinal.GetHashCode(PolicyVersion) : 0); - hash = (hash * 31) + (ContentInputHash != null ? StringComparer.Ordinal.GetHashCode(ContentInputHash) : 0); - hash = (hash * 31) + (ContentModified?.GetHashCode() ?? 0); - hash = (hash * 31) + (ExternalEventId != null ? StringComparer.Ordinal.GetHashCode(ExternalEventId) : 0); - return hash; - } - } - } -} diff --git a/src/Observability/Runtime/Tracing/Contracts/GuardrailFinding.cs b/src/Observability/Runtime/Tracing/Contracts/GuardrailFinding.cs deleted file mode 100644 index 96cef690..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/GuardrailFinding.cs +++ /dev/null @@ -1,134 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts -{ - /// - /// Represents a single security finding detected during guardian evaluation. - /// Multiple findings may be emitted for a single guardrail span. - /// - public sealed class GuardrailFinding : IEquatable - { - /// - /// Initializes a new instance of the class. - /// - /// The category of security risk detected (required). - /// The severity level of the detected risk (required). - /// The decision type for this specific policy finding. - /// Identifier of the policy that triggered the finding. - /// Human-readable name of the triggered policy. - /// Version of the policy. - /// Numeric risk/confidence score (0.0 to 1.0). - /// Non-content metadata about the detected risk (MUST NOT contain PII). - public GuardrailFinding( - string riskCategory, - string riskSeverity, - string? policyDecisionType = null, - string? policyId = null, - string? policyName = null, - string? policyVersion = null, - double? riskScore = null, - string[]? riskMetadata = null) - { - RiskCategory = riskCategory ?? throw new ArgumentNullException(nameof(riskCategory)); - RiskSeverity = riskSeverity ?? throw new ArgumentNullException(nameof(riskSeverity)); - PolicyDecisionType = policyDecisionType; - PolicyId = policyId; - PolicyName = policyName; - PolicyVersion = policyVersion; - RiskScore = riskScore; - RiskMetadata = riskMetadata; - } - - /// - /// Gets the category of security risk detected. - /// - /// - /// Free-form field aligned with OWASP LLM Top 10 2025. - /// Common values: prompt_injection, sensitive_info_disclosure, jailbreak, toxicity, pii. - /// - public string RiskCategory { get; } - - /// - /// Gets the severity level of the detected risk. - /// - /// - public string RiskSeverity { get; } - - /// - /// Gets the decision type for this specific policy finding. - /// - public string? PolicyDecisionType { get; } - - /// - /// Gets the identifier of the policy that triggered the finding. - /// - public string? PolicyId { get; } - - /// - /// Gets the human-readable name of the triggered policy. - /// - public string? PolicyName { get; } - - /// - /// Gets the version of the policy. - /// - public string? PolicyVersion { get; } - - /// - /// Gets the numeric risk/confidence score (0.0 to 1.0). - /// - public double? RiskScore { get; } - - /// - /// Gets non-content metadata about the detected risk. - /// - /// - /// This MUST NOT contain sensitive user content, PII, or other high-risk data. - /// Example values: "field:bcc", "pattern:ssn", "count:3", "position:input[0].content". - /// - public string[]? RiskMetadata { get; } - - /// - public bool Equals(GuardrailFinding? other) - { - if (other is null) - { - return false; - } - - return string.Equals(RiskCategory, other.RiskCategory, StringComparison.Ordinal) && - string.Equals(RiskSeverity, other.RiskSeverity, StringComparison.Ordinal) && - string.Equals(PolicyDecisionType, other.PolicyDecisionType, StringComparison.Ordinal) && - string.Equals(PolicyId, other.PolicyId, StringComparison.Ordinal) && - string.Equals(PolicyName, other.PolicyName, StringComparison.Ordinal) && - string.Equals(PolicyVersion, other.PolicyVersion, StringComparison.Ordinal) && - RiskScore == other.RiskScore; - } - - /// - public override bool Equals(object? obj) - { - return Equals(obj as GuardrailFinding); - } - - /// - public override int GetHashCode() - { - unchecked - { - int hash = 17; - hash = (hash * 31) + (RiskCategory != null ? StringComparer.Ordinal.GetHashCode(RiskCategory) : 0); - hash = (hash * 31) + (RiskSeverity != null ? StringComparer.Ordinal.GetHashCode(RiskSeverity) : 0); - hash = (hash * 31) + (PolicyDecisionType != null ? StringComparer.Ordinal.GetHashCode(PolicyDecisionType) : 0); - hash = (hash * 31) + (PolicyId != null ? StringComparer.Ordinal.GetHashCode(PolicyId) : 0); - hash = (hash * 31) + (PolicyName != null ? StringComparer.Ordinal.GetHashCode(PolicyName) : 0); - hash = (hash * 31) + (PolicyVersion != null ? StringComparer.Ordinal.GetHashCode(PolicyVersion) : 0); - hash = (hash * 31) + (RiskScore?.GetHashCode() ?? 0); - return hash; - } - } - } -} diff --git a/src/Observability/Runtime/Tracing/Contracts/GuardrailRiskSeverity.cs b/src/Observability/Runtime/Tracing/Contracts/GuardrailRiskSeverity.cs deleted file mode 100644 index 9576c2a9..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/GuardrailRiskSeverity.cs +++ /dev/null @@ -1,36 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts -{ - /// - /// Well-known severity levels for security risks detected by guardrails. - /// - public static class GuardrailRiskSeverity - { - /// - /// No risk detected. - /// - public const string None = "none"; - - /// - /// Low severity risk. - /// - public const string Low = "low"; - - /// - /// Medium severity risk. - /// - public const string Medium = "medium"; - - /// - /// High severity risk. - /// - public const string High = "high"; - - /// - /// Critical severity risk requiring immediate action. - /// - public const string Critical = "critical"; - } -} diff --git a/src/Observability/Runtime/Tracing/Contracts/GuardrailTargetType.cs b/src/Observability/Runtime/Tracing/Contracts/GuardrailTargetType.cs deleted file mode 100644 index 7da15ca9..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/GuardrailTargetType.cs +++ /dev/null @@ -1,109 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts -{ - /// - /// Well-known values for the type of content or action a guardrail is applied to. - /// - /// - /// This is a free-form field per the OpenTelemetry semantic conventions. - /// These static members provide discoverability for common values, but custom strings - /// are accepted via implicit conversion (e.g. GuardrailTargetType myType = "custom_target";). - /// - public readonly struct GuardrailTargetType : IEquatable - { - /// - /// Input to a language model. - /// - public static readonly GuardrailTargetType LlmInput = new GuardrailTargetType("llm_input"); - - /// - /// Output from a language model. - /// - public static readonly GuardrailTargetType LlmOutput = new GuardrailTargetType("llm_output"); - - /// - /// A tool call action. - /// - public static readonly GuardrailTargetType ToolCall = new GuardrailTargetType("tool_call"); - - /// - /// A tool definition. - /// - public static readonly GuardrailTargetType ToolDefinition = new GuardrailTargetType("tool_definition"); - - /// - /// A memory store operation. - /// - public static readonly GuardrailTargetType MemoryStore = new GuardrailTargetType("memory_store"); - - /// - /// A memory retrieval operation. - /// - public static readonly GuardrailTargetType MemoryRetrieve = new GuardrailTargetType("memory_retrieve"); - - /// - /// A knowledge query. - /// - public static readonly GuardrailTargetType KnowledgeQuery = new GuardrailTargetType("knowledge_query"); - - /// - /// A knowledge retrieval result. - /// - public static readonly GuardrailTargetType KnowledgeResult = new GuardrailTargetType("knowledge_result"); - - /// - /// A message. - /// - public static readonly GuardrailTargetType Message = new GuardrailTargetType("message"); - - /// - /// Gets the string value of this target type. - /// - public string Value { get; } - - /// - /// Initializes a new instance of the struct. - /// - /// The target type string value. - public GuardrailTargetType(string value) - { - Value = value ?? throw new ArgumentNullException(nameof(value)); - } - - /// - /// Implicitly converts a string to a . - /// - public static implicit operator GuardrailTargetType(string value) => new GuardrailTargetType(value); - - /// - /// Implicitly converts a to its string value. - /// - public static implicit operator string(GuardrailTargetType type) => type.Value; - - /// - public bool Equals(GuardrailTargetType other) => string.Equals(Value, other.Value, StringComparison.Ordinal); - - /// - public override bool Equals(object? obj) => obj is GuardrailTargetType other && Equals(other); - - /// - public override int GetHashCode() => Value != null ? StringComparer.Ordinal.GetHashCode(Value) : 0; - - /// - public override string ToString() => Value; - - /// - /// Equality operator. - /// - public static bool operator ==(GuardrailTargetType left, GuardrailTargetType right) => left.Equals(right); - - /// - /// Inequality operator. - /// - public static bool operator !=(GuardrailTargetType left, GuardrailTargetType right) => !left.Equals(right); - } -} diff --git a/src/Observability/Runtime/Tracing/Contracts/InferenceCallDetails.cs b/src/Observability/Runtime/Tracing/Contracts/InferenceCallDetails.cs deleted file mode 100644 index 4e0fac3e..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/InferenceCallDetails.cs +++ /dev/null @@ -1,145 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Collections.Generic; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts -{ - /// - /// Details of an inference call for generative AI operations. - /// - public sealed class InferenceCallDetails : IEquatable - { - /// - /// Initializes a new instance of the class. - /// - /// Telemetry identifier for the inference operation. - /// Model name used to satisfy the inference request. - /// Provider responsible for the inference call. - /// Optional count of tokens provided as input. - /// Optional count of tokens produced by the model. - /// Optional set of finish reasons supplied by the model. - /// Optional identifier for the model response. - public InferenceCallDetails( - InferenceOperationType operationName, - string model, - string providerName, - int? inputTokens = null, - int? outputTokens = null, - string[]? finishReasons = null, - string? responseId = null) - { - OperationName = operationName; - Model = model; - ProviderName = providerName; - InputTokens = inputTokens; - OutputTokens = outputTokens; - FinishReasons = finishReasons; - ResponseId = responseId; - } - - /// - /// Gets the operation name associated with the inference call. - /// - public InferenceOperationType OperationName { get; } - - /// - /// Gets the language model name used by the call. - /// - public string Model { get; } - - /// - /// Gets the provider responsible for servicing the inference request. - /// - public string ProviderName { get; } - - /// - /// Gets the number of input tokens, when available. - /// - public int? InputTokens { get; } - - /// - /// Gets the number of output tokens, when available. - /// - public int? OutputTokens { get; } - - /// - /// Gets the model-provided finish reasons, when provided. - /// - public string[]? FinishReasons { get; } - - /// - /// Gets the identifier associated with the model's response payload. - /// - public string? ResponseId { get; } - - /// - /// Deconstructs this instance into individual components. - /// - /// Receives the operation name. - /// Receives the model name. - /// Receives the provider name. - /// Receives the input token count. - /// Receives the output token count. - /// Receives the finish reasons. - /// Receives the response identifier. - public void Deconstruct( - out InferenceOperationType operationName, - out string model, - out string providerName, - out int? inputTokens, - out int? outputTokens, - out string[]? finishReasons, - out string? responseId) - { - operationName = OperationName; - model = Model; - providerName = ProviderName; - inputTokens = InputTokens; - outputTokens = OutputTokens; - finishReasons = FinishReasons; - responseId = ResponseId; - } - - /// - public bool Equals(InferenceCallDetails? other) - { - if (other is null) - { - return false; - } - - return OperationName == other.OperationName && - string.Equals(Model, other.Model, StringComparison.Ordinal) && - string.Equals(ProviderName, other.ProviderName, StringComparison.Ordinal) && - InputTokens == other.InputTokens && - OutputTokens == other.OutputTokens && - Equals(FinishReasons, other.FinishReasons) && - string.Equals(ResponseId, other.ResponseId, StringComparison.Ordinal); - } - - /// - public override bool Equals(object? obj) - { - return Equals(obj as InferenceCallDetails); - } - - /// - public override int GetHashCode() - { - unchecked - { - int hash = 17; - hash = (hash * 31) + OperationName.GetHashCode(); - hash = (hash * 31) + (Model != null ? StringComparer.Ordinal.GetHashCode(Model) : 0); - hash = (hash * 31) + (ProviderName != null ? StringComparer.Ordinal.GetHashCode(ProviderName) : 0); - hash = (hash * 31) + (InputTokens?.GetHashCode() ?? 0); - hash = (hash * 31) + (OutputTokens?.GetHashCode() ?? 0); - hash = (hash * 31) + EqualityComparer.Default.GetHashCode(FinishReasons); - hash = (hash * 31) + (ResponseId != null ? StringComparer.Ordinal.GetHashCode(ResponseId) : 0); - return hash; - } - } - } -} \ No newline at end of file diff --git a/src/Observability/Runtime/Tracing/Contracts/InferenceOperationType.cs b/src/Observability/Runtime/Tracing/Contracts/InferenceOperationType.cs deleted file mode 100644 index f494f5f4..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/InferenceOperationType.cs +++ /dev/null @@ -1,26 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts -{ - /// - /// Supported inference operation types for generative AI. - /// - public enum InferenceOperationType - { - /// - /// Chat-based inference operation. - /// - Chat, - - /// - /// Text completion inference operation. - /// - TextCompletion, - - /// - /// Content generation inference operation. - /// - GenerateContent - } -} \ No newline at end of file diff --git a/src/Observability/Runtime/Tracing/Contracts/InvokeAgentDetails.cs b/src/Observability/Runtime/Tracing/Contracts/InvokeAgentDetails.cs deleted file mode 100644 index 3f6a5085..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/InvokeAgentDetails.cs +++ /dev/null @@ -1,51 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Collections.Generic; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts -{ - /// - /// Scope-level configuration for agent invocation tracing. - /// - public sealed class InvokeAgentScopeDetails : IEquatable - { - /// - /// Initializes a new instance of the class. - /// - /// Optional endpoint URI of the agent to invoke. - public InvokeAgentScopeDetails(Uri? endpoint = null) - { - Endpoint = endpoint; - } - - /// - /// The endpoint URI for the AI agent. - /// - public Uri? Endpoint { get; } - - /// - public bool Equals(InvokeAgentScopeDetails? other) - { - if (other is null) - { - return false; - } - - return EqualityComparer.Default.Equals(Endpoint, other.Endpoint); - } - - /// - public override bool Equals(object? obj) - { - return Equals(obj as InvokeAgentScopeDetails); - } - - /// - public override int GetHashCode() - { - return EqualityComparer.Default.GetHashCode(Endpoint); - } - } -} \ No newline at end of file diff --git a/src/Observability/Runtime/Tracing/Contracts/Messages/MessageEnums.cs b/src/Observability/Runtime/Tracing/Contracts/Messages/MessageEnums.cs deleted file mode 100644 index 058d11e7..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/Messages/MessageEnums.cs +++ /dev/null @@ -1,59 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts.Messages -{ - /// - /// Role of a message participant per OTEL gen-ai semantic conventions. - /// - public enum MessageRole - { - /// System message. - System, - - /// User message. - User, - - /// Assistant message. - Assistant, - - /// Tool message. - Tool, - } - - /// - /// Reason a model stopped generating per OTEL gen-ai semantic conventions. - /// - public enum FinishReason - { - /// Normal completion. - Stop, - - /// Token limit reached. - Length, - - /// Content filter triggered. - ContentFilter, - - /// Tool call requested. - ToolCall, - - /// Error occurred. - Error, - } - - /// - /// Media modality for blob, file, and URI parts. - /// - public enum Modality - { - /// Image content. - Image, - - /// Video content. - Video, - - /// Audio content. - Audio, - } -} diff --git a/src/Observability/Runtime/Tracing/Contracts/Messages/MessageParts.cs b/src/Observability/Runtime/Tracing/Contracts/Messages/MessageParts.cs deleted file mode 100644 index 2f9ee58b..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/Messages/MessageParts.cs +++ /dev/null @@ -1,312 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Collections.Generic; -using System.Text.Json; -using System.Text.Json.Serialization; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts.Messages -{ - /// - /// Marker interface for all message part types per OTEL gen-ai semantic conventions. - /// - public interface IMessagePart - { - /// - /// Gets the discriminator string for this part type (e.g. "text", "tool_call"). - /// - string Type { get; } - } - - /// - /// Plain text content part. - /// - public sealed class TextPart : IMessagePart - { - /// - /// Initializes a new instance of the class. - /// - /// The text content. - public TextPart(string content) - { - Content = content ?? throw new ArgumentNullException(nameof(content)); - } - - /// Gets the text content. - public string Content { get; } - - /// - public string Type => "text"; - } - - /// - /// A tool call requested by the model. - /// - public sealed class ToolCallRequestPart : IMessagePart - { - /// - /// Initializes a new instance of the class. - /// - /// The tool name. - /// Optional tool call identifier. - /// Optional arguments for the tool call. - public ToolCallRequestPart(string name, string? id = null, object? arguments = null) - { - Name = name ?? throw new ArgumentNullException(nameof(name)); - Id = id; - Arguments = arguments; - } - - /// Gets the tool name. - public string Name { get; } - - /// Gets the optional tool call identifier. - public string? Id { get; } - - /// Gets the optional arguments for the tool call. - public object? Arguments { get; } - - /// - public string Type => "tool_call"; - } - - /// - /// Result of a tool call. - /// - public sealed class ToolCallResponsePart : IMessagePart - { - /// - /// Initializes a new instance of the class. - /// - /// Optional tool call identifier. - /// Optional tool response payload. - public ToolCallResponsePart(string? id = null, object? response = null) - { - Id = id; - Response = response; - } - - /// Gets the optional tool call identifier. - public string? Id { get; } - - /// Gets the optional tool response payload. - public object? Response { get; } - - /// - public string Type => "tool_call_response"; - } - - /// - /// Model reasoning / chain-of-thought content. - /// - public sealed class ReasoningPart : IMessagePart - { - /// - /// Initializes a new instance of the class. - /// - /// The reasoning content. - public ReasoningPart(string content) - { - Content = content ?? throw new ArgumentNullException(nameof(content)); - } - - /// Gets the reasoning content. - public string Content { get; } - - /// - public string Type => "reasoning"; - } - - /// - /// Inline binary data (base64-encoded). - /// - public sealed class BlobPart : IMessagePart - { - /// - /// Initializes a new instance of the class. - /// - /// The media modality. - /// The base64-encoded binary content. - /// Optional MIME type. - public BlobPart(string modality, string content, string? mimeType = null) - { - Modality = modality ?? throw new ArgumentNullException(nameof(modality)); - Content = content ?? throw new ArgumentNullException(nameof(content)); - MimeType = mimeType; - } - - /// Gets the media modality. - public string Modality { get; } - - /// Gets the base64-encoded binary content. - public string Content { get; } - - /// Gets the optional MIME type. - public string? MimeType { get; } - - /// - public string Type => "blob"; - } - - /// - /// Reference to a pre-uploaded file. - /// - public sealed class FilePart : IMessagePart - { - /// - /// Initializes a new instance of the class. - /// - /// The media modality. - /// The file identifier. - /// Optional MIME type. - public FilePart(string modality, string fileId, string? mimeType = null) - { - Modality = modality ?? throw new ArgumentNullException(nameof(modality)); - FileId = fileId ?? throw new ArgumentNullException(nameof(fileId)); - MimeType = mimeType; - } - - /// Gets the media modality. - public string Modality { get; } - - /// Gets the file identifier. - public string FileId { get; } - - /// Gets the optional MIME type. - public string? MimeType { get; } - - /// - public string Type => "file"; - } - - /// - /// External URI reference. - /// - public sealed class UriPart : IMessagePart - { - /// - /// Initializes a new instance of the class. - /// - /// The media modality. - /// The external URI. - /// Optional MIME type. - public UriPart(string modality, string uri, string? mimeType = null) - { - Modality = modality ?? throw new ArgumentNullException(nameof(modality)); - Uri = uri ?? throw new ArgumentNullException(nameof(uri)); - MimeType = mimeType; - } - - /// Gets the media modality. - public string Modality { get; } - - /// Gets the external URI. - public string Uri { get; } - - /// Gets the optional MIME type. - public string? MimeType { get; } - - /// - public string Type => "uri"; - } - - /// - /// Server-side tool invocation. - /// - public sealed class ServerToolCallPart : IMessagePart - { - /// - /// Initializes a new instance of the class. - /// - /// The tool name. - /// The tool call payload. - /// Optional tool call identifier. - public ServerToolCallPart(string name, IDictionary serverToolCall, string? id = null) - { - Name = name ?? throw new ArgumentNullException(nameof(name)); - ServerToolCall = serverToolCall ?? throw new ArgumentNullException(nameof(serverToolCall)); - Id = id; - } - - /// Gets the tool name. - public string Name { get; } - - /// Gets the tool call payload. - public IDictionary ServerToolCall { get; } - - /// Gets the optional tool call identifier. - public string? Id { get; } - - /// - public string Type => "server_tool_call"; - } - - /// - /// Server-side tool response. - /// - public sealed class ServerToolCallResponsePart : IMessagePart - { - /// - /// Initializes a new instance of the class. - /// - /// The tool response payload. - /// Optional tool call identifier. - public ServerToolCallResponsePart(IDictionary serverToolCallResponse, string? id = null) - { - ServerToolCallResponse = serverToolCallResponse ?? throw new ArgumentNullException(nameof(serverToolCallResponse)); - Id = id; - } - - /// Gets the tool response payload. - public IDictionary ServerToolCallResponse { get; } - - /// Gets the optional tool call identifier. - public string? Id { get; } - - /// - public string Type => "server_tool_call_response"; - } - - /// - /// Extensible part for custom / future types. - /// - public sealed class GenericPart : IMessagePart - { - /// - /// Initializes a new instance of the class. - /// - /// The custom type discriminator. - /// Optional payload dictionary. - public GenericPart(string type, IDictionary? data = null) - { - Type = type ?? throw new ArgumentNullException(nameof(type)); - Data = data ?? new Dictionary(); - } - - /// - public string Type { get; } - - /// Gets the payload dictionary. - public IDictionary Data { get; } - } - - /// - /// Custom JSON converter that serializes using the runtime concrete type, - /// ensuring all properties of derived types (TextPart, ToolCallRequestPart, etc.) are included. - /// - internal sealed class MessagePartConverter : JsonConverter - { - /// - public override IMessagePart Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) - { - throw new NotSupportedException("Deserialization of IMessagePart is not supported."); - } - - /// - public override void Write(Utf8JsonWriter writer, IMessagePart value, JsonSerializerOptions options) - { - JsonSerializer.Serialize(writer, value, value.GetType(), options); - } - } -} diff --git a/src/Observability/Runtime/Tracing/Contracts/Messages/MessageTypes.cs b/src/Observability/Runtime/Tracing/Contracts/Messages/MessageTypes.cs deleted file mode 100644 index edf205f2..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/Messages/MessageTypes.cs +++ /dev/null @@ -1,61 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Collections.Generic; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts.Messages -{ - /// - /// An input message sent to a model (OTEL gen-ai semantic conventions). - /// - public class ChatMessage - { - /// - /// Initializes a new instance of the class. - /// - /// The role of the message sender. - /// The message parts. - /// Optional participant name. - public ChatMessage(MessageRole role, IReadOnlyList parts, string? name = null) - { - Role = role; - Parts = parts ?? throw new ArgumentNullException(nameof(parts)); - Name = name; - } - - /// Gets the role of the message sender. - public MessageRole Role { get; } - - /// Gets the message parts. - public IReadOnlyList Parts { get; } - - /// Gets the optional participant name. - public string? Name { get; } - } - - /// - /// An output message produced by a model (OTEL gen-ai semantic conventions). - /// - public class OutputMessage : ChatMessage - { - private const string DefaultFinishReason = "stop"; - - /// - /// Initializes a new instance of the class. - /// - /// The role of the message sender. - /// The message parts. - /// Optional participant name. - /// Reason the model stopped generating. Defaults to "stop" per OTel spec. - public OutputMessage(MessageRole role, IReadOnlyList parts, string? name = null, string? finishReason = null) - : base(role, parts, name) - { - FinishReason = finishReason ?? DefaultFinishReason; - } - - /// Gets the reason the model stopped generating (required per OTel spec). - public string FinishReason { get; } - } - -} diff --git a/src/Observability/Runtime/Tracing/Contracts/Messages/MessageWrappers.cs b/src/Observability/Runtime/Tracing/Contracts/Messages/MessageWrappers.cs deleted file mode 100644 index 243887e9..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/Messages/MessageWrappers.cs +++ /dev/null @@ -1,86 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Collections.Generic; -using System.Text.Json; -using System.Text.Json.Serialization; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts.Messages -{ - /// - /// Represents the list of input messages sent to the model. - /// Serializes as a plain JSON array per OTel gen-ai semantic conventions. - /// - [JsonConverter(typeof(InputMessagesConverter))] - public sealed class InputMessages - { - /// - /// Initializes a new instance of the class. - /// - /// The input chat messages. - public InputMessages(IReadOnlyList messages) - { - Messages = messages ?? throw new ArgumentNullException(nameof(messages)); - } - - /// Gets the input chat messages. - public IReadOnlyList Messages { get; } - } - - /// - /// Represents the list of output messages generated by the model. - /// Serializes as a plain JSON array per OTel gen-ai semantic conventions. - /// - [JsonConverter(typeof(OutputMessagesConverter))] - public sealed class OutputMessages - { - /// - /// Initializes a new instance of the class. - /// - /// The output messages. - public OutputMessages(IReadOnlyList messages) - { - Messages = messages ?? throw new ArgumentNullException(nameof(messages)); - } - - /// Gets the output messages. - public IReadOnlyList Messages { get; } - } - - /// - /// Serializes as a plain JSON array of ChatMessage objects. - /// - internal sealed class InputMessagesConverter : JsonConverter - { - /// - public override InputMessages Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) - { - throw new NotSupportedException("Deserialization of InputMessages is not supported."); - } - - /// - public override void Write(Utf8JsonWriter writer, InputMessages value, JsonSerializerOptions options) - { - JsonSerializer.Serialize(writer, value.Messages, options); - } - } - - /// - /// Serializes as a plain JSON array of OutputMessage objects. - /// - internal sealed class OutputMessagesConverter : JsonConverter - { - /// - public override OutputMessages Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) - { - throw new NotSupportedException("Deserialization of OutputMessages is not supported."); - } - - /// - public override void Write(Utf8JsonWriter writer, OutputMessages value, JsonSerializerOptions options) - { - JsonSerializer.Serialize(writer, value.Messages, options); - } - } -} diff --git a/src/Observability/Runtime/Tracing/Contracts/OperationSource.cs b/src/Observability/Runtime/Tracing/Contracts/OperationSource.cs deleted file mode 100644 index 3b18afa9..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/OperationSource.cs +++ /dev/null @@ -1,26 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts -{ - /// - /// Enum representing the source of an operation. - /// - public enum OperationSource - { - /// - /// Operation executed by SDK. - /// - SDK, - - /// - /// Operation executed by Gateway. - /// - Gateway, - - /// - /// Operation executed by MCP Server. - /// - MCPServer - } -} diff --git a/src/Observability/Runtime/Tracing/Contracts/Request.cs b/src/Observability/Runtime/Tracing/Contracts/Request.cs deleted file mode 100644 index b314d20e..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/Request.cs +++ /dev/null @@ -1,201 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Collections.Generic; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts.Messages; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts -{ - /// - /// Represents channel information for agent execution context. - /// - public sealed class Channel : IEquatable - { - /// - /// Initializes a new instance of the class. - /// - /// Human-readable name of the channel. - /// Optional link for the channel. - public Channel(string? name, string? link = null) - { - Name = name; - Link = link; - } - - /// - /// Gets the human-readable name for the channel. - /// - public string? Name { get; } - - /// - /// Gets an optional link for the channel. - /// - public string? Link { get; } - - /// - /// Deconstructs this instance for tuple deconstruction support. - /// - /// Receives the channel name. - /// Receives the link. - public void Deconstruct(out string? name, out string? link) - { - name = Name; - link = Link; - } - - /// - public bool Equals(Channel? other) - { - if (other is null) - { - return false; - } - - return string.Equals(Name, other.Name, StringComparison.Ordinal) && - string.Equals(Link, other.Link, StringComparison.Ordinal); - } - - /// - public override bool Equals(object? obj) - { - return Equals(obj as Channel); - } - - /// - public override int GetHashCode() - { - unchecked - { - int hash = 17; - hash = (hash * 31) + (Name != null ? StringComparer.Ordinal.GetHashCode(Name) : 0); - hash = (hash * 31) + (Link != null ? StringComparer.Ordinal.GetHashCode(Link) : 0); - return hash; - } - } - } - - /// - /// Represents a request to an AI agent with telemetry context. - /// - public sealed class Request : IEquatable - { - /// - /// Initializes a new instance of the class. - /// - /// The payload content supplied to the agent. - /// Optional session identifier. - /// Optional channel information describing request origin. - /// Optional conversation or session correlation ID. - /// Optional source of the operation (e.g., SDK, Gateway, MCPServer). - public Request(string? content = null, string? sessionId = null, Channel? channel = null, string? conversationId = null, string? operationSource = null) - { - Content = content; - SessionId = sessionId; - Channel = channel; - ConversationId = conversationId; - OperationSource = operationSource; - } - - /// - /// Initializes a new instance of the class with structured input content. - /// - /// The structured input messages for the agent. - /// Optional session identifier. - /// Optional channel information describing request origin. - /// Optional conversation or session correlation ID. - /// Optional source of the operation (e.g., SDK, Gateway, MCPServer). - public Request(InputMessages inputContent, string? sessionId = null, Channel? channel = null, string? conversationId = null, string? operationSource = null) - { - InputContent = inputContent ?? throw new ArgumentNullException(nameof(inputContent)); - SessionId = sessionId; - Channel = channel; - ConversationId = conversationId; - OperationSource = operationSource; - } - - /// - /// Gets the textual content of the request. - /// - public string? Content { get; } - - /// - /// Gets the structured input messages, when supplied. - /// Takes precedence over for telemetry recording. - /// - public InputMessages? InputContent { get; } - - /// - /// Gets the session identifier, when supplied. - /// - public string? SessionId { get; } - - /// - /// Gets channel information describing the origin of the request. - /// - public Channel? Channel { get; } - - /// - /// Gets the conversation or session correlation ID. - /// - public string? ConversationId { get; } - - /// - /// Gets the source of the operation, when supplied. - /// - public string? OperationSource { get; } - - /// - /// Deconstructs the request for tuple deconstruction support. - /// - /// Receives the request content. - /// Receives the session identifier. - /// Receives the channel information. - /// Receives the conversation ID. - public void Deconstruct(out string? content, out string? sessionId, out Channel? channel, out string? conversationId) - { - content = Content; - sessionId = SessionId; - channel = Channel; - conversationId = ConversationId; - } - - /// - public bool Equals(Request? other) - { - if (other is null) - { - return false; - } - - return string.Equals(Content, other.Content, StringComparison.Ordinal) && - string.Equals(SessionId, other.SessionId, StringComparison.Ordinal) && - EqualityComparer.Default.Equals(Channel, other.Channel) && - string.Equals(ConversationId, other.ConversationId, StringComparison.Ordinal) && - string.Equals(OperationSource, other.OperationSource, StringComparison.Ordinal) && - ReferenceEquals(InputContent, other.InputContent); - } - - /// - public override bool Equals(object? obj) - { - return Equals(obj as Request); - } - - /// - public override int GetHashCode() - { - unchecked - { - int hash = 17; - hash = (hash * 31) + (Content != null ? StringComparer.Ordinal.GetHashCode(Content) : 0); - hash = (hash * 31) + (SessionId != null ? StringComparer.Ordinal.GetHashCode(SessionId) : 0); - hash = (hash * 31) + EqualityComparer.Default.GetHashCode(Channel); - hash = (hash * 31) + (ConversationId != null ? StringComparer.Ordinal.GetHashCode(ConversationId) : 0); - hash = (hash * 31) + (OperationSource != null ? StringComparer.Ordinal.GetHashCode(OperationSource) : 0); - hash = (hash * 31) + (InputContent != null ? InputContent.GetHashCode() : 0); - return hash; - } - } - } -} \ No newline at end of file diff --git a/src/Observability/Runtime/Tracing/Contracts/Response.cs b/src/Observability/Runtime/Tracing/Contracts/Response.cs deleted file mode 100644 index 5a7fcb59..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/Response.cs +++ /dev/null @@ -1,120 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Collections.Generic; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts.Messages; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts -{ - /// - /// Represents a response from an AI agent with output messages. - /// Accepts plain strings (backward compat) or structured . - /// - public sealed class Response : IEquatable - { - /// - /// Initializes a new instance of the class with plain string messages. - /// - /// The output messages from the agent. - public Response(IReadOnlyList? messages = null) - { - Messages = messages ?? Array.Empty(); - } - - /// - /// Initializes a new instance of the class with structured output messages. - /// - /// The structured output messages. - public Response(OutputMessages outputContent) - { - OutputContent = outputContent ?? throw new ArgumentNullException(nameof(outputContent)); - Messages = Array.Empty(); - } - - /// - /// Initializes a new instance of the class with a tool call result dictionary. - /// Per OTEL spec, tool call results are expected to be objects and are serialized to JSON. - /// - /// The tool call result as a structured dictionary. - public Response(IDictionary toolResultObject) - { - ToolResultObject = toolResultObject ?? throw new ArgumentNullException(nameof(toolResultObject)); - Messages = Array.Empty(); - } - - /// - /// Gets the output messages from the agent response (backward compat). - /// - public IReadOnlyList Messages { get; } - - /// - /// Gets the structured output messages, when supplied. - /// Takes precedence over for telemetry recording. - /// - public OutputMessages? OutputContent { get; } - - /// - /// Gets the tool call result dictionary, when supplied. - /// Per OTEL spec, tool call results are serialized to JSON directly. - /// - public IDictionary? ToolResultObject { get; } - - /// - public bool Equals(Response? other) - { - if (other is null) - { - return false; - } - - if (!ReferenceEquals(OutputContent, other.OutputContent)) - { - return false; - } - - if (!ReferenceEquals(ToolResultObject, other.ToolResultObject)) - { - return false; - } - - if (Messages.Count != other.Messages.Count) - { - return false; - } - - for (int i = 0; i < Messages.Count; i++) - { - if (!string.Equals(Messages[i], other.Messages[i], StringComparison.Ordinal)) - { - return false; - } - } - - return true; - } - - /// - public override bool Equals(object? obj) - { - return Equals(obj as Response); - } - - /// - public override int GetHashCode() - { - unchecked - { - int hash = 17; - hash = (hash * 31) + (OutputContent != null ? OutputContent.GetHashCode() : 0); - hash = (hash * 31) + (ToolResultObject != null ? ToolResultObject.GetHashCode() : 0); - foreach (var message in Messages) - { - hash = (hash * 31) + (message != null ? StringComparer.Ordinal.GetHashCode(message) : 0); - } - - return hash; - } - } - } -} diff --git a/src/Observability/Runtime/Tracing/Contracts/SpanDetails.cs b/src/Observability/Runtime/Tracing/Contracts/SpanDetails.cs deleted file mode 100644 index 599f40b8..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/SpanDetails.cs +++ /dev/null @@ -1,64 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Diagnostics; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts -{ - /// - /// Groups span configuration for scope construction. - /// - public sealed class SpanDetails - { - /// - /// Initializes a new instance of the class. - /// - /// Optional span kind override. - /// Optional parent used to link this span to an upstream operation. - /// Use - /// to obtain an from HTTP headers containing a W3C traceparent. - /// Optional explicit start time as a . - /// Optional explicit end time as a . - /// Optional span links to associate with this span, establishing causal - /// relationships to other spans (e.g. linking a batch operation to individual trigger spans). - public SpanDetails( - ActivityKind? spanKind = null, - ActivityContext? parentContext = null, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - ActivityLink[]? spanLinks = null) - { - SpanKind = spanKind; - ParentContext = parentContext; - StartTime = startTime; - EndTime = endTime; - SpanLinks = spanLinks; - } - - /// - /// Gets the optional span kind override. - /// - public ActivityKind? SpanKind { get; } - - /// - /// Gets the optional OpenTelemetry parent context used to link this span to an upstream operation. - /// - public ActivityContext? ParentContext { get; } - - /// - /// Gets the optional explicit start time. - /// - public DateTimeOffset? StartTime { get; } - - /// - /// Gets the optional explicit end time. - /// - public DateTimeOffset? EndTime { get; } - - /// - /// Gets the optional span links to associate with this span. - /// - public ActivityLink[]? SpanLinks { get; } - } -} diff --git a/src/Observability/Runtime/Tracing/Contracts/ThreatDiagnosticsSummary.cs b/src/Observability/Runtime/Tracing/Contracts/ThreatDiagnosticsSummary.cs deleted file mode 100644 index e32b6543..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/ThreatDiagnosticsSummary.cs +++ /dev/null @@ -1,69 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Text.Json; -using System.Text.Json.Serialization; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts -{ - /// - /// Represents a threat diagnostics summary containing security-related information - /// about blocked actions and their reasons. - /// - public sealed class ThreatDiagnosticsSummary - { - private static readonly JsonSerializerOptions JsonOptions = new JsonSerializerOptions - { - PropertyNamingPolicy = JsonNamingPolicy.CamelCase - }; - - /// - /// Initializes a new instance of the class. - /// - /// Indicates whether the action was blocked. - /// The numeric reason code for the action. - /// A human-readable description of the reason. - /// Optional additional diagnostic information. - public ThreatDiagnosticsSummary(bool blockAction, int reasonCode, string reason, string? diagnostics = null) - { - BlockAction = blockAction; - ReasonCode = reasonCode; - Reason = reason ?? throw new ArgumentNullException(nameof(reason)); - Diagnostics = diagnostics; - } - - /// - /// Gets a value indicating whether the action was blocked. - /// - [JsonPropertyName("blockAction")] - public bool BlockAction { get; } - - /// - /// Gets the numeric reason code for the action. - /// - [JsonPropertyName("reasonCode")] - public int ReasonCode { get; } - - /// - /// Gets the human-readable description of the reason. - /// - [JsonPropertyName("reason")] - public string Reason { get; } - - /// - /// Gets additional diagnostic information. - /// - [JsonPropertyName("diagnostics")] - public string? Diagnostics { get; } - - /// - /// Serializes this instance to a JSON string. - /// - /// A JSON string representation of this instance. - internal string ToJson() - { - return JsonSerializer.Serialize(this, JsonOptions); - } - } -} diff --git a/src/Observability/Runtime/Tracing/Contracts/ToolCallDetails.cs b/src/Observability/Runtime/Tracing/Contracts/ToolCallDetails.cs deleted file mode 100644 index 2ba369d9..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/ToolCallDetails.cs +++ /dev/null @@ -1,176 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Collections.Generic; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts -{ - /// - /// Details of a tool call made by an agent in the system. - /// - public sealed class ToolCallDetails : IEquatable - { - /// - /// Initializes a new instance of the class. - /// - /// Name of the tool being invoked. - /// Optional serialized arguments passed to the tool. - /// Optional identifier for the tool invocation. - /// Optional description of the tool call. - /// Optional type classification for the tool. - /// Optional endpoint for remote tool execution. - /// Optional server name for the tool. - public ToolCallDetails( - string toolName, - string? arguments, - string? toolCallId = null, - string? description = null, - string? toolType = null, - Uri? endpoint = null, - string? toolServerName = null) - { - ToolName = toolName; - Arguments = arguments; - ToolCallId = toolCallId; - Description = description; - ToolType = toolType; - Endpoint = endpoint; - ToolServerName = toolServerName; - } - - /// - /// Initializes a new instance of the class with structured arguments. - /// Per OTEL spec, tool arguments are expected to be an object and SHOULD be recorded in structured form. - /// The dictionary is serialized to JSON when setting the span attribute. - /// - /// Name of the tool being invoked. - /// Structured arguments passed to the tool, serialized to JSON. - /// Optional identifier for the tool invocation. - /// Optional description of the tool call. - /// Optional type classification for the tool. - /// Optional endpoint for remote tool execution. - /// Optional server name for the tool. - public ToolCallDetails( - string toolName, - IDictionary argumentsObject, - string? toolCallId = null, - string? description = null, - string? toolType = null, - Uri? endpoint = null, - string? toolServerName = null) - { - ToolName = toolName; - ArgumentsObject = argumentsObject ?? throw new ArgumentNullException(nameof(argumentsObject)); - ToolCallId = toolCallId; - Description = description; - ToolType = toolType; - Endpoint = endpoint; - ToolServerName = toolServerName; - } - - /// - /// Gets the tool name. - /// - public string ToolName { get; } - - /// - /// Gets the serialized JSON arguments supplied to the tool, when any. - /// - public string? Arguments { get; } - - /// - /// Gets the structured arguments supplied to the tool, when any. - /// Takes precedence over for telemetry recording. - /// - public IDictionary? ArgumentsObject { get; } - - /// - /// Gets the identifier for the tool call, when provided. - /// - public string? ToolCallId { get; } - - /// - /// Gets a user-facing description of the tool call, when provided. - /// - public string? Description { get; } - - /// - /// Gets the classification for the tool, when provided. - /// - public string? ToolType { get; } - - /// - /// Gets the endpoint associated with the tool, when applicable. - /// - public Uri? Endpoint { get; } - - /// - /// Gets the server name associated with the tool, when provided. - /// - public string? ToolServerName { get; } - - /// - /// Deconstructs this instance into individual tool call components. - /// - /// Receives the tool name. - /// Receives the arguments payload. - /// Receives the tool call identifier. - /// Receives the human-readable description. - /// Receives the type hint. - /// Receives the endpoint. - /// Receives the tool server name. - public void Deconstruct(out string toolName, out string? arguments, out string? toolCallId, out string? description, out string? toolType, out Uri? endpoint, out string? toolServerName) - { - toolName = ToolName; - arguments = Arguments; - toolCallId = ToolCallId; - description = Description; - toolType = ToolType; - endpoint = Endpoint; - toolServerName = ToolServerName; - } - - /// - public bool Equals(ToolCallDetails? other) - { - if (other is null) - { - return false; - } - - return string.Equals(ToolName, other.ToolName, StringComparison.Ordinal) && - string.Equals(Arguments, other.Arguments, StringComparison.Ordinal) && - ReferenceEquals(ArgumentsObject, other.ArgumentsObject) && - string.Equals(ToolCallId, other.ToolCallId, StringComparison.Ordinal) && - string.Equals(Description, other.Description, StringComparison.Ordinal) && - string.Equals(ToolType, other.ToolType, StringComparison.Ordinal) && - EqualityComparer.Default.Equals(Endpoint, other.Endpoint) && - string.Equals(ToolServerName, other.ToolServerName, StringComparison.Ordinal); - } - - /// - public override bool Equals(object? obj) - { - return Equals(obj as ToolCallDetails); - } - - /// - public override int GetHashCode() - { - unchecked - { - int hash = 17; - hash = (hash * 31) + (ToolName != null ? StringComparer.Ordinal.GetHashCode(ToolName) : 0); - hash = (hash * 31) + (Arguments != null ? StringComparer.Ordinal.GetHashCode(Arguments) : 0); - hash = (hash * 31) + (ArgumentsObject != null ? System.Runtime.CompilerServices.RuntimeHelpers.GetHashCode(ArgumentsObject) : 0); - hash = (hash * 31) + (ToolCallId != null ? StringComparer.Ordinal.GetHashCode(ToolCallId) : 0); - hash = (hash * 31) + (Description != null ? StringComparer.Ordinal.GetHashCode(Description) : 0); - hash = (hash * 31) + (ToolType != null ? StringComparer.Ordinal.GetHashCode(ToolType) : 0); - hash = (hash * 31) + EqualityComparer.Default.GetHashCode(Endpoint); - hash = (hash * 31) + (ToolServerName != null ? StringComparer.Ordinal.GetHashCode(ToolServerName) : 0); - return hash; - } - } - } -} \ No newline at end of file diff --git a/src/Observability/Runtime/Tracing/Contracts/ToolType.cs b/src/Observability/Runtime/Tracing/Contracts/ToolType.cs deleted file mode 100644 index a56f4dd0..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/ToolType.cs +++ /dev/null @@ -1,26 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts -{ - /// - /// Constants for tool type identifiers used in observability tracing. - /// - public sealed class ToolType - { - /// - /// Represents a function tool type. - /// - public static readonly string Function = "function"; - - /// - /// Represents an extension tool type. - /// - public static readonly string Extension = "extension"; - - /// - /// Represents a datastore tool type. - /// - public static readonly string Datastore = "datastore"; - } -} \ No newline at end of file diff --git a/src/Observability/Runtime/Tracing/Contracts/UserDetails.cs b/src/Observability/Runtime/Tracing/Contracts/UserDetails.cs deleted file mode 100644 index ca8f4353..00000000 --- a/src/Observability/Runtime/Tracing/Contracts/UserDetails.cs +++ /dev/null @@ -1,87 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Net; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts -{ - /// - /// Details about the human user that invoked an agent. - /// - public sealed class UserDetails : IEquatable - { - /// - /// Initializes a new instance of the class. - /// - /// The unique identifier for the user. - /// The email address of the user. - /// The human-readable name of the user. - /// The client IP address of the user. - public UserDetails( - string? userId = null, - string? userEmail = null, - string? userName = null, - IPAddress? userClientIP = null) - { - UserId = userId; - UserEmail = userEmail; - UserName = userName; - UserClientIP = userClientIP; - } - - /// - /// Gets the unique identifier for the user. - /// - public string? UserId { get; } - - /// - /// Gets the email address of the user. - /// - public string? UserEmail { get; } - - /// - /// Gets the human-readable name of the user. - /// - public string? UserName { get; } - - /// - /// Gets the client IP address of the user. - /// - public IPAddress? UserClientIP { get; } - - /// - public bool Equals(UserDetails? other) - { - if (other is null) - { - return false; - } - - return string.Equals(UserId, other.UserId, StringComparison.Ordinal) && - string.Equals(UserEmail, other.UserEmail, StringComparison.Ordinal) && - string.Equals(UserName, other.UserName, StringComparison.Ordinal) && - Equals(UserClientIP, other.UserClientIP); - } - - /// - public override bool Equals(object? obj) - { - return Equals(obj as UserDetails); - } - - /// - public override int GetHashCode() - { - unchecked - { - int hash = 17; - hash = (hash * 31) + (UserId != null ? StringComparer.Ordinal.GetHashCode(UserId) : 0); - hash = (hash * 31) + (UserEmail != null ? StringComparer.Ordinal.GetHashCode(UserEmail) : 0); - hash = (hash * 31) + (UserName != null ? StringComparer.Ordinal.GetHashCode(UserName) : 0); - hash = (hash * 31) + (UserClientIP?.GetHashCode() ?? 0); - return hash; - } - } - } -} diff --git a/src/Observability/Runtime/Tracing/Exporters/Agent365Exporter.cs b/src/Observability/Runtime/Tracing/Exporters/Agent365Exporter.cs deleted file mode 100644 index 00d3b010..00000000 --- a/src/Observability/Runtime/Tracing/Exporters/Agent365Exporter.cs +++ /dev/null @@ -1,89 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Diagnostics; -using System.Net.Http; -using System.Threading.Tasks; -using Microsoft.Extensions.Logging; -using OpenTelemetry; -using OpenTelemetry.Resources; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters -{ - /// - /// Minimal OTLP/HTTP JSON exporter for traces. - /// Sends POST {Endpoint}/v1/traces with application/json. - /// - - public sealed class Agent365Exporter : BaseExporter - { - private readonly HttpClient _httpClient; - private readonly Resource _resource; - private readonly ILogger _logger; - private readonly Agent365ExporterOptions _options; - private readonly Agent365ExporterCore _core; - - /// - /// Initializes a new instance of the class. - /// - /// The Agent365ExporterCore instance. - /// The logger instance. - /// The exporter configuration options. - /// Optional OpenTelemetry resource information. - /// Optional HttpClient instance. - public Agent365Exporter( - Agent365ExporterCore core, - ILogger logger, - Agent365ExporterOptions options, - Resource? resource = null, - HttpClient? httpClient = null) - { - _core = core ?? throw new ArgumentNullException(nameof(core)); - _logger = logger ?? throw new ArgumentNullException(nameof(logger)); - _options = options ?? throw new ArgumentNullException(nameof(options)); - - if (_options.TokenResolver == null && _options.ContextualTokenResolver == null) - throw new ArgumentNullException(nameof(options.TokenResolver), "Agent365ExporterOptions.TokenResolver or ContextualTokenResolver must provide an app-only OBS token."); - - _httpClient = httpClient ?? HttpClientFactory.CreateWithTimeout(options.ExporterTimeoutMilliseconds); - _resource = resource ?? ResourceBuilder.CreateEmpty().Build(); - } - - /// - /// Exports a batch of OpenTelemetry activities to the Microsoft Agent 365 observability platform. - /// - /// The batch of activities to export. - /// The export result indicating success or failure. - public override ExportResult Export(in Batch batch) - { - _logger.LogDebug("Agent365Exporter: Exporting batch of {Count} spans.", batch.Count); - - try - { - var groups = _core.PartitionByIdentity(batch); - if (groups.Count == 0) - { - _logger.LogDebug("Agent365Exporter: No spans with tenant/agent identity found; nothing exported."); - return ExportResult.Success; - } - - // Use the async core method, synchronously - return _core.ExportBatchCoreAsync( - groups: groups, - resource: _resource, - options: _options, - tokenResolver: (agentId, tenantId) => _options.TokenResolver != null - ? _options.TokenResolver(agentId, tenantId) - : Task.FromResult(null), - sendAsync: request => _httpClient.SendAsync(request) - ).GetAwaiter().GetResult(); - } - catch (Exception exOuter) - { - _logger.LogError(exOuter, "Agent365Exporter: Unhandled export exception."); - return ExportResult.Failure; - } - } - } -} diff --git a/src/Observability/Runtime/Tracing/Exporters/Agent365ExporterAsync.cs b/src/Observability/Runtime/Tracing/Exporters/Agent365ExporterAsync.cs deleted file mode 100644 index 3fd9f6b0..00000000 --- a/src/Observability/Runtime/Tracing/Exporters/Agent365ExporterAsync.cs +++ /dev/null @@ -1,94 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Collections.Generic; -using System.Diagnostics; -using System.Net.Http; -using System.Threading; -using System.Threading.Tasks; -using Microsoft.Extensions.Logging; -using OpenTelemetry.Resources; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters -{ - /// - /// Minimal OTLP/HTTP JSON exporter for traces. - /// Sends POST {Endpoint}/v1/traces with application/json. - /// - - public sealed class Agent365ExporterAsync : BaseExporterAsync - { - private readonly HttpClient _httpClient; - private readonly Resource _resource; - private readonly ILogger _logger; - private readonly Agent365ExporterOptions _options; - private readonly Agent365ExporterCore _core; - - /// - /// Initializes a new instance of the class. - /// - /// The Agent365ExporterCore instance. - /// The logger instance. - /// The exporter configuration options. - /// Optional OpenTelemetry resource information. - /// Optional HttpClient instance. - public Agent365ExporterAsync( - Agent365ExporterCore core, - ILogger logger, - Agent365ExporterOptions options, - Resource? resource = null, - HttpClient? httpClient = null) - { - this._core = core ?? throw new ArgumentNullException(nameof(core)); - this._logger = logger ?? throw new ArgumentNullException(nameof(logger)); - this._options = options ?? throw new ArgumentNullException(nameof(options)); - - if (_options.TokenResolver == null && _options.ContextualTokenResolver == null) - throw new ArgumentNullException(nameof(options.TokenResolver), "Agent365ExporterOptions.TokenResolver or ContextualTokenResolver must provide an app-only OBS token."); - - this._httpClient = httpClient ?? HttpClientFactory.CreateWithTimeout(options.ExporterTimeoutMilliseconds); - this._resource = resource ?? ResourceBuilder.CreateEmpty().Build(); - } - - /// - /// Exports a batch of OpenTelemetry activities to the Microsoft Agent 365 observability platform asynchronously. - /// - /// The batch of activities to export. - /// A token to monitor for cancellation requests. - /// A representing the asynchronous export operation. - public override async Task ExportAsync(IReadOnlyCollection batch, CancellationToken cancellationToken) - { - this._logger.LogDebug("Agent365ExporterAsync: Exporting batch of {Count} spans.", batch.Count); - - try - { - var groups = _core.PartitionByIdentity(batch); - if (groups.Count == 0) - { - this._logger.LogDebug("Agent365ExporterAsync: No spans with tenant/agent identity found; nothing exported."); - return; - } - - await _core.ExportBatchCoreAsync( - groups: groups, - resource: this._resource, - options: this._options, - tokenResolver: (agentId, tenantId) => this._options.TokenResolver != null - ? this._options.TokenResolver(agentId, tenantId) - : Task.FromResult(null), - sendAsync: request => this._httpClient.SendAsync(request, cancellationToken) - ).ConfigureAwait(false); - } - catch (OperationCanceledException) - { - this._logger.LogWarning("Agent365ExporterAsync: Export operation was canceled."); - throw; - } - catch (Exception exOuter) - { - this._logger.LogError(exOuter, "Agent365ExporterAsync: Unhandled export exception."); - } - } - } -} diff --git a/src/Observability/Runtime/Tracing/Exporters/Agent365ExporterCore.cs b/src/Observability/Runtime/Tracing/Exporters/Agent365ExporterCore.cs deleted file mode 100644 index 4d8cf8fa..00000000 --- a/src/Observability/Runtime/Tracing/Exporters/Agent365ExporterCore.cs +++ /dev/null @@ -1,320 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using Microsoft.Agents.A365.Observability.Runtime.Common; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Extensions.Logging; -using Microsoft.Extensions.Logging.Abstractions; -using OpenTelemetry; -using OpenTelemetry.Resources; -using System; -using System.Collections.Generic; -using System.Diagnostics; -using System.Linq; -using System.Net.Http; -using System.Net.Http.Headers; -using System.Text; -using System.Threading.Tasks; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters -{ - /// - /// Utility methods for Agent365 trace exporters. - /// Provides helpers for partitioning activities and building endpoint URIs. - /// - public class Agent365ExporterCore - { - private const string CorrelationIdHeaderKey = "x-ms-correlation-id"; - private readonly ExportFormatter _formatter; - private readonly ILogger _logger; - - // The ingest service performs a case-insensitive check for "chat", so we send the - // gen_ai.operation.name through unchanged. Both the lowercase canonical value and the - // InferenceOperationType.Chat enum name are accepted in this set so that activities - // tagged with either form are not filtered out by PartitionByIdentity. - private static readonly HashSet GenAiOperationNames = new HashSet(StringComparer.Ordinal) - { - InvokeAgentScope.OperationName, - ExecuteToolScope.OperationName, - OutputScope.OperationName, - OpenTelemetryConstants.ApplyGuardrailOperationName, - "chat", - nameof(InferenceOperationType.Chat), - }; - - private enum AddResult { Added, NonGenAI, MissingIdentity, Null } - - /// - /// Initializes a new instance of the class. - /// - /// The formatter instance used to format export payloads. - /// The logger instance used to log messages during the export process. - public Agent365ExporterCore(ExportFormatter formatter, ILogger logger) - { - _formatter = formatter ?? throw new ArgumentNullException(nameof(formatter)); - _logger = logger ?? NullLogger.Instance; - } - - /// - /// Partitions a batch of activities by tenant and agent identity. - /// Only genAI activities (those with a known gen_ai.operation.name) are included. - /// - /// The collection of activities to partition. - /// - /// A list of tuples containing TenantId, AgentId, and the corresponding activities. - /// - public List<(string TenantId, string AgentId, List Activities)> PartitionByIdentity(IEnumerable batch) - { - var map = new Dictionary<(string tenant, string agent), List>(); - int nonGenAICount = 0; - int missingIdentityCount = 0; - - foreach (var activity in batch) - { - var result = Agent365ExporterCore.TryAddActivityToMap(activity, map); - if (result == AddResult.NonGenAI) nonGenAICount++; - else if (result == AddResult.MissingIdentity) missingIdentityCount++; - } - - LogPartitionResults(map.Count, nonGenAICount, missingIdentityCount); - return map.Select(kvp => (kvp.Key.tenant, kvp.Key.agent, kvp.Value)).ToList(); - } - - /// - /// Partitions a batch of activities by tenant and agent identity. - /// Only genAI activities (those with a known gen_ai.operation.name) are included. - /// - /// The collection of activities to partition. - /// - /// A list of tuples containing TenantId, AgentId, and the corresponding activities. - /// - public List<(string TenantId, string AgentId, List Activities)> PartitionByIdentity(in Batch batch) - { - var map = new Dictionary<(string tenant, string agent), List>(); - int nonGenAICount = 0; - int missingIdentityCount = 0; - - foreach (var activity in batch) - { - var result = Agent365ExporterCore.TryAddActivityToMap(activity, map); - if (result == AddResult.NonGenAI) nonGenAICount++; - else if (result == AddResult.MissingIdentity) missingIdentityCount++; - } - - LogPartitionResults(map.Count, nonGenAICount, missingIdentityCount); - return map.Select(kvp => (kvp.Key.tenant, kvp.Key.agent, kvp.Value)).ToList(); - } - - /// - /// Builds the S2S OTLP endpoint path for the trace export request based on tenant ID and agent ID. - /// - /// The tenant identifier. - /// The agent identifier. - /// The endpoint path string. - public string BuildEndpointPath(string tenantId, string agentId) - { - var encodedTenantId = Uri.EscapeDataString(tenantId); - var encodedAgentId = Uri.EscapeDataString(agentId); - - return $"/observabilityService/tenants/{encodedTenantId}/otlp/agents/{encodedAgentId}/traces"; - } - - /// - /// Builds the S2S OTLP endpoint path for the trace export request based on tenant ID and agent ID. - /// The value is ignored for source compatibility. - /// - /// The tenant identifier. - /// The agent identifier. - /// Ignored. OBS export always uses the S2S OTLP endpoint. - /// The endpoint path string. - [Obsolete("Agent 365 OBS export always uses the service-to-service /observabilityService OTLP endpoint; this argument is ignored.", false)] - public string BuildEndpointPath(string tenantId, string agentId, bool useS2SEndpoint) - { - return BuildEndpointPath(tenantId, agentId); - } - - /// - /// Builds the full request URI for the trace export request. - /// If the endpoint already includes a scheme (https://), it is used as-is. - /// Otherwise, https:// is prepended. Plaintext http:// is not supported. - /// - /// The base endpoint (domain or full HTTPS URL). - /// The endpoint path. - /// The full request URI string. - /// Thrown when the endpoint uses an http:// (non-TLS) scheme. - public string BuildRequestUri(string endpoint, string endpointPath) - { - var normalizedEndpoint = endpoint.TrimEnd('/'); - - if (normalizedEndpoint.StartsWith("http://", StringComparison.OrdinalIgnoreCase)) - { - throw new ArgumentException("Plaintext HTTP endpoints are not supported. Use HTTPS to protect credentials in transit.", nameof(endpoint)); - } - - if (normalizedEndpoint.StartsWith("https://", StringComparison.OrdinalIgnoreCase)) - { - return $"{normalizedEndpoint}{endpointPath}?api-version=1"; - } - - return $"https://{normalizedEndpoint}{endpointPath}?api-version=1"; - } - - /// - /// Exports a batch of activities grouped by tenant and agent identity. - /// - /// - /// - /// - /// - /// - /// - public async Task ExportBatchCoreAsync( - IEnumerable<(string TenantId, string AgentId, List Activities)> groups, - Resource resource, - Agent365ExporterOptions options, - Func> tokenResolver, - Func> sendAsync) - { - foreach (var g in groups) - { - var (tenantId, agentId, activities) = g; - - // Split the per-identity batch into byte-size chunks under MaxPayloadBytes. - // Per-span truncation already caps individual spans at 250 KB; this provides - // batch-level enforcement of the 1 MB server limit. - var chunks = PayloadChunking.ChunkBySize( - activities, - PayloadChunking.EstimateActivityBytes, - options.MaxPayloadBytes); - - if (chunks.Count > 1) - { - this._logger?.LogInformation( - "Agent365ExporterCore: Split {SpanCount} spans into {ChunkCount} chunks for tenantId {TenantId}, agentId {AgentId}.", - activities.Count, chunks.Count, tenantId, agentId); - } - - var endpointOverride = Environment.GetEnvironmentVariable("A365_OBSERVABILITY_DOMAIN_OVERRIDE"); - var endpoint = !string.IsNullOrEmpty(endpointOverride) - ? endpointOverride - : options.DomainResolver.Invoke(tenantId); - - var endpointPath = BuildEndpointPath(tenantId, agentId); - var requestUri = BuildRequestUri(endpoint, endpointPath); - - string? token = null; - try - { - // Prefer ContextualTokenResolver when set; extract agentic user ID from the - // first activity in the group (1:1 relationship between agent and agentic user). - if (options.ContextualTokenResolver != null) - { - var agenticUserId = activities.Count > 0 - ? activities[0].GetAttributeOrBaggage(OpenTelemetryConstants.AgentAUIDKey) - : null; - var identity = new AgentIdentity(agentId, agenticUserId); - - var context = new TokenResolverContext(identity, tenantId); - token = await options.ContextualTokenResolver(context).ConfigureAwait(false); - } - else - { - token = await tokenResolver(agentId, tenantId).ConfigureAwait(false); - } - - this._logger?.LogDebug("Agent365ExporterCore: Obtained token for agent {AgentId} tenant {TenantId}.", agentId, tenantId); - } - catch (Exception ex) - { - this._logger?.LogError(ex, "Agent365ExporterCore: TokenResolver threw for agent {AgentId} tenant {TenantId}.", agentId, tenantId); - } - - if (string.IsNullOrEmpty(token)) - { - this._logger?.LogWarning("Agent365ExporterCore: No token obtained. Skipping export for this identity."); - return ExportResult.Failure; - } - - // Send each chunk; all-or-nothing: fail on first chunk failure so the batch processor retries. - for (int i = 0; i < chunks.Count; i++) - { - var chunk = chunks[i]; - var json = _formatter.FormatMany(chunk, resource); - using var content = new StringContent(json, Encoding.UTF8, "application/json"); - - using var request = new HttpRequestMessage(HttpMethod.Post, requestUri) - { - Content = content - }; - request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token); - - var bodyBytes = Encoding.UTF8.GetByteCount(json); - this._logger?.LogDebug( - "Agent365ExporterCore: Sending chunk {ChunkIndex} of {ChunkCount} ({SpanCount} spans, {BodyBytes} bytes) to {RequestUri}.", - i + 1, chunks.Count, chunk.Count, bodyBytes, requestUri); - - try - { - using var resp = await sendAsync(request).ConfigureAwait(false); - var correlationId = resp.Headers.Contains(CorrelationIdHeaderKey) ? resp.Headers.GetValues(CorrelationIdHeaderKey).FirstOrDefault() : null; - this._logger?.LogDebug("Agent365ExporterCore: HTTP {StatusCode} exporting spans. '{HeaderKey}': '{CorrelationId}'.", (int)resp.StatusCode, CorrelationIdHeaderKey, correlationId); - if (!resp.IsSuccessStatusCode) - { - this._logger?.LogWarning( - "Agent365ExporterCore: Chunk {ChunkIndex} of {ChunkCount} failed; aborting batch.", - i + 1, chunks.Count); - return ExportResult.Failure; - } - } - catch (HttpRequestException ex) - { - this._logger?.LogError(ex, "Agent365ExporterCore: Exception exporting spans."); - return ExportResult.Failure; - } - catch (TaskCanceledException ex) - { - this._logger?.LogError(ex, "Agent365ExporterCore: Exception exporting spans."); - return ExportResult.Failure; - } - } - } - return ExportResult.Success; - } - - private static AddResult TryAddActivityToMap(Activity activity, Dictionary<(string tenant, string agent), List> map) - { - if (activity is null) return AddResult.Null; - - var operationName = activity.GetAttributeOrBaggage(OpenTelemetryConstants.GenAiOperationNameKey); - if (string.IsNullOrEmpty(operationName) || !GenAiOperationNames.Contains(operationName!)) - return AddResult.NonGenAI; - - var tenant = activity.GetAttributeOrBaggage(OpenTelemetryConstants.TenantIdKey); - var agent = activity.GetAttributeOrBaggage(OpenTelemetryConstants.GenAiAgentIdKey) ?? activity.GetAttributeOrBaggage(OpenTelemetryConstants.AgentPlatformIdKey); - - if (string.IsNullOrEmpty(tenant) || string.IsNullOrEmpty(agent)) - return AddResult.MissingIdentity; - - var key = (tenant!, agent!); - if (!map.TryGetValue(key, out var list)) - { - list = new List(); - map[key] = list; - } - list.Add(activity); - return AddResult.Added; - } - - private void LogPartitionResults(int groupCount, int nonGenAICount, int missingIdentityCount) - { - if (nonGenAICount > 0) - _logger?.LogDebug("[Agent365Exporter] {NonGenAICount} non-genAI spans filtered out", nonGenAICount); - if (missingIdentityCount > 0) - _logger?.LogDebug("[Agent365Exporter] {MissingIdentityCount} spans skipped due to missing tenant or agent ID", missingIdentityCount); - - var skippedCount = nonGenAICount + missingIdentityCount; - if (skippedCount > 0) - _logger?.LogDebug("[Agent365Exporter] Partitioned into {GroupCount} identity groups ({SkippedCount} spans skipped)", groupCount, skippedCount); - } - } -} \ No newline at end of file diff --git a/src/Observability/Runtime/Tracing/Exporters/Agent365ExporterOptions.cs b/src/Observability/Runtime/Tracing/Exporters/Agent365ExporterOptions.cs deleted file mode 100644 index d07f71c6..00000000 --- a/src/Observability/Runtime/Tracing/Exporters/Agent365ExporterOptions.cs +++ /dev/null @@ -1,122 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Threading.Tasks; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters -{ - /// - /// Async delegate used by the exporter to obtain an app-only OBS auth token for a specific agent and tenant. - /// Must be fast and non-blocking; cache tokens in the resolver and refresh only near expiry. - /// Return null or empty to fail the export batch without sending a request. - /// - public delegate Task AsyncAuthTokenResolver(string agentId, string tenantId); - - /// - /// Async delegate used by the exporter to obtain an app-only OBS auth token using rich context. - /// Provides additional fields (e.g. ) - /// beyond what offers. - /// Must be fast and non-blocking; cache tokens in the resolver and refresh only near expiry. - /// Return null or empty to fail the export batch without sending a request. - /// - public delegate Task AsyncContextualTokenResolver(TokenResolverContext context); - - /// - /// Delegate used by the exporter to resolve the endpoint host or URL for a given tenant id. - /// The return value may be a bare host name (e.g. "agent365.svc.cloud.microsoft") or a full URL - /// (e.g. "https://agent365.svc.cloud.microsoft"). - /// - public delegate string TenantDomainResolver(string tenantId); - - /// - /// Configuration for Agent365Exporter. - /// A configured app-only token resolver is required for core operation. - /// - public sealed class Agent365ExporterOptions - { - /// - /// The default endpoint host for Agent365 observability. - /// - public const string DefaultEndpointHost = "agent365.svc.cloud.microsoft"; - - /// - /// Initializes a new instance of the class with default settings. - /// - /// The default constructor sets the DomainResolver property to return the default - /// Agent365 endpoint host (agent365.svc.cloud.microsoft). - public Agent365ExporterOptions() - { - this.DomainResolver = tenantId => DefaultEndpointHost; - } - - /// - /// Cluster region argument. Defaults to production. - /// - public string ClusterCategory { get; set; } = "production"; - - /// - /// Async delegate used to resolve the app-only OBS auth token. - /// Either this or must be set. - /// When both are set, takes precedence. - /// The exporter invokes this resolver once per tenant/agent identity group in each export batch, so a batch - /// that contains several identities invokes it several times. It never falls back to a delegated token. - /// - public AsyncAuthTokenResolver? TokenResolver { get; set; } - - /// - /// Async delegate used to resolve the app-only OBS auth token with rich context, which may include the - /// agentic user ID associated with the export batch context for cache selection or diagnostics. - /// Takes precedence over when set. - /// The exporter does not guarantee separate batching or resolver invocation per agentic user ID. - /// This resolver must not perform delegated, OBO, or user_fic authentication for OBS export. - /// - public AsyncContextualTokenResolver? ContextualTokenResolver { get; set; } - - /// - /// Delegate used to resolve the endpoint host or URL for a given tenant id. - /// Defaults to returning . - /// - public TenantDomainResolver DomainResolver { get; set; } - - /// - /// OBS export always uses the service-to-service (S2S) OTLP endpoint path: - /// /observabilityService/tenants/{tenantId}/otlp/agents/{agentId}/traces. - /// This compatibility switch is ignored even when set to false. - /// - [Obsolete("Agent 365 OBS export always uses the service-to-service /observabilityService OTLP endpoint; this option is ignored.", false)] - public bool UseS2SEndpoint { get; set; } = false; - - /// - /// Maximum queue size for the batch processor. - /// Default is 2048. - /// - public int MaxQueueSize { get; set; } = 2048; - - /// - /// Delay in milliseconds between export batches. - /// Default is 5000 (5 seconds). - /// - public int ScheduledDelayMilliseconds { get; set; } = 5000; - - /// - /// Timeout in milliseconds for the export operation. - /// Default is 30000 (30 seconds). - /// - public int ExporterTimeoutMilliseconds { get; set; } = 30000; - - /// - /// Maximum batch size for export operations. - /// Default is 512. - /// - public int MaxExportBatchSize { get; set; } = 512; - - /// - /// Upper bound on HTTP request body size in bytes used by per-request payload chunking. - /// 100 KB headroom under the 1 MB server limit accounts for estimator error and JSON/envelope - /// overhead (for example, resource attributes and scope wrappers). - /// Default is 900,000 bytes. - /// - public long MaxPayloadBytes { get; set; } = 900_000; - } -} diff --git a/src/Observability/Runtime/Tracing/Exporters/Agent365ExporterType.cs b/src/Observability/Runtime/Tracing/Exporters/Agent365ExporterType.cs deleted file mode 100644 index 3ac5ed47..00000000 --- a/src/Observability/Runtime/Tracing/Exporters/Agent365ExporterType.cs +++ /dev/null @@ -1,20 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters -{ - /// - /// Represents the supported Agent365 exporter types. - /// - public enum Agent365ExporterType - { - /// - /// Agent365 synchronous exporter type. - /// - Agent365Exporter = 0, - - /// - /// Agent365 asynchronous exporter type. - /// - Agent365ExporterAsync = 1 - } -} diff --git a/src/Observability/Runtime/Tracing/Exporters/AgentIdentity.cs b/src/Observability/Runtime/Tracing/Exporters/AgentIdentity.cs deleted file mode 100644 index 1aff1faa..00000000 --- a/src/Observability/Runtime/Tracing/Exporters/AgentIdentity.cs +++ /dev/null @@ -1,37 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters -{ - /// - /// Represents the identity of an agent and its acting user. - /// - /// is contextual metadata only. OBS export must use an app-only - /// token for and must not use this value for delegated token exchange. - /// - /// - public class AgentIdentity - { - /// - /// Initializes a new instance of the class. - /// - /// The agent identifier. - /// The agentic user identifier (AAD Object ID), or null when unavailable. - public AgentIdentity(string agentId, string? agenticUserId = null) - { - AgentId = agentId; - AgenticUserId = agenticUserId; - } - - /// - /// Gets the agent identifier. - /// - public string AgentId { get; } - - /// - /// Gets the agentic user identifier (AAD Object ID), if available. - /// This value is not an OBS authentication input. - /// - public string? AgenticUserId { get; } - } -} diff --git a/src/Observability/Runtime/Tracing/Exporters/BaseExporterAsync.cs b/src/Observability/Runtime/Tracing/Exporters/BaseExporterAsync.cs deleted file mode 100644 index ebc6df4e..00000000 --- a/src/Observability/Runtime/Tracing/Exporters/BaseExporterAsync.cs +++ /dev/null @@ -1,61 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using System; -using System.Collections.Generic; -using System.Threading; -using System.Threading.Tasks; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters -{ - /// - /// Abstract base class for asynchronous exporters of telemetry objects. - /// - /// The type of telemetry object to export. - public abstract class BaseExporterAsync : IDisposable - where T : class - { - /// - /// Exports a batch of telemetry objects asynchronously. - /// - /// The batch of telemetry objects to export. - /// A token to monitor for cancellation requests. - /// A representing the asynchronous export operation. - public abstract Task ExportAsync(IReadOnlyCollection batch, CancellationToken cancellationToken); - - /// - /// Flushes any buffered telemetry data asynchronously. - /// - /// A token to monitor for cancellation requests. - /// - /// A representing the asynchronous flush operation. - /// Returns true if flush was successful or not required. - /// - public virtual Task ForceFlushAsync(CancellationToken cancellationToken = default) - { - // Default: nothing to flush - return Task.FromResult(true); - } - - /// - /// Shuts down the exporter asynchronously, releasing any resources if necessary. - /// - /// A token to monitor for cancellation requests. - /// - /// A representing the asynchronous shutdown operation. - /// Returns true if shutdown was successful or not required. - /// - public virtual Task ShutdownAsync(CancellationToken cancellationToken = default) - { - // Default: nothing to shutdown - return Task.FromResult(true); - } - - /// - /// Releases resources used by the exporter. - /// - public virtual void Dispose() - { - // Default: nothing to dispose - } - } -} diff --git a/src/Observability/Runtime/Tracing/Exporters/HttpClientFactory.cs b/src/Observability/Runtime/Tracing/Exporters/HttpClientFactory.cs deleted file mode 100644 index 681a0c15..00000000 --- a/src/Observability/Runtime/Tracing/Exporters/HttpClientFactory.cs +++ /dev/null @@ -1,51 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Net.Http; -using System.Threading; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters -{ - /// - /// Factory for creating HttpClient instances with configured timeouts for Agent365 exporters. - /// - /// - /// This factory creates HttpClient instances without custom handlers or IHttpClientFactory integration. - /// In high-load scenarios, callers should ensure HttpClient instances are reused (e.g., passed via constructor) - /// to avoid socket exhaustion. This factory is intended for creating a single HttpClient per exporter instance - /// that is reused for the exporter's lifetime. - /// - internal static class HttpClientFactory - { - /// - /// Creates a new HttpClient with the specified timeout. - /// - /// - /// The timeout in milliseconds. Must be a positive value, or -1 for infinite timeout. - /// - /// A new HttpClient instance with the configured timeout. - /// - /// Thrown when is 0 or less than -1. - /// - public static HttpClient CreateWithTimeout(int timeoutMilliseconds) - { - if (timeoutMilliseconds == 0 || timeoutMilliseconds < -1) - { - throw new ArgumentOutOfRangeException( - nameof(timeoutMilliseconds), - timeoutMilliseconds, - "Timeout must be a positive value, or -1 for infinite timeout."); - } - - var timeout = timeoutMilliseconds == -1 - ? Timeout.InfiniteTimeSpan - : TimeSpan.FromMilliseconds(timeoutMilliseconds); - - return new HttpClient - { - Timeout = timeout - }; - } - } -} diff --git a/src/Observability/Runtime/Tracing/Exporters/ObservabilityTracerProviderBuilderExtensions.cs b/src/Observability/Runtime/Tracing/Exporters/ObservabilityTracerProviderBuilderExtensions.cs deleted file mode 100644 index f491a904..00000000 --- a/src/Observability/Runtime/Tracing/Exporters/ObservabilityTracerProviderBuilderExtensions.cs +++ /dev/null @@ -1,107 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using Microsoft.Agents.A365.Observability.Runtime.Common; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Processors; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.Logging; -using OpenTelemetry; -using OpenTelemetry.Trace; -using System; -using System.Net.Http; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters -{ - /// - /// Extension methods to add Agent365 Exporter to OpenTelemetry TracerProviderBuilder. - /// - public static class ObservabilityTracerProviderBuilderExtensions - { - private static readonly Lazy FallbackConsoleLoggerFactory = new Lazy(() => LoggerFactory.Create(b => b.AddConsole())); - - /// - /// Adds the Agent365 Exporter to the OpenTelemetry TracerProviderBuilder using deferred initialization. - /// - /// The TracerProviderBuilder to configure. - /// The Agent365 exporter type to use. - internal static TracerProviderBuilder AddAgent365Exporter(this TracerProviderBuilder builder, Agent365ExporterType exporterType = Agent365ExporterType.Agent365Exporter) - { - if (builder == null) - { - throw new ArgumentNullException(nameof(builder)); - } - - var deferredBuilder = builder as IDeferredTracerProviderBuilder; - if (deferredBuilder == null) - { - throw new InvalidOperationException("The provided TracerProviderBuilder does not implement IDeferredTracerProviderBuilder."); - } - - return deferredBuilder.Configure((sp, builder) => ObservabilityTracerProviderBuilderExtensions.ConfigureInternal(sp, builder, exporterType)); - } - - /// - /// Adds the Agent365 Exporter to the OpenTelemetry TracerProviderBuilder using the provided service collection. - /// - /// The TracerProviderBuilder to configure. - /// The service collection to use for dependency injection. - /// The Agent365 exporter type to use. - internal static TracerProviderBuilder AddAgent365Exporter(this TracerProviderBuilder builder, IServiceCollection serviceCollection, Agent365ExporterType exporterType = Agent365ExporterType.Agent365Exporter) - { - if (builder == null) - { - throw new ArgumentNullException(nameof(builder)); - } - - if (serviceCollection == null) - { - throw new ArgumentNullException(nameof(serviceCollection)); - } - - return ObservabilityTracerProviderBuilderExtensions.ConfigureInternal( - serviceProvider: serviceCollection.BuildServiceProvider(), - builder: builder, - exporterType: exporterType); - } - - private static TracerProviderBuilder ConfigureInternal(IServiceProvider serviceProvider, TracerProviderBuilder builder, Agent365ExporterType exporterType) - { - // Ensure required services are registered - var exporterOptions = serviceProvider.GetRequiredService(); - var httpClient = serviceProvider.GetService(); - - // Resolve ILoggerFactory from DI to ensure loggers have proper lifetime; fall back to NullLogger when unavailable. - var loggerFactory = serviceProvider.GetService() ?? ObservabilityTracerProviderBuilderExtensions.FallbackConsoleLoggerFactory.Value; - var logger = serviceProvider.GetService>() ?? loggerFactory.CreateLogger(); - var coreLogger = serviceProvider.GetService>() ?? loggerFactory.CreateLogger(); - var formatterLogger = serviceProvider.GetService>() ?? loggerFactory.CreateLogger(); - - // Create ExportFormatter and Agent365ExporterCore - var exportFormatter = new ExportFormatter(formatterLogger); - var exporterCore = new Agent365ExporterCore(exportFormatter, coreLogger); - - switch (exporterType) - { - case Agent365ExporterType.Agent365ExporterAsync: - builder.AddProcessor(new BatchActivityExportProcessorAsync( - new Agent365ExporterAsync(core: exporterCore, logger: logger, options: exporterOptions, resource: null, httpClient: httpClient), - maxQueueSize: exporterOptions.MaxQueueSize, - scheduledDelayMilliseconds: exporterOptions.ScheduledDelayMilliseconds, - maxExportBatchSize: exporterOptions.MaxExportBatchSize)); - break; - - case Agent365ExporterType.Agent365Exporter: - builder.AddProcessor(new BatchActivityExportProcessor( - new Agent365Exporter(core: exporterCore, logger: logger, options: exporterOptions, resource: null, httpClient: httpClient), - maxQueueSize: exporterOptions.MaxQueueSize, - scheduledDelayMilliseconds: exporterOptions.ScheduledDelayMilliseconds, - exporterTimeoutMilliseconds: exporterOptions.ExporterTimeoutMilliseconds, - maxExportBatchSize: exporterOptions.MaxExportBatchSize)); - break; - - default: - throw new ArgumentOutOfRangeException(nameof(exporterType), exporterType, "Unknown Agent365ExporterType specified."); - } - return builder; - } - } -} \ No newline at end of file diff --git a/src/Observability/Runtime/Tracing/Exporters/PayloadChunking.cs b/src/Observability/Runtime/Tracing/Exporters/PayloadChunking.cs deleted file mode 100644 index d4ad9115..00000000 --- a/src/Observability/Runtime/Tracing/Exporters/PayloadChunking.cs +++ /dev/null @@ -1,176 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Collections; -using System.Collections.Generic; -using System.Diagnostics; -using System.Text; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters -{ - /// - /// Heuristic span size estimation and byte-level chunking for OTLP/HTTP JSON payloads. - /// - /// The Agent 365 OTLP traces endpoint enforces a 1 MB request body limit. Per-span - /// truncation (250 KB cap inside ) prevents any - /// single span from being too large; this class provides the second layer of defense - /// at the batch level by splitting batches into sub-batches whose cumulative estimated - /// size stays under . - /// - public static class PayloadChunking - { - /// - /// Overhead constant for OTLP JSON span fixed fields - /// (traceId, spanId, parentSpanId, kind, timestamps, status, scope wrapper, etc.). - /// Intentionally generous to account for serializer variance. - /// - private const int SpanBaseOverhead = 2000; - - /// - /// Overhead per attribute in OTLP JSON format. Covers key/value JSON wrapping overhead. - /// - private const int AttrOverhead = 80; - - /// - /// Overhead per event in OTLP JSON. - /// - private const int EventOverhead = 200; - - /// - /// Estimates the serialized byte size of a single attribute value in OTLP/HTTP JSON. - /// - /// The attribute value to estimate. - /// An over-estimate of the serialized size in bytes. - public static long EstimateValueBytes(object? value) - { - if (value is string s) - { - return 40 + Encoding.UTF8.GetByteCount(s); - } - - if (value is IEnumerable enumerable && !(value is string)) - { - long count = 0; - bool firstIsString = false; - long stringSum = 0; - bool sawAny = false; - foreach (var item in enumerable) - { - if (!sawAny) - { - firstIsString = item is string; - sawAny = true; - } - if (firstIsString) - { - stringSum += 40 + Encoding.UTF8.GetByteCount(item?.ToString() ?? string.Empty); - } - count++; - } - - if (count == 0) return 60; - if (firstIsString) return 60 + stringSum; - return 60 + 50 * count; - } - - return 40; // bool/int/float/null/other - } - - /// - /// Heuristic estimator for the serialized size of an OTLP span produced from the - /// supplied when emitted as OTLP/HTTP JSON. - /// - /// Uses generous constants tuned to over-estimate by ~25-50%, providing headroom - /// for JSON serializer variance (whitespace, enum representation, integer-as-string). - /// - /// The activity to estimate. - /// An over-estimate of the serialized OTLP span size in bytes. - public static long EstimateActivityBytes(Activity activity) - { - if (activity is null) throw new ArgumentNullException(nameof(activity)); - - long total = SpanBaseOverhead; - - if (!string.IsNullOrEmpty(activity.DisplayName)) - { - total += Encoding.UTF8.GetByteCount(activity.DisplayName); - } - - foreach (var tag in activity.TagObjects) - { - total += AttrOverhead; - total += Encoding.UTF8.GetByteCount(tag.Key ?? string.Empty); - total += EstimateValueBytes(tag.Value); - } - - if (activity.Events != null) - { - foreach (var ev in activity.Events) - { - total += EventOverhead; - total += Encoding.UTF8.GetByteCount(ev.Name ?? string.Empty); - if (ev.Tags != null) - { - foreach (var tag in ev.Tags) - { - total += AttrOverhead; - total += Encoding.UTF8.GetByteCount(tag.Key ?? string.Empty); - total += EstimateValueBytes(tag.Value); - } - } - } - } - - return total; - } - - /// - /// Splits items into sub-batches whose cumulative estimated size stays under - /// . - /// - /// Invariants: - /// - /// Input order is preserved across chunks. - /// Empty input produces empty output. - /// A single item whose size exceeds - /// forms its own single-item chunk (never silently dropped). - /// No chunk is ever empty. - /// - /// - /// Item type. - /// The items to chunk. - /// Estimator function returning approximate serialized byte size of one item. - /// Upper bound on cumulative estimated size per chunk. - /// An ordered list of non-empty chunks. - public static List> ChunkBySize(IEnumerable items, Func getSize, long maxChunkBytes) - { - if (items is null) throw new ArgumentNullException(nameof(items)); - if (getSize is null) throw new ArgumentNullException(nameof(getSize)); - - var chunks = new List>(); - var current = new List(); - long currentBytes = 0; - - foreach (var item in items) - { - long itemBytes = getSize(item); - if (current.Count > 0 && currentBytes + itemBytes > maxChunkBytes) - { - chunks.Add(current); - current = new List(); - currentBytes = 0; - } - current.Add(item); - currentBytes += itemBytes; - } - - if (current.Count > 0) - { - chunks.Add(current); - } - - return chunks; - } - } -} diff --git a/src/Observability/Runtime/Tracing/Exporters/TokenResolverContext.cs b/src/Observability/Runtime/Tracing/Exporters/TokenResolverContext.cs deleted file mode 100644 index a47469c8..00000000 --- a/src/Observability/Runtime/Tracing/Exporters/TokenResolverContext.cs +++ /dev/null @@ -1,39 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters -{ - /// - /// Provides contextual information to the token resolver delegate. - /// - /// provides first-class access to agent identity fields (agent ID, - /// agentic user ID). and - /// together identify the cache key. Resolvers must still return app-only OBS tokens for - /// the exporting agent; the SDK never exchanges this context for delegated OBS tokens. - /// - /// - public class TokenResolverContext - { - /// - /// Initializes a new instance of the class. - /// - /// The agent identity associated with this request. - /// The tenant identifier (cache key). - public TokenResolverContext(AgentIdentity identity, string tenantId) - { - Identity = identity; - TenantId = tenantId; - } - - /// - /// Gets the agent identity associated with this token resolution request. - /// Contains the agent ID and agentic user ID (AAD Object ID) as first-class properties. - /// - public AgentIdentity Identity { get; } - - /// - /// Gets the tenant identifier (part of the cache key). - /// - public string TenantId { get; } - } -} diff --git a/src/Observability/Runtime/Tracing/MessageUtils.cs b/src/Observability/Runtime/Tracing/MessageUtils.cs deleted file mode 100644 index d2ab19cd..00000000 --- a/src/Observability/Runtime/Tracing/MessageUtils.cs +++ /dev/null @@ -1,138 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Collections.Generic; -using System.Text.Json; -using System.Text.Json.Serialization; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts.Messages; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing -{ - /// - /// Conversion and serialization helpers for OTEL gen-ai message format. - /// Provides normalization from plain string[] (backward compat) to the - /// structured array format, and non-throwing Serialize methods. - /// - internal static class MessageUtils - { - private static readonly JsonSerializerOptions SerializerOptions = CreateSerializerOptions(); - - private static readonly string DiagnosticFallback = - "[{\"role\":\"system\",\"parts\":[{\"type\":\"text\",\"content\":\"[serialization failed]\"}],\"finish_reason\":\"error\"}]"; - - private static JsonSerializerOptions CreateSerializerOptions() - { - var options = new JsonSerializerOptions - { - DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull, - PropertyNamingPolicy = JsonNamingPolicy.SnakeCaseLower, - WriteIndented = false, - }; - options.Converters.Add(new JsonStringEnumConverter(JsonNamingPolicy.SnakeCaseLower)); - options.Converters.Add(new MessagePartConverter()); - return options; - } - - // ------------------------------------------------------------------- - // Normalization: string → structured messages - // ------------------------------------------------------------------- - - /// - /// Normalizes a single string into an instance. - /// The string is wrapped as a user role . - /// - public static InputMessages NormalizeInputMessages(string content) - { - return NormalizeInputMessages(new[] { content }); - } - - /// - /// Normalizes a string array into an instance. - /// Each string is wrapped as a user role . - /// - public static InputMessages NormalizeInputMessages(IEnumerable messages) - { - var chatMessages = new List(); - foreach (var msg in messages) - { - if (msg == null) continue; - chatMessages.Add(new ChatMessage( - MessageRole.User, - new IMessagePart[] { new TextPart(msg) })); - } - - return new InputMessages(chatMessages); - } - - /// - /// Normalizes a single string into an instance. - /// The string is wrapped as an assistant role . - /// - public static OutputMessages NormalizeOutputMessages(string content) - { - return NormalizeOutputMessages(new[] { content }); - } - - /// - /// Normalizes a string array into an instance. - /// Each string is wrapped as an assistant role . - /// - public static OutputMessages NormalizeOutputMessages(IEnumerable messages) - { - var outputMsgs = new List(); - foreach (var msg in messages) - { - if (msg == null) continue; - outputMsgs.Add(new OutputMessage( - MessageRole.Assistant, - new IMessagePart[] { new TextPart(msg) })); - } - - return new OutputMessages(outputMsgs); - } - - // ------------------------------------------------------------------- - // Serialization - // ------------------------------------------------------------------- - - /// - /// Returns true if the string is a valid JSON object or array. - /// Primitives (strings, numbers, booleans, null) return false. - /// - public static bool IsJson(string? value) - { - if (string.IsNullOrWhiteSpace(value)) - { - return false; - } - - try - { - using var doc = JsonDocument.Parse(value!); - var kind = doc.RootElement.ValueKind; - return kind == JsonValueKind.Object || kind == JsonValueKind.Array; - } - catch (JsonException) - { - return false; - } - } - - /// - /// Serializes an object to a JSON string using the shared snake_case options. - /// Non-throwing; falls back to a diagnostic payload on error. - /// - public static string Serialize(object value) - { - try - { - return JsonSerializer.Serialize(value, value.GetType(), SerializerOptions); - } - catch (Exception) - { - return DiagnosticFallback; - } - } - } -} diff --git a/src/Observability/Runtime/Tracing/Processors/ActivityProcessor.cs b/src/Observability/Runtime/Tracing/Processors/ActivityProcessor.cs deleted file mode 100644 index 6b4ee325..00000000 --- a/src/Observability/Runtime/Tracing/Processors/ActivityProcessor.cs +++ /dev/null @@ -1,82 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Processors -{ - using Microsoft.Agents.A365.Observability.Runtime.Common; - using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; - using OpenTelemetry; - using System.Diagnostics; - - /// - /// Processes activity telemetry data by adding contextual baggage information. - /// - public sealed class ActivityProcessor : BaseProcessor - { - private static readonly string[] AttributeKeys = new[] - { - OpenTelemetryConstants.GenAiAgentIdKey, - OpenTelemetryConstants.GenAiAgentNameKey, - OpenTelemetryConstants.GenAiAgentDescriptionKey, - OpenTelemetryConstants.GenAiAgentVersionKey, - OpenTelemetryConstants.AgentEmailKey, - OpenTelemetryConstants.AgentBlueprintIdKey, - OpenTelemetryConstants.AgentAUIDKey, - OpenTelemetryConstants.AgentPlatformIdKey, - OpenTelemetryConstants.TenantIdKey, - OpenTelemetryConstants.GenAiConversationIdKey, - OpenTelemetryConstants.GenAiConversationItemLinkKey, - OpenTelemetryConstants.GenAiInputMessagesKey, - OpenTelemetryConstants.GenAiOutputMessagesKey, - OpenTelemetryConstants.GenAiToolCallResultKey, - OpenTelemetryConstants.GenAiToolNameKey, - OpenTelemetryConstants.GenAiToolCallIdKey, - OpenTelemetryConstants.GenAiToolDescriptionKey, - OpenTelemetryConstants.GenAiToolArgumentsKey, - OpenTelemetryConstants.GenAiToolTypeKey, - OpenTelemetryConstants.GenAiProviderNameKey, - OpenTelemetryConstants.SessionIdKey, - OpenTelemetryConstants.SessionDescriptionKey, - OpenTelemetryConstants.ChannelNameKey, - OpenTelemetryConstants.ChannelLinkKey, - OpenTelemetryConstants.UserIdKey, - OpenTelemetryConstants.UserNameKey, - OpenTelemetryConstants.UserEmailKey, - OpenTelemetryConstants.CallerClientIpKey, - OpenTelemetryConstants.ServiceNameKey, - }; - - private static readonly string[] InvokeAgentAttributeKeys = new[] - { - OpenTelemetryConstants.ServerAddressKey, - OpenTelemetryConstants.ServerPortKey, - }; - - /// - /// Called when an activity starts, adds tags for attributes listed in AttributeKeys. - /// - /// The activity that is starting. - public override void OnStart(Activity activity) - { - // Set telemetry SDK attributes - activity.CoalesceTag(OpenTelemetryConstants.TelemetrySdkNameKey, OpenTelemetryConstants.TelemetrySdkNameValue); - activity.CoalesceTag(OpenTelemetryConstants.TelemetrySdkLanguageKey, OpenTelemetryConstants.TelemetrySdkLanguageValue); - activity.CoalesceTag(OpenTelemetryConstants.TelemetrySdkVersionKey, OpenTelemetryConstants.TelemetrySdkVersionValue); - - foreach (var key in AttributeKeys) - { - activity.CoalesceTag(key, Baggage.Current.GetBaggage(key)); - } - - if (activity.OperationName == InvokeAgentScope.OperationName || - (activity.DisplayName != null && activity.DisplayName.StartsWith(InvokeAgentScope.OperationName))) - { - foreach (var key in InvokeAgentAttributeKeys) - { - activity.CoalesceTag(key, Baggage.Current.GetBaggage(key)); - } - } - - base.OnStart(activity); - } - } -} diff --git a/src/Observability/Runtime/Tracing/Processors/BatchActivityExportProcessorAsync.cs b/src/Observability/Runtime/Tracing/Processors/BatchActivityExportProcessorAsync.cs deleted file mode 100644 index 7ec72ba9..00000000 --- a/src/Observability/Runtime/Tracing/Processors/BatchActivityExportProcessorAsync.cs +++ /dev/null @@ -1,196 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters; -using OpenTelemetry; -using System; -using System.Collections.Concurrent; -using System.Collections.Generic; -using System.Diagnostics; -using System.Threading; -using System.Threading.Tasks; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Processors -{ - /// - /// Implements an async processor that batches objects before calling exporter asynchronously. - /// - public class BatchActivityExportProcessorAsync : BaseProcessor - { - internal const int DefaultMaxQueueSize = 2048; - internal const int DefaultScheduledDelayMilliseconds = 5000; - internal const int DefaultMaxExportBatchSize = 512; - - private readonly BaseExporterAsync exporter; - private readonly int maxQueueSize; - private readonly int scheduledDelayMilliseconds; - private readonly int maxExportBatchSize; - - private readonly ConcurrentQueue queue; - private readonly SemaphoreSlim signal; - private readonly CancellationTokenSource shutdownCts; - private Task? workerTask; - private bool disposed; - private readonly string friendlyTypeName; - - /// - /// Initializes a new instance of the class. - /// - /// The async exporter instance. - /// Maximum queue size. - /// Delay between exports in ms. - /// Max batch size per export. - public BatchActivityExportProcessorAsync( - BaseExporterAsync exporter, - int maxQueueSize = DefaultMaxQueueSize, - int scheduledDelayMilliseconds = DefaultScheduledDelayMilliseconds, - int maxExportBatchSize = DefaultMaxExportBatchSize) - { - this.exporter = exporter ?? throw new ArgumentNullException(nameof(exporter)); - this.maxQueueSize = maxQueueSize; - this.scheduledDelayMilliseconds = scheduledDelayMilliseconds; - this.maxExportBatchSize = maxExportBatchSize; - - this.queue = new ConcurrentQueue(); - this.signal = new SemaphoreSlim(0); - this.shutdownCts = new CancellationTokenSource(); - this.workerTask = Task.Run(ProcessLoopAsync); - this.friendlyTypeName = $"{this.GetType().Name}{{{exporter.GetType().Name}}}"; - } - - /// - /// Called when an is ended. - /// - /// The activity to export. - public override void OnEnd(Activity data) - { - if (data == null) - { - throw new ArgumentNullException(nameof(data)); - } - - if (!data.Recorded) - { - return; - } - - this.OnExport(data); - } - - /// - /// Enqueues activity data for export. If the queue is full, the data is dropped. - /// - /// The activity to export. - private void OnExport(Activity data) - { - if (disposed) throw new ObjectDisposedException(nameof(BatchActivityExportProcessorAsync)); - - if (queue.Count < maxQueueSize) - { - queue.Enqueue(data); - signal.Release(); - } - // else: drop, could count dropped - } - - /// - /// Forces the processor to flush all queued activities asynchronously. - /// - /// A token to monitor for cancellation requests. - /// A representing the asynchronous flush operation. - public async Task ForceFlushAsync(CancellationToken cancellationToken = default) - { - signal.Release(); - var sw = Stopwatch.StartNew(); - while (!queue.IsEmpty) - { - if (cancellationToken.IsCancellationRequested) break; - await Task.Delay(100, cancellationToken); - } - await exporter.ForceFlushAsync(cancellationToken); - } - - /// - /// Shuts down the processor and exporter asynchronously, releasing any resources if necessary. - /// - /// A token to monitor for cancellation requests. - /// A representing the asynchronous shutdown operation. - public async Task ShutdownAsync(CancellationToken cancellationToken = default) - { - shutdownCts.Cancel(); - signal.Release(); - - if (workerTask != null) - { - await workerTask; - } - - await exporter.ShutdownAsync(cancellationToken); - } - - /// - /// The main processing loop that batches and exports activities asynchronously. - /// - /// A representing the asynchronous processing loop. - private async Task ProcessLoopAsync() - { - var token = shutdownCts.Token; - try - { - while (!token.IsCancellationRequested) - { - await Task.WhenAny( - Task.Delay(scheduledDelayMilliseconds, token), - signal.WaitAsync(token)); - - if (queue.IsEmpty) continue; - - var batch = new List(maxExportBatchSize); - while (batch.Count < maxExportBatchSize && queue.TryDequeue(out var item)) - { - batch.Add(item); - } - - if (batch.Count > 0) - { - await exporter.ExportAsync(batch, token); - } - } - } - catch (OperationCanceledException) { } - } - - /// - /// Returns a string that represents the current processor. - /// - /// - /// A string containing the friendly type name of the processor and exporter. - /// - public override string ToString() - => this.friendlyTypeName; - - /// - /// Releases resources used by the processor and exporter. - /// - protected override void Dispose(bool disposing) - { - if (!disposed) - { - if (disposing) - { - shutdownCts.Cancel(); - signal.Dispose(); - shutdownCts.Dispose(); - try - { - exporter.Dispose(); - } - catch (Exception) - { - // handle/log as needed - } - } - disposed = true; - } - } - } -} diff --git a/src/Observability/Runtime/Tracing/Scopes/ApplyGuardrailScope.cs b/src/Observability/Runtime/Tracing/Scopes/ApplyGuardrailScope.cs deleted file mode 100644 index 03dd008d..00000000 --- a/src/Observability/Runtime/Tracing/Scopes/ApplyGuardrailScope.cs +++ /dev/null @@ -1,191 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Diagnostics; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes -{ - /// - /// Provides OpenTelemetry tracing scope for security guardrail evaluation operations. - /// - /// - /// - /// Describes a security guardian evaluation. Multiple guardian spans MAY exist under a single - /// operation span if multiple guardians are chained. - /// - /// - /// Guardian spans SHOULD be children of the operation span they are protecting - /// (e.g., inference or execute_tool spans). - /// - /// - public sealed class ApplyGuardrailScope : OpenTelemetryScope - { - - /// - /// Creates and starts a new scope for guardrail evaluation tracing. - /// - /// Details of the guardrail evaluation (target, decision, guardian info, policy). - /// Information about the agent being guarded. - /// Optional request details for conversation context. - /// Optional human user details. - /// Optional span configuration (parent context, timing, kind, span links). - /// A new ApplyGuardrailScope instance. - /// - /// - /// Certification Requirements: The following parameters must be set for the agent to pass certification requirements: - /// - /// - /// - /// - /// - /// - public static ApplyGuardrailScope Start( - GuardrailDetails details, - AgentDetails agentDetails, - Request? request = null, - UserDetails? userDetails = null, - SpanDetails? spanDetails = null) => new ApplyGuardrailScope(details, agentDetails, request, userDetails, spanDetails); - - private ApplyGuardrailScope( - GuardrailDetails details, - AgentDetails agentDetails, - Request? request, - UserDetails? userDetails, - SpanDetails? spanDetails) - : base( - operationName: OpenTelemetryConstants.ApplyGuardrailOperationName, - activityName: BuildActivityName(details), - agentDetails: agentDetails, - spanDetails: new SpanDetails(spanDetails?.SpanKind ?? ActivityKind.Internal, spanDetails?.ParentContext, spanDetails?.StartTime, spanDetails?.EndTime, spanDetails?.SpanLinks), - userDetails: userDetails) - { - // Required attributes - SetTagMaybe(OpenTelemetryConstants.GenAiSecurityDecisionTypeKey, details.DecisionType.ToString().ToLowerInvariant()); - SetTagMaybe(OpenTelemetryConstants.GenAiSecurityTargetTypeKey, details.TargetType); - - // Guardian attributes - SetTagMaybe(OpenTelemetryConstants.GenAiGuardianIdKey, details.GuardianId); - SetTagMaybe(OpenTelemetryConstants.GenAiGuardianNameKey, details.GuardianName); - SetTagMaybe(OpenTelemetryConstants.GenAiGuardianProviderNameKey, details.GuardianProviderName); - SetTagMaybe(OpenTelemetryConstants.GenAiGuardianVersionKey, details.GuardianVersion); - - // Target attributes - SetTagMaybe(OpenTelemetryConstants.GenAiSecurityTargetIdKey, details.TargetId); - - // Decision attributes - SetTagMaybe(OpenTelemetryConstants.GenAiSecurityDecisionReasonKey, details.DecisionReason); - SetTagMaybe(OpenTelemetryConstants.GenAiSecurityDecisionCodeKey, details.DecisionCode); - - // Policy attributes - SetTagMaybe(OpenTelemetryConstants.GenAiSecurityPolicyIdKey, details.PolicyId); - SetTagMaybe(OpenTelemetryConstants.GenAiSecurityPolicyNameKey, details.PolicyName); - SetTagMaybe(OpenTelemetryConstants.GenAiSecurityPolicyVersionKey, details.PolicyVersion); - - // Content attributes - SetTagMaybe(OpenTelemetryConstants.GenAiSecurityContentInputHashKey, details.ContentInputHash); - if (details.ContentModified.HasValue) - { - SetTagMaybe(OpenTelemetryConstants.GenAiSecurityContentModifiedKey, details.ContentModified.Value); - } - - // Correlation attributes - SetTagMaybe(OpenTelemetryConstants.GenAiSecurityExternalEventIdKey, details.ExternalEventId); - - // Request context - SetTagMaybe(OpenTelemetryConstants.GenAiSecurityContentInputValueKey, request?.Content); - SetTagMaybe(OpenTelemetryConstants.GenAiConversationIdKey, request?.ConversationId); - if (request?.Channel != null) - { - SetTagMaybe(OpenTelemetryConstants.ChannelNameKey, request.Channel.Name); - SetTagMaybe(OpenTelemetryConstants.ChannelLinkKey, request.Channel.Link); - } - } - - /// - /// Records an updated decision on the guardrail span. - /// Use this when the guardrail decision is determined after span creation. - /// - /// The decision type made by the guardian. - /// Optional human-readable explanation for the decision. - public void RecordDecision(GuardrailDecisionType decisionType, string? reason = null) - { - SetTagMaybe(OpenTelemetryConstants.GenAiSecurityDecisionTypeKey, decisionType.ToString().ToLowerInvariant()); - SetTagMaybe(OpenTelemetryConstants.GenAiSecurityDecisionReasonKey, reason); - } - - /// - /// Records the output content value for the guardrail evaluation (opt-in). - /// - /// The output content after guardrail processing. - public void RecordContentOutput(string outputValue) - { - SetTagMaybe(OpenTelemetryConstants.GenAiSecurityContentOutputValueKey, outputValue); - } - - /// - /// Records a security finding event on the current span. - /// Multiple findings may be recorded per guardrail evaluation. - /// - /// The security finding to record. - /// Thrown when is null. - public void RecordFinding(GuardrailFinding finding) - { - if (finding == null) - { - throw new ArgumentNullException(nameof(finding)); - } - - var tags = new ActivityTagsCollection - { - { OpenTelemetryConstants.GenAiSecurityRiskCategoryKey, finding.RiskCategory }, - { OpenTelemetryConstants.GenAiSecurityRiskSeverityKey, finding.RiskSeverity } - }; - - if (finding.PolicyDecisionType != null) - { - tags.Add(OpenTelemetryConstants.GenAiSecurityPolicyDecisionTypeKey, finding.PolicyDecisionType); - } - - if (finding.PolicyId != null) - { - tags.Add(OpenTelemetryConstants.GenAiSecurityPolicyIdKey, finding.PolicyId); - } - - if (finding.PolicyName != null) - { - tags.Add(OpenTelemetryConstants.GenAiSecurityPolicyNameKey, finding.PolicyName); - } - - if (finding.PolicyVersion != null) - { - tags.Add(OpenTelemetryConstants.GenAiSecurityPolicyVersionKey, finding.PolicyVersion); - } - - if (finding.RiskScore.HasValue) - { - tags.Add(OpenTelemetryConstants.GenAiSecurityRiskScoreKey, finding.RiskScore.Value); - } - - if (finding.RiskMetadata != null) - { - tags.Add(OpenTelemetryConstants.GenAiSecurityRiskMetadataKey, finding.RiskMetadata); - } - - var activityEvent = new ActivityEvent(OpenTelemetryConstants.GenAiSecurityFindingEventName, tags: tags); - AddEvent(activityEvent); - } - - private static string BuildActivityName(GuardrailDetails details) - { - if (!string.IsNullOrWhiteSpace(details.GuardianName)) - { - return $"{OpenTelemetryConstants.ApplyGuardrailOperationName} {details.GuardianName} {details.TargetType}"; - } - - return $"{OpenTelemetryConstants.ApplyGuardrailOperationName} {details.TargetType}"; - } - - } -} diff --git a/src/Observability/Runtime/Tracing/Scopes/AutoInstrumentationConstants.cs b/src/Observability/Runtime/Tracing/Scopes/AutoInstrumentationConstants.cs deleted file mode 100644 index 5b06d663..00000000 --- a/src/Observability/Runtime/Tracing/Scopes/AutoInstrumentationConstants.cs +++ /dev/null @@ -1,19 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes -{ - /// - /// Constants used for auto-instrumentation. - /// - public static class AutoInstrumentationConstants - { - /// The key for the input to a GenAI agent invocation. - /// Set by the Semantic Kernel OpenTelemetry integration for agent invocations. - public const string GenAiInvocationInputKey = "gen_ai.agent.invocation_input"; - - /// The key for the output of a GenAI agent invocation. - /// Set by the Semantic Kernel OpenTelemetry integration for agent invocations. - public const string GenAiInvocationOutputKey = "gen_ai.agent.invocation_output"; - } -} diff --git a/src/Observability/Runtime/Tracing/Scopes/ExecuteToolScope.cs b/src/Observability/Runtime/Tracing/Scopes/ExecuteToolScope.cs deleted file mode 100644 index 82a9e7d6..00000000 --- a/src/Observability/Runtime/Tracing/Scopes/ExecuteToolScope.cs +++ /dev/null @@ -1,142 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Collections.Generic; -using System.Diagnostics; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; - -using Microsoft.Agents.A365.Observability.Runtime.Tracing; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes -{ - /// - /// Provides OpenTelemetry tracing scope for AI tool execution operations. - /// - /// - /// Learn more about tool execution - /// - public sealed class ExecuteToolScope : OpenTelemetryScope - { - /// - /// The operation name for tool execution tracing. - /// - public const string OperationName = "execute_tool"; - - /// - /// Creates and starts a new scope for tool execution tracing. - /// - /// Request details for the tool execution. - /// Details of the tool call (name, args, type, call ID, description, endpoint). - /// Information about the agent executing the tool (service, version, identifiers). - /// Optional human user details. - /// Optional span configuration (parent context, timing, kind, span links). - /// Optional threat diagnostics summary containing security-related information about blocked actions. - /// A new ExecuteToolScope instance. - /// - /// - /// Certification Requirements: The following parameters must be set for the agent to pass certification requirements: - /// - /// - /// - /// - /// - /// - /// Learn more about certification requirements - /// - /// - public static ExecuteToolScope Start(Request request, ToolCallDetails details, AgentDetails agentDetails, UserDetails? userDetails = null, SpanDetails? spanDetails = null, ThreatDiagnosticsSummary? threatDiagnosticsSummary = null) => new ExecuteToolScope(request, details, agentDetails, userDetails, spanDetails, threatDiagnosticsSummary); - - private ExecuteToolScope(Request request, ToolCallDetails details, AgentDetails agentDetails, UserDetails? userDetails, SpanDetails? spanDetails, ThreatDiagnosticsSummary? threatDiagnosticsSummary) - : base( - operationName: OperationName, - activityName: $"{OperationName} {details.ToolName}", - agentDetails: agentDetails, - spanDetails: new SpanDetails(spanDetails?.SpanKind ?? ActivityKind.Internal, spanDetails?.ParentContext, spanDetails?.StartTime, spanDetails?.EndTime, spanDetails?.SpanLinks), - userDetails: userDetails) - { - var (toolName, arguments, toolCallId, description, toolType, endpoint, toolServerName) = details; - SetTagMaybe(OpenTelemetryConstants.GenAiToolNameKey, toolName); - - // Per OTEL spec: arguments SHOULD be recorded in structured form. - // Prefer ArgumentsObject (dict → JSON); fall back to string with JSON check. - if (details.ArgumentsObject != null) - { - SetTagMaybe(OpenTelemetryConstants.GenAiToolArgumentsKey, MessageUtils.Serialize(details.ArgumentsObject)); - } - else if (arguments != null) - { - if (MessageUtils.IsJson(arguments)) - { - SetTagMaybe(OpenTelemetryConstants.GenAiToolArgumentsKey, arguments); - } - else - { - SetTagMaybe(OpenTelemetryConstants.GenAiToolArgumentsKey, - MessageUtils.Serialize(new Dictionary { { "arguments", arguments } })); - } - } - - SetTagMaybe(OpenTelemetryConstants.GenAiToolTypeKey, toolType); - SetTagMaybe(OpenTelemetryConstants.GenAiToolCallIdKey, toolCallId); - SetTagMaybe(OpenTelemetryConstants.GenAiToolDescriptionKey, description); - SetTagMaybe(OpenTelemetryConstants.GenAiToolServerNameKey, toolServerName); - SetTagMaybe(OpenTelemetryConstants.ThreatDiagnosticsSummaryKey, threatDiagnosticsSummary?.ToJson()); - SetTagMaybe(OpenTelemetryConstants.GenAiConversationIdKey, request?.ConversationId); - - if (endpoint != null) - { - SetTagMaybe(OpenTelemetryConstants.ServerAddressKey, endpoint.Host); - if (endpoint.Port != 443) - { - SetTagMaybe(OpenTelemetryConstants.ServerPortKey, endpoint.Port.ToString()); - } - } - - if (request?.Channel != null) - { - SetTagMaybe(OpenTelemetryConstants.ChannelNameKey, request.Channel.Name); - SetTagMaybe(OpenTelemetryConstants.ChannelLinkKey, request.Channel.Link); - } - } - - /// - /// Records response information for telemetry tracking. - /// Per OTEL spec, the result SHOULD be recorded in structured form. - /// If the string is already valid JSON, it is recorded as-is. - /// Otherwise, it is wrapped as {"result":"..."}. - /// - public void RecordResponse(string response) - { - if (MessageUtils.IsJson(response)) - { - SetTagMaybe(OpenTelemetryConstants.GenAiToolCallResultKey, response); - } - else - { - SetTagMaybe(OpenTelemetryConstants.GenAiToolCallResultKey, - MessageUtils.Serialize(new Dictionary { { "result", response ?? string.Empty } })); - } - } - - /// - /// Records a structured tool call result for telemetry tracking. - /// Per OTEL spec, the result SHOULD be recorded in structured form. - /// The dictionary is serialized to JSON. - /// - /// Tool call result as a structured dictionary. - public void RecordResponse(IDictionary result) - { - SetTagMaybe(OpenTelemetryConstants.GenAiToolCallResultKey, MessageUtils.Serialize(result)); - } - - /// - /// Records threat diagnostics summary for telemetry tracking. - /// - /// The threat diagnostics summary containing security-related information about blocked actions. - public void RecordThreatDiagnosticsSummary(ThreatDiagnosticsSummary threatDiagnosticsSummary) - { - SetTagMaybe(OpenTelemetryConstants.ThreatDiagnosticsSummaryKey, threatDiagnosticsSummary.ToJson()); - } - } -} \ No newline at end of file diff --git a/src/Observability/Runtime/Tracing/Scopes/InferenceScope.cs b/src/Observability/Runtime/Tracing/Scopes/InferenceScope.cs deleted file mode 100644 index 994d4141..00000000 --- a/src/Observability/Runtime/Tracing/Scopes/InferenceScope.cs +++ /dev/null @@ -1,153 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Diagnostics; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts.Messages; -using static Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes.OpenTelemetryConstants; - -using Microsoft.Agents.A365.Observability.Runtime.Tracing; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes -{ - /// - /// Provides OpenTelemetry tracing scope for generative AI inference operations. - /// - /// - /// Learn more about inference - /// - public sealed class InferenceScope : OpenTelemetryScope - { - /// - /// Creates and starts a new scope for inference tracing. - /// - /// Request details for the inference. - /// Details of the inference call (operation name, model, provider, token usage, finish reasons, response ID). - /// Information about the agent executing the inference (service, version, identifiers). - /// Optional human user details. - /// Optional span configuration (parent context, timing, span links). - /// A new InferenceScope instance. - /// - /// - /// Certification Requirements: The following parameters must be set for the agent to pass certification requirements: - /// - /// - /// - /// - /// - /// - /// Learn more about certification requirements - /// - /// - public static InferenceScope Start(Request request, InferenceCallDetails details, AgentDetails agentDetails, UserDetails? userDetails = null, SpanDetails? spanDetails = null) => new InferenceScope(request, details, agentDetails, userDetails, spanDetails); - - private InferenceScope(Request request, InferenceCallDetails details, AgentDetails agentDetails, UserDetails? userDetails, SpanDetails? spanDetails) - : base( - operationName: details.OperationName.ToString(), - activityName: $"{details.OperationName} {details.Model}", - agentDetails: agentDetails, - spanDetails: spanDetails ?? new SpanDetails(ActivityKind.Client), - userDetails: userDetails) - { - SetTagMaybe(GenAiOperationNameKey, details.OperationName.ToString()); - SetTagMaybe(GenAiRequestModelKey, details.Model); - SetTagMaybe(GenAiProviderNameKey, details.ProviderName); - SetTagMaybe(GenAiUsageInputTokensKey, details.InputTokens?.ToString()); - SetTagMaybe(GenAiUsageOutputTokensKey, details.OutputTokens?.ToString()); - SetTagMaybe(GenAiResponseFinishReasonsKey, details.FinishReasons != null ? string.Join(",", details.FinishReasons) : null); - SetTagMaybe(GenAiConversationIdKey, request?.ConversationId); - - if (request?.InputContent != null) - { - RecordInputMessages(request.InputContent); - } - else if (request?.Content != null) - { - RecordInputMessages(new[] { request.Content }); - } - - if (request?.Channel != null) - { - SetTagMaybe(ChannelNameKey, request.Channel.Name); - SetTagMaybe(ChannelLinkKey, request.Channel.Link); - } - } - - /// - /// Records the input messages for telemetry tracking. - /// Plain strings are auto-wrapped as OTEL ChatMessage with role "user". - /// - public void RecordInputMessages(string[] messages) - { - var wrapper = MessageUtils.NormalizeInputMessages(messages); - SetTagMaybe(GenAiInputMessagesKey, MessageUtils.Serialize(wrapper)); - } - - /// - /// Records structured input messages for telemetry tracking. - /// - /// The versioned input messages wrapper. - public void RecordInputMessages(InputMessages messages) - { - SetTagMaybe(GenAiInputMessagesKey, MessageUtils.Serialize(messages)); - } - - /// - /// Records the output messages for telemetry tracking. - /// Plain strings are auto-wrapped as OTEL OutputMessage with role "assistant". - /// - public void RecordOutputMessages(string[] messages) - { - var wrapper = MessageUtils.NormalizeOutputMessages(messages); - SetTagMaybe(GenAiOutputMessagesKey, MessageUtils.Serialize(wrapper)); - } - - /// - /// Records structured output messages for telemetry tracking. - /// - /// The versioned output messages wrapper. - public void RecordOutputMessages(OutputMessages messages) - { - SetTagMaybe(GenAiOutputMessagesKey, MessageUtils.Serialize(messages)); - } - - /// - /// Records the number of input tokens for telemetry tracking. - /// - public void RecordInputTokens(int inputTokens) - { - SetTagMaybe(GenAiUsageInputTokensKey, inputTokens.ToString()); - } - - /// - /// Records the number of output tokens for telemetry tracking. - /// - public void RecordOutputTokens(int outputTokens) - { - SetTagMaybe(GenAiUsageOutputTokensKey, outputTokens.ToString()); - } - - /// - /// Records the finish reasons for telemetry tracking. - /// - public void RecordFinishReasons(string[] finishReasons) - { - if (finishReasons != null) - { - SetTagMaybe(GenAiResponseFinishReasonsKey, string.Join(",", finishReasons)); - } - } - - /// - /// Records the agent's thought process for telemetry tracking. - /// - public void RecordThoughtProcess(string thoughtProcess) - { - if (!string.IsNullOrEmpty(thoughtProcess)) - { - SetTagMaybe(GenAiAgentThoughtProcessKey, thoughtProcess); - } - } - } -} \ No newline at end of file diff --git a/src/Observability/Runtime/Tracing/Scopes/InvokeAgentScope.cs b/src/Observability/Runtime/Tracing/Scopes/InvokeAgentScope.cs deleted file mode 100644 index bc8be4c5..00000000 --- a/src/Observability/Runtime/Tracing/Scopes/InvokeAgentScope.cs +++ /dev/null @@ -1,177 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Diagnostics; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts.Messages; - -using Microsoft.Agents.A365.Observability.Runtime.Tracing; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes -{ - /// - /// Provides OpenTelemetry tracing scope for AI agent invocation operations. - /// - public sealed class InvokeAgentScope : OpenTelemetryScope - { - /// - /// The operation name for agent invocation tracing. - /// - /// - /// - /// Learn more about Agent Invocation - /// - /// - public const string OperationName = "invoke_agent"; - - /// - /// Creates and starts a new scope for agent invocation tracing. - /// - /// The request details for the invocation. - /// Scope-level configuration (endpoint). - /// The details of the agent being invoked. - /// Optional composite caller details (human user and/or calling agent for A2A scenarios). - /// Optional span configuration (parent context, timing, kind, span links). - /// Optional threat diagnostics summary containing security-related information about blocked actions. - /// A new InvokeAgentScope instance. - /// - /// - /// Certification Requirements: The following parameters must be set (i.e., not null) for the agent to pass certification requirements: - /// - /// - /// - /// - /// - /// - /// - /// Learn more about certification requirements - /// - /// - public static InvokeAgentScope Start( - Request request, - InvokeAgentScopeDetails scopeDetails, - AgentDetails agentDetails, - CallerDetails? callerDetails = null, - SpanDetails? spanDetails = null, - ThreatDiagnosticsSummary? threatDiagnosticsSummary = null) => new InvokeAgentScope(request, scopeDetails, agentDetails, callerDetails, spanDetails, threatDiagnosticsSummary); - - private InvokeAgentScope( - Request request, - InvokeAgentScopeDetails scopeDetails, - AgentDetails agentDetails, - CallerDetails? callerDetails, - SpanDetails? spanDetails, - ThreatDiagnosticsSummary? threatDiagnosticsSummary) - : base( - operationName: OperationName, - activityName: string.IsNullOrWhiteSpace(agentDetails?.AgentName) - ? OperationName - : $"invoke_agent {agentDetails!.AgentName}", - agentDetails: agentDetails!, - spanDetails: new SpanDetails(spanDetails?.SpanKind ?? ActivityKind.Client, spanDetails?.ParentContext, spanDetails?.StartTime, spanDetails?.EndTime, spanDetails?.SpanLinks), - userDetails: callerDetails?.UserDetails) - { - SetTagMaybe(OpenTelemetryConstants.SessionIdKey, request?.SessionId); - SetTagMaybe(OpenTelemetryConstants.GenAiConversationIdKey, request?.ConversationId); - SetTagMaybe(OpenTelemetryConstants.ThreatDiagnosticsSummaryKey, threatDiagnosticsSummary?.ToJson()); - - var endpoint = scopeDetails?.Endpoint; - if (endpoint != null) - { - SetTagMaybe(OpenTelemetryConstants.ServerAddressKey, endpoint.Host); - if (endpoint.Port != 443) - { - SetTagMaybe(OpenTelemetryConstants.ServerPortKey, endpoint.Port.ToString()); - } - } - - // Set request metadata - if (request?.Channel != null) - { - SetTagMaybe(OpenTelemetryConstants.ChannelNameKey, request.Channel.Name); - SetTagMaybe(OpenTelemetryConstants.ChannelLinkKey, request.Channel.Link); - } - - if (request?.InputContent != null) - { - RecordInputMessages(request.InputContent); - } - else if (request?.Content != null) - { - RecordInputMessages(new[] { request.Content }); - } - - // Set caller details tags - if (callerDetails != null) - { - var callerAgentDetails = callerDetails.CallerAgentDetails; - if (callerAgentDetails != null) - { - SetTagMaybe(OpenTelemetryConstants.CallerAgentNameKey, callerAgentDetails.AgentName); - SetTagMaybe(OpenTelemetryConstants.CallerAgentIdKey, callerAgentDetails.AgentId); - SetTagMaybe(OpenTelemetryConstants.CallerAgentBlueprintIdKey, callerAgentDetails.AgentBlueprintId); - SetTagMaybe(OpenTelemetryConstants.CallerAgentAUIDKey, callerAgentDetails.AgenticUserId); - SetTagMaybe(OpenTelemetryConstants.CallerAgentEmailKey, callerAgentDetails.AgenticUserEmail); - SetTagMaybe(OpenTelemetryConstants.CallerAgentPlatformIdKey, callerAgentDetails.AgentPlatformId); - SetTagMaybe(OpenTelemetryConstants.CallerAgentVersionKey, callerAgentDetails.AgentVersion); - } - } - } - - /// - /// Records response information for telemetry tracking. - /// - public void RecordResponse(string response) - { - this.RecordOutputMessages(messages: new string[] { response }); - } - - /// - /// Records the input messages for telemetry tracking. - /// Plain strings are auto-wrapped as OTEL ChatMessage with role "user". - /// - public void RecordInputMessages(string[] messages) - { - var wrapper = MessageUtils.NormalizeInputMessages(messages); - SetTagMaybe(OpenTelemetryConstants.GenAiInputMessagesKey, MessageUtils.Serialize(wrapper)); - } - - /// - /// Records structured input messages for telemetry tracking. - /// - /// The versioned input messages wrapper. - public void RecordInputMessages(InputMessages messages) - { - SetTagMaybe(OpenTelemetryConstants.GenAiInputMessagesKey, MessageUtils.Serialize(messages)); - } - - /// - /// Records the output messages for telemetry tracking. - /// Plain strings are auto-wrapped as OTEL OutputMessage with role "assistant". - /// - public void RecordOutputMessages(string[] messages) - { - var wrapper = MessageUtils.NormalizeOutputMessages(messages); - SetTagMaybe(OpenTelemetryConstants.GenAiOutputMessagesKey, MessageUtils.Serialize(wrapper)); - } - - /// - /// Records structured output messages for telemetry tracking. - /// - /// The versioned output messages wrapper. - public void RecordOutputMessages(OutputMessages messages) - { - SetTagMaybe(OpenTelemetryConstants.GenAiOutputMessagesKey, MessageUtils.Serialize(messages)); - } - - /// - /// Records threat diagnostics summary for telemetry tracking. - /// - /// The threat diagnostics summary containing security-related information about blocked actions. - public void RecordThreatDiagnosticsSummary(ThreatDiagnosticsSummary threatDiagnosticsSummary) - { - SetTagMaybe(OpenTelemetryConstants.ThreatDiagnosticsSummaryKey, threatDiagnosticsSummary.ToJson()); - } - } -} \ No newline at end of file diff --git a/src/Observability/Runtime/Tracing/Scopes/OpenTelemetryConstants.cs b/src/Observability/Runtime/Tracing/Scopes/OpenTelemetryConstants.cs deleted file mode 100644 index c51053f8..00000000 --- a/src/Observability/Runtime/Tracing/Scopes/OpenTelemetryConstants.cs +++ /dev/null @@ -1,291 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System.Runtime.Serialization; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes -{ - #pragma warning disable CS1591 // XML documentation not required for constant definitions. - /// - /// OpenTelemetry constant keys and values used across the Microsoft Agent 365 SDK. - /// - public static class OpenTelemetryConstants - { - public const string EnableOpenTelemetrySwitch = "Azure.Experimental.EnableActivitySource"; - public const string SourceName = "Agent365Sdk"; - - public const string ServerAddressKey = "server.address"; - public const string ServerPortKey = "server.port"; - public const string SessionIdKey = "microsoft.session.id"; - public const string SessionDescriptionKey = "microsoft.session.description"; - public const string TenantIdKey = "microsoft.tenant.id"; - - public const string GenAiClientOperationDurationMetricName = "gen_ai.client.operation.duration"; - public const string GenAiRequestModelKey = "gen_ai.request.model"; - public const string GenAiResponseFinishReasonsKey = "gen_ai.response.finish_reasons"; - - public const string GenAiConversationIdKey = "gen_ai.conversation.id"; - public const string GenAiConversationItemLinkKey = "microsoft.conversation.item.link"; - public const string GenAiUsageInputTokensKey = "gen_ai.usage.input_tokens"; - public const string GenAiUsageOutputTokensKey = "gen_ai.usage.output_tokens"; - public const string GenAiProviderNameKey = "gen_ai.provider.name"; - public const string GenAiInputMessagesKey = "gen_ai.input.messages"; - public const string GenAiOutputMessagesKey = "gen_ai.output.messages"; - public const string GenAiUserMessageEventName = "gen_ai.user.message"; - public const string GenAiChoiceEventName = "gen_ai.choice"; - public const string GenAiAgentInvocationInputKey = "gen_ai.agent.invocation_input"; - public const string GenAiAgentInvocationOutputKey = "gen_ai.agent.invocation_output"; - - [DataContract] - public enum OperationNames - { - [EnumMember(Value = "InvokeAgent")] - InvokeAgent, - - [EnumMember(Value = "ExecuteInference")] - ExecuteInference, - - [EnumMember(Value = "ExecuteTool")] - ExecuteTool, - - [EnumMember(Value = "OutputMessages")] - OutputMessages, - - [EnumMember(Value = "ApplyGuardrail")] - ApplyGuardrail - } - - /// - /// The operation name value for apply_guardrail spans. - /// - public const string ApplyGuardrailOperationName = "apply_guardrail"; - - // Channel dimensions (renamed from gen_ai.channel.* to microsoft.channel.*) - public const string ChannelNameKey = "microsoft.channel.name"; - public const string ChannelLinkKey = "microsoft.channel.link"; - - // Target agent dimensions - public const string GenAiAgentIdKey = "gen_ai.agent.id"; - public const string GenAiAgentNameKey = "gen_ai.agent.name"; - public const string GenAiAgentDescriptionKey = "gen_ai.agent.description"; - public const string GenAiAgentVersionKey = "gen_ai.agent.version"; - public const string AgentAUIDKey = "microsoft.agent.user.id"; - public const string AgentEmailKey = "microsoft.agent.user.email"; - public const string AgentBlueprintIdKey = "microsoft.a365.agent.blueprint.id"; - public const string AgentPlatformIdKey = "microsoft.a365.agent.platform.id"; - - // Human caller dimensions (OTel user.* namespace) - public const string UserIdKey = "user.id"; - public const string UserEmailKey = "user.email"; - public const string UserNameKey = "user.name"; - public const string CallerClientIpKey = "client.address"; - - // Caller agent dimensions (renamed from gen_ai.caller.agent.* to microsoft.a365.caller.agent.*) - public const string CallerAgentNameKey = "microsoft.a365.caller.agent.name"; - public const string CallerAgentIdKey = "microsoft.a365.caller.agent.id"; - public const string CallerAgentBlueprintIdKey = "microsoft.a365.caller.agent.blueprint.id"; - public const string CallerAgentAUIDKey = "microsoft.a365.caller.agent.user.id"; - public const string CallerAgentEmailKey = "microsoft.a365.caller.agent.user.email"; - public const string CallerAgentPlatformIdKey = "microsoft.a365.caller.agent.platform.id"; - public const string CallerAgentVersionKey = "microsoft.a365.caller.agent.version"; - - // Service attributes - public const string ServiceNameKey = "service.name"; - - // Telemetry SDK attributes - public const string TelemetrySdkNameKey = "telemetry.sdk.name"; - public const string TelemetrySdkLanguageKey = "telemetry.sdk.language"; - public const string TelemetrySdkVersionKey = "telemetry.sdk.version"; - public const string TelemetrySdkNameValue = "A365ObservabilitySDK"; - public const string TelemetrySdkLanguageValue = "dotnet"; - - /// - /// Gets the telemetry SDK version dynamically from the assembly. - /// - public static string TelemetrySdkVersionValue => - typeof(OpenTelemetryConstants).Assembly.GetName().Version?.ToString() ?? "unknown"; - - #region Public Constants - /// - /// The GenAI operation name key. - /// - public const string GenAiOperationNameKey = "gen_ai.operation.name"; - - /// - /// The error message key. - /// - public const string ErrorMessageKey = "error.message"; - - /// - /// The error type key. - /// - public const string ErrorTypeKey = "error.type"; - - #region tool call keys - /// - /// The GenAI tool name key. - /// - public const string GenAiToolNameKey = "gen_ai.tool.name"; - - /// - /// The GenAI tool call identifier key. - /// - public const string GenAiToolCallIdKey = "gen_ai.tool.call.id"; - - /// - /// The GenAI tool description key. - /// - public const string GenAiToolDescriptionKey = "gen_ai.tool.description"; - - /// - /// The GenAI tool call arguments key. - /// - public const string GenAiToolArgumentsKey = "gen_ai.tool.call.arguments"; - - /// - /// The GenAI tool type key. - /// - public const string GenAiToolTypeKey = "gen_ai.tool.type"; - - /// - /// The GenAI tool server name key. - /// - public const string GenAiToolServerNameKey = "gen_ai.tool.server.name"; - - /// - /// The GenAI tool call result key. - /// - public const string GenAiToolCallResultKey = "gen_ai.tool.call.result"; - #endregion - - /// - /// The threat diagnostics summary key. - /// - public const string ThreatDiagnosticsSummaryKey = "threat.diagnostics.summary"; - - /// - /// The GenAI agent thought process key. - /// - public const string GenAiAgentThoughtProcessKey = "microsoft.a365.agent.thought.process"; - - #endregion - - #region guardrail keys - /// - /// The guardian identifier key. - /// - public const string GenAiGuardianIdKey = "microsoft.guardian.id"; - - /// - /// The guardian name key. - /// - public const string GenAiGuardianNameKey = "microsoft.guardian.name"; - - /// - /// The guardian provider name key. - /// - public const string GenAiGuardianProviderNameKey = "microsoft.guardian.provider.name"; - - /// - /// The guardian version key. - /// - public const string GenAiGuardianVersionKey = "microsoft.guardian.version"; - - /// - /// The security decision type key. - /// - public const string GenAiSecurityDecisionTypeKey = "microsoft.security.decision.type"; - - /// - /// The security decision reason key. - /// - public const string GenAiSecurityDecisionReasonKey = "microsoft.security.decision.reason"; - - /// - /// The security decision code key. - /// - public const string GenAiSecurityDecisionCodeKey = "microsoft.security.decision.code"; - - /// - /// The security target type key. - /// - public const string GenAiSecurityTargetTypeKey = "microsoft.security.target.type"; - - /// - /// The security target identifier key. - /// - public const string GenAiSecurityTargetIdKey = "microsoft.security.target.id"; - - /// - /// The security policy identifier key. - /// - public const string GenAiSecurityPolicyIdKey = "microsoft.security.policy.id"; - - /// - /// The security policy name key. - /// - public const string GenAiSecurityPolicyNameKey = "microsoft.security.policy.name"; - - /// - /// The security policy version key. - /// - public const string GenAiSecurityPolicyVersionKey = "microsoft.security.policy.version"; - - /// - /// The security content input hash key. - /// - public const string GenAiSecurityContentInputHashKey = "microsoft.security.content.input.hash"; - - /// - /// The security content modified key. - /// - public const string GenAiSecurityContentModifiedKey = "microsoft.security.content.modified"; - - /// - /// The security external event identifier key for SIEM correlation. - /// - public const string GenAiSecurityExternalEventIdKey = "microsoft.security.external_event_id"; - - /// - /// The security content input value key (opt-in). - /// - public const string GenAiSecurityContentInputValueKey = "microsoft.security.content.input.value"; - - /// - /// The security content output value key (opt-in). - /// - public const string GenAiSecurityContentOutputValueKey = "microsoft.security.content.output.value"; - - /// - /// The security finding event name. - /// - public const string GenAiSecurityFindingEventName = "microsoft.security.finding"; - - /// - /// The security risk category key. - /// - public const string GenAiSecurityRiskCategoryKey = "microsoft.security.risk.category"; - - /// - /// The security risk severity key. - /// - public const string GenAiSecurityRiskSeverityKey = "microsoft.security.risk.severity"; - - /// - /// The security risk score key. - /// - public const string GenAiSecurityRiskScoreKey = "microsoft.security.risk.score"; - - /// - /// The security risk metadata key. - /// - public const string GenAiSecurityRiskMetadataKey = "microsoft.security.risk.metadata"; - - /// - /// The security policy decision type key (per-finding decision). - /// - public const string GenAiSecurityPolicyDecisionTypeKey = "microsoft.security.policy.decision.type"; - #endregion - } - #pragma warning restore CS1591 -} diff --git a/src/Observability/Runtime/Tracing/Scopes/OpenTelemetryScope.cs b/src/Observability/Runtime/Tracing/Scopes/OpenTelemetryScope.cs deleted file mode 100644 index cffaa376..00000000 --- a/src/Observability/Runtime/Tracing/Scopes/OpenTelemetryScope.cs +++ /dev/null @@ -1,303 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Collections.Generic; -using System.Diagnostics; -using System.Diagnostics.Metrics; -using System.Threading; -using System.Threading.Tasks; -using Azure; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using static Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes.OpenTelemetryConstants; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes -{ - /// - /// Base class for OpenTelemetry tracing scopes in the AI SDK, providing common telemetry functionality. - /// - public abstract class OpenTelemetryScope : IDisposable - { - private static readonly ActivitySource ActivitySource = new ActivitySource(SourceName); - private static readonly Meter Meter = new Meter(SourceName); - - private static readonly Histogram Duration = Meter.CreateHistogram( - GenAiClientOperationDurationMetricName, "s", "Measures GenAI operation duration."); - - private readonly Activity? activity; - private readonly Stopwatch? duration; - private readonly TagList commonTags; - private DateTimeOffset? customStartTime; - private DateTimeOffset? customEndTime; - - private string? errorType; - private Exception? exception; - private int hasEnded = 0; - - /// - /// Initializes a new instance of the OpenTelemetryScope class. - /// - /// The name of the operation being traced. - /// The name of the activity for display purposes. - /// Optional agent details. Tenant ID is read from . - /// Optional span configuration including parent context, start/end times, - /// span kind, and span links. Subclasses may override before calling this constructor; - /// defaults to . - /// Optional human caller identity details (id, email, name, client IP). - protected OpenTelemetryScope(string operationName, string activityName, AgentDetails agentDetails, SpanDetails? spanDetails = null, UserDetails? userDetails = null) - { - var kind = spanDetails?.SpanKind ?? ActivityKind.Client; - var parentContext = spanDetails?.ParentContext; - var startTime = spanDetails?.StartTime; - var endTime = spanDetails?.EndTime; - var spanLinks = spanDetails?.SpanLinks; - - customStartTime = startTime; - customEndTime = endTime; - activity = parentContext.HasValue && parentContext.Value.TraceId != default - ? ActivitySource.CreateActivity(activityName, kind, parentContext.Value, links: spanLinks) - : ActivitySource.CreateActivity(activityName, kind, default(ActivityContext), links: spanLinks); - - if (startTime != null) - { - activity?.SetStartTime(startTime.Value.UtcDateTime); - } - - commonTags = new TagList - { - { GenAiOperationNameKey, operationName }, - }; - - foreach (var kv in commonTags) - { - activity?.SetTag(kv.Key, kv.Value); - } - - if (agentDetails != null) - { - SetTagMaybe(GenAiAgentIdKey, agentDetails.AgentId); - SetTagMaybe(GenAiAgentNameKey, agentDetails.AgentName); - SetTagMaybe(GenAiAgentDescriptionKey, agentDetails.AgentDescription); - SetTagMaybe(GenAiAgentVersionKey, agentDetails.AgentVersion); - SetTagMaybe(AgentAUIDKey, agentDetails.AgenticUserId); - SetTagMaybe(AgentEmailKey, agentDetails.AgenticUserEmail); - SetTagMaybe(AgentBlueprintIdKey, agentDetails.AgentBlueprintId); - SetTagMaybe(AgentPlatformIdKey, agentDetails.AgentPlatformId); - SetTagMaybe(TenantIdKey, agentDetails.TenantId); - } - - if (userDetails != null) - { - SetTagMaybe(UserIdKey, userDetails.UserId); - SetTagMaybe(UserEmailKey, userDetails.UserEmail); - SetTagMaybe(UserNameKey, userDetails.UserName); - SetTagMaybe(CallerClientIpKey, userDetails.UserClientIP?.ToString()); - } - - // Only start the stopwatch if no custom start time is provided - if (!customStartTime.HasValue) - { - duration = Stopwatch.StartNew(); - } - - activity?.Start(); - } - - /// - /// Log the error. - /// - /// Exception thrown by completion call. - public void RecordError(Exception e) - { - if (e is RequestFailedException requestFailed && requestFailed.Status != 0) - { - errorType = requestFailed.Status.ToString(); - } - else - { - errorType = e.GetType().FullName ?? "error"; - } - - exception = e; - } - - /// - /// Record the task cancellation event. - /// - public void RecordCancellation() - { - errorType = typeof(TaskCanceledException).FullName; - exception = null; - } - - /// - /// Sets a custom start time for the scope. This allows for manual control of the scope start time. - /// Can be used in addition to or instead of setting start time via constructor. - /// - /// The start time to set for this scope. - public void SetStartTime(DateTimeOffset startTime) - { - customStartTime = startTime; - activity?.SetStartTime(startTime.UtcDateTime); - } - - /// - /// Sets a custom end time for the scope. This allows for manual control of the scope duration. - /// - /// The end time to set for this scope. - public void SetEndTime(DateTimeOffset endTime) - { - customEndTime = endTime; - } - - /// - /// Record the events and metrics associated with the response. - /// - private void End() - { - var finalTags = commonTags; - if (errorType != null) - { - finalTags.Add(ErrorTypeKey, errorType); - activity?.SetTag(ErrorTypeKey, errorType); - activity?.SetStatus(ActivityStatusCode.Error, exception?.Message); - } - - // Calculate duration based on custom times if provided, otherwise use stopwatch - double durationSeconds; - if (customStartTime.HasValue && customEndTime.HasValue) - { - durationSeconds = (customEndTime.Value - customStartTime.Value).TotalSeconds; - // Set the end time on the activity if we have custom times - activity?.SetEndTime(customEndTime.Value.UtcDateTime); - } - else if (customStartTime.HasValue && !customEndTime.HasValue) - { - // Start time was custom but end time is now, calculate from custom start to now - var endTime = DateTimeOffset.UtcNow; - durationSeconds = (endTime - customStartTime.Value).TotalSeconds; - activity?.SetEndTime(endTime.UtcDateTime); - } - else - { - // Use stopwatch for normal operation - durationSeconds = duration?.Elapsed.TotalSeconds ?? 0; - } - - Duration.Record(durationSeconds, finalTags); - } - - /// - /// Disposes the scope and finalizes telemetry data collection. - /// - public void Dispose() - { - // check if the scope has already ended - if (Interlocked.Exchange(ref hasEnded, 1) == 0) - { - End(); - activity?.Dispose(); - } - } - - /// - /// Set the tag on the activity if the tag is present. - /// - /// The name of tag to set. - /// Nullable value to be set. - public void SetTagMaybe(string name, object? value) - { - if (value != null) - { - activity?.SetTag(name, value); - } - } - - /// - /// Records multiple attribute key/value pairs for telemetry tracking. - /// - /// Collection of attribute key/value pairs. - public void RecordAttributes(IEnumerable> attributes) - { - if (attributes is null) return; - foreach (var kv in attributes) - { - if (string.IsNullOrWhiteSpace(kv.Key)) continue; - activity?.SetTag(kv.Key, kv.Value); - } - } - - /// - /// Adds baggage to the current activity for distributed tracing context propagation. - /// - /// The baggage key. - /// The baggage value. - protected void AddBaggage(string key, string value) - { - activity?.AddBaggage(key, value); - } - - /// - /// Adds an event to the current activity. - /// - /// The event to add. - protected void AddEvent(ActivityEvent activityEvent) - { - activity?.AddEvent(activityEvent); - } - - /// - /// Gets the for this scope's span. - /// - /// - /// The returned context can be passed as parentContext to child scope - /// Start methods to establish a parent-child relationship within the - /// same process. - /// - /// - /// An for this scope's span, or null if - /// no activity exists. - /// - public ActivityContext? GetActivityContext() - { - return activity?.Context; - } - - /// - /// Injects this span's trace context into W3C HTTP headers. - /// - /// - /// Returns a dictionary containing traceparent and optionally - /// tracestate headers that can be forwarded to downstream services - /// for distributed trace propagation. - /// - /// - /// A dictionary containing W3C trace context headers. Returns an empty - /// dictionary if no activity exists. - /// - public Dictionary InjectTraceContext() - { - var headers = new Dictionary(); - if (activity != null && activity.Id != null) - { - headers["traceparent"] = activity.Id; - if (!string.IsNullOrEmpty(activity.TraceStateString)) - { - headers["tracestate"] = activity.TraceStateString!; - } - } - - return headers; - } - - /// - /// Gets the span ID for the current activity. - /// - public string Id => activity?.Id ?? string.Empty; - - /// - /// Gets the trace ID for the current activity. - /// - public string TraceId => activity?.TraceId.ToHexString().ToLowerInvariant() ?? string.Empty; - } -} \ No newline at end of file diff --git a/src/Observability/Runtime/Tracing/Scopes/OutputScope.cs b/src/Observability/Runtime/Tracing/Scopes/OutputScope.cs deleted file mode 100644 index 95353cf5..00000000 --- a/src/Observability/Runtime/Tracing/Scopes/OutputScope.cs +++ /dev/null @@ -1,129 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Collections.Generic; -using System.Diagnostics; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts.Messages; - -using Microsoft.Agents.A365.Observability.Runtime.Tracing; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes -{ - /// - /// Provides OpenTelemetry tracing scope for AI agent output operations. - /// - /// - /// Output messages are set once (via the constructor or ) - /// rather than accumulated. For streaming scenarios, the agent developer should collect all output - /// and pass the final result to . - /// - public sealed class OutputScope : OpenTelemetryScope - { - /// - /// The operation name for output tracing. - /// - public const string OperationName = "output_messages"; - - /// - /// Creates and starts a new scope for output tracing. - /// - /// Request details for the output operation. - /// Response containing output messages. - /// Information about the agent producing the output. - /// Optional human user details. - /// Optional span configuration (parent context, timing, span links). - /// A new OutputScope instance. - public static OutputScope Start(Request request, Response response, AgentDetails agentDetails, UserDetails? userDetails = null, SpanDetails? spanDetails = null) - => new OutputScope(request, response, agentDetails, userDetails, spanDetails); - - private OutputScope(Request request, Response response, AgentDetails agentDetails, UserDetails? userDetails, SpanDetails? spanDetails) - : base( - operationName: OperationName, - activityName: $"{OperationName} {agentDetails.AgentId}", - agentDetails: agentDetails, - spanDetails: spanDetails ?? new SpanDetails(ActivityKind.Client), - userDetails: userDetails) - { - SetTagMaybe(OpenTelemetryConstants.GenAiConversationIdKey, request?.ConversationId); - - if (request?.Channel != null) - { - SetTagMaybe(OpenTelemetryConstants.ChannelNameKey, request.Channel.Name); - SetTagMaybe(OpenTelemetryConstants.ChannelLinkKey, request.Channel.Link); - } - - if (response?.ToolResultObject != null) - { - SetTagMaybe(OpenTelemetryConstants.GenAiOutputMessagesKey, MessageUtils.Serialize(response.ToolResultObject)); - } - else if (response?.OutputContent != null) - { - SetOutput(response.OutputContent); - } - else if (response?.Messages != null && response.Messages.Count > 0) - { - SetOutput(response.Messages); - } - } - - /// - /// Records the output messages for telemetry tracking. - /// Overwrites any previously set output messages. - /// Plain strings are auto-wrapped as OTEL OutputMessage with role "assistant". - /// - /// The messages to set as output. - public void RecordOutputMessages(IEnumerable messages) - { - if (messages == null) - { - return; - } - - SetOutput(messages); - } - - /// - /// Records structured output messages for telemetry tracking. - /// Overwrites any previously set output messages. - /// - /// The versioned output messages wrapper. - public void RecordOutputMessages(OutputMessages messages) - { - if (messages == null) - { - return; - } - - SetOutput(messages); - } - - /// - /// Records a tool call result dictionary for telemetry tracking. - /// Per OTEL spec, tool call results are expected to be objects and are serialized to JSON. - /// Overwrites any previously set output messages. - /// - /// The tool call result as a structured dictionary. - public void RecordOutputMessages(IDictionary toolResult) - { - if (toolResult == null) - { - return; - } - - SetTagMaybe(OpenTelemetryConstants.GenAiOutputMessagesKey, MessageUtils.Serialize(toolResult)); - } - - private void SetOutput(OutputMessages messages) - { - SetTagMaybe(OpenTelemetryConstants.GenAiOutputMessagesKey, MessageUtils.Serialize(messages)); - } - - private void SetOutput(IEnumerable messages) - { - var normalized = MessageUtils.NormalizeOutputMessages(messages); - SetTagMaybe(OpenTelemetryConstants.GenAiOutputMessagesKey, MessageUtils.Serialize(normalized)); - } - } -} \ No newline at end of file diff --git a/src/Observability/Runtime/Tracing/Scopes/TraceContextHelper.cs b/src/Observability/Runtime/Tracing/Scopes/TraceContextHelper.cs deleted file mode 100644 index 88f3a31b..00000000 --- a/src/Observability/Runtime/Tracing/Scopes/TraceContextHelper.cs +++ /dev/null @@ -1,60 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System.Collections.Generic; -using System.Diagnostics; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes -{ - /// - /// Utility methods for W3C trace context propagation via HTTP headers. - /// - public static class TraceContextHelper - { - /// - /// Extracts an from W3C trace HTTP headers. - /// - /// - /// Parses the traceparent (and optionally tracestate) headers - /// and returns an that can be passed as - /// parentContext to any scope's Start() method. - /// - /// Dictionary of HTTP headers containing trace context. - /// Expected keys include traceparent and optionally tracestate. - /// - /// An containing the extracted trace information. - /// Returns default if no valid trace context is found. - /// - public static ActivityContext ExtractContextFromHeaders(IDictionary headers) - { - if (headers != null - && headers.TryGetValue("traceparent", out var traceparent) - && !string.IsNullOrEmpty(traceparent)) - { - headers.TryGetValue("tracestate", out var tracestate); - return ActivityContext.Parse(traceparent, tracestate); - } - - return default; - } - - /// - /// Returns the W3C traceparent value from a headers dictionary. - /// - /// Dictionary of HTTP headers, typically obtained from - /// . - /// - /// The traceparent string (e.g. "00-{trace_id}-{span_id}-{flags}"), - /// or null if the key is not present. - /// - public static string? GetTraceparent(IDictionary headers) - { - if (headers != null && headers.TryGetValue("traceparent", out var traceparent)) - { - return traceparent; - } - - return null; - } - } -} diff --git a/src/Observability/Runtime/docs/design.md b/src/Observability/Runtime/docs/design.md deleted file mode 100644 index ee84efde..00000000 --- a/src/Observability/Runtime/docs/design.md +++ /dev/null @@ -1,486 +0,0 @@ -# Microsoft.Agents.A365.Observability.Runtime - Design Documentation - -## Overview - -The `Microsoft.Agents.A365.Observability.Runtime` package provides OpenTelemetry-based distributed tracing infrastructure for Microsoft Agent 365 applications. It includes a fluent builder for configuration, scope classes for tracing operations, and custom exporters for Agent365 telemetry. - -## Architecture - -``` -Microsoft.Agents.A365.Observability.Runtime -├── Public API -│ ├── Builder # Fluent OpenTelemetry configuration -│ └── Tracing/ -│ ├── Scopes/ -│ │ ├── InvokeAgentScope # Agent invocation tracing -│ │ ├── InferenceScope # LLM inference tracing -│ │ ├── ExecuteToolScope # Tool execution tracing -│ │ └── OpenTelemetryScope # Base scope class -│ └── Contracts/ -│ ├── AgentDetails # Agent metadata -│ ├── TenantDetails # Tenant context -│ ├── CallerDetails # Caller information -│ └── InferenceCallDetails # Inference metadata -├── Common/ -│ ├── ActivityExtensions # Activity helper methods -│ ├── BaggageBuilder # Baggage context management -│ └── EnvironmentUtils # Environment detection -├── DTOs/ -│ ├── BaseData # Base telemetry data -│ ├── ExecuteInferenceData # Inference telemetry -│ ├── ExecuteToolData # Tool telemetry -│ └── InvokeAgentData # Agent invocation telemetry -├── Etw/ -│ ├── A365EtwLogger # ETW logging implementation -│ ├── EtwEventSource # ETW event source -│ └── EtwLoggingBuilder # ETW configuration -└── Tracing/ - ├── Processors/ - │ └── ActivityProcessor # Base span processor - └── Exporters/ - └── Agent365Exporter # Custom telemetry exporter -``` - -## Key Components - -### Builder - -**Source**: [Builder.cs](../Builder.cs) - -The main entry point for configuring OpenTelemetry tracing with Agent365 extensions. - -```csharp -// Basic configuration -new Builder(services, configuration) - .Build(); - -// With framework extensions (added via extension packages) -new Builder(services, configuration) - .WithAgentFramework() // From Extensions.AgentFramework - .WithSemanticKernel() // From Extensions.SemanticKernel - .WithOpenAI() // From Extensions.OpenAI - .Build(); - -// Without OpenTelemetry builder (standalone tracer provider) -new Builder(services, configuration, useOpenTelemetryBuilder: false) - .Build(); -``` - -**Configuration Options:** - -| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `services` | `IServiceCollection` | Required | DI service collection | -| `configuration` | `IConfiguration` | Required | App configuration | -| `useOpenTelemetryBuilder` | `bool` | `true` | Use OpenTelemetry DI integration | -| `agent365ExporterType` | `Agent365ExporterType` | `Agent365Exporter` | Exporter type | - -**Environment Variables:** - -| Variable | Description | -|----------|-------------| -| `EnableAgent365Exporter` | Set to `true` to enable Agent365 exporter | - -### Agent365Exporter - -The Agent 365 exporter is S2S-only. It always builds -`/observabilityService/tenants/{tenantId}/otlp/agents/{agentId}/traces?api-version=1` -and ignores the obsolete `UseS2SEndpoint` compatibility property. Domain override and -custom tenant domain resolution still control only the host, not the path. - -Exporter authentication comes only from the configured `TokenResolver` or -`ContextualTokenResolver`. Both resolvers must return an app-only OBS token for the -exporting agent identity; the default app-only OBS scope is -`api://9b975845-388f-4429-889e-eab1ef63949c/.default`. The exporter never reads -per-request delegated tokens, never performs OBO/user_fic token exchange, and never retries -401/403/404 on the delegated `/observability` route. A missing resolver fails -configuration, and a null/empty token or resolver exception fails the export batch before -sending data. - -Resolvers are invoked once per tenant/agent identity group in each export batch, so a batch -that contains several identities invokes them several times; cache tokens per agent and -tenant. They must validate -the final token before returning it: accept `idtyp=app`, or, when `idtyp` is absent, a -non-empty `roles` array or a non-empty `oid` equal to `sub`; reject any other `idtyp` and -any `scp` claim. Also verify the OBS audience -(`api://9b975845-388f-4429-889e-eab1ef63949c`) and token lifetime. - -### InvokeAgentScope - -**Source**: [InvokeAgentScope.cs](../Tracing/Scopes/InvokeAgentScope.cs) - -Tracing scope for agent invocation operations. Creates an OpenTelemetry span with standardized attributes. - -```csharp -using var scope = InvokeAgentScope.Start( - invokeAgentDetails: new InvokeAgentDetails( - endpoint: new Uri("https://agent.example.com"), - details: new AgentDetails { AgentName = "MyAgent", AgentId = "agent-123" }, - sessionId: "session-456" - ), - tenantDetails: new TenantDetails { TenantId = "tenant-789" }, - request: new Request { Content = "User message", ExecutionType = ExecutionType.User }, - callerDetails: new CallerDetails { CallerId = "user-abc", CallerUpn = "user@contoso.com" } -); - -// Record response -scope.RecordResponse("Agent response text"); - -// Record input/output messages -scope.RecordInputMessages(new[] { "message1", "message2" }); -scope.RecordOutputMessages(new[] { "response1", "response2" }); -``` - -**Certification Requirements:** - -The following parameters must be provided (not null) for certification: -- `invokeAgentDetails` -- `tenantDetails` -- `request` -- `callerDetails` - -### InferenceScope - -**Source**: [InferenceScope.cs](../Tracing/Scopes/InferenceScope.cs) - -Tracing scope for LLM inference operations. - -```csharp -using var scope = InferenceScope.Start( - details: new InferenceCallDetails - { - OperationName = InferenceOperationType.Chat, - Model = "gpt-4", - ProviderName = "Azure OpenAI", - InputTokens = 150, - OutputTokens = 200, - FinishReasons = new[] { "stop" }, - ResponseId = "chatcmpl-123" - }, - agentDetails: new AgentDetails { AgentName = "MyAgent" }, - tenantDetails: new TenantDetails { TenantId = "tenant-789" } -); - -// Record additional metrics -scope.RecordInputMessages(new[] { "User: Hello" }); -scope.RecordOutputMessages(new[] { "Assistant: Hi there!" }); -scope.RecordInputTokens(150); -scope.RecordOutputTokens(200); -``` - -### ExecuteToolScope - -Tracing scope for tool/function execution operations. - -```csharp -using var scope = ExecuteToolScope.Start( - toolDetails: new ToolDetails { ToolName = "search_web", ToolId = "tool-123" }, - agentDetails: agentDetails, - tenantDetails: tenantDetails -); - -scope.RecordToolResult("Search results..."); -``` - -### BaggageBuilder - -**Source**: [BaggageBuilder.cs](../Common/BaggageBuilder.cs) - -Manages OpenTelemetry baggage context for propagating request metadata across distributed traces. - -```csharp -// Set request context for the current async context -using (BaggageBuilder.SetRequestContext(tenantId: "tenant-123", agentId: "agent-456")) -{ - // All spans created here will have tenant/agent baggage - await ProcessRequestAsync(); -} - -// Add custom baggage -BaggageBuilder.AddBaggage("correlation_id", correlationId); -``` - -**Standard Baggage Keys:** - -| Key | Description | -|-----|-------------| -| `tenant_id` | Tenant identifier | -| `gen_ai.agent.id` | Agent identifier | -| `correlation_id` | Request correlation ID | - -### ActivityExtensions - -**Source**: [ActivityExtensions.cs](../Common/ActivityExtensions.cs) - -Extension methods for `System.Diagnostics.Activity`. - -```csharp -// Get attribute or fallback to baggage -string? value = activity.GetAttributeOrBaggage("tenant_id"); - -// Set tag only if not already present (coalesce) -activity.CoalesceTag("gen_ai.agent.name", agentName, fallbackName); -``` - -### OpenTelemetryConstants - -Defines standard attribute keys following OpenTelemetry semantic conventions and Agent365 extensions. - -```csharp -public static class OpenTelemetryConstants -{ - public const string SourceName = "Microsoft.Agents.A365.Observability"; - - // GenAI semantic conventions - public const string GenAiOperationNameKey = "gen_ai.operation.name"; - public const string GenAiRequestModelKey = "gen_ai.request.model"; - public const string GenAiProviderNameKey = "gen_ai.system"; - public const string GenAiUsageInputTokensKey = "gen_ai.usage.input_tokens"; - public const string GenAiUsageOutputTokensKey = "gen_ai.usage.output_tokens"; - - // Agent365 extensions - public const string GenAiAgentNameKey = "gen_ai.agent.name"; - public const string GenAiAgentIdKey = "gen_ai.agent.id"; - public const string TenantIdKey = "tenant_id"; -} -``` - -## Design Patterns - -### Builder Pattern - -The `Builder` class implements the fluent builder pattern for configuring OpenTelemetry: - -```csharp -public sealed class Builder -{ - private readonly IServiceCollection _services; - private bool _isBuilt = false; - - public Builder(IServiceCollection services, IConfiguration? configuration) { ... } - - // Extension methods add framework support - // public Builder WithAgentFramework() { ... } // From extension package - - public IServiceCollection Build() - { - EnsureBuilt(); - return _services; - } - - private void EnsureBuilt() - { - if (_isBuilt) return; - - // Configure OpenTelemetry - _services.AddOpenTelemetry() - .WithTracing(tracerProviderBuilder => Configure(tracerProviderBuilder)); - - _isBuilt = true; - } -} -``` - -### Disposable Pattern - -Scope classes implement `IDisposable` to automatically end spans: - -```csharp -public sealed class InvokeAgentScope : OpenTelemetryScope -{ - private readonly Activity? _activity; - - private InvokeAgentScope(...) - : base(kind: ActivityKind.Client, ...) - { - // Activity is started in base constructor - } - - // Dispose ends the activity/span -} - -// Usage - span automatically ended when scope is disposed -using var scope = InvokeAgentScope.Start(...); -``` - -### Strategy Pattern - -Exporter selection based on environment: - -```csharp -private void Configure(TracerProviderBuilder builder) -{ - builder.AddSource(OpenTelemetryConstants.SourceName) - .AddProcessor(new ActivityProcessor()); - - if (IsAgent365ExporterEnabled()) - builder.AddAgent365Exporter(exporterType: _agent365ExporterType); - else if (EnvironmentUtils.IsDevelopmentEnvironment()) - builder.AddConsoleExporter(); -} -``` - -## Data Flow - -``` -┌──────────────────┐ ┌─────────────────┐ ┌─────────────────┐ -│ Application Code │────►│ Scope Classes │────►│ Activity/Span │ -│ │ │ │ │ │ -│ using var scope │ │ InvokeAgentScope│ │ OpenTelemetry │ -│ = Scope.Start() │ │ InferenceScope │ │ Activity │ -└──────────────────┘ └─────────────────┘ └────────┬────────┘ - │ - ┌─────────────────────────────────────┤ - │ │ - ▼ ▼ - ┌─────────────────┐ ┌─────────────────────┐ - │ SpanProcessor │ │ Baggage Context │ - │ │ │ │ - │ Add tags │ │ Propagate tenant_id │ - │ Transform spans │ │ agent_id, etc. │ - └────────┬────────┘ └─────────────────────┘ - │ - ▼ - ┌─────────────────┐ - │ Exporter │ - │ │ - │ Agent365 / OTLP │ - │ / Console │ - └─────────────────┘ -``` - -## File Structure - -``` -src/Observability/Runtime/ -├── Builder.cs # Main configuration builder -├── Common/ -│ ├── ActivityExtensions.cs # Activity helper methods -│ ├── BaggageBuilder.cs # Baggage context management -│ ├── EnvironmentUtils.cs # Environment detection -│ └── PowerPlatformApiDiscovery.cs # API endpoint discovery -├── DTOs/ -│ ├── BaseData.cs # Base telemetry DTO -│ ├── ExecuteInferenceData.cs # Inference telemetry -│ ├── ExecuteToolData.cs # Tool execution telemetry -│ └── InvokeAgentData.cs # Agent invocation telemetry -├── Etw/ -│ ├── A365EtwLogger.cs # ETW logger implementation -│ ├── IA365EtwLogger.cs # ETW logger interface -│ ├── EtwEventSource.cs # ETW event source -│ ├── EtwLoggingBuilder.cs # ETW logging configuration -│ ├── EtwTracingBuilder.cs # ETW tracing configuration -│ ├── EtwLogProcessor.cs # ETW log processor -│ └── EtwScopeEventProcessor.cs # ETW scope processor -├── Tracing/ -│ ├── Contracts/ -│ │ ├── AgentDetails.cs # Agent metadata contract -│ │ ├── TenantDetails.cs # Tenant context contract -│ │ ├── CallerDetails.cs # Caller information -│ │ ├── InvokeAgentDetails.cs # Agent invocation details -│ │ ├── InferenceCallDetails.cs # Inference call details -│ │ └── SourceMetadata.cs # Source metadata -│ ├── Scopes/ -│ │ ├── OpenTelemetryScope.cs # Base scope class -│ │ ├── OpenTelemetryConstants.cs # Attribute key constants -│ │ ├── InvokeAgentScope.cs # Agent invocation scope -│ │ ├── InferenceScope.cs # Inference operation scope -│ │ └── ExecuteToolScope.cs # Tool execution scope -│ ├── Processors/ -│ │ └── ActivityProcessor.cs # Base activity processor -│ └── Exporters/ -│ └── ObservabilityTracerProviderBuilderExtensions.cs -├── Microsoft.Agents.A365.Observability.Runtime.csproj -└── docs/ - └── design.md # This file -``` - -## Dependencies - -| Package | Purpose | -|---------|---------| -| `OpenTelemetry` | Core tracing functionality | -| `OpenTelemetry.Extensions.Hosting` | DI integration | -| `OpenTelemetry.Exporter.Console` | Development exporter | -| `OpenTelemetry.Exporter.OpenTelemetryProtocol` | OTLP exporter | -| `OpenTelemetry.Instrumentation.Runtime` | Runtime metrics | -| `Azure.Core` | Azure SDK integration | - -## Usage Examples - -### Basic Configuration - -```csharp -// In Program.cs or Startup.cs -builder.Services.AddOpenTelemetry() - .WithTracing(tracerBuilder => - { - new Builder(builder.Services, builder.Configuration) - .Build(); - }); -``` - -### Manual Span Creation - -```csharp -public async Task ProcessAgentRequestAsync(AgentRequest request) -{ - using var scope = InvokeAgentScope.Start( - invokeAgentDetails: new InvokeAgentDetails( - endpoint: _agentEndpoint, - details: new AgentDetails - { - AgentName = _agentName, - AgentId = _agentId - }, - sessionId: request.SessionId - ), - tenantDetails: new TenantDetails { TenantId = request.TenantId }, - request: new Request - { - Content = request.Message, - ExecutionType = ExecutionType.User - }, - callerDetails: new CallerDetails - { - CallerId = request.UserId, - CallerUpn = request.UserPrincipalName - } - ); - - try - { - var response = await _agent.ProcessAsync(request); - scope.RecordResponse(response.Content); - } - catch (Exception ex) - { - scope.SetStatus(ActivityStatusCode.Error, ex.Message); - throw; - } -} -``` - -### Context Propagation - -```csharp -public class ObservabilityMiddleware -{ - public async Task InvokeAsync(HttpContext context, RequestDelegate next) - { - var tenantId = context.User?.FindFirst("tenant_id")?.Value; - var agentId = context.Request.Headers["X-Agent-Id"].FirstOrDefault(); - - using (BaggageBuilder.SetRequestContext(tenantId, agentId)) - { - await next(context); - } - } -} -``` - -## External Resources - -- [OpenTelemetry .NET Documentation](https://opentelemetry.io/docs/languages/net/) -- [OpenTelemetry Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/) -- [Microsoft Agent 365 Observability](https://learn.microsoft.com/microsoft-agent-365/developer/observability) diff --git a/src/Observability/dirs.proj b/src/Observability/dirs.proj deleted file mode 100644 index 3d14079c..00000000 --- a/src/Observability/dirs.proj +++ /dev/null @@ -1,9 +0,0 @@ - - - net8.0 - - - - - - diff --git a/src/PACKAGE-SETUP.md b/src/PACKAGE-SETUP.md index 31b40d6d..d0afab6c 100644 --- a/src/PACKAGE-SETUP.md +++ b/src/PACKAGE-SETUP.md @@ -1,234 +1,65 @@ -# Microsoft Agent 365 SDK - Package Setup Guide +# Microsoft Agent 365 SDK - Package Setup Guide -This document provides a complete guide for building, validating, and publishing the Microsoft Agent 365 SDK NuGet packages. +## Maintained packages -## Package Information +This repository produces six .NET 8 packages: -The Microsoft Agent 365 SDK is organized into multiple NuGet packages: +- `Microsoft.Agents.A365.Runtime` +- `Microsoft.Agents.A365.Notifications` +- `Microsoft.Agents.A365.Tooling` +- `Microsoft.Agents.A365.Tooling.Extensions.AgentFramework` +- `Microsoft.Agents.A365.Tooling.Extensions.SemanticKernel` +- `Microsoft.Agents.A365.Tooling.Extensions.AzureAIFoundry` -- **Microsoft.Agents A365.Observability.Common** - Core observability and tracing infrastructure -- **Microsoft.Agents.A365.Observability.SemanticKernel** - SemanticKernel observability integration -- **Microsoft.Agents.A365.Observability.OpenAI** - OpenAI observability integration -- **Microsoft.Agents.A365.Runtime.SemanticKernel** - Runtime components for SemanticKernel -- **Microsoft.Agents.A365.Runtime.Common.AspNetCore** - ASP.NET Core runtime components -- **Microsoft.Agents.A365.DevTools.Analyzer.SemanticKernel** - Roslyn analyzers for code governance -- **Microsoft.Agents.A365.Tooling.SemanticKernel** - Tooling and utilities for SemanticKernel -- **Version**: 1.0.0 -- **Target Framework**: .NET 8.0 -- **License**: MIT -- **Repository**: https://github.com/microsoft/Agent365-dotnet +Observability packages have been retired here. Applications should migrate to `Microsoft.OpenTelemetry` using the [migration guide](../docs/observability-migration.md), which also documents owner-operated NuGet deprecation and external publishing controls. -## Building the Package +## Build and package -### Prerequisites +Use the SDK selected by `src/global.json` and PowerShell for the build script. Run these commands from the repository root: -- .NET 8.0 SDK or later -- PowerShell 5.1 or later (for build scripts) +```powershell +dotnet restore src\Microsoft.Agents.A365.Sdk.sln +dotnet build src\Microsoft.Agents.A365.Sdk.sln --configuration Release --no-restore +dotnet test src\Microsoft.Agents.A365.Sdk.sln --configuration Release --no-build +dotnet format src\Microsoft.Agents.A365.Sdk.sln --verify-no-changes -### Manual Build - -```bash -# Clean build entire solution -dotnet clean Microsoft.Agents.A365.sln - -# Build the entire solution -dotnet build Microsoft.Agents.A365.sln --configuration Release - -# Create NuGet packages for all projects -dotnet pack Microsoft.Agents.A365.sln --configuration Release --output ../NuGetPackages +# Traversal build, tests, and packages: +.\build\build.ps1 -Restore -Test -Pack ``` -### Using Build Scripts - -#### PowerShell (Recommended) -```bash -# Build only -.\build.ps1 +The build script creates a fresh `NuGetPackages\` directory and prints its absolute path. It rejects retired Observability filenames. Do not gather release artifacts from the parent directory or previous runs. -# Build and create package -.\build.ps1 -Pack - -# Build with custom version -.\build.ps1 -Pack -Version "1.1.0-preview" - -# Build in Debug mode -.\build.ps1 -Configuration Debug -Pack -``` +To pack the solution directly into an isolated directory: -#### Batch Script (Windows) -```cmd -build.cmd -Pack +```powershell +$output = Join-Path $env:TEMP "agent365-packages-$([guid]::NewGuid().ToString('N'))" +New-Item -ItemType Directory -Path $output | Out-Null +dotnet pack src\Microsoft.Agents.A365.Sdk.sln --configuration Release --no-build --output $output ``` -## Package Validation - -Use the validation script to verify package contents: - -```bash -.\validate.ps1 -``` - -The validation script checks for: -- Required assemblies in each package -- XML documentation for each assembly -- Package metadata (.nuspec files) -- Documentation files (README.md, CHANGELOG.md) -- Symbol packages (.snupkg) - -## Publishing the Package - -### Using the Publish Script - -```bash -# Publish with API key parameter -.\publish.ps1 -ApiKey "your-nuget-api-key" - -# Publish using environment variable -$env:NUGET_API_KEY = "your-nuget-api-key" -.\publish.ps1 - -# Publish to custom source -.\publish.ps1 -ApiKey "your-key" -Source "https://custom-nuget-server.com/v3/index.json" -``` - -### Manual Publishing - -```bash -# Publish all packages -dotnet nuget push "../NuGetPackages/*.nupkg" \ - --source https://api.nuget.org/v3/index.json \ - --api-key "your-nuget-api-key" -``` - -## Package Contents - -The generated package includes: - -### Assemblies -- `lib/net8.0/Microsoft.Agents.A365.dll` - Main assembly -- `lib/net8.0/Microsoft.Agents.A365.xml` - XML documentation - -### Documentation -- `README.md` - Package documentation -- `CHANGELOG.md` - Release notes and version history - -### Symbol Package -- `Microsoft.Agents.A365.1.0.0.snupkg` - Debug symbols for source link support +## Package validation -## Automated CI/CD +Inspect the fresh output before signing or publishing: -The repository includes a GitHub Actions workflow (`.github/workflows/build-and-package.yml`) that: +1. Verify the six package IDs above and their git-derived versions in the `.nuspec` metadata. +2. Verify each package's assembly and XML documentation under `lib/net8.0`, README, license metadata, and symbols where enabled. +3. Check package dependencies and assemblies for references to retired Observability packages. +4. Reject any `Microsoft.Agents.A365.Observability.*` package, including symbol packages. -1. **Builds** the project on every push and pull request -2. **Tests** the project (if tests exist) -3. **Packages** the NuGet package as artifacts -4. **Publishes** to NuGet.org on release events +Tests and traversal projects are not distributable packages. Do not hardcode a package version or edit project versions to perform this retirement. Nerdbank.GitVersioning uses git history and the repository's version configuration. -### Setting up CI/CD +## CI artifacts and external publishing -1. **Add NuGet API Key**: Add your NuGet API key as a repository secret named `NUGET_API_KEY` -2. **Create Release**: Create a GitHub release to trigger automatic publishing -3. **Monitor**: Check the Actions tab for build and publish status +`.github/workflows/ci.yml` restores, builds, tests, and packs the solution. Its pack step creates a unique temporary directory, rejects retired filenames, and uploads only the current run's output. Upload fails if there are no packages. -## Version Management +The checked-in workflow does **not** publish to nuget.org. Authorized maintainers must update external signing/publishing pipelines separately. Limit those pipelines to the six maintained IDs and current-run artifacts. Disable publishing from old branches/tags and remove queued or cached release artifacts that could reintroduce retired packages. Do not use a broad recursive wildcard over old package directories. -### Version Strategy -- **Major.Minor.Patch** for stable releases (e.g., 1.0.0) -- **Major.Minor.Patch-preview** for preview releases (e.g., 1.1.0-preview) -- **Major.Minor.Patch-alpha** for alpha releases (e.g., 1.1.0-alpha) +Deprecation and optional unlisting of already-published packages are separate package-owner actions on nuget.org. Existing exact-version restores remain available; there is no automatic replacement redirect. -### Updating Version - -Update the version in `Directory.Build.props` to apply to all projects: - -```xml - - 1.1.0 - -``` - -Or update individual project files as needed. - -Or override at build time: - -```bash -.\build.ps1 -Pack -Version "1.1.0-preview" -``` - -## Dependencies - -The package depends on the following NuGet packages: - -- **Azure.Monitor.OpenTelemetry.Exporter** (1.4.0) -- **OpenTelemetry.Exporter.Console** (1.12.0) -- **OpenTelemetry.Exporter.OpenTelemetryProtocol** (1.12.0) -- **OpenTelemetry.Extensions.Hosting** (1.12.0) -- **OpenTelemetry.Instrumentation.AspNetCore** (1.12.0) -- **OpenTelemetry.Instrumentation.Http** (1.12.0) -- **OpenTelemetry.Instrumentation.Runtime** (1.12.0) -- **Microsoft.SourceLink.GitHub** (8.0.0) - Build-time only - -## Troubleshooting - -### Common Issues - -1. **MSB1011 Error**: Multiple project files in folder - - Solution: Use `dotnet build Microsoft.Agents.A365.sln` instead of `dotnet build` - -2. **NU5046 Error**: Missing icon file - - Solution: Remove `` property or add the icon file - -3. **Build Warnings**: Missing XML documentation - - Solution: Add XML documentation comments to public APIs - -4. **Missing Dependencies**: Package references not found - - Solution: Run `dotnet restore` before building - -### Debug Build Issues - -Enable MSBuild verbosity for detailed output: - -```bash -dotnet build Microsoft.Agents.A365.sln --verbosity detailed -``` - -## File Structure - -``` -/ -├── Microsoft.Agents.A365.sln # Main solution file -├── Directory.Build.props # Common build properties -├── build.ps1 # Build script -├── build.cmd # Build script (Windows batch) -├── publish.ps1 # Publish script -├── validate.ps1 # Validation script -├── PACKAGE-SETUP.md # This file -├── Observability/ # Observability and tracing packages -│ ├── Common/ # Core observability infrastructure -│ │ └── Microsoft.Agents.A365.Observability.Common -│ ├── SemanticKernel/ # SemanticKernel observability -│ │ └── Microsoft.Agents.A365.Observability.SemanticKernel -│ └── OpenAI/ # OpenAI observability -│ └── Microsoft.Agents.A365.Observability.OpenAI -├── Runtime/ # Runtime components -│ ├── Common/ # Common runtime functionality -│ │ └── Microsoft.Agents.A365.Runtime.Common.AspNetCore -│ └── SemanticKernel/ # SemanticKernel runtime -│ └── Microsoft.Agents.A365.Runtime.SemanticKernel -├── DevTools/ # Development tools -│ └── Analyzer/ # Roslyn analyzers -│ └── Microsoft.Agents.A365.DevTools.Analyzer.SemanticKernel -├── Tooling/ # Utilities and tooling -│ └── Microsoft.Agents.A365.Tooling.SemanticKernel -└── Notification/ # Agent notifications - └── AgentNotification -``` +## Dependencies and project maintenance -## Support +All dependency versions belong in `src/Directory.Packages.props`; `.csproj` references must omit versions. Preserve central transitive/security pins needed by remaining projects even if no direct reference uses them. -For issues with the package setup or build process: +Traversal projects discover source projects within their modules, but `src/Microsoft.Agents.A365.Sdk.sln` must still be updated explicitly when projects change. Regenerate `DEPENDENCIES.md` with `.\generate-dependencies.ps1` after dependency changes. -1. Check the [GitHub Issues](https://github.com/microsoft/Agent365-dotnet/issues) -2. Review the build logs in GitHub Actions -3. Run validation script to verify package integrity -4. Check NuGet.org for package availability after publishing +See [build internals](../build/BUILD.md) and [architecture](../docs/design.md) for additional details. diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Extension.Tests/Microsoft.Agents.A365.Observability.Extension.Tests.csproj b/src/Tests/Microsoft.Agents.A365.Observability.Extension.Tests/Microsoft.Agents.A365.Observability.Extension.Tests.csproj deleted file mode 100644 index 377864cf..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Extension.Tests/Microsoft.Agents.A365.Observability.Extension.Tests.csproj +++ /dev/null @@ -1,27 +0,0 @@ - - - net8.0 - false - false - true - - - - - - - - - - - - - - - - - - - - - diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Extension.Tests/SemanticKernelSpanProcessorHelperTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Extension.Tests/SemanticKernelSpanProcessorHelperTests.cs deleted file mode 100644 index 6a937a51..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Extension.Tests/SemanticKernelSpanProcessorHelperTests.cs +++ /dev/null @@ -1,209 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System.Diagnostics; -using System.Text.Json; -using Microsoft.Agents.A365.Observability.Extensions.SemanticKernel; -using Microsoft.Agents.A365.Observability.Extensions.SemanticKernel.Models; -using Microsoft.Agents.A365.Observability.Extensions.SemanticKernel.Utils; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; - -namespace Microsoft.Agents.A365.Observability.Extension.Tests -{ - [TestClass] - public class SemanticKernelSpanProcessorHelperTests - { - [TestMethod] - public void ProcessInvocationInputOutputTag_MapsMessagesToStructuredInputFormat() - { - var activity = new Activity("test"); - var messages = new List - { - JsonSerializer.Serialize(new MessageContent { Role = "system", Content = "System message" }), - JsonSerializer.Serialize(new MessageContent { Role = "user", Content = "Message:User message" }) - }; - activity.SetTag(OpenTelemetryConstants.GenAiAgentInvocationInputKey, JsonSerializer.Serialize(messages)); - - SemanticKernelSpanProcessorHelper.ProcessInvocationInputOutputTag(activity); - - var invocationTag = activity.Tags.FirstOrDefault(t => t.Key == OpenTelemetryConstants.GenAiAgentInvocationInputKey).Value; - Assert.IsNull(invocationTag, "Invocation input key should be removed"); - - var result = activity.Tags.FirstOrDefault(t => t.Key == OpenTelemetryConstants.GenAiInputMessagesKey).Value as string; - Assert.IsNotNull(result); - Assert.IsTrue(result.StartsWith("["), "Should be a JSON array"); - Assert.IsTrue(result.Contains("\"role\":\"system\""), "System messages should be preserved"); - Assert.IsTrue(result.Contains("\"role\":\"user\""), "Should contain user role"); - Assert.IsTrue(result.Contains("System message"), "System message content should be preserved"); - Assert.IsTrue(result.Contains("User message"), "User message content should be preserved"); - Assert.IsTrue(result.Contains("\"type\":\"text\""), "Should contain text part type"); - } - - [TestMethod] - public void ProcessInvocationInputOutputTag_HandlesJsonArrayOfObjects() - { - var activity = new Activity("test"); - - // JSON array of objects directly (not wrapped in strings) - var jsonArrayOfObjects = @"[{""role"":""system"",""name"":""Agent365Agent"",""content"":""You are a friendly assistant."",""tool_calls"":[]},{""role"":""user"",""name"":null,""content"":""hi"",""tool_calls"":[]}]"; - - activity.SetTag(OpenTelemetryConstants.GenAiAgentInvocationInputKey, jsonArrayOfObjects); - - SemanticKernelSpanProcessorHelper.ProcessInvocationInputOutputTag(activity); - - var invocationTag = activity.Tags.FirstOrDefault(t => t.Key == OpenTelemetryConstants.GenAiAgentInvocationInputKey).Value; - Assert.IsNull(invocationTag, "Invocation input key should be removed"); - - var result = activity.Tags.FirstOrDefault(t => t.Key == OpenTelemetryConstants.GenAiInputMessagesKey).Value as string; - Assert.IsNotNull(result); - Assert.IsTrue(result.StartsWith("["), "Should be a JSON array"); - Assert.IsTrue(result.Contains("\"role\":\"system\""), "System message should be preserved"); - Assert.IsTrue(result.Contains("\"role\":\"user\""), "Should contain user role"); - Assert.IsTrue(result.Contains("hi"), "User message should be preserved"); - } - - [TestMethod] - public void ProcessInvocationInputOutputTag_SuppressInvocationInput_RemovesInputTagsAndPreservesOutput() - { - var activity = new Activity("test"); - var agentMessages = new List - { - JsonSerializer.Serialize(new MessageContent { Role = "system", Content = "System message" }), - JsonSerializer.Serialize(new MessageContent { Role = "user", Content = "Message:Sensitive user message" }) - }; - var inputMessages = JsonSerializer.Serialize(new[] { "Sensitive input message 1", "Sensitive input message 2" }); - var outputMessages = new List - { - JsonSerializer.Serialize(new MessageContent { Role = "assistant", Content = "Output message" }) - }; - - activity.SetTag(OpenTelemetryConstants.GenAiAgentInvocationInputKey, JsonSerializer.Serialize(agentMessages)); - activity.SetTag(OpenTelemetryConstants.GenAiInputMessagesKey, inputMessages); - activity.SetTag(OpenTelemetryConstants.GenAiAgentInvocationOutputKey, JsonSerializer.Serialize(outputMessages)); - - SemanticKernelSpanProcessorHelper.ProcessInvocationInputOutputTag(activity, suppressInvocationInput: true); - - var removedAgentInput = activity.Tags.FirstOrDefault(t => t.Key == OpenTelemetryConstants.GenAiAgentInvocationInputKey).Value; - var removedInputMessages = activity.Tags.FirstOrDefault(t => t.Key == OpenTelemetryConstants.GenAiInputMessagesKey).Value; - var removedInvocationOutput = activity.Tags.FirstOrDefault(t => t.Key == OpenTelemetryConstants.GenAiAgentInvocationOutputKey).Value; - var output = activity.Tags.FirstOrDefault(t => t.Key == OpenTelemetryConstants.GenAiOutputMessagesKey).Value as string; - - Assert.IsNull(removedAgentInput, "Invocation input key should be removed"); - Assert.IsNull(removedInputMessages, "Input messages key should be removed"); - Assert.IsNull(removedInvocationOutput, "Invocation output key should be removed"); - Assert.IsNotNull(output, "Output should be on gen_ai.output.messages"); - Assert.IsTrue(output.StartsWith("["), "Output should be a JSON array"); - Assert.IsTrue(output.Contains("\"role\":\"assistant\""), "Should contain assistant role"); - Assert.IsTrue(output.Contains("Output message"), "Output content should be preserved"); - } - - [TestMethod] - public void ProcessInvocationInputOutputTag_SuppressInvocationInput_HandlesMissingTags() - { - var activity = new Activity("test"); - // GenAiInputMessagesKey and GenAiAgentInvocationInputKey intentionally not set - - SemanticKernelSpanProcessorHelper.ProcessInvocationInputOutputTag(activity, suppressInvocationInput: true); - - var agentInput = activity.Tags.FirstOrDefault(t => t.Key == OpenTelemetryConstants.GenAiAgentInvocationInputKey).Value; - var inputMessages = activity.Tags.FirstOrDefault(t => t.Key == OpenTelemetryConstants.GenAiInputMessagesKey).Value; - - Assert.IsNull(agentInput); - Assert.IsNull(inputMessages); - } - - [TestMethod] - public void ProcessInvocationInputOutputTag_SuppressInvocationInput_RemovesEmptyStringInputTags() - { - var activity = new Activity("test"); - activity.SetTag(OpenTelemetryConstants.GenAiAgentInvocationInputKey, ""); - activity.SetTag(OpenTelemetryConstants.GenAiInputMessagesKey, ""); - - SemanticKernelSpanProcessorHelper.ProcessInvocationInputOutputTag(activity, suppressInvocationInput: true); - - var agentInput = activity.Tags.FirstOrDefault(t => t.Key == OpenTelemetryConstants.GenAiAgentInvocationInputKey).Value; - var inputMessages = activity.Tags.FirstOrDefault(t => t.Key == OpenTelemetryConstants.GenAiInputMessagesKey).Value; - - Assert.IsNull(agentInput); - Assert.IsNull(inputMessages); - } - - [TestMethod] - public void TryDeserializeMessageContent_HandlesUnquotedPropertyValues() - { - var unquotedJson = "{\u0022role\u0022: \u0022Assistant\u0022, \u0022content\u0022: \u0022\\u003Cp\\u003EHello Jian Han,\\u003C/p\\u003E\\n\\u003Cp\\u003EHow may I assist you today?\\u003C/p\\u003E\u0022, \u0022name\u0022: ShippingAgent1efe6ed@a365preview005.onmicrosoft.com}"; - var result = typeof(SemanticKernelSpanProcessorHelper) - .GetMethod("TryDeserializeMessageContent", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Static) - ?.Invoke(null, new object[] { unquotedJson }) as MessageContent; - - Assert.IsNotNull(result); - Assert.AreEqual("Assistant", result.Role); - Assert.AreEqual("

Hello Jian Han,

\n

How may I assist you today?

", result.Content); - } - - [TestMethod] - public void ProcessInvocationInputOutputTag_ExtractsNestedContentFromAssistantMessages() - { - var activity = new Activity("test"); - - // JSON array with an assistant message containing nested content structure - var nestedContentJson = @"{""contentType"": ""'Text'"", ""content"": ""Hello! Before I can assist you, you must accept the terms and conditions.""}"; - var jsonArrayOfObjects = $@"[{{""role"":""Assistant"",""name"":""Agent365Agent"",""content"":{JsonSerializer.Serialize(nestedContentJson)}}}]"; - - activity.SetTag(OpenTelemetryConstants.GenAiAgentInvocationOutputKey, jsonArrayOfObjects); - - SemanticKernelSpanProcessorHelper.ProcessInvocationInputOutputTag(activity); - - var invocationTag = activity.Tags.FirstOrDefault(t => t.Key == OpenTelemetryConstants.GenAiAgentInvocationOutputKey).Value; - Assert.IsNull(invocationTag, "Invocation output key should be removed"); - - var filtered = activity.Tags.FirstOrDefault(t => t.Key == OpenTelemetryConstants.GenAiOutputMessagesKey).Value as string; - Assert.IsNotNull(filtered); - Assert.IsTrue(filtered.Contains("Hello! Before I can assist you, you must accept the terms and conditions."), "Nested content should be extracted"); - Assert.IsFalse(filtered.Contains("contentType"), "contentType property should be removed"); - } - - [TestMethod] - public void ProcessInvocationInputOutputTag_PreservesNonNestedContent() - { - var activity = new Activity("test"); - - // JSON array with a user message that has simple string content (not nested) - var jsonArrayOfObjects = @"[{""role"":""user"",""name"":null,""content"":""Message:Simple text message""}]"; - - activity.SetTag(OpenTelemetryConstants.GenAiAgentInvocationInputKey, jsonArrayOfObjects); - - SemanticKernelSpanProcessorHelper.ProcessInvocationInputOutputTag(activity); - - var invocationTag = activity.Tags.FirstOrDefault(t => t.Key == OpenTelemetryConstants.GenAiAgentInvocationInputKey).Value; - Assert.IsNull(invocationTag, "Invocation input key should be removed"); - - var filtered = activity.Tags.FirstOrDefault(t => t.Key == OpenTelemetryConstants.GenAiInputMessagesKey).Value as string; - Assert.IsNotNull(filtered); - Assert.IsTrue(filtered.Contains("Simple text message"), "Simple content should be preserved after Message: trimming"); - } - - [TestMethod] - public void ProcessInvocationInputOutputTag_HandlesNestedContentWithUserRole() - { - var activity = new Activity("test"); - - // JSON array with a user message containing nested content structure AND Message: prefix - var nestedContentJson = @"{""contentType"": ""'Text'"", ""content"": ""Message:User query with nested structure""}"; - var jsonArrayOfObjects = $@"[{{""role"":""user"",""name"":null,""content"":{JsonSerializer.Serialize(nestedContentJson)}}}]"; - - activity.SetTag(OpenTelemetryConstants.GenAiAgentInvocationInputKey, jsonArrayOfObjects); - - SemanticKernelSpanProcessorHelper.ProcessInvocationInputOutputTag(activity); - - var invocationTag = activity.Tags.FirstOrDefault(t => t.Key == OpenTelemetryConstants.GenAiAgentInvocationInputKey).Value; - Assert.IsNull(invocationTag, "Invocation input key should be removed"); - - var filtered = activity.Tags.FirstOrDefault(t => t.Key == OpenTelemetryConstants.GenAiInputMessagesKey).Value as string; - Assert.IsNotNull(filtered); - Assert.IsTrue(filtered.Contains("User query with nested structure"), "Nested content should be extracted and Message: prefix trimmed"); - Assert.IsFalse(filtered.Contains("contentType"), "contentType property should be removed"); - Assert.IsFalse(filtered.Contains("Message:"), "Message: prefix should be trimmed"); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/ActivityCapturingExporter.cs b/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/ActivityCapturingExporter.cs deleted file mode 100644 index 52644fbb..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/ActivityCapturingExporter.cs +++ /dev/null @@ -1,28 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System.Collections.Generic; -using System.Diagnostics; -using OpenTelemetry; - -namespace Microsoft.Agents.A365.Observability.Extensions.IntegrationTests; - -/// -/// Exporter that captures activities at export time — after all processors have run. -/// Use with to capture spans at the final stage. -/// -internal sealed class ActivityCapturingExporter : BaseExporter -{ - private readonly List _activities; - - public ActivityCapturingExporter(List activities) => _activities = activities; - - public override ExportResult Export(in Batch batch) - { - foreach (var activity in batch) - { - _activities.Add(activity); - } - return ExportResult.Success; - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/AgentFrameworkInvokeAgentPipelineTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/AgentFrameworkInvokeAgentPipelineTests.cs deleted file mode 100644 index 374e27b9..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/AgentFrameworkInvokeAgentPipelineTests.cs +++ /dev/null @@ -1,185 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System.Diagnostics; -using System.Text.Json; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Extensions.AgentFramework; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.Logging; -using OpenTelemetry; -using OpenTelemetry.Trace; - -namespace Microsoft.Agents.A365.Observability.Extensions.IntegrationTests; - -/// -/// Pipeline integration tests for handling invoke_agent spans. -/// Verifies the full Builder → WithAgentFramework → Processor → Exporter chain produces A365 versioned messages. -/// Does NOT require Azure OpenAI credentials. -/// -[TestClass] -public class AgentFrameworkInvokeAgentPipelineTests -{ - private const string AfSourceName = BuilderExtensions.AgentFrameworkSource; - private List _exportedActivities = new(); - private ServiceProvider? _serviceProvider; - private ActivitySource? _activitySource; - - [TestInitialize] - public void Setup() - { - _exportedActivities = new List(); - - var services = new ServiceCollection(); - services.AddLogging(); - - new Runtime.Builder(services, configuration: null, useOpenTelemetryBuilder: true) - .WithAgentFramework() - .Build(); - - services.AddOpenTelemetry() - .WithTracing(tracing => tracing - .AddSource(AfSourceName) - .AddProcessor(new SimpleActivityExportProcessor(new ActivityCapturingExporter(_exportedActivities)))); - - _serviceProvider = services.BuildServiceProvider(); - _serviceProvider.GetService(); - - _activitySource = new ActivitySource(AfSourceName); - } - - [TestCleanup] - public void Cleanup() - { - _activitySource?.Dispose(); - _serviceProvider?.Dispose(); - } - - [TestMethod] - public void InvokeAgent_MapsInputAndOutputToA365StructuredFormat() - { - // Agent Framework emits messages with parts array - var inputJson = JsonSerializer.Serialize(new[] - { - new - { - role = "system", - parts = new[] { new { type = "text", content = "You are a helpful agent." } } - }, - new - { - role = "user", - parts = new[] { new { type = "text", content = "What can you do?" } } - } - }); - var outputJson = JsonSerializer.Serialize(new object[] - { - new - { - role = "assistant", - parts = new object[] { new { type = "text", content = "I can help with many tasks!" } }, - finish_reason = "stop" - } - }); - - using (var activity = _activitySource!.StartActivity("invoke_agent TestAgent")) - { - activity!.SetTag("gen_ai.operation.name", "invoke_agent"); - activity.SetTag(OpenTelemetryConstants.GenAiInputMessagesKey, inputJson); - activity.SetTag(OpenTelemetryConstants.GenAiOutputMessagesKey, outputJson); - } - - ForceFlush(); - - var span = FindInvokeAgentSpan(); - span.Should().NotBeNull("invoke_agent span should be captured by pipeline"); - var tags = GetTags(span!); - - // Input: versioned, all roles preserved, TextPart format - var input = tags[OpenTelemetryConstants.GenAiInputMessagesKey] as string; - input.Should().StartWith("["); - input.Should().Contain("\"role\":\"system\"", "system messages should be preserved"); - input.Should().Contain("\"role\":\"user\"", "user messages should be mapped"); - input.Should().Contain("\"type\":\"text\"", "should use TextPart format"); - input.Should().Contain("You are a helpful agent."); - input.Should().Contain("What can you do?"); - - // Output: versioned, assistant role, finish reason, TextPart format - var output = tags[OpenTelemetryConstants.GenAiOutputMessagesKey] as string; - output.Should().StartWith("["); - output.Should().Contain("\"role\":\"assistant\""); - output.Should().Contain("\"type\":\"text\""); - output.Should().Contain("I can help with many tasks!"); - output.Should().Contain("\"finish_reason\":\"stop\"", "finish reason should be preserved"); - } - - [TestMethod] - public void InvokeAgent_MapsToolCallAndResponseParts() - { - var inputJson = JsonSerializer.Serialize(new object[] - { - new - { - role = "user", - parts = new object[] { new { type = "text", content = "What is the weather?" } } - }, - new - { - role = "assistant", - parts = new object[] - { - new { type = "tool_call", name = "GetWeather", id = "call_123", arguments = "{\"location\":\"Seattle\"}" } - } - }, - new - { - role = "tool", - parts = new object[] - { - new { type = "tool_call_response", id = "call_123", response = "Sunny, 72°F" } - } - } - }); - - using (var activity = _activitySource!.StartActivity("invoke_agent WeatherAgent")) - { - activity!.SetTag("gen_ai.operation.name", "invoke_agent"); - activity.SetTag(OpenTelemetryConstants.GenAiInputMessagesKey, inputJson); - } - - ForceFlush(); - - var span = FindInvokeAgentSpan(); - span.Should().NotBeNull(); - - var tags = GetTags(span!); - var input = tags[OpenTelemetryConstants.GenAiInputMessagesKey] as string; - - input.Should().StartWith("["); - input.Should().Contain("\"type\":\"tool_call\"", "tool call request should be mapped"); - input.Should().Contain("GetWeather", "function name should be preserved"); - input.Should().Contain("\"type\":\"tool_call_response\"", "tool response should be mapped"); - input.Should().Contain("Sunny", "tool response content should be preserved"); - } - - #region Helpers - - private void ForceFlush() - { - _serviceProvider?.GetService()?.ForceFlush(); - } - - private Activity? FindInvokeAgentSpan() - { - return _exportedActivities.FirstOrDefault(a => - a.GetTagItem("gen_ai.operation.name") as string == "invoke_agent"); - } - - private static Dictionary GetTags(Activity activity) - { - return activity.TagObjects.ToDictionary(t => t.Key, t => t.Value); - } - - #endregion -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/AgentFrameworkSpanProcessorTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/AgentFrameworkSpanProcessorTests.cs deleted file mode 100644 index 39431a1a..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/AgentFrameworkSpanProcessorTests.cs +++ /dev/null @@ -1,296 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System.ClientModel; -using System.Diagnostics; -using System.Text.Json; -using Azure.AI.OpenAI; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Extensions.AgentFramework; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Extensions.AI; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.Logging; -using OpenTelemetry; -using OpenTelemetry.Trace; - -namespace Microsoft.Agents.A365.Observability.Extensions.IntegrationTests; - -/// -/// Integration tests for running against real Azure OpenAI -/// via with UseOpenTelemetry() — the same pipeline used by Agent Framework. -/// Uses the real A365 SDK initialization: Builder → WithAgentFramework → Build. -/// Requires: AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY, AZURE_OPENAI_DEPLOYMENT env vars. -/// -[TestClass] -public class AgentFrameworkSpanProcessorTests -{ - private static readonly JsonSerializerOptions JsonPrint = new() { WriteIndented = true }; - private const string OTelSourceName = BuilderExtensions.AgentFrameworkChatClientSource; - - private static string? Endpoint => Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT"); - private static string? ApiKey => Environment.GetEnvironmentVariable("AZURE_OPENAI_API_KEY"); - private static string? Deployment => Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT"); - - private static bool HasCredentials => - !string.IsNullOrEmpty(Endpoint) && - !string.IsNullOrEmpty(ApiKey) && - !string.IsNullOrEmpty(Deployment); - - private List _exportedActivities = new(); - private ServiceProvider? _serviceProvider; - - [TestInitialize] - public void Setup() - { - _exportedActivities = new List(); - - // Use the real A365 SDK initialization path: Builder → WithAgentFramework → Build - var services = new ServiceCollection(); - services.AddLogging(); - - new Runtime.Builder(services, configuration: null, useOpenTelemetryBuilder: true) - .WithAgentFramework() - .Build(); - - // Add a capturing exporter — runs after all processors, at export time - services.AddOpenTelemetry() - .WithTracing(tracing => tracing - .AddProcessor(new SimpleActivityExportProcessor(new ActivityCapturingExporter(_exportedActivities)))); - - _serviceProvider = services.BuildServiceProvider(); - - // Force the TracerProvider to be created by resolving it - _serviceProvider.GetService(); - } - - [TestCleanup] - public void Cleanup() - { - _serviceProvider?.Dispose(); - } - - [TestMethod] - public async Task SimpleChat_ProcessorMapsToVersionedFormat() - { - SkipIfNoCredentials(); - - var chatClient = CreateChatClient(); - var messages = new List - { - new(ChatRole.System, "You are a helpful assistant. Reply in one sentence."), - new(ChatRole.User, "What is the capital of France?") - }; - - await chatClient.GetResponseAsync(messages); - - ForceFlush(); - - var chatSpan = FindChatSpan(); - chatSpan.Should().NotBeNull("IChatClient should emit a chat span"); - DumpActivity(chatSpan!, "AF SimpleChat"); - - var tags = GetTags(chatSpan!); - - // Verify the processor mapped to A365 versioned format - tags.Should().ContainKey("gen_ai.input.messages"); - var input = tags["gen_ai.input.messages"] as string; - input.Should().StartWith("["); - input.Should().Contain("\"type\":\"text\"", "should use TextPart format"); - input.Should().Contain("capital of France"); - - tags.Should().ContainKey("gen_ai.output.messages"); - var output = tags["gen_ai.output.messages"] as string; - output.Should().StartWith("["); - output.Should().Contain("\"type\":\"text\""); - output.Should().Contain("\"role\":\"assistant\""); - } - - [TestMethod] - public async Task ChatWithToolCall_ProcessorMapsToolCallParts() - { - SkipIfNoCredentials(); - - var chatClient = CreateChatClientWithTools(); - var messages = new List - { - new(ChatRole.System, "You are a weather assistant. Always use the GetWeather function."), - new(ChatRole.User, "What's the weather in Seattle?") - }; - - await chatClient.GetResponseAsync(messages); - - ForceFlush(); - - // Dump all spans to see what the full pipeline emits - Console.WriteLine($"\n All captured activities ({_exportedActivities.Count}):"); - foreach (var act in _exportedActivities) - { - var op = act.GetTagItem("gen_ai.operation.name") as string ?? "(none)"; - Console.WriteLine($" {act.Source.Name} | {act.DisplayName} | op={op}"); - DumpActivity(act, $"AF ToolCall — {act.DisplayName}"); - } - - // Find a chat span that has input messages - var chatSpan = _exportedActivities.LastOrDefault(a => - { - var input = a.GetTagItem("gen_ai.input.messages") as string; - return input != null && input.StartsWith("["); - }); - - chatSpan.Should().NotBeNull("should have at least one span with structured input messages"); - - var tags = GetTags(chatSpan!); - var input = tags["gen_ai.input.messages"] as string; - input.Should().StartWith("["); - input.Should().Contain("weather"); - } - - [TestMethod] - public async Task SimpleChat_RawFormatBeforeProcessor() - { - SkipIfNoCredentials(); - - // Capture spans WITHOUT the A365 processor — just raw IChatClient format - var rawActivities = new List(); - var rawServices = new ServiceCollection(); - rawServices.AddLogging(); - rawServices.AddOpenTelemetry() - .WithTracing(tracing => tracing - .AddSource(OTelSourceName) - .AddProcessor(new SimpleActivityExportProcessor(new ActivityCapturingExporter(rawActivities)))); - using var rawProvider = rawServices.BuildServiceProvider(); - rawProvider.GetService(); - - var chatClient = CreateChatClient(); - var messages = new List - { - new(ChatRole.User, "Say hello") - }; - - await chatClient.GetResponseAsync(messages); - - rawProvider.GetRequiredService().ForceFlush(); - - var chatSpan = rawActivities.FirstOrDefault(a => - a.GetTagItem("gen_ai.operation.name") as string == "chat"); - - chatSpan.Should().NotBeNull("IChatClient should emit a chat span"); - DumpActivity(chatSpan!, "AF Raw (no processor)"); - - // Document the raw format that Microsoft.Extensions.AI emits - var tags = GetTags(chatSpan!); - tags.Should().ContainKey("gen_ai.input.messages", - "Microsoft.Extensions.AI UseOpenTelemetry should emit input messages"); - tags.Should().ContainKey("gen_ai.output.messages", - "Microsoft.Extensions.AI UseOpenTelemetry should emit output messages"); - - // Log the raw format for analysis - Console.WriteLine($"\n Raw gen_ai.input.messages:\n{tags["gen_ai.input.messages"]}"); - Console.WriteLine($"\n Raw gen_ai.output.messages:\n{tags["gen_ai.output.messages"]}"); - } - - #region Helpers - - private void ForceFlush() - { - var tracerProvider = _serviceProvider?.GetService(); - tracerProvider?.ForceFlush(); - } - - private IChatClient CreateChatClient() - { - return new AzureOpenAIClient( - new Uri(Endpoint!), - new ApiKeyCredential(ApiKey!)) - .GetChatClient(Deployment!) - .AsIChatClient() - .AsBuilder() - .UseOpenTelemetry( - sourceName: OTelSourceName, - configure: cfg => cfg.EnableSensitiveData = true) - .Build(); - } - - private IChatClient CreateChatClientWithTools() - { - return new AzureOpenAIClient( - new Uri(Endpoint!), - new ApiKeyCredential(ApiKey!)) - .GetChatClient(Deployment!) - .AsIChatClient() - .AsBuilder() - .UseFunctionInvocation() - .UseOpenTelemetry( - sourceName: OTelSourceName, - configure: cfg => cfg.EnableSensitiveData = true) - .Build(); - } - - private Activity? FindChatSpan() - { - return _exportedActivities.FirstOrDefault(a => - a.GetTagItem("gen_ai.operation.name") as string == "chat"); - } - - private static void SkipIfNoCredentials() - { - if (!HasCredentials) - { - Assert.Inconclusive( - "Skipped: set AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY, AZURE_OPENAI_DEPLOYMENT env vars to run."); - } - } - - private static Dictionary GetTags(Activity activity) - { - return activity.TagObjects.ToDictionary(t => t.Key, t => t.Value); - } - - private static void DumpActivity(Activity activity, string label) - { - Console.WriteLine($"\n=== {label} ==="); - Console.WriteLine($" Source: {activity.Source.Name} Kind: {activity.Kind} Duration: {activity.Duration}"); - - Console.WriteLine(" Attributes:"); - foreach (var tag in activity.TagObjects) - Console.WriteLine($" {tag.Key} = {FormatValue(tag.Value)}"); - - if (activity.Events.Any()) - { - Console.WriteLine($" Events ({activity.Events.Count()}):"); - foreach (var ev in activity.Events) - { - Console.WriteLine($" '{ev.Name}'"); - foreach (var attr in ev.Tags) - Console.WriteLine($" {attr.Key} = {FormatValue(attr.Value)}"); - } - } - Console.WriteLine("===\n"); - } - - private static string FormatValue(object? value) - { - string val = value switch - { - string s => s, - string[] arr => $"[{string.Join(", ", arr)}]", - null => "(null)", - _ => value.ToString() ?? "(null)" - }; - - if (val.Length > 120) - { - try - { - var doc = JsonDocument.Parse(val); - val = "\n " + JsonSerializer.Serialize(doc.RootElement, JsonPrint).Replace("\n", "\n "); - } - catch { } - } - - return val; - } - - #endregion -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests.csproj b/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests.csproj deleted file mode 100644 index 97a6d0fb..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests.csproj +++ /dev/null @@ -1,32 +0,0 @@ - - - net8.0 - false - false - true - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/OpenAISpanProcessorTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/OpenAISpanProcessorTests.cs deleted file mode 100644 index 5fcddfc4..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/OpenAISpanProcessorTests.cs +++ /dev/null @@ -1,288 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System.ClientModel; -using System.Diagnostics; -using System.Text.Json; -using Azure.AI.OpenAI; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Extensions.OpenAI; -using OpenAI.Chat; -using OpenTelemetry; -using OpenTelemetry.Trace; - -namespace Microsoft.Agents.A365.Observability.Extensions.IntegrationTests; - -/// -/// Integration tests for running against real Azure OpenAI spans. -/// The tests wire up the full OTel pipeline: OpenAI SDK → TracerProvider → OpenAISpanProcessor → captured spans. -/// This lets us assert on the span attributes after the processor runs and test any mapping logic. -/// Requires: AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY, AZURE_OPENAI_DEPLOYMENT env vars. -/// -[TestClass] -public class OpenAISpanProcessorTests -{ - private static readonly JsonSerializerOptions JsonPrint = new() { WriteIndented = true }; - - private static string? Endpoint => Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT"); - private static string? ApiKey => Environment.GetEnvironmentVariable("AZURE_OPENAI_API_KEY"); - private static string? Deployment => Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT"); - - private static bool HasCredentials => - !string.IsNullOrEmpty(Endpoint) && - !string.IsNullOrEmpty(ApiKey) && - !string.IsNullOrEmpty(Deployment); - - private List _exportedActivities = new(); - private TracerProvider? _tracerProvider; - - [TestInitialize] - public void Setup() - { - AppContext.SetSwitch("OpenAI.Experimental.EnableOpenTelemetry", true); - - _exportedActivities = new List(); - - // Wire up the full pipeline: OpenAI source → OpenAISpanProcessor → capture exporter - _tracerProvider = Sdk.CreateTracerProviderBuilder() - .AddSource("OpenAI.*") - .AddProcessor(new OpenAISpanProcessor()) - .AddProcessor(new SimpleActivityExportProcessor(new ActivityCapturingExporter(_exportedActivities))) - .Build(); - } - - [TestCleanup] - public void Cleanup() - { - _tracerProvider?.Dispose(); - } - - [TestMethod] - public async Task SimpleChat_ProcessorReceivesSpanWithExpectedAttributes() - { - SkipIfNoCredentials(); - - var chatClient = CreateChatClient(); - var messages = new List - { - new SystemChatMessage("You are a helpful assistant. Reply in one sentence."), - new UserChatMessage("What is the capital of France?") - }; - - await chatClient.CompleteChatAsync(messages); - - // Force flush to ensure processor has run - _tracerProvider!.ForceFlush(); - - _exportedActivities.Should().HaveCount(1); - var activity = _exportedActivities[0]; - - DumpActivity(activity, "SimpleChat — after OpenAISpanProcessor"); - - // Verify the processor received a valid OpenAI span - activity.Source.Name.Should().StartWith("OpenAI"); - activity.Kind.Should().Be(ActivityKind.Client); - - // Attributes present after processor - var tags = GetTags(activity); - tags.Should().ContainKey("gen_ai.system").WhoseValue.Should().Be("openai"); - tags.Should().ContainKey("gen_ai.operation.name").WhoseValue.Should().Be("chat"); - tags.Should().ContainKey("gen_ai.request.model"); - tags.Should().ContainKey("gen_ai.response.model"); - tags.Should().ContainKey("gen_ai.response.id"); - tags.Should().ContainKey("gen_ai.usage.input_tokens"); - tags.Should().ContainKey("gen_ai.usage.output_tokens"); - tags.Should().ContainKey("gen_ai.response.finish_reasons"); - tags.Should().ContainKey("server.address"); - tags.Should().ContainKey("server.port"); - - // Current gap: processor does not add message content - tags.Should().NotContainKey("gen_ai.input.messages"); - tags.Should().NotContainKey("gen_ai.output.messages"); - } - - [TestMethod] - public async Task ChatWithToolCall_ProcessorReceivesToolCallSpan() - { - SkipIfNoCredentials(); - - var chatClient = CreateChatClient(); - var weatherTool = ChatTool.CreateFunctionTool( - functionName: "get_weather", - functionDescription: "Get the current weather for a location", - functionParameters: BinaryData.FromString(""" - { - "type": "object", - "properties": { - "location": { "type": "string", "description": "City name" } - }, - "required": ["location"] - } - """)); - - var messages = new List - { - new SystemChatMessage("You are a weather assistant. Always use the get_weather tool."), - new UserChatMessage("What's the weather in Seattle?") - }; - - var options = new ChatCompletionOptions { Tools = { weatherTool } }; - await chatClient.CompleteChatAsync(messages, options); - - _tracerProvider!.ForceFlush(); - - _exportedActivities.Should().HaveCount(1); - var activity = _exportedActivities[0]; - - DumpActivity(activity, "ChatWithToolCall — after OpenAISpanProcessor"); - - var tags = GetTags(activity); - tags.Should().ContainKey("gen_ai.operation.name").WhoseValue.Should().Be("chat"); - - var finishReasons = activity.GetTagItem("gen_ai.response.finish_reasons") as string[]; - finishReasons.Should().Contain("tool_calls"); - - // Current gap: no message/tool call content captured - tags.Should().NotContainKey("gen_ai.input.messages"); - tags.Should().NotContainKey("gen_ai.output.messages"); - } - - [TestMethod] - public async Task ToolCallRoundTrip_ProcessorHandlesMultiTurnConversation() - { - SkipIfNoCredentials(); - - var chatClient = CreateChatClient(); - var weatherTool = ChatTool.CreateFunctionTool( - functionName: "get_weather", - functionDescription: "Get the current weather for a location", - functionParameters: BinaryData.FromString(""" - { - "type": "object", - "properties": { - "location": { "type": "string", "description": "City name" } - }, - "required": ["location"] - } - """)); - - // First call: trigger tool call - var messages = new List - { - new SystemChatMessage("You are a weather assistant. Always use the get_weather tool."), - new UserChatMessage("What's the weather in Seattle?") - }; - - var options = new ChatCompletionOptions { Tools = { weatherTool } }; - var response1 = await chatClient.CompleteChatAsync(messages, options); - response1.Value.ToolCalls.Should().NotBeEmpty("model should request tool call"); - - _exportedActivities.Clear(); - - // Second call: provide tool response and get final answer - var assistantMessage = new AssistantChatMessage(response1.Value); - var toolCallId = response1.Value.ToolCalls[0].Id; - var toolResponse = new ToolChatMessage(toolCallId, "Sunny, 72°F, light breeze from the west"); - - var followUp = new List - { - new SystemChatMessage("You are a weather assistant."), - new UserChatMessage("What's the weather in Seattle?"), - assistantMessage, - toolResponse - }; - - await chatClient.CompleteChatAsync(followUp); - - _tracerProvider!.ForceFlush(); - - _exportedActivities.Should().HaveCount(1); - var activity = _exportedActivities[0]; - - DumpActivity(activity, "ToolCallRoundTrip — after OpenAISpanProcessor"); - - var tags = GetTags(activity); - tags.Should().ContainKey("gen_ai.operation.name").WhoseValue.Should().Be("chat"); - tags.Should().ContainKey("gen_ai.usage.input_tokens"); - tags.Should().ContainKey("gen_ai.usage.output_tokens"); - - var finishReasons = activity.GetTagItem("gen_ai.response.finish_reasons") as string[]; - finishReasons.Should().Contain("stop"); - - // Current gap: multi-turn messages with tool call/response not captured - tags.Should().NotContainKey("gen_ai.input.messages"); - tags.Should().NotContainKey("gen_ai.output.messages"); - } - - #region Helpers - - private ChatClient CreateChatClient() - { - var client = new AzureOpenAIClient( - new Uri(Endpoint!), - new ApiKeyCredential(ApiKey!)); - return client.GetChatClient(Deployment!); - } - - private static void SkipIfNoCredentials() - { - if (!HasCredentials) - { - Assert.Inconclusive( - "Skipped: set AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY, AZURE_OPENAI_DEPLOYMENT env vars to run."); - } - } - - private static Dictionary GetTags(Activity activity) - { - return activity.TagObjects.ToDictionary(t => t.Key, t => t.Value); - } - - private static void DumpActivity(Activity activity, string label) - { - Console.WriteLine($"\n=== {label} ==="); - Console.WriteLine($" Source: {activity.Source.Name} Kind: {activity.Kind} Duration: {activity.Duration}"); - - Console.WriteLine(" Attributes:"); - foreach (var tag in activity.TagObjects) - Console.WriteLine($" {tag.Key} = {FormatValue(tag.Value)}"); - - if (activity.Events.Any()) - { - Console.WriteLine($" Events ({activity.Events.Count()}):"); - foreach (var ev in activity.Events) - { - Console.WriteLine($" '{ev.Name}'"); - foreach (var attr in ev.Tags) - Console.WriteLine($" {attr.Key} = {FormatValue(attr.Value)}"); - } - } - - Console.WriteLine("===\n"); - } - - private static string FormatValue(object? value) - { - string val = value switch - { - string s => s, - string[] arr => $"[{string.Join(", ", arr)}]", - null => "(null)", - _ => value.ToString() ?? "(null)" - }; - - if (val.Length > 120) - { - try - { - var doc = JsonDocument.Parse(val); - val = "\n " + JsonSerializer.Serialize(doc.RootElement, JsonPrint).Replace("\n", "\n "); - } - catch { } - } - - return val; - } - - #endregion -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/SemanticKernelInvokeAgentPipelineTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/SemanticKernelInvokeAgentPipelineTests.cs deleted file mode 100644 index a0b55f1f..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/SemanticKernelInvokeAgentPipelineTests.cs +++ /dev/null @@ -1,155 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System.Diagnostics; -using System.Text.Json; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Extensions.SemanticKernel; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.Logging; -using OpenTelemetry; -using OpenTelemetry.Trace; - -namespace Microsoft.Agents.A365.Observability.Extensions.IntegrationTests; - -/// -/// Pipeline integration tests for handling invoke_agent spans. -/// Verifies the full Builder → WithSemanticKernel → Processor → Exporter chain produces A365 versioned messages. -/// Does NOT require Azure OpenAI credentials. -/// -[TestClass] -public class SemanticKernelInvokeAgentPipelineTests -{ - private const string SkSourceName = "Microsoft.SemanticKernel.Agent"; - private List _exportedActivities = new(); - private ServiceProvider? _serviceProvider; - private ActivitySource? _activitySource; - - [TestInitialize] - public void Setup() - { - _exportedActivities = new List(); - - var services = new ServiceCollection(); - services.AddLogging(); - - new Runtime.Builder(services, configuration: null, useOpenTelemetryBuilder: true) - .WithSemanticKernel() - .Build(); - - services.AddOpenTelemetry() - .WithTracing(tracing => tracing - .AddSource(SkSourceName) - .AddProcessor(new SimpleActivityExportProcessor(new ActivityCapturingExporter(_exportedActivities)))); - - _serviceProvider = services.BuildServiceProvider(); - _serviceProvider.GetService(); - - _activitySource = new ActivitySource(SkSourceName); - } - - [TestCleanup] - public void Cleanup() - { - _activitySource?.Dispose(); - _serviceProvider?.Dispose(); - } - - [TestMethod] - public void InvokeAgent_MapsInvocationTagsToA365StructuredFormat() - { - // SK emits invoke_agent spans with invocation-specific keys containing JSON arrays of objects - var inputJson = @"[{""role"":""system"",""name"":""TestAgent"",""content"":""You are a helpful assistant."",""tool_calls"":[]},{""role"":""user"",""name"":null,""content"":""What can you do?"",""tool_calls"":[]}]"; - - var nestedContentJson = @"{""contentType"":""Text"",""content"":""I can help with many tasks!""}"; - var outputJson = $@"[{{""role"":""Assistant"",""name"":""TestAgent"",""content"":{JsonSerializer.Serialize(nestedContentJson)}}}]"; - - using (var activity = _activitySource!.StartActivity("invoke_agent TestAgent")) - { - activity!.SetTag(OpenTelemetryConstants.GenAiOperationNameKey, "invoke_agent"); - activity.SetTag(OpenTelemetryConstants.GenAiAgentInvocationInputKey, inputJson); - activity.SetTag(OpenTelemetryConstants.GenAiAgentInvocationOutputKey, outputJson); - } - - ForceFlush(); - - var span = FindInvokeAgentSpan(); - span.Should().NotBeNull("invoke_agent span should be captured by pipeline"); - var tags = GetTags(span!); - - // Invocation keys should be removed, results on standard keys - tags.Should().NotContainKey(OpenTelemetryConstants.GenAiAgentInvocationInputKey); - tags.Should().NotContainKey(OpenTelemetryConstants.GenAiAgentInvocationOutputKey); - - // Input: versioned, all roles preserved, TextPart format - var input = tags[OpenTelemetryConstants.GenAiInputMessagesKey] as string; - input.Should().StartWith("["); - input.Should().Contain("\"role\":\"system\"", "system messages should be preserved"); - input.Should().Contain("\"role\":\"user\"", "user messages should be mapped"); - input.Should().Contain("\"type\":\"text\"", "should use TextPart format"); - input.Should().Contain("You are a helpful assistant."); - input.Should().Contain("What can you do?"); - - // Output: versioned, nested content extracted, TextPart format - var output = tags[OpenTelemetryConstants.GenAiOutputMessagesKey] as string; - output.Should().StartWith("["); - output.Should().Contain("\"role\":\"assistant\""); - output.Should().Contain("\"type\":\"text\""); - output.Should().Contain("I can help with many tasks!", "nested content should be extracted"); - output.Should().NotContain("contentType", "wrapper should be unwrapped"); - } - - [TestMethod] - public void InvokeAgent_MapsStringArrayInputWithMessagePrefix() - { - // SK also emits invocation input as JSON array of serialized message strings (not objects) - var messages = new List - { - JsonSerializer.Serialize(new { role = "system", content = "System prompt" }), - JsonSerializer.Serialize(new { role = "user", content = "Message:Hello agent" }) - }; - var inputJson = JsonSerializer.Serialize(messages); - - using (var activity = _activitySource!.StartActivity("invoke_agent TestAgent")) - { - activity!.SetTag(OpenTelemetryConstants.GenAiOperationNameKey, "invoke_agent"); - activity.SetTag(OpenTelemetryConstants.GenAiAgentInvocationInputKey, inputJson); - } - - ForceFlush(); - - var span = FindInvokeAgentSpan(); - span.Should().NotBeNull(); - - var tags = GetTags(span!); - tags.Should().NotContainKey(OpenTelemetryConstants.GenAiAgentInvocationInputKey); - - var input = tags[OpenTelemetryConstants.GenAiInputMessagesKey] as string; - input.Should().StartWith("["); - input.Should().Contain("\"role\":\"system\"", "system messages should be preserved"); - input.Should().Contain("\"role\":\"user\""); - input.Should().Contain("Hello agent", "Message: prefix should be trimmed"); - input.Should().NotContain("Message:", "Message: prefix should be removed"); - } - - #region Helpers - - private void ForceFlush() - { - _serviceProvider?.GetService()?.ForceFlush(); - } - - private Activity? FindInvokeAgentSpan() - { - return _exportedActivities.FirstOrDefault(a => - a.GetTagItem(OpenTelemetryConstants.GenAiOperationNameKey) as string == "invoke_agent"); - } - - private static Dictionary GetTags(Activity activity) - { - return activity.TagObjects.ToDictionary(t => t.Key, t => t.Value); - } - - #endregion -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/SemanticKernelSpanProcessorTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/SemanticKernelSpanProcessorTests.cs deleted file mode 100644 index 572326c0..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Extensions.IntegrationTests/SemanticKernelSpanProcessorTests.cs +++ /dev/null @@ -1,342 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System.Diagnostics; -using System.Text.Json; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Extensions.SemanticKernel; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.Logging; -using Microsoft.SemanticKernel; -using Microsoft.SemanticKernel.ChatCompletion; -using Microsoft.SemanticKernel.Connectors.AzureOpenAI; -using OpenTelemetry; -using OpenTelemetry.Trace; - -namespace Microsoft.Agents.A365.Observability.Extensions.IntegrationTests; - -/// -/// Integration tests for running against real Azure OpenAI -/// via Semantic Kernel's . -/// Pipeline: SK SDK → TracerProvider → SemanticKernelSpanProcessor → captured spans. -/// Requires: AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY, AZURE_OPENAI_DEPLOYMENT env vars. -/// -[TestClass] -public class SemanticKernelSpanProcessorTests -{ - private static readonly JsonSerializerOptions JsonPrint = new() { WriteIndented = true }; - - private static string? Endpoint => Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT"); - private static string? ApiKey => Environment.GetEnvironmentVariable("AZURE_OPENAI_API_KEY"); - private static string? Deployment => Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT"); - - private static bool HasCredentials => - !string.IsNullOrEmpty(Endpoint) && - !string.IsNullOrEmpty(ApiKey) && - !string.IsNullOrEmpty(Deployment); - - private List _exportedActivities = new(); - private ServiceProvider? _serviceProvider; - - [TestInitialize] - public void Setup() - { - _exportedActivities = new List(); - - // Use the real A365 SDK initialization path: Builder → WithSemanticKernel → Build - var services = new ServiceCollection(); - services.AddLogging(); - - new Runtime.Builder(services, configuration: null, useOpenTelemetryBuilder: true) - .WithSemanticKernel() - .Build(); - - // Add a capturing exporter — runs after all processors, at export time - services.AddOpenTelemetry() - .WithTracing(tracing => tracing - .AddProcessor(new SimpleActivityExportProcessor(new ActivityCapturingExporter(_exportedActivities)))); - - _serviceProvider = services.BuildServiceProvider(); - - // Force the TracerProvider to be created by resolving it - _serviceProvider.GetService(); - } - - [TestCleanup] - public void Cleanup() - { - _serviceProvider?.Dispose(); - } - - [TestMethod] - public async Task SimpleChat_ProcessorSetsInputAndOutputMessages() - { - SkipIfNoCredentials(); - - var kernel = CreateKernel(); - var chatService = kernel.GetRequiredService(); - - var history = new ChatHistory(); - history.AddSystemMessage("You are a helpful assistant. Reply in one sentence."); - history.AddUserMessage("What is the capital of France?"); - - _ = await chatService.GetChatMessageContentAsync(history); - - ForceFlush(); - - // Find the chat completion span (SK may emit multiple spans) - var chatSpan = _exportedActivities.FirstOrDefault(a => - { - var op = a.GetTagItem(OpenTelemetryConstants.GenAiOperationNameKey) as string; - return op == "Chat" || op == "chat" || op == "chat.completions"; - }); - - chatSpan.Should().NotBeNull("SK should emit a chat completion span"); - DumpActivity(chatSpan!, "SK SimpleChat — after SemanticKernelSpanProcessor"); - - var tags = GetTags(chatSpan!); - - // Processor should rename operation to "Chat" - tags.Should().ContainKey("gen_ai.operation.name"); - - // Processor should produce versioned A365 input messages - tags.Should().ContainKey("gen_ai.input.messages", - "SemanticKernelSpanProcessor should map input events to A365 versioned format"); - - var inputMessages = tags["gen_ai.input.messages"] as string; - inputMessages.Should().StartWith("["); - inputMessages.Should().Contain("capital of France", "should contain user message text"); - inputMessages.Should().Contain("\"type\":\"text\"", "should use TextPart format"); - inputMessages.Should().Contain("\"role\":\"system\"", "should include system message"); - inputMessages.Should().Contain("\"role\":\"user\"", "should include user message"); - - // Processor should produce versioned A365 output messages - tags.Should().ContainKey("gen_ai.output.messages", - "SemanticKernelSpanProcessor should map choice events to A365 versioned format"); - - var outputMessages = tags["gen_ai.output.messages"] as string; - outputMessages.Should().StartWith("["); - outputMessages.Should().Contain("\"type\":\"text\"", "should use TextPart format"); - outputMessages.Should().Contain("\"role\":\"assistant\"", "should have assistant role"); - outputMessages.Should().Contain("\"finish_reason\":\"stop\"", "should map SK Stop → stop"); - - // Verify standard attributes are present - tags.Should().ContainKey("gen_ai.request.model"); - tags.Should().ContainKey("gen_ai.usage.input_tokens"); - tags.Should().ContainKey("gen_ai.usage.output_tokens"); - - // Dump all activities for visibility - Console.WriteLine($"\n All captured activities ({_exportedActivities.Count}):"); - foreach (var act in _exportedActivities) - { - var op = act.GetTagItem(OpenTelemetryConstants.GenAiOperationNameKey) as string ?? "(none)"; - Console.WriteLine($" {act.Source.Name} | {act.DisplayName} | op={op}"); - } - } - - [TestMethod] - public async Task ChatWithToolCall_ProcessorHandlesToolCallSpan() - { - SkipIfNoCredentials(); - - var kernel = CreateKernel(); - kernel.Plugins.AddFromFunctions("Weather", - [ - KernelFunctionFactory.CreateFromMethod( - ([System.ComponentModel.Description("City name")] string location) => - $"Sunny, 72°F in {location}", - "GetWeather", - "Get the current weather for a location") - ]); - - var chatService = kernel.GetRequiredService(); - - var history = new ChatHistory(); - history.AddSystemMessage("You are a weather assistant. Use the GetWeather function when asked about weather."); - history.AddUserMessage("What's the weather in Seattle?"); - - var settings = new AzureOpenAIPromptExecutionSettings - { - FunctionChoiceBehavior = FunctionChoiceBehavior.Auto() - }; - - _ = await chatService.GetChatMessageContentAsync(history, settings, kernel); - - ForceFlush(); - - // Dump all spans - Console.WriteLine($"\n All captured activities ({_exportedActivities.Count}):"); - foreach (var act in _exportedActivities) - { - var op = act.GetTagItem(OpenTelemetryConstants.GenAiOperationNameKey) as string ?? "(none)"; - Console.WriteLine($" {act.Source.Name} | {act.DisplayName} | op={op}"); - DumpActivity(act, $"SK ToolCall — {act.DisplayName}"); - } - - // Find the chat span - var chatSpan = _exportedActivities.FirstOrDefault(a => - { - var op = a.GetTagItem(OpenTelemetryConstants.GenAiOperationNameKey) as string; - return op == "Chat" || op == "chat" || op == "chat.completions"; - }); - - chatSpan.Should().NotBeNull("SK should emit a chat completion span"); - - var tags = GetTags(chatSpan!); - - // Input messages should be in versioned format with tool call parts - tags.Should().ContainKey("gen_ai.input.messages"); - var inputMessages = tags["gen_ai.input.messages"] as string; - inputMessages.Should().StartWith("["); - inputMessages.Should().Contain("\"role\":\"user\""); - inputMessages.Should().Contain("weather"); - - // The last chat span (after tool response) should have full round-trip in input - var lastChatSpan = _exportedActivities.LastOrDefault(a => - { - var op = a.GetTagItem(OpenTelemetryConstants.GenAiOperationNameKey) as string; - return op == "Chat"; - }); - - if (lastChatSpan != null) - { - var lastTags = GetTags(lastChatSpan); - var lastInput = lastTags["gen_ai.input.messages"] as string; - lastInput.Should().Contain("\"type\":\"tool_call\"", - "assistant tool call request should be mapped as ToolCallRequestPart"); - lastInput.Should().Contain("\"type\":\"tool_call_response\"", - "tool response should be mapped as ToolCallResponsePart"); - lastInput.Should().Contain("GetWeather", "should contain function name"); - - var lastOutput = lastTags["gen_ai.output.messages"] as string; - lastOutput.Should().StartWith("["); - lastOutput.Should().Contain("\"finish_reason\":\"stop\""); - } - - tags.Should().ContainKey("gen_ai.usage.input_tokens"); - } - - [TestMethod] - public async Task SimpleChat_SpanEventsContainMessageContent() - { - SkipIfNoCredentials(); - - var kernel = CreateKernel(); - var chatService = kernel.GetRequiredService(); - - var history = new ChatHistory(); - history.AddUserMessage("Say hello"); - - _ = await chatService.GetChatMessageContentAsync(history); - - ForceFlush(); - - // Find chat span and examine raw events before processor - var chatSpan = _exportedActivities.FirstOrDefault(a => - { - var op = a.GetTagItem(OpenTelemetryConstants.GenAiOperationNameKey) as string; - return op == "Chat" || op == "chat" || op == "chat.completions"; - }); - - chatSpan.Should().NotBeNull(); - - // SK emits events — log them to see the format - Console.WriteLine($"\n Span events ({chatSpan!.Events.Count()}):"); - foreach (var ev in chatSpan.Events) - { - Console.WriteLine($" Event: '{ev.Name}'"); - foreach (var attr in ev.Tags) - { - string val = FormatValue(attr.Value); - Console.WriteLine($" {attr.Key} = {val}"); - } - } - - // SK should emit gen_ai.user.message and gen_ai.choice events - var eventNames = chatSpan.Events.Select(e => e.Name).ToList(); - eventNames.Should().Contain("gen_ai.user.message", - "SK should emit user message events when sensitive diagnostics are enabled"); - eventNames.Should().Contain("gen_ai.choice", - "SK should emit choice events when sensitive diagnostics are enabled"); - } - - #region Helpers - - private void ForceFlush() - { - var tracerProvider = _serviceProvider?.GetService(); - tracerProvider?.ForceFlush(); - } - - private Kernel CreateKernel() - { - var builder = Kernel.CreateBuilder(); - builder.AddAzureOpenAIChatCompletion( - deploymentName: Deployment!, - endpoint: Endpoint!, - apiKey: ApiKey!); - return builder.Build(); - } - - private static void SkipIfNoCredentials() - { - if (!HasCredentials) - { - Assert.Inconclusive( - "Skipped: set AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY, AZURE_OPENAI_DEPLOYMENT env vars to run."); - } - } - - private static Dictionary GetTags(Activity activity) - { - return activity.TagObjects.ToDictionary(t => t.Key, t => t.Value); - } - - private static void DumpActivity(Activity activity, string label) - { - Console.WriteLine($"\n=== {label} ==="); - Console.WriteLine($" Source: {activity.Source.Name} Kind: {activity.Kind} Duration: {activity.Duration}"); - - Console.WriteLine(" Attributes:"); - foreach (var tag in activity.TagObjects) - Console.WriteLine($" {tag.Key} = {FormatValue(tag.Value)}"); - - if (activity.Events.Any()) - { - Console.WriteLine($" Events ({activity.Events.Count()}):"); - foreach (var ev in activity.Events) - { - Console.WriteLine($" '{ev.Name}'"); - foreach (var attr in ev.Tags) - Console.WriteLine($" {attr.Key} = {FormatValue(attr.Value)}"); - } - } - Console.WriteLine("===\n"); - } - - private static string FormatValue(object? value) - { - string val = value switch - { - string s => s, - string[] arr => $"[{string.Join(", ", arr)}]", - null => "(null)", - _ => value.ToString() ?? "(null)" - }; - - if (val.Length > 120) - { - try - { - var doc = JsonDocument.Parse(val); - val = "\n " + JsonSerializer.Serialize(doc.RootElement, JsonPrint).Replace("\n", "\n "); - } - catch { } - } - - return val; - } - - #endregion -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Caching/AgenticTokenCacheTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Caching/AgenticTokenCacheTests.cs deleted file mode 100644 index bd6f3057..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Caching/AgenticTokenCacheTests.cs +++ /dev/null @@ -1,464 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System.Reflection; -using System.Text; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Hosting.Caching; - -namespace Microsoft.Agents.A365.Observability.Hosting.Tests.Caching; - -[TestClass] -public sealed class AgenticTokenCacheTests -{ - private const string TestAgentId = "test-agent"; - private const string TestTenantId = "test-tenant"; - private const string ScopeOverrideEnvVar = "A365_OBSERVABILITY_SCOPE_OVERRIDE"; - private const string DefaultObsScope = "api://9b975845-388f-4429-889e-eab1ef63949c/.default"; - private static readonly string[] TestScopes = new[] { "api://9b975845-388f-4429-889e-eab1ef63949c/.default" }; - - [TestCleanup] - public void TestCleanup() - { - Environment.SetEnvironmentVariable(ScopeOverrideEnvVar, null); - } - - [TestMethod] - public async Task RegisterObservability_WithAppOnlyResolver_CachesTokenPerAgentTenant() - { - var resolverCalls = 0; - var cache = new AgenticTokenCache(TimeSpan.Zero, () => DateTimeOffset.UtcNow); - ObservabilityTokenResolver resolver = (agentId, tenantId, scopes) => - { - Interlocked.Increment(ref resolverCalls); - agentId.Should().Be(TestAgentId); - tenantId.Should().Be(TestTenantId); - scopes.Should().Equal(TestScopes); - return Task.FromResult("app-only-token"); - }; - - cache.RegisterObservability(TestAgentId, TestTenantId, resolver, TestScopes); - - var firstToken = await cache.GetObservabilityToken(TestAgentId, TestTenantId); - var secondToken = await cache.GetObservabilityToken(TestAgentId, TestTenantId); - - firstToken.Should().Be("app-only-token"); - secondToken.Should().Be("app-only-token"); - resolverCalls.Should().Be(1, "the cache should reuse a usable token for the same agent/tenant key"); - } - - [TestMethod] - public async Task RegisterObservability_RepeatedRegistration_DoesNotReplaceResolverOrClearToken() - { - var firstResolverCalls = 0; - var secondResolverCalls = 0; - var cache = new AgenticTokenCache(TimeSpan.Zero, () => DateTimeOffset.UtcNow); - - cache.RegisterObservability(TestAgentId, TestTenantId, (_, _, _) => - { - Interlocked.Increment(ref firstResolverCalls); - return Task.FromResult("first-token"); - }, TestScopes); - - cache.RegisterObservability(TestAgentId, TestTenantId, (_, _, _) => - { - Interlocked.Increment(ref secondResolverCalls); - return Task.FromResult("second-token"); - }, TestScopes); - - (await cache.GetObservabilityToken(TestAgentId, TestTenantId)).Should().Be("first-token"); - - cache.RegisterObservability(TestAgentId, TestTenantId, (_, _, _) => - { - Interlocked.Increment(ref secondResolverCalls); - return Task.FromResult("second-token"); - }, TestScopes); - - (await cache.GetObservabilityToken(TestAgentId, TestTenantId)).Should().Be("first-token"); - firstResolverCalls.Should().Be(1); - secondResolverCalls.Should().Be(0, "RegisterObservability is idempotent and first registration wins"); - } - - [TestMethod] - public async Task RegisterObservability_MultipleAgentTenantKeys_AreIndependent() - { - var cache = new AgenticTokenCache(TimeSpan.Zero, () => DateTimeOffset.UtcNow); - - cache.RegisterObservability("agent-1", "tenant-1", (_, _, _) => Task.FromResult("token-11"), TestScopes); - cache.RegisterObservability("agent-2", "tenant-1", (_, _, _) => Task.FromResult("token-21"), TestScopes); - cache.RegisterObservability("agent-1", "tenant-2", (_, _, _) => Task.FromResult("token-12"), TestScopes); - - (await cache.GetObservabilityToken("agent-1", "tenant-1")).Should().Be("token-11"); - (await cache.GetObservabilityToken("agent-2", "tenant-1")).Should().Be("token-21"); - (await cache.GetObservabilityToken("agent-1", "tenant-2")).Should().Be("token-12"); - } - - [TestMethod] - public async Task RefreshObservabilityToken_UpdatesCachedTokenAndExpiry() - { - var now = DateTimeOffset.Parse("2026-09-29T00:00:00Z"); - var cache = new AgenticTokenCache(TimeSpan.Zero, () => now); - var firstToken = CreateJwt(now.AddMinutes(4)); - var secondToken = CreateJwt(now.AddMinutes(20)); - - await cache.RefreshObservabilityToken(TestAgentId, TestTenantId, (_, _, _) => Task.FromResult(firstToken), TestScopes); - await cache.RefreshObservabilityToken(TestAgentId, TestTenantId, (_, _, _) => Task.FromResult(secondToken), TestScopes); - - var token = await cache.GetObservabilityToken(TestAgentId, TestTenantId); - - token.Should().Be(secondToken); - } - - [TestMethod] - public async Task RefreshObservabilityToken_ReturnsCachedTokenUntilNearExpiry_AndReplacesFutureResolver() - { - var now = DateTimeOffset.Parse("2026-09-29T00:00:00Z"); - var firstToken = CreateJwt(now.AddMinutes(10)); - var secondToken = CreateJwt(now.AddMinutes(20)); - var firstResolverCalls = 0; - var secondResolverCalls = 0; - var cache = new AgenticTokenCache(TimeSpan.Zero, () => now); - - await cache.RefreshObservabilityToken(TestAgentId, TestTenantId, (_, _, _) => - { - Interlocked.Increment(ref firstResolverCalls); - return Task.FromResult(firstToken); - }, TestScopes); - - var cachedToken = await cache.RefreshObservabilityToken(TestAgentId, TestTenantId, (_, _, _) => - { - Interlocked.Increment(ref secondResolverCalls); - return Task.FromResult(secondToken); - }, TestScopes); - - cachedToken.Should().Be(firstToken); - firstResolverCalls.Should().Be(1); - secondResolverCalls.Should().Be(0, "RefreshObservabilityToken should not call the resolver while a token is usable"); - - now = now.AddMinutes(6); - (await cache.GetObservabilityToken(TestAgentId, TestTenantId)).Should().Be(secondToken); - secondResolverCalls.Should().Be(1, "the resolver passed to RefreshObservabilityToken replaces the resolver for future refreshes"); - } - - [TestMethod] - public async Task RefreshObservabilityToken_ThreeArgumentOverload_UsesDefaultAppOnlyScope() - { - Environment.SetEnvironmentVariable(ScopeOverrideEnvVar, null); - string[]? capturedScopes = null; - var cache = new AgenticTokenCache(TimeSpan.Zero, () => DateTimeOffset.UtcNow); - - await cache.RefreshObservabilityToken(TestAgentId, TestTenantId, (_, _, scopes) => - { - capturedScopes = scopes; - return Task.FromResult("app-only-token"); - }); - - capturedScopes.Should().Equal(DefaultObsScope); - } - - [TestMethod] - public async Task RefreshObservabilityToken_ThreeArgumentOverload_HonorsScopeOverride() - { - const string overrideScope = "api://override-resource/.default"; - Environment.SetEnvironmentVariable(ScopeOverrideEnvVar, overrideScope); - string[]? capturedScopes = null; - var cache = new AgenticTokenCache(TimeSpan.Zero, () => DateTimeOffset.UtcNow); - - await cache.RefreshObservabilityToken(TestAgentId, TestTenantId, (_, _, scopes) => - { - capturedScopes = scopes; - return Task.FromResult("app-only-token"); - }); - - capturedScopes.Should().Equal(overrideScope); - } - - [TestMethod] - public async Task RefreshObservabilityToken_OpaqueTokenClearsStaleExpiryMetadata() - { - var now = DateTimeOffset.Parse("2026-09-29T00:00:00Z"); - var cache = new AgenticTokenCache(TimeSpan.Zero, () => now); - var expiredJwt = CreateJwt(now.AddMinutes(-10)); - - await cache.RefreshObservabilityToken(TestAgentId, TestTenantId, (_, _, _) => Task.FromResult(expiredJwt), TestScopes); - await cache.RefreshObservabilityToken(TestAgentId, TestTenantId, (_, _, _) => Task.FromResult("opaque-token"), TestScopes); - - var removed = cache.RemoveExpiredTokens(); - - removed.Should().Be(0, "refreshing with an opaque token should clear the prior JWT expiry metadata"); - cache.Count.Should().Be(1); - (await cache.GetObservabilityToken(TestAgentId, TestTenantId)).Should().Be("opaque-token"); - } - - [TestMethod] - public async Task OpaqueToken_IsUsableBeforeFallbackMaxAge_AndRefreshesAtBoundary() - { - var acquiredAt = DateTimeOffset.Parse("2026-09-29T00:00:00Z"); - var now = acquiredAt; - var resolverCalls = 0; - var cache = new AgenticTokenCache(TimeSpan.Zero, () => now); - ObservabilityTokenResolver resolver = (_, _, _) => - { - var call = Interlocked.Increment(ref resolverCalls); - return Task.FromResult(call == 1 ? "opaque-token-1" : "opaque-token-2"); - }; - - await cache.RefreshObservabilityToken(TestAgentId, TestTenantId, resolver, TestScopes); - - now = acquiredAt.AddHours(1).AddTicks(-1); - (await cache.GetObservabilityToken(TestAgentId, TestTenantId)).Should().Be("opaque-token-1"); - resolverCalls.Should().Be(1); - - now = acquiredAt.AddHours(1); - (await cache.GetObservabilityToken(TestAgentId, TestTenantId)).Should().Be("opaque-token-2"); - resolverCalls.Should().Be(2); - } - - [TestMethod] - public async Task RefreshObservabilityToken_ResolverFailure_PropagatesAndClearsCachedToken() - { - var now = DateTimeOffset.Parse("2026-09-29T00:00:00Z"); - var cache = new AgenticTokenCache(TimeSpan.Zero, () => now); - await cache.RefreshObservabilityToken(TestAgentId, TestTenantId, (_, _, _) => Task.FromResult(CreateJwt(now.AddMinutes(4))), TestScopes); - - Func act = () => cache.RefreshObservabilityToken( - TestAgentId, - TestTenantId, - (_, _, _) => Task.FromException(new InvalidOperationException("acquisition failed")), - TestScopes); - - await act.Should().ThrowAsync().WithMessage("acquisition failed"); - - cache.RemoveExpiredTokens().Should().Be(0, "a refresh failure must clear stale token and expiry metadata"); - Func get = async () => await cache.GetObservabilityToken(TestAgentId, TestTenantId); - await get.Should().ThrowAsync() - .WithMessage("acquisition failed"); - } - - [TestMethod] - public async Task RefreshObservabilityToken_EmptyToken_PropagatesFailureAndClearsCachedToken() - { - var now = DateTimeOffset.Parse("2026-09-29T00:00:00Z"); - var cache = new AgenticTokenCache(TimeSpan.Zero, () => now); - await cache.RefreshObservabilityToken(TestAgentId, TestTenantId, (_, _, _) => Task.FromResult(CreateJwt(now.AddMinutes(4))), TestScopes); - - Func act = () => cache.RefreshObservabilityToken( - TestAgentId, - TestTenantId, - (_, _, _) => Task.FromResult(string.Empty), - TestScopes); - - await act.Should().ThrowAsync() - .WithMessage("The observability token resolver returned an empty token."); - - cache.RemoveExpiredTokens().Should().Be(0, "an empty resolver result must clear stale token and expiry metadata"); - Func get = async () => await cache.GetObservabilityToken(TestAgentId, TestTenantId); - await get.Should().ThrowAsync() - .WithMessage("The observability token resolver returned an empty token."); - } - - [TestMethod] - public async Task RefreshObservabilityToken_ConcurrentCallers_ShareOneAcquisition() - { - var resolverCalls = 0; - var release = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); - var cache = new AgenticTokenCache(TimeSpan.Zero, () => DateTimeOffset.UtcNow); - ObservabilityTokenResolver resolver = (_, _, _) => - { - Interlocked.Increment(ref resolverCalls); - return release.Task; - }; - - var first = cache.RefreshObservabilityToken(TestAgentId, TestTenantId, resolver, TestScopes); - var second = cache.RefreshObservabilityToken(TestAgentId, TestTenantId, resolver, TestScopes); - release.SetResult("app-only-token"); - - (await Task.WhenAll(first, second)).Should().Equal("app-only-token", "app-only-token"); - resolverCalls.Should().Be(1, "concurrent refreshes for one agent and tenant should share one acquisition"); - } - - [TestMethod] - public async Task RefreshObservabilityToken_ConcurrentFailure_DoesNotClearTokenCachedByAnotherCaller() - { - var failingResolverCalls = 0; - var release = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); - var cache = new AgenticTokenCache(TimeSpan.Zero, () => DateTimeOffset.UtcNow); - - var succeeding = cache.RefreshObservabilityToken(TestAgentId, TestTenantId, (_, _, _) => release.Task, TestScopes); - var failing = cache.RefreshObservabilityToken(TestAgentId, TestTenantId, (_, _, _) => - { - Interlocked.Increment(ref failingResolverCalls); - return Task.FromException(new InvalidOperationException("acquisition failed")); - }, TestScopes); - release.SetResult("app-only-token"); - - (await succeeding).Should().Be("app-only-token"); - (await failing).Should().Be("app-only-token", "a waiting caller should reuse the token cached by the refresh ahead of it"); - failingResolverCalls.Should().Be(0); - (await cache.GetObservabilityToken(TestAgentId, TestTenantId)).Should().Be("app-only-token"); - } - - [TestMethod] - public async Task RemoveExpiredTokens_ClearsExpiredTokenButKeepsResolverRegistration() - { - var now = DateTimeOffset.Parse("2026-09-29T00:00:00Z"); - var resolverCalls = 0; - var cache = new AgenticTokenCache(TimeSpan.Zero, () => now); - ObservabilityTokenResolver resolver = (_, _, _) => - { - Interlocked.Increment(ref resolverCalls); - return Task.FromResult(CreateJwt(now.AddHours(1))); - }; - - cache.RegisterObservability(TestAgentId, TestTenantId, resolver, TestScopes); - var firstToken = await cache.GetObservabilityToken(TestAgentId, TestTenantId); - - now = now.AddHours(2); - cache.RemoveExpiredTokens().Should().Be(1); - cache.Count.Should().Be(1, "cleanup must keep the app-only resolver registration"); - - var secondToken = await cache.GetObservabilityToken(TestAgentId, TestTenantId); - secondToken.Should().NotBeNull().And.NotBe(firstToken); - resolverCalls.Should().Be(2); - } - - [TestMethod] - public async Task RemoveExpiredTokens_SkipsEntryWhileRefreshIsInFlight() - { - var now = DateTimeOffset.Parse("2026-09-29T00:00:00Z"); - var cache = new AgenticTokenCache(TimeSpan.Zero, () => now); - await cache.RefreshObservabilityToken(TestAgentId, TestTenantId, (_, _, _) => Task.FromResult(CreateJwt(now.AddMinutes(10))), TestScopes); - - now = now.AddMinutes(30); - var release = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); - var refresh = cache.RefreshObservabilityToken(TestAgentId, TestTenantId, (_, _, _) => release.Task, TestScopes); - - cache.RemoveExpiredTokens().Should().Be(0, "cleanup must not race an in-flight refresh"); - release.SetResult(CreateJwt(now.AddHours(1))); - - var refreshed = await refresh; - (await cache.GetObservabilityToken(TestAgentId, TestTenantId)).Should().Be(refreshed); - } - - [TestMethod] - public async Task GetObservabilityToken_QueuedBehindExplicitRefresh_DoesNotRestoreReplacedResolver() - { - var now = DateTimeOffset.Parse("2026-09-29T00:00:00Z"); - var registeredResolverCalls = 0; - var replacementResolverCalls = 0; - var firstToken = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); - var cache = new AgenticTokenCache(TimeSpan.Zero, () => now); - cache.RegisterObservability(TestAgentId, TestTenantId, (_, _, _) => - Interlocked.Increment(ref registeredResolverCalls) == 1 - ? firstToken.Task - : Task.FromResult(CreateJwt(now.AddHours(1))), TestScopes); - ObservabilityTokenResolver replacementResolver = (_, _, _) => - { - Interlocked.Increment(ref replacementResolverCalls); - return Task.FromResult(CreateJwt(now.AddHours(1))); - }; - - var inFlightGet = cache.GetObservabilityToken(TestAgentId, TestTenantId); - var explicitRefresh = cache.RefreshObservabilityToken(TestAgentId, TestTenantId, replacementResolver, TestScopes); - var queuedGet = cache.GetObservabilityToken(TestAgentId, TestTenantId); - firstToken.SetResult(CreateJwt(now.AddHours(1))); - await Task.WhenAll(new Task[] { inFlightGet, explicitRefresh, queuedGet }); - - now = now.AddHours(2); - await cache.GetObservabilityToken(TestAgentId, TestTenantId); - - replacementResolverCalls.Should().Be(1, "the resolver installed by an explicit refresh is used for the next refresh"); - registeredResolverCalls.Should().Be(1, "a cache-driven refresh must not restore the resolver an explicit refresh replaced"); - } - - [TestMethod] - [DataRow("opaque.token")] - [DataRow("not.a.jwt")] - [DataRow("header.@@@.signature")] - public async Task RefreshObservabilityToken_UnparseableTokenWithPeriods_UsesOpaqueFallback(string unparseableToken) - { - var acquiredAt = DateTimeOffset.Parse("2026-09-29T00:00:00Z"); - var now = acquiredAt; - var resolverCalls = 0; - var cache = new AgenticTokenCache(TimeSpan.Zero, () => now); - ObservabilityTokenResolver resolver = (_, _, _) => - { - Interlocked.Increment(ref resolverCalls); - return Task.FromResult(unparseableToken); - }; - - (await cache.RefreshObservabilityToken(TestAgentId, TestTenantId, resolver, TestScopes)).Should().Be(unparseableToken); - - now = acquiredAt.AddMinutes(59); - (await cache.GetObservabilityToken(TestAgentId, TestTenantId)).Should().Be(unparseableToken); - resolverCalls.Should().Be(1, "tokens the JWT handler cannot parse use the opaque one-hour fallback"); - } - - [TestMethod] - public async Task RefreshObservabilityToken_JwtWithNonNumericExp_UsesOpaqueFallback() - { - var acquiredAt = DateTimeOffset.Parse("2026-09-29T00:00:00Z"); - var now = acquiredAt; - var token = $"{Base64Url("{\"alg\":\"none\",\"typ\":\"JWT\"}")}.{Base64Url("{\"exp\":\"soon\"}")}."; - var cache = new AgenticTokenCache(TimeSpan.Zero, () => now); - - (await cache.RefreshObservabilityToken(TestAgentId, TestTenantId, (_, _, _) => Task.FromResult(token), TestScopes)).Should().Be(token); - - now = acquiredAt.AddMinutes(59); - (await cache.GetObservabilityToken(TestAgentId, TestTenantId)).Should().Be(token); - } - - [TestMethod] - public async Task RefreshObservabilityToken_SeparatorBearingIds_DoNotShareCacheEntry() - { - var secondResolverCalls = 0; - var cache = new AgenticTokenCache(TimeSpan.Zero, () => DateTimeOffset.UtcNow); - - (await cache.RefreshObservabilityToken("a:b", "c", (_, _, _) => Task.FromResult("token-for-first"), TestScopes)) - .Should().Be("token-for-first"); - (await cache.RefreshObservabilityToken("a", "b:c", (_, _, _) => - { - Interlocked.Increment(ref secondResolverCalls); - return Task.FromResult("token-for-second"); - }, TestScopes)).Should().Be("token-for-second"); - - secondResolverCalls.Should().Be(1, "a different agent/tenant pair must not reuse another pair's token"); - (await cache.GetObservabilityToken("a:b", "c")).Should().Be("token-for-first"); - (await cache.GetObservabilityToken("a", "b:c")).Should().Be("token-for-second"); - cache.Count.Should().Be(2); - } - - [TestMethod] - public async Task RemovedDelegatedRegisterObservabilityShape_DoesNotRegisterOrThrowWhenInvokedDynamically() - { - var cache = new AgenticTokenCache(TimeSpan.Zero, () => DateTimeOffset.UtcNow); - var removedOverload = typeof(AgenticTokenCache).GetMethod( - nameof(AgenticTokenCache.RegisterObservability), - BindingFlags.Instance | BindingFlags.Public, - binder: null, - types: new[] { typeof(string), typeof(string), typeof(AgenticTokenStruct), typeof(string[]) }, - modifiers: null); - - removedOverload.Should().NotBeNull(); - var act = () => removedOverload!.Invoke(cache, new object?[] { TestAgentId, TestTenantId, null, TestScopes }); - - act.Should().NotThrow("untyped callers should get no delegated exchange and no crash"); - cache.Count.Should().Be(0); - (await cache.GetObservabilityToken(TestAgentId, TestTenantId)).Should().BeNull(); - } - - private static string CreateJwt(DateTimeOffset expiresAt) - { - var header = Base64Url("{\"alg\":\"none\",\"typ\":\"JWT\"}"); - var payload = Base64Url($"{{\"exp\":{expiresAt.ToUnixTimeSeconds()}}}"); - return $"{header}.{payload}."; - } - - private static string Base64Url(string value) - { - return Convert.ToBase64String(Encoding.UTF8.GetBytes(value)) - .TrimEnd('=') - .Replace('+', '-') - .Replace('/', '_'); - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Caching/ServiceTokenCacheTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Caching/ServiceTokenCacheTests.cs deleted file mode 100644 index be2b92be..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Caching/ServiceTokenCacheTests.cs +++ /dev/null @@ -1,425 +0,0 @@ -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Hosting.Caching; - -namespace Microsoft.Agents.A365.Observability.Hosting.Tests.Caching; - -[TestClass] -public sealed class ServiceTokenCacheTests -{ - private const string TestAgentId = "test-agent"; - private const string TestTenantId = "test-tenant"; - private const string TestToken = "test-token-12345"; - private readonly string[] TestScopes = new[] { "https://example.com/.default" }; - - [TestMethod] - public void Constructor_WithDefaultExpiration_ShouldSucceed() - { - var cache = new ServiceTokenCache(); - cache.Should().NotBeNull(); - } - - [TestMethod] - public void Constructor_WithCustomExpiration_ShouldSucceed() - { - var cache = new ServiceTokenCache(TimeSpan.FromMinutes(30)); - cache.Should().NotBeNull(); - } - - [TestMethod] - public void Constructor_WithZeroExpiration_ShouldThrow() - { - Action act = () => new ServiceTokenCache(TimeSpan.Zero); - act.Should().Throw() - .WithMessage("*Default expiration must be greater than zero*"); - } - - [TestMethod] - public void Constructor_WithNegativeExpiration_ShouldThrow() - { - Action act = () => new ServiceTokenCache(TimeSpan.FromMinutes(-1)); - act.Should().Throw() - .WithMessage("*Default expiration must be greater than zero*"); - } - - [TestMethod] - public async Task RegisterObservability_WithValidParameters_ShouldSucceed() - { - var cache = new ServiceTokenCache(); - - cache.RegisterObservability(TestAgentId, TestTenantId, TestToken, TestScopes); - - var token = await cache.GetObservabilityToken(TestAgentId, TestTenantId); - token.Should().Be(TestToken); - } - - [TestMethod] - public void RegisterObservability_WithNullAgentId_ShouldThrow() - { - var cache = new ServiceTokenCache(); - - Action act = () => cache.RegisterObservability(null!, TestTenantId, TestToken, TestScopes); - - act.Should().Throw() - .WithMessage("*Value cannot be null or whitespace*") - .WithParameterName("agentId"); - } - - [TestMethod] - public void RegisterObservability_WithEmptyAgentId_ShouldThrow() - { - var cache = new ServiceTokenCache(); - - Action act = () => cache.RegisterObservability("", TestTenantId, TestToken, TestScopes); - - act.Should().Throw() - .WithMessage("*Value cannot be null or whitespace*") - .WithParameterName("agentId"); - } - - [TestMethod] - public void RegisterObservability_WithNullTenantId_ShouldThrow() - { - var cache = new ServiceTokenCache(); - - Action act = () => cache.RegisterObservability(TestAgentId, null!, TestToken, TestScopes); - - act.Should().Throw() - .WithMessage("*Value cannot be null or whitespace*") - .WithParameterName("tenantId"); - } - - [TestMethod] - public void RegisterObservability_WithEmptyToken_ShouldThrow() - { - var cache = new ServiceTokenCache(); - - Action act = () => cache.RegisterObservability(TestAgentId, TestTenantId, "", TestScopes); - - act.Should().Throw() - .WithMessage("*Value cannot be null or whitespace*") - .WithParameterName("token"); - } - - [TestMethod] - public void RegisterObservability_WithNullScopes_ShouldThrow() - { - var cache = new ServiceTokenCache(); - - Action act = () => cache.RegisterObservability(TestAgentId, TestTenantId, TestToken, null!); - - act.Should().Throw() - .WithMessage("*Observability scopes cannot be null or empty*") - .WithParameterName("observabilityScopes"); - } - - [TestMethod] - public void RegisterObservability_WithEmptyScopes_ShouldThrow() - { - var cache = new ServiceTokenCache(); - - Action act = () => cache.RegisterObservability(TestAgentId, TestTenantId, TestToken, Array.Empty()); - - act.Should().Throw() - .WithMessage("*Observability scopes cannot be null or empty*") - .WithParameterName("observabilityScopes"); - } - - [TestMethod] - public async Task RegisterObservability_WithCustomExpiration_ShouldRespectExpiration() - { - var cache = new ServiceTokenCache(); - var customExpiration = TimeSpan.FromSeconds(1); - - cache.RegisterObservability(TestAgentId, TestTenantId, TestToken, TestScopes, customExpiration); - - var tokenBefore = await cache.GetObservabilityToken(TestAgentId, TestTenantId); - tokenBefore.Should().Be(TestToken); - - Thread.Sleep(1100); - - var tokenAfter = await cache.GetObservabilityToken(TestAgentId, TestTenantId); - tokenAfter.Should().BeNull(); - } - - [TestMethod] - public void RegisterObservability_WithZeroCustomExpiration_ShouldThrow() - { - var cache = new ServiceTokenCache(); - - Action act = () => cache.RegisterObservability(TestAgentId, TestTenantId, TestToken, TestScopes, TimeSpan.Zero); - - act.Should().Throw() - .WithMessage("*Expiration time must be greater than zero*") - .WithParameterName("expiresIn"); - } - - [TestMethod] - public async Task RegisterObservability_Twice_ShouldUpdateToken() - { - var cache = new ServiceTokenCache(); - const string firstToken = "first-token"; - const string secondToken = "second-token"; - - cache.RegisterObservability(TestAgentId, TestTenantId, firstToken, TestScopes); - cache.RegisterObservability(TestAgentId, TestTenantId, secondToken, TestScopes); - - var token = await cache.GetObservabilityToken(TestAgentId, TestTenantId); - token.Should().Be(secondToken); - } - - [TestMethod] - public async Task GetObservabilityToken_WithNonExistentKey_ShouldReturnNull() - { - var cache = new ServiceTokenCache(); - - var token = await cache.GetObservabilityToken("non-existent-agent", "non-existent-tenant"); - - token.Should().BeNull(); - } - - [TestMethod] - public async Task GetObservabilityToken_WithNullAgentId_ShouldReturnNull() - { - var cache = new ServiceTokenCache(); - - var token = await cache.GetObservabilityToken(null!, TestTenantId); - - token.Should().BeNull(); - } - - [TestMethod] - public async Task GetObservabilityToken_WithEmptyTenantId_ShouldReturnNull() - { - var cache = new ServiceTokenCache(); - - var token = await cache.GetObservabilityToken(TestAgentId, ""); - - token.Should().BeNull(); - } - - [TestMethod] - public async Task GetObservabilityToken_AfterExpiration_ShouldReturnNull() - { - var cache = new ServiceTokenCache(TimeSpan.FromMilliseconds(500)); - - cache.RegisterObservability(TestAgentId, TestTenantId, TestToken, TestScopes); - - Thread.Sleep(600); - - var token = await cache.GetObservabilityToken(TestAgentId, TestTenantId); - token.Should().BeNull(); - } - - [TestMethod] - public async Task GetObservabilityToken_BeforeExpiration_ShouldReturnToken() - { - var cache = new ServiceTokenCache(TimeSpan.FromSeconds(10)); - - cache.RegisterObservability(TestAgentId, TestTenantId, TestToken, TestScopes); - - var token = await cache.GetObservabilityToken(TestAgentId, TestTenantId); - token.Should().Be(TestToken); - } - - [TestMethod] - public async Task InvalidateToken_WithExistingToken_ShouldReturnTrueAndRemoveToken() - { - var cache = new ServiceTokenCache(); - cache.RegisterObservability(TestAgentId, TestTenantId, TestToken, TestScopes); - - var result = cache.InvalidateToken(TestAgentId, TestTenantId); - - result.Should().BeTrue(); - - var token = await cache.GetObservabilityToken(TestAgentId, TestTenantId); - token.Should().BeNull(); - } - - [TestMethod] - public void InvalidateToken_WithNonExistentToken_ShouldReturnFalse() - { - var cache = new ServiceTokenCache(); - - var result = cache.InvalidateToken("non-existent-agent", "non-existent-tenant"); - - result.Should().BeFalse(); - } - - [TestMethod] - public void InvalidateToken_WithNullAgentId_ShouldReturnFalse() - { - var cache = new ServiceTokenCache(); - - var result = cache.InvalidateToken(null!, TestTenantId); - - result.Should().BeFalse(); - } - - [TestMethod] - public void InvalidateToken_WithEmptyTenantId_ShouldReturnFalse() - { - var cache = new ServiceTokenCache(); - - var result = cache.InvalidateToken(TestAgentId, ""); - - result.Should().BeFalse(); - } - - [TestMethod] - public async Task InvalidateAll_ShouldRemoveAllTokens() - { - var cache = new ServiceTokenCache(); - cache.RegisterObservability("agent1", "tenant1", "token1", TestScopes); - cache.RegisterObservability("agent2", "tenant2", "token2", TestScopes); - cache.RegisterObservability("agent3", "tenant3", "token3", TestScopes); - - cache.InvalidateAll(); - - (await cache.GetObservabilityToken("agent1", "tenant1")).Should().BeNull(); - (await cache.GetObservabilityToken("agent2", "tenant2")).Should().BeNull(); - (await cache.GetObservabilityToken("agent3", "tenant3")).Should().BeNull(); - } - - [TestMethod] - public async Task RemoveExpiredTokens_WithNoExpiredTokens_ShouldReturnZero() - { - var cache = new ServiceTokenCache(TimeSpan.FromHours(1)); - cache.RegisterObservability(TestAgentId, TestTenantId, TestToken, TestScopes); - - var removedCount = cache.RemoveExpiredTokens(); - - removedCount.Should().Be(0); - (await cache.GetObservabilityToken(TestAgentId, TestTenantId)).Should().Be(TestToken); - } - - [TestMethod] - public async Task RemoveExpiredTokens_WithExpiredTokens_ShouldRemoveThemAndReturnCount() - { - var cache = new ServiceTokenCache(TimeSpan.FromMilliseconds(500)); - cache.RegisterObservability("agent1", "tenant1", "token1", TestScopes); - cache.RegisterObservability("agent2", "tenant2", "token2", TestScopes); - - Thread.Sleep(600); - - cache.RegisterObservability("agent3", "tenant3", "token3", TestScopes, TimeSpan.FromHours(1)); - - var removedCount = cache.RemoveExpiredTokens(); - - removedCount.Should().Be(2); - (await cache.GetObservabilityToken("agent1", "tenant1")).Should().BeNull(); - (await cache.GetObservabilityToken("agent2", "tenant2")).Should().BeNull(); - (await cache.GetObservabilityToken("agent3", "tenant3")).Should().Be("token3"); - } - - [TestMethod] - public async Task RemoveExpiredTokens_WithMixedExpirations_ShouldOnlyRemoveExpired() - { - var cache = new ServiceTokenCache(); - cache.RegisterObservability("agent1", "tenant1", "token1", TestScopes, TimeSpan.FromMilliseconds(500)); - cache.RegisterObservability("agent2", "tenant2", "token2", TestScopes, TimeSpan.FromHours(1)); - cache.RegisterObservability("agent3", "tenant3", "token3", TestScopes, TimeSpan.FromMilliseconds(500)); - - Thread.Sleep(600); - - var removedCount = cache.RemoveExpiredTokens(); - - removedCount.Should().Be(2); - (await cache.GetObservabilityToken("agent1", "tenant1")).Should().BeNull(); - (await cache.GetObservabilityToken("agent2", "tenant2")).Should().Be("token2"); - (await cache.GetObservabilityToken("agent3", "tenant3")).Should().BeNull(); - } - - [TestMethod] - public async Task MultipleAgentsTenants_ShouldBeIndependent() - { - var cache = new ServiceTokenCache(); - cache.RegisterObservability("agent1", "tenant1", "token1", TestScopes); - cache.RegisterObservability("agent2", "tenant1", "token2", TestScopes); - cache.RegisterObservability("agent1", "tenant2", "token3", TestScopes); - - (await cache.GetObservabilityToken("agent1", "tenant1")).Should().Be("token1"); - (await cache.GetObservabilityToken("agent2", "tenant1")).Should().Be("token2"); - (await cache.GetObservabilityToken("agent1", "tenant2")).Should().Be("token3"); - } - - [TestMethod] - public async Task ConcurrentAccess_ShouldBeThreadSafe() - { - var cache = new ServiceTokenCache(); - var tasks = new List(); - var tokenCount = 100; - - for (int i = 0; i < tokenCount; i++) - { - var agentId = $"agent-{i}"; - var tenantId = $"tenant-{i}"; - var token = $"token-{i}"; - - tasks.Add(Task.Run(() => cache.RegisterObservability(agentId, tenantId, token, TestScopes))); - } - - Task.WaitAll(tasks.ToArray()); - - for (int i = 0; i < tokenCount; i++) - { - var agentId = $"agent-{i}"; - var tenantId = $"tenant-{i}"; - var expectedToken = $"token-{i}"; - - (await cache.GetObservabilityToken(agentId, tenantId)).Should().Be(expectedToken); - } - } - - [TestMethod] - public void Constructor_WithCustomCleanupInterval_ShouldSucceed() - { - using var cache = new ServiceTokenCache(TimeSpan.FromHours(1), TimeSpan.FromMinutes(10)); - cache.Should().NotBeNull(); - } - - [TestMethod] - public void Constructor_WithDisabledCleanupInterval_ShouldSucceed() - { - using var cache = new ServiceTokenCache(TimeSpan.FromHours(1), TimeSpan.Zero); - cache.Should().NotBeNull(); - } - - [TestMethod] - public void Count_ShouldReturnCorrectNumber() - { - using var cache = new ServiceTokenCache(); - cache.RegisterObservability("agent1", "tenant1", "token1", TestScopes); - cache.RegisterObservability("agent2", "tenant2", "token2", TestScopes); - - cache.Count.Should().Be(2); - } - - [TestMethod] - public void Dispose_ShouldClearAllTokens() - { - var cache = new ServiceTokenCache(); - cache.RegisterObservability(TestAgentId, TestTenantId, TestToken, TestScopes); - - cache.Dispose(); - - cache.Count.Should().Be(0); - } - - [TestMethod] - public void Dispose_CalledMultipleTimes_ShouldNotThrow() - { - var cache = new ServiceTokenCache(); - cache.RegisterObservability(TestAgentId, TestTenantId, TestToken, TestScopes); - - cache.Dispose(); - cache.Dispose(); // Second dispose should not throw - - cache.Count.Should().Be(0); - } - - [TestMethod] - public void DefaultCleanupInterval_ShouldBeFiveMinutes() - { - ServiceTokenCache.DefaultCleanupInterval.Should().Be(TimeSpan.FromMinutes(5)); - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Extensions/ObservabilityServiceCollectionExtensionsTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Extensions/ObservabilityServiceCollectionExtensionsTests.cs deleted file mode 100644 index 6a07b90d..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Extensions/ObservabilityServiceCollectionExtensionsTests.cs +++ /dev/null @@ -1,156 +0,0 @@ -using FluentAssertions; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters; -using Microsoft.Agents.A365.Observability.Hosting.Caching; - -namespace Microsoft.Agents.A365.Observability.Hosting.Tests.Extensions; - -/// -/// Tests for Agent 365 extension methods. -/// -[TestClass] -public sealed class ObservabilityServiceCollectionExtensionsTests -{ - [TestMethod] - public void AddAgenticTracingExporter_RegistersRequiredServices() - { - // Arrange - var services = new ServiceCollection(); - - // Act - services.AddAgenticTracingExporter(); - var serviceProvider = services.BuildServiceProvider(); - - // Assert - var tokenCache = serviceProvider.GetService>(); - tokenCache.Should().NotBeNull(); - tokenCache.Should().BeOfType(); - - serviceProvider.GetService>() - .Should().BeNull("the delegated AgenticTokenStruct cache contract was removed"); - - var options = serviceProvider.GetService(); - options.Should().NotBeNull(); - options!.TokenResolver.Should().NotBeNull(); - } - - [TestMethod] - public void AddAgenticTracingExporter_DefaultClusterCategory_IsProduction() - { - // Arrange - var services = new ServiceCollection(); - - // Act - services.AddAgenticTracingExporter(); - var serviceProvider = services.BuildServiceProvider(); - - // Assert - var options = serviceProvider.GetRequiredService(); - options.ClusterCategory.Should().Be("production"); - } - - [TestMethod] - public void AddAgenticTracingExporter_UsesIgnoredLegacyEndpointFlag() - { - // Arrange - var services = new ServiceCollection(); - - // Act - services.AddAgenticTracingExporter(); - var serviceProvider = services.BuildServiceProvider(); - - // Assert - var options = serviceProvider.GetRequiredService(); -#pragma warning disable CS0618 - options.UseS2SEndpoint.Should().BeFalse("the compatibility property default is preserved but ignored"); -#pragma warning restore CS0618 - } - - [TestMethod] - public void AddServiceTracingExporter_RegistersRequiredServices() - { - // Arrange - var services = new ServiceCollection(); - - // Act - services.AddServiceTracingExporter(); - var serviceProvider = services.BuildServiceProvider(); - - // Assert - var tokenCache = serviceProvider.GetService>(); - tokenCache.Should().NotBeNull(); - tokenCache.Should().BeOfType(); - - var options = serviceProvider.GetService(); - options.Should().NotBeNull(); - options!.TokenResolver.Should().NotBeNull(); - } - - [TestMethod] - public void AddServiceTracingExporter_DefaultClusterCategory_IsProduction() - { - // Arrange - var services = new ServiceCollection(); - - // Act - services.AddServiceTracingExporter(); - var serviceProvider = services.BuildServiceProvider(); - - // Assert - var options = serviceProvider.GetRequiredService(); - options.ClusterCategory.Should().Be("production"); - } - - [TestMethod] - public void AddServiceTracingExporter_UsesIgnoredLegacyEndpointFlag() - { - // Arrange - var services = new ServiceCollection(); - - // Act - services.AddServiceTracingExporter(); - var serviceProvider = services.BuildServiceProvider(); - - // Assert - var options = serviceProvider.GetRequiredService(); -#pragma warning disable CS0618 - options.UseS2SEndpoint.Should().BeFalse("OBS export always routes to S2S regardless of this compatibility property"); -#pragma warning restore CS0618 - } - - [TestMethod] - public void AddServiceTracingExporter_TokenResolver_CanBeCalled() - { - // Arrange - var services = new ServiceCollection(); - services.AddServiceTracingExporter(); - var serviceProvider = services.BuildServiceProvider(); - var options = serviceProvider.GetRequiredService(); - - // Act - var token = options.TokenResolver!("test-agent", "test-tenant"); - - // Assert - // Token resolver should not throw (actual token retrieval logic is in the cache) - // This just verifies the resolver is wired up - options.TokenResolver.Should().NotBeNull(); - } - - [TestMethod] - public void AddAgenticTracingExporter_TokenResolver_CanBeCalled() - { - // Arrange - var services = new ServiceCollection(); - services.AddAgenticTracingExporter(); - var serviceProvider = services.BuildServiceProvider(); - var options = serviceProvider.GetRequiredService(); - - // Act - var token = options.TokenResolver!("test-agent", "test-tenant"); - - // Assert - // Token resolver should not throw (actual token retrieval logic is in the cache) - // This just verifies the resolver is wired up - options.TokenResolver.Should().NotBeNull(); - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Microsoft.Agents.A365.Observability.Hosting.Tests.csproj b/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Microsoft.Agents.A365.Observability.Hosting.Tests.csproj deleted file mode 100644 index a8168905..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Microsoft.Agents.A365.Observability.Hosting.Tests.csproj +++ /dev/null @@ -1,26 +0,0 @@ - - - net8.0 - false - false - true - - - - - - - - - - - - - - - - - - - - diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Middleware/BaggageTurnMiddlewareTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Middleware/BaggageTurnMiddlewareTests.cs deleted file mode 100644 index e5cde121..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Middleware/BaggageTurnMiddlewareTests.cs +++ /dev/null @@ -1,143 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Hosting.Middleware; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Agents.Builder; -using Microsoft.Agents.Core.Models; -using Moq; -using OpenTelemetry; - -namespace Microsoft.Agents.A365.Observability.Hosting.Tests.Middleware; - -[TestClass] -public class BaggageTurnMiddlewareTests -{ - [TestMethod] - public async Task OnTurnAsync_SetsOpenTelemetryBaggage() - { - // Arrange - var middleware = new BaggageTurnMiddleware(); - var turnContext = CreateTurnContext(); - - string? capturedTenantId = null; - string? capturedCallerId = null; - - NextDelegate next = (ct) => - { - capturedTenantId = Baggage.Current.GetBaggage(OpenTelemetryConstants.TenantIdKey); - capturedCallerId = Baggage.Current.GetBaggage(OpenTelemetryConstants.UserIdKey); - return Task.CompletedTask; - }; - - // Act - await middleware.OnTurnAsync(turnContext, next); - - // Assert - capturedTenantId.Should().Be("tenant-123"); - capturedCallerId.Should().Be("caller-aad"); - } - - [TestMethod] - public async Task OnTurnAsync_SkipsBaggageForContinueConversation() - { - // Arrange - var middleware = new BaggageTurnMiddleware(); - var turnContext = CreateTurnContext( - activityType: ActivityTypes.Event, - activityName: ActivityEventNames.ContinueConversation); - - bool logicCalled = false; - string? capturedCallerId = null; - - NextDelegate next = (ct) => - { - logicCalled = true; - capturedCallerId = Baggage.Current.GetBaggage(OpenTelemetryConstants.UserIdKey); - return Task.CompletedTask; - }; - - // Act - await middleware.OnTurnAsync(turnContext, next); - - // Assert - logicCalled.Should().BeTrue(); - // Baggage should NOT be set because the middleware skipped it - capturedCallerId.Should().BeNull(); - } - - [TestMethod] - public async Task OnTurnAsync_CallsNextDelegate() - { - // Arrange - var middleware = new BaggageTurnMiddleware(); - var turnContext = CreateTurnContext(); - - bool nextCalled = false; - NextDelegate next = (ct) => - { - nextCalled = true; - return Task.CompletedTask; - }; - - // Act - await middleware.OnTurnAsync(turnContext, next); - - // Assert - nextCalled.Should().BeTrue(); - } - - [TestMethod] - public async Task OnTurnAsync_RestoresBaggageAfterNext() - { - // Arrange - var middleware = new BaggageTurnMiddleware(); - var turnContext = CreateTurnContext(); - - string? baggageBeforeMiddleware = Baggage.Current.GetBaggage(OpenTelemetryConstants.TenantIdKey); - - NextDelegate next = (ct) => Task.CompletedTask; - - // Act - await middleware.OnTurnAsync(turnContext, next); - - // Assert – the baggage scope should be disposed after OnTurnAsync returns - string? baggageAfterMiddleware = Baggage.Current.GetBaggage(OpenTelemetryConstants.TenantIdKey); - baggageAfterMiddleware.Should().Be(baggageBeforeMiddleware); - } - - private static ITurnContext CreateTurnContext( - string activityType = "message", - string? activityName = null) - { - var mockActivity = new Mock(); - mockActivity.Setup(a => a.Type).Returns(activityType); - if (activityName != null) - { - mockActivity.Setup(a => a.Name).Returns(activityName); - } - mockActivity.Setup(a => a.Text).Returns("Hello"); - mockActivity.Setup(a => a.From).Returns(new ChannelAccount - { - Id = "caller-id", - Name = "Caller", - AadObjectId = "caller-aad", - }); - mockActivity.Setup(a => a.Recipient).Returns(new ChannelAccount - { - Id = "agent-id", - Name = "Agent", - TenantId = "tenant-123", - Role = "user", - }); - mockActivity.Setup(a => a.Conversation).Returns(new ConversationAccount { Id = "conv-id" }); - mockActivity.Setup(a => a.ServiceUrl).Returns("https://example.com"); - mockActivity.Setup(a => a.ChannelId).Returns(new ChannelId("test-channel")); - - var mockTurnContext = new Mock(); - mockTurnContext.Setup(tc => tc.Activity).Returns(mockActivity.Object); - - return mockTurnContext.Object; - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Middleware/ObservabilityBaggageMiddlewareTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Middleware/ObservabilityBaggageMiddlewareTests.cs deleted file mode 100644 index d73ca91a..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Middleware/ObservabilityBaggageMiddlewareTests.cs +++ /dev/null @@ -1,50 +0,0 @@ -using Microsoft.AspNetCore.Builder; -using Microsoft.AspNetCore.Hosting; -using Microsoft.AspNetCore.Http; -using Microsoft.AspNetCore.TestHost; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Agents.A365.Observability.Hosting.Middleware; -using System.Text.Json; - -namespace Microsoft.Agents.A365.Observability.Hosting.Tests.Middleware; - -[TestClass] -public class ObservabilityBaggageMiddlewareTests -{ - [TestMethod] - public async Task Middleware_UsesResolverValues() - { - // Arrange - var builder = new WebHostBuilder() - .ConfigureServices(services => - { - services.AddRouting(); - }) - .Configure(app => - { - app.UseObservabilityRequestContext(ctx => ("tenant-abc", "agent-xyz")); - app.UseRouting(); - app.UseEndpoints(endpoints => - { - endpoints.MapGet("/", (HttpContext ctx) => - { - var (tenant, agent) = ("tenant-abc", "agent-xyz"); - return Results.Json(new { tenant, agent }); - }); - }); - }); - - using var server = new TestServer(builder); - var client = server.CreateClient(); - - // Act - var response = await client.GetAsync("/"); - var json = await response.Content.ReadAsStringAsync(); - - // Assert - using var doc = JsonDocument.Parse(json); - var root = doc.RootElement; - Assert.AreEqual("tenant-abc", root.GetProperty("tenant").GetString()); - Assert.AreEqual("agent-xyz", root.GetProperty("agent").GetString()); - } -} \ No newline at end of file diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Middleware/OutputLoggingMiddlewareTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Middleware/OutputLoggingMiddlewareTests.cs deleted file mode 100644 index bfe66328..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/Middleware/OutputLoggingMiddlewareTests.cs +++ /dev/null @@ -1,146 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Hosting.Middleware; -using Microsoft.Agents.Builder; -using Microsoft.Agents.Core.Models; -using Moq; - -namespace Microsoft.Agents.A365.Observability.Hosting.Tests.Middleware; - -[TestClass] -public class OutputLoggingMiddlewareTests -{ - [TestMethod] - public async Task OnTurnAsync_CallsNextDelegate() - { - // Arrange - var middleware = new OutputLoggingMiddleware(); - var turnContext = CreateTurnContext(); - - bool nextCalled = false; - NextDelegate next = (ct) => - { - nextCalled = true; - return Task.CompletedTask; - }; - - // Act - await middleware.OnTurnAsync(turnContext, next); - - // Assert - nextCalled.Should().BeTrue(); - } - - [TestMethod] - public async Task OnTurnAsync_RegistersSendHandler_WhenRecipientHasDetails() - { - // Arrange - var middleware = new OutputLoggingMiddleware(); - var mockTurnContext = new Mock(); - SetupTurnContext(mockTurnContext); - - NextDelegate next = (ct) => Task.CompletedTask; - - // Act - await middleware.OnTurnAsync(mockTurnContext.Object, next); - - // Assert - mockTurnContext.Verify(tc => tc.OnSendActivities(It.IsAny()), Times.Once); - } - - [TestMethod] - public async Task OnTurnAsync_PassesThrough_WhenRecipientIsNull() - { - // Arrange - var middleware = new OutputLoggingMiddleware(); - var mockActivity = new Mock(); - mockActivity.Setup(a => a.Recipient).Returns((ChannelAccount)null!); - mockActivity.Setup(a => a.Type).Returns("message"); - - var mockTurnContext = new Mock(); - mockTurnContext.Setup(tc => tc.Activity).Returns(mockActivity.Object); - - bool nextCalled = false; - NextDelegate next = (ct) => - { - nextCalled = true; - return Task.CompletedTask; - }; - - // Act - await middleware.OnTurnAsync(mockTurnContext.Object, next); - - // Assert - nextCalled.Should().BeTrue(); - mockTurnContext.Verify(tc => tc.OnSendActivities(It.IsAny()), Times.Never); - } - - [TestMethod] - public async Task OnTurnAsync_RegistersHandler_WhenTenantIdIsMissing() - { - // Arrange - var middleware = new OutputLoggingMiddleware(); - var mockActivity = new Mock(); - mockActivity.Setup(a => a.Type).Returns("message"); - mockActivity.Setup(a => a.Recipient).Returns(new ChannelAccount - { - Id = "agent-id", - Name = "Agent", - // No TenantId set - middleware no longer gates on TenantId - }); - - var mockTurnContext = new Mock(); - mockTurnContext.Setup(tc => tc.Activity).Returns(mockActivity.Object); - - bool nextCalled = false; - NextDelegate next = (ct) => - { - nextCalled = true; - return Task.CompletedTask; - }; - - // Act - await middleware.OnTurnAsync(mockTurnContext.Object, next); - - // Assert - nextCalled.Should().BeTrue(); - mockTurnContext.Verify(tc => tc.OnSendActivities(It.IsAny()), Times.Once); - } - - private static ITurnContext CreateTurnContext() - { - var mockTurnContext = new Mock(); - SetupTurnContext(mockTurnContext); - return mockTurnContext.Object; - } - - private static void SetupTurnContext(Mock mockTurnContext) - { - var mockActivity = new Mock(); - mockActivity.Setup(a => a.Type).Returns("message"); - mockActivity.Setup(a => a.Text).Returns("Hello"); - mockActivity.Setup(a => a.From).Returns(new ChannelAccount - { - Id = "caller-id", - Name = "Caller", - AadObjectId = "caller-aad", - }); - mockActivity.Setup(a => a.Recipient).Returns(new ChannelAccount - { - Id = "agent-id", - Name = "Agent", - TenantId = "badf1f56-284d-4dc5-ac59-0dd53900e743", - Role = "agenticAppInstance", - }); - mockActivity.Setup(a => a.Conversation).Returns(new ConversationAccount { Id = "conv-id" }); - mockActivity.Setup(a => a.ServiceUrl).Returns("https://example.com"); - mockActivity.Setup(a => a.ChannelId).Returns(new ChannelId("test-channel")); - - mockTurnContext.Setup(tc => tc.Activity).Returns(mockActivity.Object); - mockTurnContext.Setup(tc => tc.StackState).Returns(new TurnContextStateCollection()); - mockTurnContext.Setup(tc => tc.OnSendActivities(It.IsAny())) - .Returns(mockTurnContext.Object); - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/ObservabilityBuilderExtensionsTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/ObservabilityBuilderExtensionsTests.cs deleted file mode 100644 index 9495d4f3..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Hosting.Tests/ObservabilityBuilderExtensionsTests.cs +++ /dev/null @@ -1,189 +0,0 @@ -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters; -using Microsoft.AspNetCore.Builder; -using Microsoft.AspNetCore.Hosting; -using Microsoft.Extensions.Configuration; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.Hosting; -using OpenTelemetry.Trace; - -#pragma warning disable CS0618 // Tests intentionally pass the ignored legacy S2S compatibility switch. - -namespace Microsoft.Agents.A365.Observability.Hosting.Tests -{ - [TestClass] - public class ObservabilityBuilderExtensionsTests - { - private class MinimalStartup - { - public void Configure(IApplicationBuilder app) - { - // no-op - } - } - - private class MarkerService { } - - [TestMethod] - public void AddA365Tracing_InvokesConfigureDelegate_RegistersCustomService() - { - var webHostBuilder = new WebHostBuilder(); - - webHostBuilder.AddA365Tracing(configure: builder => - { - builder.Services.AddSingleton(); - }); - webHostBuilder.UseStartup(); - - using var host = webHostBuilder.Build(); - var service = host.Services.GetService(); - service.Should().NotBeNull("custom services registered via the configure delegate should be available"); - } - - [TestMethod] - public void AddA365Tracing_WithOpenTelemetryBuilderTrue_BuildsSuccessfully() - { - var webHostBuilder = new WebHostBuilder(); - webHostBuilder.AddA365Tracing(useOpenTelemetryBuilder: true); - webHostBuilder.UseStartup(); - - using var host = webHostBuilder.Build(); - host.Should().NotBeNull(); - } - - [TestMethod] - public void AddA365Tracing_WithOpenTelemetryBuilderTrue_RegistersOpenTelemetryServices() - { - var webHostBuilder = new WebHostBuilder(); - webHostBuilder.AddA365Tracing(useOpenTelemetryBuilder: true); - webHostBuilder.UseStartup(); - - using var host = webHostBuilder.Build(); - var tracerProvider = host.Services.GetService(); - tracerProvider.Should().NotBeNull("OpenTelemetry tracing services should be registered when using the OpenTelemetry builder"); - } - - [TestMethod] - public void AddA365Tracing_WithOpenTelemetryBuilderTrue_AndExporterEnabled_RegistersTracerProvider() - { - // Enable exporter via configuration - Environment.SetEnvironmentVariable("EnableAgent365Exporter", "true"); - - var webHostBuilder = new WebHostBuilder(); - - // Ensure environment variables are part of configuration - var configuration = new ConfigurationBuilder() - .AddEnvironmentVariables() - .Build(); - webHostBuilder.UseConfiguration(configuration); - - // Provide dependencies needed by exporter - webHostBuilder.AddA365Tracing(useOpenTelemetryBuilder: true, agent365ExporterType: Agent365ExporterType.Agent365Exporter, - configure: builder => - { - builder.Services.AddSingleton(sp => new HttpClient()); - builder.Services.AddSingleton(sp => new Agent365ExporterOptions - { - UseS2SEndpoint = false, - TokenResolver = (_, _) => Task.FromResult("test-token") - }); - }); - webHostBuilder.UseStartup(); - - using var host = webHostBuilder.Build(); - var tracerProvider = host.Services.GetService(); - tracerProvider.Should().NotBeNull("TracerProvider should be registered when exporter is enabled via OpenTelemetry builder"); - } - - [TestMethod] - public void AddA365Tracing_WithOpenTelemetryBuilderFalse_BuildsSuccessfully() - { - var webHostBuilder = new WebHostBuilder(); - webHostBuilder.AddA365Tracing(useOpenTelemetryBuilder: false); - webHostBuilder.UseStartup(); - - using var host = webHostBuilder.Build(); - host.Should().NotBeNull(); - } - - [TestMethod] - public void AddA365Tracing_IHostBuilder_InvokesConfigureDelegate_RegistersCustomService() - { - var hostBuilder = new HostBuilder(); - - hostBuilder.AddA365Tracing(configure: b => - { - b.Services.AddSingleton(); - }); - - using var host = hostBuilder.Build(); - var service = host.Services.GetService(); - service.Should().NotBeNull("custom services registered via the configure delegate should be available"); - } - - [TestMethod] - public void AddA365Tracing_IHostBuilder_WithOpenTelemetryBuilderTrue_BuildsSuccessfully() - { - var hostBuilder = new HostBuilder(); - hostBuilder.AddA365Tracing(useOpenTelemetryBuilder: true); - - using var host = hostBuilder.Build(); - host.Should().NotBeNull(); - } - - [TestMethod] - public void AddA365Tracing_IHostBuilder_WithOpenTelemetryBuilderTrue_RegistersOpenTelemetryServices() - { - var hostBuilder = new HostBuilder(); - hostBuilder.AddA365Tracing(useOpenTelemetryBuilder: true); - - using var host = hostBuilder.Build(); - var tracerProvider = host.Services.GetService(); - tracerProvider.Should().NotBeNull("OpenTelemetry tracing services should be registered when using the OpenTelemetry builder"); - } - - [TestMethod] - public void AddA365Tracing_IHostBuilder_WithOpenTelemetryBuilderTrue_AndExporterEnabled_RegistersTracerProvider() - { - try - { - Environment.SetEnvironmentVariable("EnableAgent365Exporter", "true"); - - var hostBuilder = new HostBuilder(); - hostBuilder.ConfigureHostConfiguration(cfg => - { - cfg.AddEnvironmentVariables(); - }); - - hostBuilder.AddA365Tracing(useOpenTelemetryBuilder: true, agent365ExporterType: Agent365ExporterType.Agent365Exporter, - configure: builder => - { - builder.Services.AddSingleton(_ => new HttpClient()); - builder.Services.AddSingleton(_ => new Agent365ExporterOptions - { - UseS2SEndpoint = false, - TokenResolver = (_, _) => Task.FromResult("test-token") - }); - }); - - using var host = hostBuilder.Build(); - var tracerProvider = host.Services.GetService(); - tracerProvider.Should().NotBeNull("TracerProvider should be registered when exporter is enabled via OpenTelemetry builder"); - } - finally - { - Environment.SetEnvironmentVariable("EnableAgent365Exporter", null); - } - } - - [TestMethod] - public void AddA365Tracing_IHostBuilder_WithOpenTelemetryBuilderFalse_BuildsSuccessfully() - { - var hostBuilder = new HostBuilder(); - hostBuilder.AddA365Tracing(useOpenTelemetryBuilder: false); - - using var host = hostBuilder.Build(); - host.Should().NotBeNull(); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.IntegrationTests/Agent365ExporterAsyncE2ETests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.IntegrationTests/Agent365ExporterAsyncE2ETests.cs deleted file mode 100644 index 9cc245b8..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.IntegrationTests/Agent365ExporterAsyncE2ETests.cs +++ /dev/null @@ -1,417 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.Hosting; -using System.Net; -using System.Text.Json; - -#pragma warning disable CS0618 // Tests intentionally pass the ignored legacy S2S compatibility switch. - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.IntegrationTests -{ - [TestClass] - public class Agent365ExporterAsyncE2ETests - { - private TestHttpMessageHandler? _handler; - private ServiceProvider? _provider; - private bool _receivedRequest; - private string? _receivedContent; - - [TestMethod] - public async Task AddTracing_And_InvokeAgentScope_ExporterMakesExpectedRequest() - { - // Arrange - this.SetupExporterTest(); - this._receivedRequest = false; - this._receivedContent = null; - var expectedAgentType = AgentType.EntraEmbodied; - var expectedAgentDetails = new AgentDetails( - agentId: Guid.NewGuid().ToString(), - agentName: "Test Agent", - agentDescription: "Agent for testing.", - agenticUserId: Guid.NewGuid().ToString(), - agenticUserEmail: "testagent@ztaitest12.onmicrosoft.com", - agentBlueprintId: Guid.NewGuid().ToString(), - tenantId: Guid.NewGuid().ToString(), - agentType: expectedAgentType); - var endpoint = new Uri("https://test-agent-endpoint"); - var invokeAgentScopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - - var expectedRequest = new Request( - content: "Test request content", - channel: new Channel( - name: "msteams", - link: "https://testchannel.link")); - - var expectedCallerDetails = new CallerDetails( - userDetails: new UserDetails( - userId: "caller-123", - userName: "Test Caller", - userEmail: "caller-123@ztaitest12.onmicrosoft.com", - userClientIP: IPAddress.Parse("203.0.113.42")), - callerAgentDetails: null); - - // Act - using (var scope = InvokeAgentScope.Start( - request: expectedRequest, - scopeDetails: invokeAgentScopeDetails, - agentDetails: expectedAgentDetails, - callerDetails: expectedCallerDetails)) - { - scope.RecordInputMessages(new[] { "Input message 1", "Input message 2" }); - scope.RecordOutputMessages(new[] { "Output message 1" }); - } - - var timeout = TimeSpan.FromSeconds(30); - var start = DateTime.UtcNow; - while (!this._receivedRequest && DateTime.UtcNow - start < timeout) - { - await Task.Delay(1000).ConfigureAwait(false); - } - - this._receivedRequest.Should().BeTrue("Exporter should make the expected HTTP request."); - this._receivedContent.Should().NotBeNull("Exporter should send a request body."); - - using var doc = JsonDocument.Parse(this._receivedContent!); - var root = doc.RootElement; - - var attributes = root - .GetProperty("resourceSpans")[0] - .GetProperty("scopeSpans")[0] - .GetProperty("spans")[0] - .GetProperty("attributes"); - this.GetAttribute(attributes, "server.address").Should().Be(invokeAgentScopeDetails.Endpoint?.Host); - this.GetAttribute(attributes, "microsoft.channel.name").Should().Be(expectedRequest.Channel?.Name); - this.GetAttribute(attributes, "microsoft.channel.link").Should().Be(expectedRequest.Channel?.Link); - this.GetAttribute(attributes, "microsoft.tenant.id").Should().Be(expectedAgentDetails.TenantId); - this.GetAttribute(attributes, "user.id").Should().Be(expectedCallerDetails.UserDetails?.UserId); - this.GetAttribute(attributes, "user.email").Should().Be(expectedCallerDetails.UserDetails?.UserEmail); - this.GetAttribute(attributes, "user.name").Should().Be(expectedCallerDetails.UserDetails?.UserName); - this.GetAttribute(attributes, "gen_ai.input.messages").Should().Contain("Input message 1").And.Contain("Input message 2"); - this.GetAttribute(attributes, "gen_ai.output.messages").Should().Contain("Output message 1"); - this.GetAttribute(attributes, "gen_ai.agent.id").Should().Be(expectedAgentDetails.AgentId); - this.GetAttribute(attributes, "gen_ai.agent.name").Should().Be(expectedAgentDetails.AgentName); - this.GetAttribute(attributes, "gen_ai.agent.description").Should().Be(expectedAgentDetails.AgentDescription); - this.GetAttribute(attributes, "microsoft.agent.user.id").Should().Be(expectedAgentDetails.AgenticUserId); - this.GetAttribute(attributes, "microsoft.agent.user.email").Should().Be(expectedAgentDetails.AgenticUserEmail); - this.GetAttribute(attributes, "microsoft.a365.agent.blueprint.id").Should().Be(expectedAgentDetails.AgentBlueprintId); - this.GetAttribute(attributes, "microsoft.tenant.id").Should().Be(expectedAgentDetails.TenantId); - this.GetAttribute(attributes, "gen_ai.operation.name").Should().Be("invoke_agent"); - } - - [TestMethod] - public async Task AddTracing_And_ExecuteToolScope_ExporterMakesExpectedRequest() - { - // Arrange - this.SetupExporterTest(); - this._receivedRequest = false; - this._receivedContent = null; - var expectedAgentType = AgentType.EntraEmbodied; - var expectedAgentDetails = new AgentDetails( - agentId: Guid.NewGuid().ToString(), - agentName: "Tool Agent", - agentDescription: "Agent for tool execution.", - agenticUserId: Guid.NewGuid().ToString(), - agenticUserEmail: "toolagent@ztaitest12.onmicrosoft.com", - agentBlueprintId: Guid.NewGuid().ToString(), - tenantId: Guid.NewGuid().ToString(), - agentType: expectedAgentType); - var endpoint = new Uri("https://tool-endpoint:8443"); - var toolCallDetails = new ToolCallDetails( - toolName: "TestTool", - arguments: "{\"param\":\"value\"}", - toolCallId: "call-456", - description: "Test tool call description", - toolType: "custom-type", - endpoint: endpoint); - - var expectedRequest = new Request(); - - // Act - using (var scope = ExecuteToolScope.Start(expectedRequest, toolCallDetails, expectedAgentDetails)) - { - scope.RecordResponse("Tool response content"); - } - - var timeout = TimeSpan.FromSeconds(30); - var start = DateTime.UtcNow; - while (!this._receivedRequest && DateTime.UtcNow - start < timeout) - { - await Task.Delay(1000).ConfigureAwait(false); - } - - this._receivedRequest.Should().BeTrue("Exporter should make the expected HTTP request."); - this._receivedContent.Should().NotBeNull("Exporter should send a request body."); - - using var doc = JsonDocument.Parse(this._receivedContent!); - var root = doc.RootElement; - - var attributes = root - .GetProperty("resourceSpans")[0] - .GetProperty("scopeSpans")[0] - .GetProperty("spans")[0] - .GetProperty("attributes"); - - this.GetAttribute(attributes, "gen_ai.operation.name").Should().Be("execute_tool"); - this.GetAttribute(attributes, "gen_ai.agent.id").Should().Be(expectedAgentDetails.AgentId); - this.GetAttribute(attributes, "gen_ai.agent.name").Should().Be(expectedAgentDetails.AgentName); - this.GetAttribute(attributes, "gen_ai.agent.description").Should().Be(expectedAgentDetails.AgentDescription); - this.GetAttribute(attributes, "microsoft.agent.user.id").Should().Be(expectedAgentDetails.AgenticUserId); - this.GetAttribute(attributes, "microsoft.agent.user.email").Should().Be(expectedAgentDetails.AgenticUserEmail); - this.GetAttribute(attributes, "microsoft.a365.agent.blueprint.id").Should().Be(expectedAgentDetails.AgentBlueprintId); - this.GetAttribute(attributes, "microsoft.tenant.id").Should().Be(expectedAgentDetails.TenantId); - this.GetAttribute(attributes, "gen_ai.tool.name").Should().Be(toolCallDetails.ToolName); - this.GetAttribute(attributes, "gen_ai.tool.call.arguments").Should().Be(toolCallDetails.Arguments); - this.GetAttribute(attributes, "gen_ai.tool.call.id").Should().Be(toolCallDetails.ToolCallId); - this.GetAttribute(attributes, "gen_ai.tool.description").Should().Be(toolCallDetails.Description); - this.GetAttribute(attributes, "gen_ai.tool.type").Should().Be(toolCallDetails.ToolType); - this.GetAttribute(attributes, "server.address").Should().Be(endpoint.Host); - this.GetAttribute(attributes, "server.port").Should().Be(endpoint.Port.ToString()); - this.GetAttribute(attributes, "gen_ai.tool.call.result").Should().Contain("Tool response content"); - } - - [TestMethod] - public async Task AddTracing_And_InferenceScope_ExporterMakesExpectedRequest() - { - // Arrange - this.SetupExporterTest(); - this._receivedRequest = false; - this._receivedContent = null; - var expectedAgentType = AgentType.EntraEmbodied; - var expectedAgentDetails = new AgentDetails( - agentId: Guid.NewGuid().ToString(), - agentName: "Inference Agent", - agentDescription: "Agent for inference testing.", - agenticUserId: Guid.NewGuid().ToString(), - agenticUserEmail: "inferenceagent@ztaitest12.onmicrosoft.com", - agentBlueprintId: Guid.NewGuid().ToString(), - tenantId: Guid.NewGuid().ToString(), - agentType: expectedAgentType); - - var inferenceDetails = new InferenceCallDetails( - operationName: InferenceOperationType.Chat, - model: "gpt-4", - providerName: "OpenAI", - inputTokens: 42, - outputTokens: 84, - finishReasons: new[] { "stop", "length" }, - responseId: "response-xyz"); - - var expectedRequest = new Request(); - - // Act - using (var scope = InferenceScope.Start(expectedRequest, inferenceDetails, expectedAgentDetails)) - { - scope.RecordInputMessages(new[] { "Hello", "World" }); - scope.RecordOutputMessages(new[] { "Hi there!" }); - scope.RecordInputTokens(42); - scope.RecordOutputTokens(84); - scope.RecordFinishReasons(new[] { "stop", "length" }); - } - - var timeout = TimeSpan.FromSeconds(30); - var start = DateTime.UtcNow; - while (!this._receivedRequest && DateTime.UtcNow - start < timeout) - { - await Task.Delay(1000).ConfigureAwait(false); - } - - this._receivedRequest.Should().BeTrue("Exporter should make the expected HTTP request."); - this._receivedContent.Should().NotBeNull("Exporter should send a request body."); - - using var doc = JsonDocument.Parse(this._receivedContent!); - var root = doc.RootElement; - var attributes = root - .GetProperty("resourceSpans")[0] - .GetProperty("scopeSpans")[0] - .GetProperty("spans")[0] - .GetProperty("attributes"); - - this.GetAttribute(attributes, "gen_ai.operation.name").Should().Be(inferenceDetails.OperationName.ToString()); - this.GetAttribute(attributes, "gen_ai.agent.id").Should().Be(expectedAgentDetails.AgentId); - this.GetAttribute(attributes, "gen_ai.agent.name").Should().Be(expectedAgentDetails.AgentName); - this.GetAttribute(attributes, "gen_ai.agent.description").Should().Be(expectedAgentDetails.AgentDescription); - this.GetAttribute(attributes, "microsoft.agent.user.id").Should().Be(expectedAgentDetails.AgenticUserId); - this.GetAttribute(attributes, "microsoft.agent.user.email").Should().Be(expectedAgentDetails.AgenticUserEmail); - this.GetAttribute(attributes, "microsoft.a365.agent.blueprint.id").Should().Be(expectedAgentDetails.AgentBlueprintId); - this.GetAttribute(attributes, "microsoft.tenant.id").Should().Be(expectedAgentDetails.TenantId); - this.GetAttribute(attributes, "gen_ai.request.model").Should().Be(inferenceDetails.Model); - this.GetAttribute(attributes, "gen_ai.provider.name").Should().Be(inferenceDetails.ProviderName); - this.GetAttribute(attributes, "gen_ai.usage.input_tokens").Should().Be("42"); - this.GetAttribute(attributes, "gen_ai.usage.output_tokens").Should().Be("84"); - this.GetAttribute(attributes, "gen_ai.response.finish_reasons").Should().Be("stop,length"); - this.GetAttribute(attributes, "gen_ai.input.messages").Should().Contain("Hello").And.Contain("World"); - this.GetAttribute(attributes, "gen_ai.output.messages").Should().Contain("Hi there!"); - } - - [TestMethod] - public async Task AddTracing_NestedScopes_AllExporterRequestsReceived() - { - // Arrange - List receivedContents = new(); - - var agentType = AgentType.EntraEmbodied; - var agentDetails = new AgentDetails( - agentId: Guid.NewGuid().ToString(), - agentName: "Nested Agent", - agentDescription: "Agent for nested scope testing.", - agenticUserId: Guid.NewGuid().ToString(), - agenticUserEmail: "nestedagent@ztaitest12.onmicrosoft.com", - agentBlueprintId: Guid.NewGuid().ToString(), - tenantId: Guid.NewGuid().ToString(), - agentType: agentType); - - var endpoint = new Uri("https://nested-endpoint"); - - var handler = new TestHttpMessageHandler(req => - { - receivedContents.Add(req.Content?.ReadAsStringAsync().GetAwaiter().GetResult() ?? ""); - return new HttpResponseMessage(System.Net.HttpStatusCode.OK); - }); - var httpClient = new HttpClient(handler); - - this.CreateTestServiceProvider(httpClient); - - var invokeAgentScopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var request = new Request( - content: "Nested request", - channel: new Channel(name: "nested", link: "https://nestedchannel.link")); - - var toolCallDetails = new ToolCallDetails( - toolName: "NestedTool", - arguments: "{\"param\":\"nested\"}", - toolCallId: "call-nested", - description: "Nested tool call", - toolType: "nested-type", - endpoint: endpoint); - - var inferenceDetails = new InferenceCallDetails( - operationName: InferenceOperationType.Chat, - model: "gpt-nested", - providerName: "OpenAI", - inputTokens: 10, - outputTokens: 20, - finishReasons: new[] { "stop" }, - responseId: "response-nested"); - - // Act - using (var agentScope = InvokeAgentScope.Start(request, invokeAgentScopeDetails, agentDetails)) - { - agentScope.RecordInputMessages(new[] { "Agent input" }); - agentScope.RecordOutputMessages(new[] { "Agent output" }); - - using (var toolScope = ExecuteToolScope.Start(request, toolCallDetails, agentDetails)) - { - toolScope.RecordResponse("Tool response"); - - using (var inferenceScope = InferenceScope.Start(request, inferenceDetails, agentDetails)) - { - inferenceScope.RecordInputMessages(new[] { "Inference input" }); - inferenceScope.RecordOutputMessages(new[] { "Inference output" }); - inferenceScope.RecordInputTokens(10); - inferenceScope.RecordOutputTokens(20); - inferenceScope.RecordFinishReasons(new[] { "stop" }); - } - } - } - - // Wait for up to 5 seconds for all spans to be exported - await Task.Delay(5000).ConfigureAwait(false); - - // Assert - var allOperationNames = new List(); - foreach (var content in receivedContents) - { - using var doc = JsonDocument.Parse(content); - var root = doc.RootElement; - var spans = root - .GetProperty("resourceSpans")[0] - .GetProperty("scopeSpans")[0] - .GetProperty("spans") - .EnumerateArray(); - - foreach (var span in spans) - { - var opName = this.GetAttribute(span.GetProperty("attributes"), "gen_ai.operation.name"); - if (opName != null) - allOperationNames.Add(opName); - } - } - allOperationNames.Should().Contain(new[] { "invoke_agent", "execute_tool", InferenceOperationType.Chat.ToString() }, "All three nested scopes should be exported, even if batched in fewer requests."); - } - - private class TestHttpMessageHandler : HttpMessageHandler - { - private Func _handler; - public TestHttpMessageHandler(Func handler) - { - this._handler = handler; - } - public void SetHandler(Func handler) - { - this._handler = handler; - } - protected override Task SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) - { - return Task.FromResult(this._handler(request)); - } - } - - private string? GetAttribute(JsonElement attributes, string key) - { - if (attributes.TryGetProperty(key, out var value)) - { - if (value.ValueKind == JsonValueKind.String) - { - return value.GetString(); - } - if (value.ValueKind == JsonValueKind.Number) - { - return value.GetRawText(); - } - if (value.ValueKind == JsonValueKind.Object && value.TryGetProperty("stringValue", out var sv)) - { - return sv.GetString(); - } - } - return null; - } - - private ServiceProvider CreateTestServiceProvider(HttpClient httpClient) - { - HostApplicationBuilder builder = new HostApplicationBuilder(); - - builder.Configuration["EnableAgent365Exporter"] = "true"; - builder.Services.AddSingleton(httpClient); - builder.Services.AddSingleton(sp => - { - return new Agent365ExporterOptions - { - UseS2SEndpoint = false, - TokenResolver = (_, _) => Task.FromResult("test-token") - }; - }); - - builder.AddA365Tracing(useOpenTelemetryBuilder: false, agent365ExporterType: Agent365ExporterType.Agent365ExporterAsync); - return builder.Services.BuildServiceProvider(); - } - private void SetupExporterTest() - { - this._handler = new TestHttpMessageHandler(req => - { - this._receivedRequest = true; - this._receivedContent = req.Content?.ReadAsStringAsync().GetAwaiter().GetResult(); - req.RequestUri.Should().NotBeNull(); - req.Headers.Authorization.Should().NotBeNull(); - return new HttpResponseMessage(System.Net.HttpStatusCode.OK); - }); - var httpClient = new HttpClient(this._handler); - this._provider = this.CreateTestServiceProvider(httpClient); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.IntegrationTests/Agent365ExporterE2ETests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.IntegrationTests/Agent365ExporterE2ETests.cs deleted file mode 100644 index d71cd37c..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.IntegrationTests/Agent365ExporterE2ETests.cs +++ /dev/null @@ -1,811 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.Hosting; -using System.Linq; -using System.Net; -using System.Text.Json; - -#pragma warning disable CS0618 // Tests intentionally pass the ignored legacy S2S compatibility switch. - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.IntegrationTests -{ - [TestClass] - public class Agent365ExporterE2ETests - { - private TestHttpMessageHandler? _handler; - private ServiceProvider? _provider; - private bool _receivedRequest; - private string? _receivedContent; - - [TestMethod] - public async Task AddTracing_And_InvokeAgentScope_ExporterMakesExpectedRequest() - { - // Arrange - this.SetupExporterTest(); - this._receivedRequest = false; - this._receivedContent = null; - var expectedAgentType = AgentType.EntraEmbodied; - var expectedAgentDetails = new AgentDetails( - agentId: Guid.NewGuid().ToString(), - agentName: "Test Agent", - agentDescription: "Agent for testing.", - agenticUserId: Guid.NewGuid().ToString(), - agenticUserEmail: "testagent@ztaittest12.onmicrosoft.com", - agentBlueprintId: Guid.NewGuid().ToString(), - tenantId: Guid.NewGuid().ToString(), - agentType: expectedAgentType); - var endpoint = new Uri("https://test-agent-endpoint"); - var invokeAgentScopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - - var expectedRequest = new Request( - content: "Test request content", - channel: new Channel( - name: "msteams", - link: "https://testchannel.link")); - - var expectedCallerDetails = new CallerDetails( - userDetails: new UserDetails( - userId: "caller-123", - userName: "Test Caller", - userEmail: "caller-123@ztaitest12.onmicrosoft.com", - userClientIP: IPAddress.Parse("203.0.113.42")), - callerAgentDetails: null); - - var expectedThreatDiagnosticsSummary = new ThreatDiagnosticsSummary( - blockAction: true, - reasonCode: 112, - reason: "The action was blocked due to security policy.", - diagnostics: "{\"flaggedField\":\"bcc\"}"); - - // Act - using (var scope = InvokeAgentScope.Start( - request: expectedRequest, - scopeDetails: invokeAgentScopeDetails, - agentDetails: expectedAgentDetails, - callerDetails: expectedCallerDetails, - threatDiagnosticsSummary: expectedThreatDiagnosticsSummary)) - { - scope.RecordInputMessages(new[] { "Input message 1", "Input message 2" }); - scope.RecordOutputMessages(new[] { "Output message 1" }); - } - - var timeout = TimeSpan.FromSeconds(30); - var start = DateTime.UtcNow; - while (!this._receivedRequest && DateTime.UtcNow - start < timeout) - { - await Task.Delay(1000).ConfigureAwait(false); - } - - this._receivedRequest.Should().BeTrue("Exporter should make the expected HTTP request."); - this._receivedContent.Should().NotBeNull("Exporter should send a request body."); - - using var doc = JsonDocument.Parse(this._receivedContent!); - var root = doc.RootElement; - var attributes = root - .GetProperty("resourceSpans")[0] - .GetProperty("scopeSpans")[0] - .GetProperty("spans")[0] - .GetProperty("attributes"); - this.GetAttribute(attributes, "server.address").Should().Be(invokeAgentScopeDetails.Endpoint?.Host); - this.GetAttribute(attributes, "microsoft.channel.name").Should().Be(expectedRequest.Channel?.Name); - this.GetAttribute(attributes, "microsoft.channel.link").Should().Be(expectedRequest.Channel?.Link); - this.GetAttribute(attributes, "microsoft.tenant.id").Should().Be(expectedAgentDetails.TenantId); - this.GetAttribute(attributes, "user.id").Should().Be(expectedCallerDetails.UserDetails?.UserId); - this.GetAttribute(attributes, "user.email").Should().Be(expectedCallerDetails.UserDetails?.UserEmail); - this.GetAttribute(attributes, "user.name").Should().Be(expectedCallerDetails.UserDetails?.UserName); - this.GetAttribute(attributes, "client.address").Should().Be(expectedCallerDetails.UserDetails?.UserClientIP?.ToString()); - this.GetAttribute(attributes, "gen_ai.input.messages").Should().Contain("Input message 1").And.Contain("Input message 2"); - this.GetAttribute(attributes, "gen_ai.output.messages").Should().Contain("Output message 1"); - this.GetAttribute(attributes, "gen_ai.agent.id").Should().Be(expectedAgentDetails.AgentId); - this.GetAttribute(attributes, "gen_ai.agent.name").Should().Be(expectedAgentDetails.AgentName); - this.GetAttribute(attributes, "gen_ai.agent.description").Should().Be(expectedAgentDetails.AgentDescription); - this.GetAttribute(attributes, "microsoft.agent.user.id").Should().Be(expectedAgentDetails.AgenticUserId); - this.GetAttribute(attributes, "microsoft.agent.user.email").Should().Be(expectedAgentDetails.AgenticUserEmail); - this.GetAttribute(attributes, "microsoft.a365.agent.blueprint.id").Should().Be(expectedAgentDetails.AgentBlueprintId); - this.GetAttribute(attributes, "microsoft.tenant.id").Should().Be(expectedAgentDetails.TenantId); - this.GetAttribute(attributes, "gen_ai.operation.name").Should().Be("invoke_agent"); - var threatSummaryJson = this.GetAttribute(attributes, "threat.diagnostics.summary"); - threatSummaryJson.Should().Contain("\"blockAction\":true"); - threatSummaryJson.Should().Contain("\"reasonCode\":112"); - threatSummaryJson.Should().Contain("\"reason\":\"The action was blocked due to security policy.\""); - threatSummaryJson.Should().Contain("flaggedField"); - threatSummaryJson.Should().Contain("bcc"); - } - - [TestMethod] - public async Task AddTracing_And_ExecuteToolScope_ExporterMakesExpectedRequest() - { - // Arrange - this.SetupExporterTest(); - this._receivedRequest = false; - this._receivedContent = null; - var expectedAgentType = AgentType.EntraEmbodied; - var expectedAgentDetails = new AgentDetails( - agentId: Guid.NewGuid().ToString(), - agentName: "Tool Agent", - agentDescription: "Agent for tool execution.", - agenticUserId: Guid.NewGuid().ToString(), - agenticUserEmail: "toolagent@ztaittest12.onmicrosoft.com", - agentBlueprintId: Guid.NewGuid().ToString(), - tenantId: Guid.NewGuid().ToString(), - agentType: expectedAgentType); - var endpoint = new Uri("https://tool-endpoint:8443"); - var toolCallDetails = new ToolCallDetails( - toolName: "TestTool", - arguments: "{\"param\":\"value\"}", - toolCallId: "call-456", - description: "Test tool call description", - toolType: "custom-type", - endpoint: endpoint, - toolServerName: "test-tool-server"); - - var expectedThreatDiagnosticsSummary = new ThreatDiagnosticsSummary( - blockAction: false, - reasonCode: 200, - reason: "No threats detected during tool execution.", - diagnostics: null); - - var expectedToolUserDetails = new UserDetails( - userId: "tool-caller-456", - userName: "Tool Caller", - userEmail: "tool-caller@ztaitest12.onmicrosoft.com", - userClientIP: IPAddress.Parse("10.0.0.42")); - - var expectedToolRequest = new Request( - content: "Tool execution request", - channel: new Channel(name: "tool-channel")); - - // Act - using (var scope = ExecuteToolScope.Start(expectedToolRequest, toolCallDetails, expectedAgentDetails, userDetails: expectedToolUserDetails, threatDiagnosticsSummary: expectedThreatDiagnosticsSummary)) - { - scope.RecordResponse("Tool response content"); - } - - var timeout = TimeSpan.FromSeconds(30); - var start = DateTime.UtcNow; - while (!this._receivedRequest && DateTime.UtcNow - start < timeout) - { - await Task.Delay(1000).ConfigureAwait(false); - } - - this._receivedRequest.Should().BeTrue("Exporter should make the expected HTTP request."); - this._receivedContent.Should().NotBeNull("Exporter should send a request body."); - - using var doc = JsonDocument.Parse(this._receivedContent!); - var root = doc.RootElement; - - var attributes = root - .GetProperty("resourceSpans")[0] - .GetProperty("scopeSpans")[0] - .GetProperty("spans")[0] - .GetProperty("attributes"); - - this.GetAttribute(attributes, "gen_ai.operation.name").Should().Be("execute_tool"); - this.GetAttribute(attributes, "gen_ai.agent.id").Should().Be(expectedAgentDetails.AgentId); - this.GetAttribute(attributes, "gen_ai.agent.name").Should().Be(expectedAgentDetails.AgentName); - this.GetAttribute(attributes, "gen_ai.agent.description").Should().Be(expectedAgentDetails.AgentDescription); - this.GetAttribute(attributes, "microsoft.agent.user.id").Should().Be(expectedAgentDetails.AgenticUserId); - this.GetAttribute(attributes, "microsoft.agent.user.email").Should().Be(expectedAgentDetails.AgenticUserEmail); - this.GetAttribute(attributes, "microsoft.a365.agent.blueprint.id").Should().Be(expectedAgentDetails.AgentBlueprintId); - this.GetAttribute(attributes, "microsoft.tenant.id").Should().Be(expectedAgentDetails.TenantId); - this.GetAttribute(attributes, "gen_ai.tool.name").Should().Be(toolCallDetails.ToolName); - this.GetAttribute(attributes, "gen_ai.tool.call.arguments").Should().Be(toolCallDetails.Arguments); - this.GetAttribute(attributes, "gen_ai.tool.call.id").Should().Be(toolCallDetails.ToolCallId); - this.GetAttribute(attributes, "gen_ai.tool.description").Should().Be(toolCallDetails.Description); - this.GetAttribute(attributes, "gen_ai.tool.type").Should().Be(toolCallDetails.ToolType); - this.GetAttribute(attributes, "gen_ai.tool.server.name").Should().Be(toolCallDetails.ToolServerName); - this.GetAttribute(attributes, "server.address").Should().Be(endpoint.Host); - this.GetAttribute(attributes, "server.port").Should().Be(endpoint.Port.ToString()); - this.GetAttribute(attributes, "gen_ai.tool.call.result").Should().Contain("Tool response content"); - this.GetAttribute(attributes, "user.id").Should().Be(expectedToolUserDetails.UserId); - this.GetAttribute(attributes, "user.email").Should().Be(expectedToolUserDetails.UserEmail); - this.GetAttribute(attributes, "user.name").Should().Be(expectedToolUserDetails.UserName); - this.GetAttribute(attributes, "client.address").Should().Be(expectedToolUserDetails.UserClientIP?.ToString()); - var toolThreatSummaryJson = this.GetAttribute(attributes, "threat.diagnostics.summary"); - toolThreatSummaryJson.Should().Contain("\"blockAction\":false"); - toolThreatSummaryJson.Should().Contain("\"reasonCode\":200"); - toolThreatSummaryJson.Should().Contain("\"reason\":\"No threats detected during tool execution.\""); - toolThreatSummaryJson.Should().Contain("\"diagnostics\":null"); - } - - [TestMethod] - public async Task AddTracing_And_InferenceScope_ExporterMakesExpectedRequest() - { - // Arrange - this.SetupExporterTest(); - this._receivedRequest = false; - this._receivedContent = null; - var expectedAgentType = AgentType.EntraEmbodied; - var expectedAgentDetails = new AgentDetails( - agentId: Guid.NewGuid().ToString(), - agentName: "Inference Agent", - agentDescription: "Agent for inference testing.", - agenticUserId: Guid.NewGuid().ToString(), - agenticUserEmail: "inferenceagent@ztaittest12.onmicrosoft.com", - agentBlueprintId: Guid.NewGuid().ToString(), - tenantId: Guid.NewGuid().ToString(), - agentType: expectedAgentType); - - var inferenceDetails= new InferenceCallDetails( - operationName: InferenceOperationType.Chat, - model: "gpt-4", - providerName: "OpenAI", - inputTokens: 42, - outputTokens: 84, - finishReasons: new[] { "stop", "length" }, - responseId: "response-xyz"); - - var expectedInferenceUserDetails = new UserDetails( - userId: "inference-caller-789", - userName: "Inference Caller", - userEmail: "inference-caller@ztaitest12.onmicrosoft.com", - userClientIP: IPAddress.Parse("172.16.0.42")); - - var expectedInferenceRequest = new Request( - content: "Inference request", - channel: new Channel(name: "inference-channel")); - - // Act - using (var scope = InferenceScope.Start(expectedInferenceRequest, inferenceDetails, expectedAgentDetails, userDetails: expectedInferenceUserDetails)) - { - scope.RecordInputMessages(new[] { "Hello", "World" }); - scope.RecordOutputMessages(new[] { "Hi there!" }); - scope.RecordInputTokens(42); - scope.RecordOutputTokens(84); - scope.RecordFinishReasons(new[] { "stop", "length" }); - } - - var timeout = TimeSpan.FromSeconds(30); - var start = DateTime.UtcNow; - while (!this._receivedRequest && DateTime.UtcNow - start < timeout) - { - await Task.Delay(1000).ConfigureAwait(false); - } - - this._receivedRequest.Should().BeTrue("Exporter should make the expected HTTP request."); - this._receivedContent.Should().NotBeNull("Exporter should send a request body."); - - using var doc = JsonDocument.Parse(this._receivedContent!); - var root = doc.RootElement; - var attributes = root - .GetProperty("resourceSpans")[0] - .GetProperty("scopeSpans")[0] - .GetProperty("spans")[0] - .GetProperty("attributes"); - - this.GetAttribute(attributes, "gen_ai.operation.name").Should().Be(inferenceDetails.OperationName.ToString()); - this.GetAttribute(attributes, "gen_ai.agent.id").Should().Be(expectedAgentDetails.AgentId); - this.GetAttribute(attributes, "gen_ai.agent.name").Should().Be(expectedAgentDetails.AgentName); - this.GetAttribute(attributes, "gen_ai.agent.description").Should().Be(expectedAgentDetails.AgentDescription); - this.GetAttribute(attributes, "microsoft.agent.user.id").Should().Be(expectedAgentDetails.AgenticUserId); - this.GetAttribute(attributes, "microsoft.agent.user.email").Should().Be(expectedAgentDetails.AgenticUserEmail); - this.GetAttribute(attributes, "microsoft.a365.agent.blueprint.id").Should().Be(expectedAgentDetails.AgentBlueprintId); - this.GetAttribute(attributes, "microsoft.tenant.id").Should().Be(expectedAgentDetails.TenantId); - this.GetAttribute(attributes, "gen_ai.request.model").Should().Be(inferenceDetails.Model); - this.GetAttribute(attributes, "gen_ai.provider.name").Should().Be(inferenceDetails.ProviderName); - this.GetAttribute(attributes, "gen_ai.usage.input_tokens").Should().Be("42"); - this.GetAttribute(attributes, "gen_ai.usage.output_tokens").Should().Be("84"); - this.GetAttribute(attributes, "gen_ai.response.finish_reasons").Should().Be("stop,length"); - this.GetAttribute(attributes, "gen_ai.input.messages").Should().Contain("Hello").And.Contain("World"); - this.GetAttribute(attributes, "gen_ai.output.messages").Should().Contain("Hi there!"); - this.GetAttribute(attributes, "user.id").Should().Be(expectedInferenceUserDetails.UserId); - this.GetAttribute(attributes, "user.email").Should().Be(expectedInferenceUserDetails.UserEmail); - this.GetAttribute(attributes, "user.name").Should().Be(expectedInferenceUserDetails.UserName); - this.GetAttribute(attributes, "client.address").Should().Be(expectedInferenceUserDetails.UserClientIP?.ToString()); - } - - [TestMethod] - public async Task AddTracing_NestedScopes_AllExporterRequestsReceived_UsesAgentId() - { - await this.RunNestedScopes_AllExporterRequestsReceived(useAgentId: true); - } - - [TestMethod] - public async Task AddTracing_NestedScopes_AllExporterRequestsReceived_UsesAgentPlatformId() - { - await this.RunNestedScopes_AllExporterRequestsReceived(useAgentId: false); - } - - private async Task RunNestedScopes_AllExporterRequestsReceived(bool useAgentId) - { - // Arrange - List receivedContents = new(); - - var agentType = AgentType.EntraEmbodied; - var agentDetails = useAgentId - ? new AgentDetails( - agentId: Guid.NewGuid().ToString(), - agentName: "Nested Agent", - agentDescription: "Agent for nested scope testing.", - agenticUserId: Guid.NewGuid().ToString(), - agenticUserEmail: "nestedagent@ztaittest12.onmicrosoft.com", - agentBlueprintId: Guid.NewGuid().ToString(), - tenantId: Guid.NewGuid().ToString(), - agentType: agentType) - : new AgentDetails( - agentId: null, - agentName: "Nested Agent", - agentDescription: "Agent for nested scope testing.", - agenticUserId: Guid.NewGuid().ToString(), - agenticUserEmail: "nestedagent@ztaittest12.onmicrosoft.com", - agentBlueprintId: Guid.NewGuid().ToString(), - tenantId: Guid.NewGuid().ToString(), - agentType: agentType, - agentClientIP: null, - agentPlatformId: Guid.NewGuid().ToString()); - - var endpoint = new Uri("https://nested-endpoint"); - - var handler = new TestHttpMessageHandler(req => - { - receivedContents.Add(req.Content?.ReadAsStringAsync().GetAwaiter().GetResult() ?? ""); - return new HttpResponseMessage(System.Net.HttpStatusCode.OK); - }); - var httpClient = new HttpClient(handler); - - var provider = this.CreateTestServiceProvider(httpClient); - - var invokeAgentScopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var request = new Request( - content: "Nested request", - channel: new Channel(name: "nested", link: "https://nestedchannel.link")); - - var toolCallDetails = new ToolCallDetails( - toolName: "NestedTool", - arguments: "{\"param\":\"nested\"}", - toolCallId: "call-nested", - description: "Nested tool call", - toolType: "nested-type", - endpoint: endpoint); - - var inferenceDetails = new InferenceCallDetails( - operationName: InferenceOperationType.Chat, - model: "gpt-nested", - providerName: "OpenAI", - inputTokens: 10, - outputTokens: 20, - finishReasons: new[] { "stop" }, - responseId: "response-nested"); - - // Act - using (var agentScope = InvokeAgentScope.Start(request, invokeAgentScopeDetails, agentDetails)) - { - agentScope.RecordInputMessages(new[] { "Agent input" }); - agentScope.RecordOutputMessages(new[] { "Agent output" }); - - using (var toolScope = ExecuteToolScope.Start(request, toolCallDetails, agentDetails)) - { - toolScope.RecordResponse("Tool response"); - - using (var inferenceScope = InferenceScope.Start(request, inferenceDetails, agentDetails)) - { - inferenceScope.RecordInputMessages(new[] { "Inference input" }); - inferenceScope.RecordOutputMessages(new[] { "Inference output" }); - inferenceScope.RecordInputTokens(10); - inferenceScope.RecordOutputTokens(20); - inferenceScope.RecordFinishReasons(new[] { "stop" }); - } - } - } - - // Wait for up to 10 seconds for all spans to be exported - await Task.Delay(10000).ConfigureAwait(false); - - // Assert - var allOperationNames = new List(); - foreach (var content in receivedContents) - { - using var doc = JsonDocument.Parse(content); - var root = doc.RootElement; - var spans = root - .GetProperty("resourceSpans")[0] - .GetProperty("scopeSpans")[0] - .GetProperty("spans") - .EnumerateArray(); - - foreach (var span in spans) - { - var opName = this.GetAttribute(span.GetProperty("attributes"), "gen_ai.operation.name"); - if (opName != null) - allOperationNames.Add(opName); - } - } - allOperationNames.Should().Contain(new[] { "invoke_agent", "execute_tool", InferenceOperationType.Chat.ToString() }, "All three nested scopes should be exported, even if batched in fewer requests."); - } - - [TestMethod] - public async Task Exporter_Truncates_Scope() - { - // Arrange - this.SetupExporterTest(); - this._receivedRequest = false; - this._receivedContent = null; - - // Create a sample text file >250KB and base64 encode it - var tempFile = Path.GetTempFileName(); - var fileBytes = new byte[300 * 1024]; // 300KB - new Random(42).NextBytes(fileBytes); - await File.WriteAllBytesAsync(tempFile, fileBytes); - var base64 = Convert.ToBase64String(await File.ReadAllBytesAsync(tempFile)); - File.Delete(tempFile); - - var agentDetails = new AgentDetails( - agentId: Guid.NewGuid().ToString(), - agentName: "Test Agent", - agentDescription: "Agent for truncation test.", - agenticUserId: Guid.NewGuid().ToString(), - agenticUserEmail: "testagent@contoso.com", - agentBlueprintId: Guid.NewGuid().ToString(), - tenantId: Guid.NewGuid().ToString()); - var endpoint = new Uri("https://test-endpoint"); - var invokeAgentScopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var request = new Request( - content: "Test request content", - channel: new Channel(name: "test")); - - var toolCallDetails = new ToolCallDetails( - toolName: "LargeFileTool", - arguments: base64, - toolCallId: "call-123", - description: "Test tool with large file content", - toolType: "file-upload", - endpoint: endpoint); - - // Act: Start nested scopes - using (var agentScope = InvokeAgentScope.Start(request, invokeAgentScopeDetails, agentDetails)) - { - agentScope.RecordInputMessages(new[] { "Agent input" }); - agentScope.RecordOutputMessages(new[] { "Agent output" }); - using (var toolScope = ExecuteToolScope.Start(request, toolCallDetails, agentDetails)) - { - toolScope.RecordResponse("Tool response"); - } - } - - // Wait for export - var timeout = TimeSpan.FromSeconds(10); - var start = DateTime.UtcNow; - while (!this._receivedRequest && DateTime.UtcNow - start < timeout) - { - await Task.Delay(500).ConfigureAwait(false); - } - - this._receivedRequest.Should().BeTrue("Exporter should make the expected HTTP request."); - this._receivedContent.Should().NotBeNull("Exporter should send a request body."); - - // Assert: Find both activities in the exported payload - using var doc = JsonDocument.Parse(this._receivedContent!); - var root = doc.RootElement; - var spans = root - .GetProperty("resourceSpans")[0] - .GetProperty("scopeSpans")[0] - .GetProperty("spans") - .EnumerateArray(); - - bool foundInvokeAgent = false; - bool foundExecuteTool = false; - foreach (var span in spans) - { - var attrs = span.GetProperty("attributes"); - var opName = this.GetAttribute(attrs, "gen_ai.operation.name"); - if (opName == "invoke_agent") - { - foundInvokeAgent = true; - // Should NOT be truncated - var input = this.GetAttribute(attrs, "gen_ai.input.messages"); - input.Should().Contain("Agent input"); - } - if (opName == "execute_tool") - { - foundExecuteTool = true; - // Should be truncated - var args = this.GetAttribute(attrs, "gen_ai.tool.call.arguments"); - args.Should().Be("TRUNCATED"); - } - } - foundInvokeAgent.Should().BeTrue(); - foundExecuteTool.Should().BeTrue(); - } - - /// - /// Verifies that multiple registrations of the Agent365 exporter through different APIs - /// (direct OpenTelemetry configuration and AddA365Tracing) result in only a single - /// exporter instance being registered, preventing duplicate telemetry exports. - /// This test ensures the exporter registry correctly implements singleton behavior - /// to avoid data duplication when the same exporter type is configured multiple times. - /// - [TestMethod] - public void AddTracing_And_ApplyGuardrailScope_ExporterMakesExpectedRequest() - { - // Arrange - var expectedAgentDetails = new AgentDetails( - agentId: Guid.NewGuid().ToString(), - agentName: "Guardrail Agent", - agentDescription: "Agent for guardrail testing.", - agenticUserId: Guid.NewGuid().ToString(), - agenticUserEmail: "guardrailagent@ztaittest12.onmicrosoft.com", - agentBlueprintId: Guid.NewGuid().ToString(), - tenantId: Guid.NewGuid().ToString(), - agentType: AgentType.EntraEmbodied); - - var guardrailDetails = new GuardrailDetails( - targetType: GuardrailTargetType.LlmInput, - decisionType: GuardrailDecisionType.Deny, - guardianId: "azure-content-safety-001", - guardianName: "Azure Content Safety", - guardianProviderName: "Azure", - guardianVersion: "2.0.0", - targetId: "msg-12345", - decisionReason: "Content violates hate speech policy", - decisionCode: "HATE_SPEECH_001", - policyId: "policy-abc", - policyName: "Content Safety Policy", - policyVersion: "1.2.0", - contentInputHash: "sha256:abc123def456", - contentModified: false, - externalEventId: "ext-event-789"); - - var expectedUserDetails = new UserDetails( - userId: "guardrail-caller-123", - userName: "Guardrail Caller", - userEmail: "guardrail-caller@ztaitest12.onmicrosoft.com", - userClientIP: IPAddress.Parse("192.168.1.42")); - - var expectedRequest = new Request( - content: "Check this content for safety", - channel: new Channel( - name: "msteams", - link: "https://guardrail-channel.link")); - - // Use ActivityListener to capture the activity directly - System.Diagnostics.Activity? capturedActivity = null; - using var listener = new System.Diagnostics.ActivityListener(); - listener.ShouldListenTo = source => source.Name == OpenTelemetryConstants.SourceName; - listener.Sample = (ref System.Diagnostics.ActivityCreationOptions _) => - System.Diagnostics.ActivitySamplingResult.AllDataAndRecorded; - listener.ActivityStarted = a => capturedActivity = a; - System.Diagnostics.ActivitySource.AddActivityListener(listener); - - // Act - using (var scope = ApplyGuardrailScope.Start( - details: guardrailDetails, - agentDetails: expectedAgentDetails, - request: expectedRequest, - userDetails: expectedUserDetails)) - { - scope.RecordDecision(GuardrailDecisionType.Deny); - scope.RecordContentOutput("sanitized-hash-output"); - scope.RecordFinding(new GuardrailFinding( - riskCategory: "hate_speech", - riskSeverity: GuardrailRiskSeverity.High, - policyDecisionType: "deny", - policyId: "policy-abc", - riskScore: 0.95, - riskMetadata: new[] { "{\"category\":\"hate\",\"confidence\":0.95}" })); - } - - // Assert activity was captured - capturedActivity.Should().NotBeNull("ApplyGuardrailScope should create an activity"); - - // Format activity as OTLP-like JSON for inspection - var spanJson = new - { - name = capturedActivity!.DisplayName, - traceId = capturedActivity.TraceId.ToHexString(), - spanId = capturedActivity.SpanId.ToHexString(), - parentSpanId = capturedActivity.ParentSpanId.ToHexString(), - kind = capturedActivity.Kind.ToString(), - startTimeUnixNano = capturedActivity.StartTimeUtc.Ticks, - endTimeUnixNano = (capturedActivity.StartTimeUtc + capturedActivity.Duration).Ticks, - attributes = capturedActivity.TagObjects.ToDictionary(t => t.Key, t => t.Value), - events = capturedActivity.Events.Select(e => new - { - name = e.Name, - timeUnixNano = e.Timestamp.Ticks, - attributes = e.Tags.ToDictionary(t => t.Key, t => t.Value) - }).ToArray() - }; - - var prettyJson = JsonSerializer.Serialize(spanJson, new JsonSerializerOptions { WriteIndented = true }); - Console.WriteLine("=== ApplyGuardrail Span JSON ==="); - Console.WriteLine(prettyJson); - Console.WriteLine("=== End ApplyGuardrail Span JSON ==="); - - // Verify span name - capturedActivity.DisplayName.Should().Be("apply_guardrail Azure Content Safety llm_input"); - - // Verify operation name - capturedActivity.GetTagItem("gen_ai.operation.name").Should().Be("apply_guardrail"); - - // Verify agent details - capturedActivity.GetTagItem("gen_ai.agent.id").Should().Be(expectedAgentDetails.AgentId); - capturedActivity.GetTagItem("gen_ai.agent.name").Should().Be(expectedAgentDetails.AgentName); - capturedActivity.GetTagItem("gen_ai.agent.description").Should().Be(expectedAgentDetails.AgentDescription); - capturedActivity.GetTagItem("microsoft.agent.user.id").Should().Be(expectedAgentDetails.AgenticUserId); - capturedActivity.GetTagItem("microsoft.agent.user.email").Should().Be(expectedAgentDetails.AgenticUserEmail); - capturedActivity.GetTagItem("microsoft.a365.agent.blueprint.id").Should().Be(expectedAgentDetails.AgentBlueprintId); - capturedActivity.GetTagItem("microsoft.tenant.id").Should().Be(expectedAgentDetails.TenantId); - - // Verify guardrail-specific attributes - capturedActivity.GetTagItem("microsoft.security.decision.type").Should().Be("deny"); - capturedActivity.GetTagItem("microsoft.security.target.type").Should().Be("llm_input"); - capturedActivity.GetTagItem("microsoft.guardian.id").Should().Be("azure-content-safety-001"); - capturedActivity.GetTagItem("microsoft.guardian.name").Should().Be("Azure Content Safety"); - capturedActivity.GetTagItem("microsoft.guardian.provider.name").Should().Be("Azure"); - capturedActivity.GetTagItem("microsoft.guardian.version").Should().Be("2.0.0"); - capturedActivity.GetTagItem("microsoft.security.target.id").Should().Be("msg-12345"); - capturedActivity.GetTagItem("microsoft.security.decision.reason").Should().Be("Content violates hate speech policy"); - capturedActivity.GetTagItem("microsoft.security.decision.code").Should().Be("HATE_SPEECH_001"); - capturedActivity.GetTagItem("microsoft.security.policy.id").Should().Be("policy-abc"); - capturedActivity.GetTagItem("microsoft.security.policy.name").Should().Be("Content Safety Policy"); - capturedActivity.GetTagItem("microsoft.security.policy.version").Should().Be("1.2.0"); - capturedActivity.GetTagItem("microsoft.security.content.input.hash").Should().Be("sha256:abc123def456"); - capturedActivity.GetTagItem("microsoft.security.content.output.value").Should().Be("sanitized-hash-output"); - capturedActivity.GetTagItem("microsoft.security.content.modified").Should().Be(false); - capturedActivity.GetTagItem("microsoft.security.external_event_id").Should().Be("ext-event-789"); - - // Verify caller details - capturedActivity.GetTagItem("user.id").Should().Be(expectedUserDetails.UserId); - capturedActivity.GetTagItem("user.email").Should().Be(expectedUserDetails.UserEmail); - capturedActivity.GetTagItem("user.name").Should().Be(expectedUserDetails.UserName); - capturedActivity.GetTagItem("client.address").Should().Be(expectedUserDetails.UserClientIP?.ToString()); - - // Verify channel - capturedActivity.GetTagItem("microsoft.channel.name").Should().Be(expectedRequest.Channel?.Name); - capturedActivity.GetTagItem("microsoft.channel.link").Should().Be(expectedRequest.Channel?.Link); - - // Verify events (finding) - capturedActivity.Events.Should().HaveCount(1); - var findingEvent = capturedActivity.Events.First(); - findingEvent.Name.Should().Be("microsoft.security.finding"); - var findingTags = findingEvent.Tags.ToDictionary(t => t.Key, t => t.Value); - findingTags["microsoft.security.risk.category"].Should().Be("hate_speech"); - findingTags["microsoft.security.risk.severity"].Should().Be(GuardrailRiskSeverity.High); - findingTags["microsoft.security.policy.decision.type"].Should().Be("deny"); - findingTags["microsoft.security.policy.id"].Should().Be("policy-abc"); - findingTags["microsoft.security.risk.score"].Should().Be(0.95); - } - - [TestMethod] - public async Task AddTracing_MultipleInvocations_NoDuplicateExports() - { - List receivedContents = new(); - var requestCount = 0; - - var handler = new TestHttpMessageHandler(req => - { - Interlocked.Increment(ref requestCount); - var content = req.Content?.ReadAsStringAsync().GetAwaiter().GetResult() ?? ""; - lock (receivedContents) - { - receivedContents.Add(content); - } - return new HttpResponseMessage(System.Net.HttpStatusCode.OK); - }); - var httpClient = new HttpClient(handler); - - // Create builder and simulate mixed API usage that could cause duplicates - HostApplicationBuilder builder = new HostApplicationBuilder(); - builder.Configuration["EnableAgent365Exporter"] = "true"; - builder.Services.AddSingleton(httpClient); - builder.Services.AddSingleton(_ => new Agent365ExporterOptions - { - UseS2SEndpoint = false, - TokenResolver = (_, _) => Task.FromResult("test-token") - }); - - // AddA365Tracing call - builder.AddA365Tracing(useOpenTelemetryBuilder: true, agent365ExporterType: Agent365ExporterType.Agent365Exporter); - - // Duplicate AddA365Tracing call - builder.AddA365Tracing(useOpenTelemetryBuilder: true, agent365ExporterType: Agent365ExporterType.Agent365Exporter); - - var serviceProvider = builder.Services.BuildServiceProvider(); - - // Initialize the OpenTelemetry infrastructure by accessing the TracerProvider - // This ensures the tracing pipeline is built and exporters are initialized - _ = serviceProvider.GetService(); - - var agentDetails = new AgentDetails( - agentId: Guid.NewGuid().ToString(), - agentName: "Singleton Test Agent", - agentDescription: "Testing singleton exporter behavior.", - agenticUserId: Guid.NewGuid().ToString(), - agenticUserEmail: "singleton@test.com", - agentBlueprintId: Guid.NewGuid().ToString(), - tenantId: Guid.NewGuid().ToString()); - - var endpoint = new Uri("https://singleton-test-endpoint"); - var invokeAgentScopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var request = new Request( - content: "Singleton test request", - channel: new Channel(name: "test", link: "singleton test")); - using (var scope = InvokeAgentScope.Start(request, invokeAgentScopeDetails, agentDetails)) - { - scope.RecordInputMessages(new[] { "Test input" }); - scope.RecordOutputMessages(new[] { "Test output" }); - } - - // Wait for export(s) to complete - await Task.Delay(10000).ConfigureAwait(false); - - // Assert - Should have exactly one export request, not two - requestCount.Should().Be(1, "Exporter registry should prevent duplicate exporter registration, resulting in only one export request"); - receivedContents.Should().HaveCount(1, "Should receive exactly one export payload"); - } - - private class TestHttpMessageHandler : HttpMessageHandler - { - private Func _handler; - public TestHttpMessageHandler(Func handler) - { - this._handler = handler; - } - public void SetHandler(Func handler) - { - this._handler = handler; - } - protected override Task SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) - { - return Task.FromResult(this._handler(request)); - } - } - - private string? GetAttribute(JsonElement attributes, string key) - { - if (attributes.TryGetProperty(key, out var value)) - { - if (value.ValueKind == JsonValueKind.String) - { - return value.GetString(); - } - if (value.ValueKind == JsonValueKind.Number) - { - return value.GetRawText(); - } - if (value.ValueKind == JsonValueKind.Object && value.TryGetProperty("stringValue", out var sv)) - { - return sv.GetString(); - } - } - return null; - } - - private ServiceProvider CreateTestServiceProvider(HttpClient httpClient) - { - HostApplicationBuilder builder = new HostApplicationBuilder(); - - builder.Configuration["EnableAgent365Exporter"] = "true"; - builder.Services.AddSingleton(httpClient); - builder.Services.AddSingleton(sp => - { - return new Agent365ExporterOptions - { - UseS2SEndpoint = false, - TokenResolver = (_, _) => Task.FromResult("test-token") - }; - }); - builder.AddA365Tracing(useOpenTelemetryBuilder: false, agent365ExporterType: Agent365ExporterType.Agent365Exporter); - return builder.Services.BuildServiceProvider(); - } - private void SetupExporterTest() - { - this._handler = new TestHttpMessageHandler(req => - { - this._receivedRequest = true; - this._receivedContent = req.Content?.ReadAsStringAsync().GetAwaiter().GetResult(); - req.RequestUri.Should().NotBeNull(); - req.Headers.Authorization.Should().NotBeNull(); - return new HttpResponseMessage(System.Net.HttpStatusCode.OK); - }); - var httpClient = new HttpClient(this._handler); - this._provider = this.CreateTestServiceProvider(httpClient); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.IntegrationTests/Microsoft.Agents.A365.Observability.Runtime.IntegrationTests.csproj b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.IntegrationTests/Microsoft.Agents.A365.Observability.Runtime.IntegrationTests.csproj deleted file mode 100644 index 1a6f9480..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.IntegrationTests/Microsoft.Agents.A365.Observability.Runtime.IntegrationTests.csproj +++ /dev/null @@ -1,25 +0,0 @@ - - - net8.0 - false - false - true - - - - - - - - - - - - - - - - - - - diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/BuilderPatternTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/BuilderPatternTests.cs deleted file mode 100644 index aa529e45..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/BuilderPatternTests.cs +++ /dev/null @@ -1,50 +0,0 @@ -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.Hosting; - -namespace Microsoft.Agents.A365.Observability.Tests; - -/// -/// Test to verify that the new builder pattern works as expected in the issue example. -/// -[TestClass] -public sealed class BuilderPatternTests -{ - [TestMethod] - public void AddTracing_WithLambdaConfiguration_ShouldWork() - { - HostApplicationBuilder builder = new HostApplicationBuilder(); - - // Use the new lambda configuration approach - var result = builder.AddA365Tracing(); - - // Should return the configured service collection directly (no Build() needed) - result.Should().NotBeNull(); - result.Should().BeSameAs(builder); - result.Services.Should().BeAssignableTo(); - } - - [TestMethod] - public void AddTracing_WithNullLambda_ShouldWork() - { - HostApplicationBuilder builder = new HostApplicationBuilder(); - - var result = builder.AddA365Tracing(null); - - result.Should().NotBeNull(); - result.Should().BeSameAs(builder); - } - - [TestMethod] - public void AddTracing_WithEmptyLambda_ShouldWork() - { - HostApplicationBuilder builder = new HostApplicationBuilder(); - - // Pass empty lambda - should work like no configuration - var result = builder.AddA365Tracing(_ => { }); - - result.Should().NotBeNull(); - result.Should().BeSameAs(builder); - } -} \ No newline at end of file diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/BuilderTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/BuilderTests.cs deleted file mode 100644 index 6b8d1347..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/BuilderTests.cs +++ /dev/null @@ -1,206 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Extensions.Configuration; -using Microsoft.Extensions.DependencyInjection; -using OpenTelemetry.Trace; - -#pragma warning disable CS0618 // Tests intentionally pass the ignored legacy S2S compatibility switch. - -namespace Microsoft.Agents.A365.Observability.Tests; - -/// -/// Tests to verify direct usage of the public Builder constructor and Build(). -/// -[TestClass] -public sealed class BuilderTests -{ - [TestMethod] - public void Builder_WithUseOpenTelemetryBuilder_True_ShouldBuild() - { - var services = new ServiceCollection(); - var configuration = new ConfigurationBuilder().Build(); - - var builder = new Builder(services: services, configuration: configuration, useOpenTelemetryBuilder: true); - - var result = builder.Build(); - - result.Should().NotBeNull(); - result.Should().BeSameAs(services); - } - - [TestMethod] - public void Builder_WithUseOpenTelemetryBuilder_False_ShouldBuild() - { - var services = new ServiceCollection(); - var configuration = new ConfigurationBuilder().Build(); - - var builder = new Builder(services: services, configuration: configuration, useOpenTelemetryBuilder: false); - - var result = builder.Build(); - - result.Should().NotBeNull(); - result.Should().BeSameAs(services); - } - - [TestMethod] - public void Builder_WithNullConfiguration_ShouldBuild() - { - var services = new ServiceCollection(); - - var builder = new Builder(services: services, configuration: null, useOpenTelemetryBuilder: true); - - var result = builder.Build(); - - result.Should().NotBeNull(); - result.Should().BeSameAs(services); - } - - [TestMethod] - public void Builder_WithExporterType_Sync_RegistersTracerProvider() - { - try - { - Environment.SetEnvironmentVariable("EnableAgent365Exporter", "true"); - - var services = new ServiceCollection(); - var configuration = new ConfigurationBuilder().AddEnvironmentVariables().Build(); - - // Provide required dependencies for exporter - services.AddSingleton(_ => new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("unit-test-token"), - UseS2SEndpoint = false - }); - - var builder = new Builder( - services: services, - configuration: configuration, - useOpenTelemetryBuilder: true, - agent365ExporterType: Agent365ExporterType.Agent365Exporter); - - builder.Build(); - - var provider = services.BuildServiceProvider(); - var tracerProvider = provider.GetService(); - tracerProvider.Should().NotBeNull(); - } - finally - { - Environment.SetEnvironmentVariable("EnableAgent365Exporter", null); - } - } - - [TestMethod] - public void Builder_WithExporterType_Async_RegistersTracerProvider() - { - try - { - Environment.SetEnvironmentVariable("EnableAgent365Exporter", "true"); - - var services = new ServiceCollection(); - var configuration = new ConfigurationBuilder().AddEnvironmentVariables().Build(); - - // Provide required dependencies for exporter - services.AddSingleton(_ => new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("unit-test-token"), - UseS2SEndpoint = false - }); - - var builder = new Builder( - services: services, - configuration: configuration, - useOpenTelemetryBuilder: true, - agent365ExporterType: Agent365ExporterType.Agent365ExporterAsync); - - builder.Build(); - - var provider = services.BuildServiceProvider(); - var tracerProvider = provider.GetService(); - tracerProvider.Should().NotBeNull(); - } - finally - { - Environment.SetEnvironmentVariable("EnableAgent365Exporter", null); - } - } - - [TestMethod] - public void Builder_UseOpenTelemetryBuilder_False_WithExporterType_Sync_CreatesTracerProvider() - { - try - { - Environment.SetEnvironmentVariable("EnableAgent365Exporter", "true"); - - var services = new ServiceCollection(); - var configuration = new ConfigurationBuilder().AddEnvironmentVariables().Build(); - - services.AddSingleton(_ => new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("unit-test-token"), - UseS2SEndpoint = false - }); - - var builder = new Builder( - services: services, - configuration: configuration, - useOpenTelemetryBuilder: false, - agent365ExporterType: Agent365ExporterType.Agent365Exporter); - - builder.Build(); - - var provider = services.BuildServiceProvider(); - var tracerProvider = provider.GetService(); - tracerProvider.Should().BeNull("useOpenTelemetryBuilder=false should not register TracerProvider in DI"); - - using var source = new System.Diagnostics.ActivitySource(OpenTelemetryConstants.SourceName); - source.HasListeners().Should().BeTrue("Agent365Exporter should register ActivitySource listeners even when not using OpenTelemetryBuilder"); - } - finally - { - Environment.SetEnvironmentVariable("EnableAgent365Exporter", null); - } - } - - [TestMethod] - public void Builder_UseOpenTelemetryBuilder_False_WithExporterType_Async_ExportsAndNoTracerProviderInDI() - { - try - { - Environment.SetEnvironmentVariable("EnableAgent365Exporter", "true"); - - var services = new ServiceCollection(); - var configuration = new ConfigurationBuilder().AddEnvironmentVariables().Build(); - - services.AddSingleton(_ => new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("unit-test-token"), - UseS2SEndpoint = false - }); - - var builder = new Builder( - services: services, - configuration: configuration, - useOpenTelemetryBuilder: false, - agent365ExporterType: Agent365ExporterType.Agent365ExporterAsync); - - builder.Build(); - - var provider = services.BuildServiceProvider(); - var tracerProvider = provider.GetService(); - tracerProvider.Should().BeNull("useOpenTelemetryBuilder=false should not register TracerProvider in DI"); - - using var source = new System.Diagnostics.ActivitySource(OpenTelemetryConstants.SourceName); - source.HasListeners().Should().BeTrue("Agent365ExporterAsync should register ActivitySource listeners even when not using OpenTelemetryBuilder"); - } - finally - { - Environment.SetEnvironmentVariable("EnableAgent365Exporter", null); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Common/EnvironmentUtilsTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Common/EnvironmentUtilsTests.cs deleted file mode 100644 index af2e0e2f..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Common/EnvironmentUtilsTests.cs +++ /dev/null @@ -1,50 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using Microsoft.Agents.A365.Observability.Runtime.Common; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.Common; - -[TestClass] -public class EnvironmentUtilsTests -{ - private const string OverrideEnvVarName = "A365_OBSERVABILITY_SCOPE_OVERRIDE"; - - [TestInitialize] - public void TestInitialize() - { - // Ensure clean environment before each test - Environment.SetEnvironmentVariable(OverrideEnvVarName, null); - } - - [TestMethod] - public void GetObservabilityAuthenticationScope_ReturnsOverride_WhenEnvVarIsSet() - { - // Arrange - var expectedOverride = "https://override.example.com/.default"; - Environment.SetEnvironmentVariable(OverrideEnvVarName, expectedOverride); - - // Act - var scopes = EnvironmentUtils.GetObservabilityAuthenticationScope(); - - // Assert - Assert.IsNotNull(scopes); - Assert.AreEqual(1, scopes.Length); - Assert.AreEqual(expectedOverride, scopes[0]); - } - - [TestMethod] - public void GetObservabilityAuthenticationScope_ReturnsDefault_WhenEnvVarIsNotSet() - { - // Arrange - Environment.SetEnvironmentVariable(OverrideEnvVarName, null); - - // Act - var scopes = EnvironmentUtils.GetObservabilityAuthenticationScope(); - - // Assert - Assert.IsNotNull(scopes); - Assert.AreEqual(1, scopes.Length); - Assert.AreEqual("api://9b975845-388f-4429-889e-eab1ef63949c/.default", scopes[0]); - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Common/ExportFormatterStatusTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Common/ExportFormatterStatusTests.cs deleted file mode 100644 index 9441e20c..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Common/ExportFormatterStatusTests.cs +++ /dev/null @@ -1,76 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System.Text.Json; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.DTOs; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.Common; - -public partial class ExportFormatterTests -{ - [TestMethod] - public void FormatLogData_NoError_EmitsUnsetStatus() - { - // Arrange - var data = new InvokeAgentData( - new Dictionary { { "key", "val" } }, - spanId: "span-1"); - var formatter = CreateFormatter(); - - // Act - var json = formatter.FormatLogData(data.ToDictionary()); - - // Assert - using var doc = JsonDocument.Parse(json); - var status = doc.RootElement.GetProperty("Status"); - status.GetProperty("code").GetInt32().Should().Be(0); - status.GetProperty("message").GetString().Should().Be(""); - } - - [TestMethod] - public void FormatLogData_WithErrorStatus_EmitsErrorCodeAndMessage() - { - // Arrange - var data = new InvokeAgentData( - new Dictionary { { "key", "val" } }, - spanId: "span-2") - { - StatusCode = SpanStatusCode.Error, - StatusMessage = "something failed" - }; - var formatter = CreateFormatter(); - - // Act - var json = formatter.FormatLogData(data.ToDictionary()); - - // Assert - using var doc = JsonDocument.Parse(json); - var status = doc.RootElement.GetProperty("Status"); - status.GetProperty("code").GetInt32().Should().Be(2); - status.GetProperty("message").GetString().Should().Be("something failed"); - } - - [TestMethod] - public void FormatLogData_MissingStatusKey_DefaultsToUnset() - { - // Arrange — a dictionary without a "Status" entry - var bareData = new Dictionary - { - { "Name", "InvokeAgent" }, - { "Attributes", new Dictionary() }, - { "SpanId", "span-3" }, - { "ParentSpanId", null }, - }; - var formatter = CreateFormatter(); - - // Act - var json = formatter.FormatLogData(bareData); - - // Assert - using var doc = JsonDocument.Parse(json); - var status = doc.RootElement.GetProperty("Status"); - status.GetProperty("code").GetInt32().Should().Be(0); - status.GetProperty("message").GetString().Should().Be(""); - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Common/ExportFormatterTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Common/ExportFormatterTests.cs deleted file mode 100644 index 0865575a..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Common/ExportFormatterTests.cs +++ /dev/null @@ -1,622 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System.Diagnostics; -using System.Text.Json; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.Common; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Agents.A365.Observability.Tests.Tracing; -using Microsoft.Agents.A365.Observability.Tests.Tracing.Scopes; -using Microsoft.Agents.A365.Observability.Runtime.DTOs; -using Microsoft.Extensions.Logging; -using Microsoft.Extensions.Logging.Abstractions; -using OpenTelemetry.Resources; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.Common; - -public sealed class TestScope : OpenTelemetryScope -{ - public TestScope(string operationName, string activityName, AgentDetails agentDetails, SpanDetails? spanDetails = null) - : base(operationName, activityName, agentDetails, spanDetails) { } -} - -[TestClass] -public partial class ExportFormatterTests : ActivityTest -{ - private static readonly NullLogger NullLogger = NullLogger.Instance; - private static ExportFormatter CreateFormatter() => new ExportFormatter(NullLogger); - private static Resource CreateResource(Dictionary? attributes = null) - { - var builder = ResourceBuilder.CreateEmpty(); - if (attributes != null) - { - builder.AddAttributes(attributes.AsEnumerable()); - } - return builder.Build(); - } - - [TestMethod] - public void Format_EmptyActivities_ReturnsValidJson() - { - // Arrange - var activities = new List(); - var resource = CreateResource(new Dictionary { { "env", "test" } }); - var formatter = CreateFormatter(); - - // Act - var json = formatter.FormatMany(activities, resource); - - // Assert - json.Should().NotBeNullOrWhiteSpace(); - var doc = JsonDocument.Parse(json); - doc.RootElement.TryGetProperty("resourceSpans", out var resourceSpans).Should().BeTrue(); - resourceSpans.GetArrayLength().Should().Be(1); - var spans = resourceSpans[0].GetProperty("scopeSpans"); - spans.GetArrayLength().Should().Be(0); - } - - [TestMethod] - public void Format_SingleActivity_AllFieldsMapped() - { - // Arrange - var startTime = DateTime.UtcNow; - var duration = TimeSpan.FromMilliseconds(123); - var tags = new Dictionary - { - { "tag1", "value1" }, - { "tag2", 42 } - }; - var events = new List - { - new ActivityEvent("ev1", startTime, new ActivityTagsCollection { { "evtag", "evval" } }) - }; - var links = new List - { - new ActivityLink(new ActivityContext(ActivityTraceId.CreateRandom(), ActivitySpanId.CreateRandom(), ActivityTraceFlags.Recorded), new ActivityTagsCollection { { "linktag", "linkval" } }) - }; - - var activity = CreateActivity( - sourceName: "TestSource", - displayName: "span1", - kind: ActivityKind.Client, - startTimeUtc: startTime, - duration: duration, - tags: tags, - events: events, - links: links, - status: ActivityStatusCode.Error, - statusDescription: "fail" - ); - - var activities = new List { activity }; - var resource = CreateResource(new Dictionary { { "env", "test" } }); - var formatter = CreateFormatter(); - - // Act - var json = formatter.FormatMany(activities, resource); - - // Assert - var doc = JsonDocument.Parse(json); - var resourceSpans = doc.RootElement.GetProperty("resourceSpans"); - var scopeSpans = resourceSpans[0].GetProperty("scopeSpans"); - scopeSpans.GetArrayLength().Should().Be(1); - - var scope = scopeSpans[0].GetProperty("scope"); - scope.GetProperty("name").GetString().Should().Be("TestSource"); - - var spans = scopeSpans[0].GetProperty("spans"); - spans.GetArrayLength().Should().Be(1); - - var span = spans[0]; - span.GetProperty("name").GetString().Should().Be("span1"); - span.GetProperty("kind").GetInt32().Should().Be((int)ActivityKind.Client); - span.GetProperty("startTimeUnixNano").GetUInt64().Should().BeGreaterThan(0); - span.GetProperty("endTimeUnixNano").GetUInt64().Should().BeGreaterThan(span.GetProperty("startTimeUnixNano").GetUInt64()); - - var attributes = span.GetProperty("attributes"); - attributes.GetProperty("tag1").GetString().Should().Be("value1"); - attributes.GetProperty("tag2").GetInt32().Should().Be(42); - - var eventsJson = span.GetProperty("events"); - eventsJson.GetArrayLength().Should().Be(1); - var eventJson = eventsJson[0]; - eventJson.GetProperty("name").GetString().Should().Be("ev1"); - eventJson.GetProperty("attributes").GetProperty("evtag").GetString().Should().Be("evval"); - - var linksJson = span.GetProperty("links"); - linksJson.GetArrayLength().Should().Be(1); - var linkJson = linksJson[0]; - linkJson.GetProperty("attributes").GetProperty("linktag").GetString().Should().Be("linkval"); - - var status = span.GetProperty("status"); - status.GetProperty("code").GetInt32().Should().Be((int)ActivityStatusCode.Error); - status.GetProperty("message").GetString().Should().Be("fail"); - } - - [TestMethod] - public void Format_MultipleActivities_GroupedBySource() - { - // Arrange - var act1 = CreateActivity(sourceName: "SourceA", displayName: "spanA"); - var act2 = CreateActivity(sourceName: "SourceA", displayName: "spanB"); - var act3 = CreateActivity(sourceName: "SourceB", displayName: "spanC"); - - var activities = new List { act1, act2, act3 }; - var resource = CreateResource(new Dictionary { { "env", "test" } }); - var formatter = CreateFormatter(); - - // Act - var json = formatter.FormatMany(activities, resource); - - // Assert - var doc = JsonDocument.Parse(json); - var resourceSpans = doc.RootElement.GetProperty("resourceSpans"); - var scopeSpans = resourceSpans[0].GetProperty("scopeSpans"); - scopeSpans.GetArrayLength().Should().Be(2); - - var scopeA = scopeSpans[0].GetProperty("scope"); - var scopeB = scopeSpans[1].GetProperty("scope"); - - var names = new[] { scopeA.GetProperty("name").GetString(), scopeB.GetProperty("name").GetString() }; - names.Should().Contain("SourceA"); - names.Should().Contain("SourceB"); - - var spansA = scopeSpans[0].GetProperty("spans"); - var spansB = scopeSpans[1].GetProperty("spans"); - (spansA.GetArrayLength() + spansB.GetArrayLength()).Should().Be(3); - } - - [TestMethod] - public void Format_ResourceAttributes_AreMapped() - { - // Arrange - var act = CreateActivity(); - var resource = CreateResource(new Dictionary - { - { "custom1", "val1" }, - { "custom2", 123 } - }); - var formatter = CreateFormatter(); - - // Act - var json = formatter.FormatMany(new[] { act }, resource); - - // Assert - var doc = JsonDocument.Parse(json); - var resourceSpans = doc.RootElement.GetProperty("resourceSpans"); - var resourceObj = resourceSpans[0].GetProperty("resource"); - var attrs = resourceObj.GetProperty("attributes"); - attrs.GetProperty("custom1").GetString().Should().Be("val1"); - attrs.GetProperty("custom2").GetInt32().Should().Be(123); - } - - [TestMethod] - public void Format_ManuallySetParentContext_IsReflectedInExport() - { - // Arrange - var manualParentActivity = CreateActivity(); - var parentContext = manualParentActivity.Context; - var parentSpanId = manualParentActivity.SpanId.ToString(); - var activity = ListenForActivity(() => - { - using var toolScope = ExecuteToolScope.Start(new Request(), new ToolCallDetails("TestTool", "Input: 42"), Util.GetAgentDetails(), spanDetails: new SpanDetails(parentContext: parentContext)); - }); - - var resource = ResourceBuilder.CreateDefault().Build(); - var formatter = CreateFormatter(); - - // Act - var json = formatter.FormatMany(new[] { activity! }, resource); - - // Assert - using var doc = JsonDocument.Parse(json); - var resourceSpans = doc.RootElement.GetProperty("resourceSpans"); - var scopeSpans = resourceSpans[0].GetProperty("scopeSpans"); - var spans = scopeSpans[0].GetProperty("spans"); - var span = spans[0]; - var parentSpanIdJson = span.GetProperty("parentSpanId").GetString()?.ToLowerInvariant(); - parentSpanIdJson.Should().Be(parentSpanId.ToLowerInvariant()); - } - - [TestMethod] - public void Format_NullOrEmptyEventsAndLinks_AreOmitted() - { - // Arrange - var act = CreateActivity(); - var resource = CreateResource(); - var formatter = CreateFormatter(); - - // Act - var json = formatter.FormatMany(new[] { act }, resource); - - // Assert - var doc = JsonDocument.Parse(json); - var resourceSpans = doc.RootElement.GetProperty("resourceSpans"); - var scopeSpans = resourceSpans[0].GetProperty("scopeSpans"); - var span = scopeSpans[0].GetProperty("spans")[0]; - - span.TryGetProperty("events", out var eventsProp).Should().BeFalse(); - span.TryGetProperty("links", out var linksProp).Should().BeFalse(); - } - - [TestMethod] - public void Format_NullOrEmptyAttributes_AreOmitted() - { - // Arrange - var act = CreateActivity(tags: new Dictionary()); - var resource = CreateResource(); - var formatter = CreateFormatter(); - - // Act - var json = formatter.FormatMany(new[] { act }, resource); - - // Assert - var doc = JsonDocument.Parse(json); - var resourceSpans = doc.RootElement.GetProperty("resourceSpans"); - var scopeSpans = resourceSpans[0].GetProperty("scopeSpans"); - var span = scopeSpans[0].GetProperty("spans")[0]; - - span.TryGetProperty("attributes", out var attrsProp).Should().BeTrue(); - attrsProp.EnumerateObject().Should().BeEmpty(); - } - - [TestMethod] - public void FormatSingle_Activity_AllFieldsMapped() - { - // Arrange - var startTime = DateTime.UtcNow; - var duration = TimeSpan.FromMilliseconds(123); - var tags = new Dictionary - { - { "tag1", "value1" }, - { "tag2", 42 } - }; - var events = new List - { - new ActivityEvent("ev1", startTime, new ActivityTagsCollection { { "evtag", "evval" } }) - }; - var links = new List - { - new ActivityLink(new ActivityContext(ActivityTraceId.CreateRandom(), ActivitySpanId.CreateRandom(), ActivityTraceFlags.Recorded), new ActivityTagsCollection { { "linktag", "linkval" } }) - }; - - var activity = CreateActivity( - sourceName: "TestSource", - displayName: "span1", - kind: ActivityKind.Client, - startTimeUtc: startTime, - duration: duration, - tags: tags, - events: events, - links: links, - status: ActivityStatusCode.Error, - statusDescription: "fail" - ); - var resource = CreateResource(new Dictionary { { "env", "test" } }); - var formatter = CreateFormatter(); - - // Act - var json = formatter.FormatSingle(activity, resource); - - // Assert - var doc = JsonDocument.Parse(json); - var root = doc.RootElement; - var resourceSpan = root.GetProperty("resourceSpan"); - var scopeSpan = resourceSpan.GetProperty("scopeSpan"); - var scope = scopeSpan.GetProperty("scope"); - scope.GetProperty("name").GetString().Should().Be("TestSource"); - var span = scopeSpan.GetProperty("span"); - span.GetProperty("name").GetString().Should().Be("span1"); - span.GetProperty("kind").GetInt32().Should().Be((int)ActivityKind.Client); - span.GetProperty("startTimeUnixNano").GetUInt64().Should().BeGreaterThan(0); - span.GetProperty("endTimeUnixNano").GetUInt64().Should().BeGreaterThan(span.GetProperty("startTimeUnixNano").GetUInt64()); - var attributes = span.GetProperty("attributes"); - attributes.GetProperty("tag1").GetString().Should().Be("value1"); - attributes.GetProperty("tag2").GetInt32().Should().Be(42); - var eventsJson = span.GetProperty("events"); - var eventJson = eventsJson[0]; - eventJson.GetProperty("name").GetString().Should().Be("ev1"); - eventJson.GetProperty("attributes").GetProperty("evtag").GetString().Should().Be("evval"); - var linksJson = span.GetProperty("links"); - var linkJson = linksJson[0]; - linkJson.GetProperty("attributes").GetProperty("linktag").GetString().Should().Be("linkval"); - var status = span.GetProperty("status"); - status.GetProperty("code").GetInt32().Should().Be((int)ActivityStatusCode.Error); - status.GetProperty("message").GetString().Should().Be("fail"); - } - - [TestMethod] - public void FormatSingle_Activity_ResourceAttributesMapped() - { - // Arrange - var act = CreateActivity(); - var resource = CreateResource(new Dictionary - { - { "custom1", "val1" }, - { "custom2", 123 } - }); - var formatter = CreateFormatter(); - - // Act - var json = formatter.FormatSingle(act, resource); - - // Assert - var doc = JsonDocument.Parse(json); - var root = doc.RootElement; - var resourceSpan = root.GetProperty("resourceSpan"); - var resourceObj = resourceSpan.GetProperty("resource"); - var attrs = resourceObj.GetProperty("attributes"); - attrs.GetProperty("custom1").GetString().Should().Be("val1"); - attrs.GetProperty("custom2").GetInt32().Should().Be(123); - } - - [TestMethod] - public void FormatSingle_Activity_ParentSpanIdIsMapped() - { - // Arrange - var parentSpanId = ActivitySpanId.CreateRandom(); - var act = CreateActivity(parentSpanId: parentSpanId); - var resource = CreateResource(); - var formatter = CreateFormatter(); - - // Act - var json = formatter.FormatSingle(act, resource); - - // Assert - var doc = JsonDocument.Parse(json); - var root = doc.RootElement; - var resourceSpan = root.GetProperty("resourceSpan"); - var scopeSpan = resourceSpan.GetProperty("scopeSpan"); - var span = scopeSpan.GetProperty("span"); - var parentSpanIdJson = span.GetProperty("parentSpanId").GetString(); - parentSpanIdJson.Should().Be(parentSpanId.ToHexString().ToLowerInvariant()); - } - - [TestMethod] - public void FormatSingle_Activity_NullOrEmptyEventsAndLinksAreOmitted() - { - // Arrange - var act = CreateActivity(); - var resource = CreateResource(); - var formatter = CreateFormatter(); - - // Act - var json = formatter.FormatSingle(act, resource); - - // Assert - var doc = JsonDocument.Parse(json); - var root = doc.RootElement; - var resourceSpan = root.GetProperty("resourceSpan"); - var scopeSpan = resourceSpan.GetProperty("scopeSpan"); - var span = scopeSpan.GetProperty("span"); - span.TryGetProperty("events", out var eventsProp).Should().BeFalse(); - span.TryGetProperty("links", out var linksProp).Should().BeFalse(); - } - - [TestMethod] - public void FormatSingle_Activity_NullOrEmptyAttributesAreOmitted() - { - // Arrange - var act = CreateActivity(tags: new Dictionary()); - var resource = CreateResource(); - var formatter = CreateFormatter(); - - // Act - var json = formatter.FormatSingle(act, resource); - - // Assert - var doc = JsonDocument.Parse(json); - var root = doc.RootElement; - var resourceSpan = root.GetProperty("resourceSpan"); - var scopeSpan = resourceSpan.GetProperty("scopeSpan"); - var span = scopeSpan.GetProperty("span"); - span.TryGetProperty("attributes", out var attrsProp).Should().BeTrue(); - attrsProp.EnumerateObject().Should().BeEmpty(); - } - - private static ulong ToUnixNanos(DateTimeOffset dto) - { - var epoch = new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc); - return (ulong)((dto.UtcDateTime - epoch).Ticks * 100); - } - - [TestMethod] - public void FormatLogData_WithAllFields_ProducesExpectedJson() - { - // Arrange - var start = DateTimeOffset.UtcNow.AddMinutes(-1); - var end = DateTimeOffset.UtcNow; - var spanId = "span-123"; - var parentSpanId = "parent-456"; - var data = new InvokeAgentData( - new Dictionary - { - { "attr1", "value1" }, - { "attr2", 42 } - }, - start, - end, - spanId, - parentSpanId, - spanKind: SpanKindConstants.Client); - var formatter = CreateFormatter(); - - // Act - var json = formatter.FormatLogData(data.ToDictionary()); - - // Assert - json.Should().NotBeNullOrWhiteSpace(); - using var doc = JsonDocument.Parse(json); - var root = doc.RootElement; - - root.GetProperty("Name").GetString().Should().Be("InvokeAgent"); - root.GetProperty("SpanId").GetString().Should().Be(spanId); - root.GetProperty("ParentSpanId").GetString().Should().Be(parentSpanId); - root.GetProperty("Kind").GetString().Should().Be(SpanKindConstants.Client); - - var attrs = root.GetProperty("Attributes"); - attrs.GetProperty("attr1").GetString().Should().Be("value1"); - attrs.GetProperty("attr2").GetInt32().Should().Be(42); - - var startNs = root.GetProperty("StartTimeUnixNano").GetUInt64(); - var endNs = root.GetProperty("EndTimeUnixNano").GetUInt64(); - startNs.Should().Be(ToUnixNanos(start)); - endNs.Should().Be(ToUnixNanos(end)); - endNs.Should().BeGreaterThan(startNs); - - // Duration is not part of the serialized payload - root.TryGetProperty("Duration", out _).Should().BeFalse(); - } - - [TestMethod] - public void FormatLogData_WithMissingOptionalFields_ProducesDefaults() - { - // Arrange - var explicitSpanId = "explicit-span"; - var data = new InvokeAgentData( - new Dictionary { { "key", "val" } }, - startTime: null, - endTime: null, - spanId: explicitSpanId, - parentSpanId: null); - var formatter = CreateFormatter(); - - // Act - var json = formatter.FormatLogData(data.ToDictionary()); - - // Assert - json.Should().NotBeNullOrWhiteSpace(); - using var doc = JsonDocument.Parse(json); - var root = doc.RootElement; - - root.GetProperty("Name").GetString().Should().Be("InvokeAgent"); - root.GetProperty("SpanId").GetString().Should().Be(explicitSpanId); - root.GetProperty("StartTimeUnixNano").GetUInt64().Should().Be(0); - root.GetProperty("EndTimeUnixNano").GetUInt64().Should().Be(0); - - // ParentSpanId should be omitted due to null (ignore when writing null) - root.TryGetProperty("ParentSpanId", out _).Should().BeFalse(); - - // Kind defaults to Client when SpanKind is null - root.GetProperty("Kind").GetString().Should().Be(SpanKindConstants.Client); - - var attrs = root.GetProperty("Attributes"); - attrs.GetProperty("key").GetString().Should().Be("val"); - } - -#region ExportFormatter FormatMany Truncation Tests - -private class ListLogger : ILogger -{ - private readonly List _logs; - public ListLogger(List logs) => _logs = logs; - public IDisposable BeginScope(TState state) where TState : notnull => null!; - public bool IsEnabled(LogLevel logLevel) => true; - public void Log(LogLevel logLevel, EventId eventId, TState state, Exception? exception, Func formatter) - { - _logs.Add(formatter(state, exception)); - } -} - -[TestMethod] -public void FormatMany_DoesNothing_WhenUnderLimit() -{ - // Arrange - using var activity = CreateActivity("tenant-1", "agent-1"); - activity.SetTag("gen_ai.tool.call.arguments", new string('a', 1024)); // 1KB - var resource = ResourceBuilder.CreateEmpty().Build(); - var logs = new List(); - var formatter = new ExportFormatter(new ListLogger(logs)); - - // Act - var json = formatter.FormatMany(new[] { activity }, resource); - - // Assert - var doc = JsonDocument.Parse(json); - var resourceSpans = doc.RootElement.GetProperty("resourceSpans"); - var scopeSpans = resourceSpans[0].GetProperty("scopeSpans"); - var span = scopeSpans[0].GetProperty("spans")[0]; - span.GetProperty("attributes").GetProperty("gen_ai.tool.call.arguments").GetString().Should().NotBe("TRUNCATED"); - logs.Should().NotContain(l => l.Contains("Truncated")); -} - -[TestMethod] -public void FormatMany_TruncatesSingleLargeKey() -{ - // Arrange - using var activity = CreateActivity("tenant-1", "agent-2"); - activity.SetTag("gen_ai.tool.call.arguments", new string('b', 300 * 1024)); // 300KB - var resource = ResourceBuilder.CreateEmpty().Build(); - var logs = new List(); - var formatter = new ExportFormatter(new ListLogger(logs)); - - // Act - var json = formatter.FormatMany(new[] { activity }, resource); - - // Assert - var doc = JsonDocument.Parse(json); - var resourceSpans = doc.RootElement.GetProperty("resourceSpans"); - var scopeSpans = resourceSpans[0].GetProperty("scopeSpans"); - var span = scopeSpans[0].GetProperty("spans")[0]; - span.GetProperty("attributes").GetProperty("gen_ai.tool.call.arguments").GetString().Should().Be("TRUNCATED"); - logs.Should().Contain(l => l.Contains("Key 'gen_ai.tool.call.arguments' size = ")); - logs.Should().Contain(l => l.Contains("Truncated 'gen_ai.tool.call.arguments'")); -} - -[TestMethod] -public void FormatMany_TruncatesMultipleKeys_LargestFirst() -{ - // Arrange - using var activity = CreateActivity("tenant-1", "agent-1"); - activity.SetTag("gen_ai.tool.call.arguments", new string('c', 200 * 1024)); - activity.SetTag("gen_ai.tool.call.result", new string('d', 100 * 1024)); - var resource = ResourceBuilder.CreateEmpty().Build(); - var logs = new List(); - var formatter = new ExportFormatter(new ListLogger(logs)); - - // Act - var json = formatter.FormatMany(new[] { activity }, resource); - - // Assert - var doc = JsonDocument.Parse(json); - var resourceSpans = doc.RootElement.GetProperty("resourceSpans"); - var scopeSpans = resourceSpans[0].GetProperty("scopeSpans"); - var span = scopeSpans[0].GetProperty("spans")[0]; - var attr = span.GetProperty("attributes"); - attr.GetProperty("gen_ai.tool.call.arguments").GetString().Should().Be("TRUNCATED"); - logs.Should().Contain(l => l.Contains("Truncated 'gen_ai.tool.call.arguments'")); - logs.Should().Contain(l => l.Contains("Key 'gen_ai.tool.call.arguments' size = ")); - logs.Should().Contain(l => l.Contains("Key 'gen_ai.tool.call.result' size = ")); -} - -[TestMethod] -public void FormatMany_LogsAllKeySizes() -{ - // Arrange - using var activity = CreateActivity("tenant-1", "agent-1"); - activity.SetTag("gen_ai.tool.call.arguments", new string('x', 100 * 1024)); - activity.SetTag("gen_ai.tool.call.result", new string('y', 125 * 1024)); - activity.SetTag("gen_ai.input.messages", new string('z', 75 * 1024)); - activity.SetTag("gen_ai.agent.invocation_input", new string('z', 0)); - activity.SetTag("gen_ai.agent.invocation_output", new string('z', 0)); - activity.SetTag("gen_ai.output.messages", new string('z', 0)); - var resource = ResourceBuilder.CreateEmpty().Build(); - var logs = new List(); - var formatter = new ExportFormatter(new ListLogger(logs)); - - // Act - formatter.FormatMany(new[] { activity }, resource); - - // Assert - logs.Should().Contain(l => l.Contains("Key 'gen_ai.tool.call.arguments' size = 100")); - logs.Should().Contain(l => l.Contains("Key 'gen_ai.tool.call.result' size = 125")); - logs.Should().Contain(l => l.Contains("Key 'gen_ai.input.messages' size = 75")); - logs.Should().Contain(l => l.Contains("Key 'gen_ai.agent.invocation_input' size = 0")); - logs.Should().Contain(l => l.Contains("Key 'gen_ai.agent.invocation_output' size = 0")); - logs.Should().Contain(l => l.Contains("Key 'gen_ai.output.messages' size = 0")); -} - -#endregion -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Common/PowerPlatformApiDiscoveryTest.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Common/PowerPlatformApiDiscoveryTest.cs deleted file mode 100644 index c349b716..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Common/PowerPlatformApiDiscoveryTest.cs +++ /dev/null @@ -1,125 +0,0 @@ -using Microsoft.Agents.A365.Observability.Runtime.Common; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.Common; - - -[TestClass] -public class PowerPlatformApiDiscoveryTests -{ - [TestMethod] - public void GetTokenAudience_Mapping_IsCorrect() - { - var expected = new Dictionary - { - ["firstrelease"] = "https://api.powerplatform.com", - ["prod"] = "https://api.powerplatform.com", - ["gov"] = "https://api.gov.powerplatform.microsoft.us", - ["high"] = "https://api.high.powerplatform.microsoft.us", - ["dod"] = "https://api.appsplatform.us", - ["mooncake"] = "https://api.powerplatform.partner.microsoftonline.cn", - ["ex"] = "https://api.powerplatform.eaglex.ic.gov", - ["rx"] = "https://api.powerplatform.microsoft.scloud", - }; - - foreach (var kv in expected) - { - var disc = new PowerPlatformApiDiscovery(kv.Key); - Assert.AreEqual(kv.Value, disc.GetTokenAudience()); - } - } - - [TestMethod] - public void GetTokenEndpointHost_Mapping_IsCorrect() - { - var expected = new Dictionary - { - ["firstrelease"] = "api.powerplatform.com", - ["prod"] = "api.powerplatform.com", - ["gov"] = "api.gov.powerplatform.microsoft.us", - ["high"] = "api.high.powerplatform.microsoft.us", - ["dod"] = "api.appsplatform.us", - ["mooncake"] = "api.powerplatform.partner.microsoftonline.cn", - ["ex"] = "api.powerplatform.eaglex.ic.gov", - ["rx"] = "api.powerplatform.microsoft.scloud", - }; - - foreach (var kv in expected) - { - var disc = new PowerPlatformApiDiscovery(kv.Key); - Assert.AreEqual(kv.Value, disc.GetTokenEndpointHost()); - } - } - - [TestMethod] - public void GetTenantEndpoint_GeneratesExpectedValues() - { - var tenantId = "e3064512-cc6d-4703-be71-a2ecaecaa98a"; - var expected = new Dictionary - { - ["firstrelease"] = "e3064512cc6d4703be71a2ecaecaa9.8a.tenant.api.powerplatform.com", - ["prod"] = "e3064512cc6d4703be71a2ecaecaa9.8a.tenant.api.powerplatform.com", - ["gov"] = "e3064512cc6d4703be71a2ecaecaa98.a.tenant.api.gov.powerplatform.microsoft.us", - ["high"] = "e3064512cc6d4703be71a2ecaecaa98.a.tenant.api.high.powerplatform.microsoft.us", - ["dod"] = "e3064512cc6d4703be71a2ecaecaa98.a.tenant.api.appsplatform.us", - ["mooncake"] = "e3064512cc6d4703be71a2ecaecaa98.a.tenant.api.powerplatform.partner.microsoftonline.cn", - ["ex"] = "e3064512cc6d4703be71a2ecaecaa98.a.tenant.api.powerplatform.eaglex.ic.gov", - ["rx"] = "e3064512cc6d4703be71a2ecaecaa98.a.tenant.api.powerplatform.microsoft.scloud", - }; - - foreach (var kv in expected) - { - var disc = new PowerPlatformApiDiscovery(kv.Key); - Assert.AreEqual(kv.Value, disc.GetTenantEndpoint(tenantId)); - } - } - - [TestMethod] - public void GetTenantIslandClusterEndpoint_GeneratesExpectedValues() - { - var tenantId = "e3064512-cc6d-4703-be71-a2ecaecaa98a"; - var expected = new Dictionary - { - ["firstrelease"] = "il-e3064512cc6d4703be71a2ecaecaa9.8a.tenant.api.powerplatform.com", - ["prod"] = "il-e3064512cc6d4703be71a2ecaecaa9.8a.tenant.api.powerplatform.com", - ["gov"] = "il-e3064512cc6d4703be71a2ecaecaa98.a.tenant.api.gov.powerplatform.microsoft.us", - ["high"] = "il-e3064512cc6d4703be71a2ecaecaa98.a.tenant.api.high.powerplatform.microsoft.us", - ["dod"] = "il-e3064512cc6d4703be71a2ecaecaa98.a.tenant.api.appsplatform.us", - ["mooncake"] = "il-e3064512cc6d4703be71a2ecaecaa98.a.tenant.api.powerplatform.partner.microsoftonline.cn", - ["ex"] = "il-e3064512cc6d4703be71a2ecaecaa98.a.tenant.api.powerplatform.eaglex.ic.gov", - ["rx"] = "il-e3064512cc6d4703be71a2ecaecaa98.a.tenant.api.powerplatform.microsoft.scloud", - }; - - foreach (var kv in expected) - { - var disc = new PowerPlatformApiDiscovery(kv.Key); - Assert.AreEqual(kv.Value, disc.GetTenantIslandClusterEndpoint(tenantId)); - } - } - - [TestMethod] - public void GetTenantEndpoint_InvalidCharacters_ThrowsExactMessage() - { - var disc = new PowerPlatformApiDiscovery("prod"); - var ex = Assert.Throws(() => disc.GetTenantEndpoint("invalid?")); - Assert.Contains("invalid host name characters", ex.Message); - } - - [TestMethod] - public void GetTenantEndpoint_InsufficientLength_ThrowsExactMessage() - { - var disc = new PowerPlatformApiDiscovery("prod"); - var ex1 = Assert.Throws(() => disc.GetTenantEndpoint("a")); - Assert.Contains("must be at least 3 characters in length", ex1.Message); - - var ex2 = Assert.Throws(() => disc.GetTenantEndpoint("a-")); - Assert.Contains("must be at least 3 characters in length", ex2.Message); - - var discProd = new PowerPlatformApiDiscovery("prod"); - var ex3 = Assert.Throws(() => discProd.GetTenantEndpoint("aa")); - Assert.Contains("must be at least 3 characters in length", ex3.Message); - - var ex4 = Assert.Throws(() => discProd.GetTenantEndpoint("a-a")); - Assert.Contains("must be at least 3 characters in length", ex4.Message); - } -} - diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/BaseDataTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/BaseDataTests.cs deleted file mode 100644 index fb9c5331..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/BaseDataTests.cs +++ /dev/null @@ -1,234 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.DTOs; -using System.Diagnostics; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.DTOs -{ - [TestClass] - public class BaseDataTests - { - private sealed class TestData : BaseData - { - public TestData( - IDictionary? attributes = null, - DateTimeOffset? startTime = null, - DateTimeOffset? endTime = null, - string? spanId = null, - string? parentSpanId = null, - string? spanKind = null, - string? traceId = null) : base(attributes, startTime, endTime, spanId, parentSpanId, spanKind, traceId) { } - public override string Name => "Test"; - } - - [TestMethod] - public void Constructor_WithAttributes_AssignsDictionary() - { - var attrs = new Dictionary { { "k", "v" } }; - var data = new TestData(attrs); - data.Attributes.Should().ContainKey("k").WhoseValue.Should().Be("v"); - } - - [TestMethod] - public void SpanId_Generated_WhenNotProvided() - { - var data = new TestData(); - data.SpanId.Should().NotBeNullOrEmpty(); - ActivitySpanId.CreateFromString(data.SpanId).ToString().Should().Be(data.SpanId); - } - - [TestMethod] - public void SpanId_UsesProvidedValue() - { - var id = "custom-span"; - var data = new TestData(spanId: id); - data.SpanId.Should().Be(id); - } - - [TestMethod] - public void ParentSpanId_DefaultsToNull() - { - var data = new TestData(); - data.ParentSpanId.Should().BeNull(); - } - - [TestMethod] - public void ParentSpanId_UsesProvidedValue() - { - var parent = "parent-span"; - var data = new TestData(parentSpanId: parent); - data.ParentSpanId.Should().Be(parent); - } - - [TestMethod] - public void Duration_BothTimesSet_ComputesPositive() - { - var start = DateTimeOffset.UtcNow.AddMinutes(-5); - var end = DateTimeOffset.UtcNow; - var data = new TestData(startTime: start, endTime: end); - data.Duration.Should().BeCloseTo(end - start, TimeSpan.FromMilliseconds(50)); - } - - [TestMethod] - public void Duration_OnlyStartTime_Zero() - { - var start = DateTimeOffset.UtcNow; - var data = new TestData(startTime: start); - data.Duration.Should().Be(TimeSpan.Zero); - } - - [TestMethod] - public void Duration_OnlyEndTime_Zero() - { - var end = DateTimeOffset.UtcNow; - var data = new TestData(endTime: end); - data.Duration.Should().Be(TimeSpan.Zero); - } - - [TestMethod] - public void Duration_NoTimes_Zero() - { - var data = new TestData(); - data.Duration.Should().Be(TimeSpan.Zero); - } - - [TestMethod] - public void Duration_EndBeforeStart_Negative() - { - var start = DateTimeOffset.UtcNow; - var end = start.AddMinutes(-2); - var data = new TestData(startTime: start, endTime: end); - data.Duration.Should().BeNegative(); - data.Duration.Should().BeCloseTo(TimeSpan.FromMinutes(-2), TimeSpan.FromMilliseconds(50)); - } - - [TestMethod] - public void MultipleInstances_GenerateUniqueSpanIds() - { - var d1 = new TestData(); - var d2 = new TestData(); - var d3 = new TestData(); - d1.SpanId.Should().NotBe(d2.SpanId); - d1.SpanId.Should().NotBe(d3.SpanId); - d2.SpanId.Should().NotBe(d3.SpanId); - } - - [TestMethod] - public void Attributes_PreserveTypes() - { - var attrs = new Dictionary - { - { "string", "s" }, - { "int", 1 }, - { "double", 2.5 }, - { "bool", true }, - { "null", null } - }; - var data = new TestData(attrs); - data.Attributes["string"].Should().BeOfType(); - data.Attributes["int"].Should().BeOfType(); - data.Attributes["double"].Should().BeOfType(); - data.Attributes["bool"].Should().BeOfType(); - data.Attributes["null"].Should().BeNull(); - } - - [TestMethod] - public void SpanKind_DefaultsToNull() - { - var data = new TestData(); - data.SpanKind.Should().BeNull(); - } - - [TestMethod] - public void SpanKind_UsesProvidedValue() - { - var data = new TestData(spanKind: SpanKindConstants.Client); - data.SpanKind.Should().Be(SpanKindConstants.Client); - } - - [TestMethod] - public void SpanKind_IncludedInToDictionary() - { - var data = new TestData(spanKind: SpanKindConstants.Server); - var dict = data.ToDictionary(); - dict.Should().ContainKey("SpanKind").WhoseValue.Should().Be(SpanKindConstants.Server); - } - - [TestMethod] - public void SpanKind_NullInToDictionary_WhenNotProvided() - { - var data = new TestData(); - var dict = data.ToDictionary(); - dict.Should().ContainKey("SpanKind").WhoseValue.Should().BeNull(); - } - - [TestMethod] - public void TraceId_DefaultsToNull() - { - var data = new TestData(); - data.TraceId.Should().BeNull(); - } - - [TestMethod] - public void TraceId_UsesProvidedValue() - { - var traceId = "1234567890abcdef1234567890abcdef"; - var data = new TestData(traceId: traceId); - data.TraceId.Should().Be(traceId); - } - - [TestMethod] - public void TraceId_IncludedInToDictionary() - { - var traceId = "abcdef1234567890abcdef1234567890"; - var data = new TestData(traceId: traceId); - var dict = data.ToDictionary(); - dict.Should().ContainKey("TraceId").WhoseValue.Should().Be(traceId); - } - - [TestMethod] - public void TraceId_NullInToDictionary_WhenNotProvided() - { - var data = new TestData(); - var dict = data.ToDictionary(); - dict.Should().ContainKey("TraceId").WhoseValue.Should().BeNull(); - } - - [TestMethod] - public void StatusCode_DefaultsToUnset() - { - var data = new TestData(); - data.StatusCode.Should().Be(SpanStatusCode.Unset); - data.StatusMessage.Should().BeNull(); - } - - [TestMethod] - public void Status_DefaultInToDictionary_IsUnsetWithEmptyMessage() - { - var data = new TestData(); - var dict = data.ToDictionary(); - - dict.Should().ContainKey("Status"); - var status = dict["Status"].Should().BeAssignableTo>().Subject; - status["code"].Should().Be(0); - status["message"].Should().Be(""); - } - - [TestMethod] - public void Status_ErrorInToDictionary_EmitsCodeAndMessage() - { - var data = new TestData - { - StatusCode = SpanStatusCode.Error, - StatusMessage = "kaboom" - }; - var dict = data.ToDictionary(); - - var status = dict["Status"].Should().BeAssignableTo>().Subject; - status["code"].Should().Be(2); - status["message"].Should().Be("kaboom"); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/Builders/BaseDataBuilderTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/Builders/BaseDataBuilderTests.cs deleted file mode 100644 index e7a66560..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/Builders/BaseDataBuilderTests.cs +++ /dev/null @@ -1,149 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.DTOs; -using Microsoft.Agents.A365.Observability.Runtime.DTOs.Builders; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.DTOs.Builders -{ - [TestClass] - public class BaseDataBuilderTests - { - private sealed class TestBuilder : BaseDataBuilder - { - public static IDictionary BuildAll( - AgentDetails? agent = null, - Uri? endpoint = null, - Request? request = null, - CallerDetails? caller = null, - AgentDetails? callerAgent = null, - string[]? input = null, - string[]? output = null) - { - var dict = new Dictionary(); - if (agent != null) AddAgentDetails(dict, agent); - if (endpoint != null) AddEndpointDetails(dict, endpoint); - if (request != null) AddRequestDetails(dict, request); - if (caller != null) AddCallerDetails(dict, caller); - if (callerAgent != null) AddCallerAgentDetails(dict, callerAgent); - AddInputMessagesAttributes(dict, input); - AddOutputMessagesAttributes(dict, output); - return dict; - } - } - - [TestMethod] - public void AddAgentDetails_PopulatesExpectedKeys() - { - var agent = new AgentDetails("agent-1", "AgentName", "Desc", agenticUserId: "auid", agenticUserEmail: "upn", agentBlueprintId: "bp", tenantId: "tenant-x", agentPlatformId: "platform-123"); - var dict = TestBuilder.BuildAll(agent: agent); - dict.Should().ContainKey(OpenTelemetryConstants.GenAiAgentIdKey); - dict.Should().ContainKey(OpenTelemetryConstants.GenAiAgentNameKey); - dict.Should().ContainKey(OpenTelemetryConstants.GenAiAgentDescriptionKey); - dict.Should().ContainKey(OpenTelemetryConstants.AgentAUIDKey); - dict.Should().ContainKey(OpenTelemetryConstants.AgentEmailKey); - dict.Should().ContainKey(OpenTelemetryConstants.AgentBlueprintIdKey); - dict.Should().ContainKey(OpenTelemetryConstants.AgentPlatformIdKey); - dict.Should().ContainKey(OpenTelemetryConstants.TenantIdKey); - } - - [TestMethod] - public void AddAgentDetails_AddsTenantId() - { - var agent = new AgentDetails("agent-1", tenantId: Guid.NewGuid().ToString()); - var dict = TestBuilder.BuildAll(agent: agent); - dict.Should().ContainKey(OpenTelemetryConstants.TenantIdKey); - } - - [TestMethod] - public void AddEndpointDetails_AddsHostAndPort() - { - var endpoint = new Uri("https://example.com:8080"); - var dict = TestBuilder.BuildAll(endpoint: endpoint); - dict.Should().ContainKey(OpenTelemetryConstants.ServerAddressKey); - dict.Should().ContainKey(OpenTelemetryConstants.ServerPortKey); - } - - [TestMethod] - public void AddEndpointDetails_StandardPort_OmitsPort() - { - var endpoint = new Uri("https://example.com:443"); - var dict = TestBuilder.BuildAll(endpoint: endpoint); - dict.Should().ContainKey(OpenTelemetryConstants.ServerAddressKey); - dict.Should().NotContainKey(OpenTelemetryConstants.ServerPortKey); - } - - [TestMethod] - public void AddRequestDetails_PopulatesRequestKeys() - { - var request = new Request("content", sessionId: "session", channel: new Channel(name: "src-name", link: "src-desc")); - var dict = TestBuilder.BuildAll(request: request); - dict.Should().ContainKey(OpenTelemetryConstants.ChannelLinkKey); - dict.Should().ContainKey(OpenTelemetryConstants.ChannelNameKey); - } - - [TestMethod] - public void AddCallerDetails_PopulatesCallerKeys() - { - var caller = new CallerDetails( - userDetails: new UserDetails(userId: "caller-1", userName: "Caller Name", userEmail: "caller@upn")); - var dict = TestBuilder.BuildAll(caller: caller); - dict.Should().ContainKey(OpenTelemetryConstants.UserIdKey); - dict.Should().ContainKey(OpenTelemetryConstants.UserEmailKey); - dict.Should().ContainKey(OpenTelemetryConstants.UserNameKey); - } - - [TestMethod] - public void AddCallerAgentDetails_PopulatesCallerAgentKeys() - { - var callerAgent = new AgentDetails("c-agent", "CallerAgent", agenticUserId: "ca-uid", agenticUserEmail: "ca-upn", agentBlueprintId: "ca-bp", tenantId: "ca-tenant", agentPlatformId: "ca-platform"); - var dict = TestBuilder.BuildAll(callerAgent: callerAgent); - dict.Should().ContainKey(OpenTelemetryConstants.CallerAgentIdKey); - dict.Should().ContainKey(OpenTelemetryConstants.CallerAgentNameKey); - dict.Should().ContainKey(OpenTelemetryConstants.CallerAgentBlueprintIdKey); - dict.Should().ContainKey(OpenTelemetryConstants.CallerAgentAUIDKey); - dict.Should().ContainKey(OpenTelemetryConstants.CallerAgentEmailKey); - dict.Should().ContainKey(OpenTelemetryConstants.CallerAgentPlatformIdKey); - } - - [TestMethod] - public void AddInputMessagesAttributes_JoinsMessages() - { - var dict = TestBuilder.BuildAll(input: new[] { "one", "two" }); - dict.Should().ContainKey(OpenTelemetryConstants.GenAiInputMessagesKey); - dict[OpenTelemetryConstants.GenAiInputMessagesKey]!.ToString()!.Should().Contain("one").And.Contain("two"); - } - - [TestMethod] - public void AddOutputMessagesAttributes_JoinsMessages() - { - var dict = TestBuilder.BuildAll(output: new[] { "out1", "out2" }); - dict.Should().ContainKey(OpenTelemetryConstants.GenAiOutputMessagesKey); - dict[OpenTelemetryConstants.GenAiOutputMessagesKey]!.ToString()!.Should().Contain("out1").And.Contain("out2"); - } - - [TestMethod] - public void AddInputMessagesAttributes_EmptyArray_OmitsKey() - { - var dict = TestBuilder.BuildAll(input: Array.Empty()); - dict.Should().NotContainKey(OpenTelemetryConstants.GenAiInputMessagesKey); - } - - [TestMethod] - public void AddOutputMessagesAttributes_EmptyArray_OmitsKey() - { - var dict = TestBuilder.BuildAll(output: Array.Empty()); - dict.Should().NotContainKey(OpenTelemetryConstants.GenAiOutputMessagesKey); - } - - [TestMethod] - public void AddIfNotNull_DoesNotAddNullValues() - { - var dict = TestBuilder.BuildAll(); - dict.Should().BeEmpty(); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/Builders/ExecuteInferenceDataBuilderTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/Builders/ExecuteInferenceDataBuilderTests.cs deleted file mode 100644 index 4e0a0287..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/Builders/ExecuteInferenceDataBuilderTests.cs +++ /dev/null @@ -1,288 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.DTOs.Builders; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.DTOs.Builders -{ - [TestClass] - public class ExecuteInferenceDataBuilderTests - { - [TestMethod] - public void Build_WithMinimalParameters_SetsBasicAttributes() - { - // Arrange - var details = new InferenceCallDetails(InferenceOperationType.Chat, "gpt-4o", "openai"); - var agent = new AgentDetails("agent-1", "AgentOne"); - var conversationId = "conv-min"; - - // Act - var data = ExecuteInferenceDataBuilder.Build(details, agent, conversationId); - - // Assert - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiOperationNameKey); - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiRequestModelKey).WhoseValue.Should().Be("gpt-4o"); - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiProviderNameKey).WhoseValue.Should().Be("openai"); - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiConversationIdKey).WhoseValue.Should().Be(conversationId); - } - - [TestMethod] - public void Build_WithChannel_IncludesChannelAttributes() - { - // Arrange - var details = new InferenceCallDetails(InferenceOperationType.Chat, "gpt-4o", "openai"); - var agent = new AgentDetails("agent-src"); - var conversationId = "conv-src-inf"; - var source = new Channel(name: "ChannelInf", link: "https://channel/inf"); - - // Act - var data = ExecuteInferenceDataBuilder.Build(details, agent, conversationId, channel: source); - - // Assert - data.Attributes.Should().ContainKey(OpenTelemetryConstants.ChannelNameKey).WhoseValue.Should().Be("ChannelInf"); - data.Attributes.Should().ContainKey(OpenTelemetryConstants.ChannelLinkKey).WhoseValue.Should().Be("https://channel/inf"); - } - - [TestMethod] - public void Build_WithTokensAndFinishReasons_IncludesUsageAndReasons() - { - // Arrange - var details = new InferenceCallDetails(InferenceOperationType.Chat, "gpt-4o", "openai", 10, 20, new[]{"stop"}, "resp-1"); - var agent = new AgentDetails("agent-2"); - var conversationId = "conv-tokens"; - - // Act - var data = ExecuteInferenceDataBuilder.Build(details, agent, conversationId); - - // Assert - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiUsageInputTokensKey).WhoseValue.Should().Be("10"); - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiUsageOutputTokensKey).WhoseValue.Should().Be("20"); - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiResponseFinishReasonsKey).WhoseValue.Should().Be("stop"); - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiConversationIdKey).WhoseValue.Should().Be(conversationId); - } - - [TestMethod] - public void Build_WithMessages_IncludesInputAndOutput() - { - // Arrange - var details = new InferenceCallDetails(InferenceOperationType.Chat, "gpt-4o", "openai"); - var agent = new AgentDetails("agent-3"); - var input = new[]{"Hello"}; - var output = new[]{"Hi"}; - var conversationId = "conv-msg"; - - // Act - var data = ExecuteInferenceDataBuilder.Build(details, agent, conversationId, inputMessages: input, outputMessages: output); - - // Assert - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiInputMessagesKey).WhoseValue!.ToString()!.Should().Contain("Hello"); - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiOutputMessagesKey).WhoseValue!.ToString()!.Should().Contain("Hi"); - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiConversationIdKey).WhoseValue.Should().Be(conversationId); - } - - [TestMethod] - public void Build_WithConversationId_IncludesConversationId() - { - // Arrange - var details = new InferenceCallDetails(InferenceOperationType.Chat, "gpt-4o", "openai"); - var agent = new AgentDetails("agent-4"); - var conversationId = "conv-456"; - - // Act - var data = ExecuteInferenceDataBuilder.Build(details, agent, conversationId); - - // Assert - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiConversationIdKey).WhoseValue.Should().Be("conv-456"); - } - - [TestMethod] - public void Build_WithNullOptionalParameters_OmitsThoseAttributes() - { - // Arrange - var details = new InferenceCallDetails(InferenceOperationType.Chat, "gpt-4o", "openai"); - var agent = new AgentDetails("agent-5"); - var conversationId = "conv-null"; - - // Act - var data = ExecuteInferenceDataBuilder.Build(details, agent, conversationId); - - // Assert - data.Attributes.Should().NotContainKey(OpenTelemetryConstants.GenAiUsageInputTokensKey); - data.Attributes.Should().NotContainKey(OpenTelemetryConstants.GenAiUsageOutputTokensKey); - data.Attributes.Should().NotContainKey(OpenTelemetryConstants.GenAiResponseFinishReasonsKey); - } - - [TestMethod] - public void Build_SetsTimingInformation_WhenProvided() - { - // Arrange - var details = new InferenceCallDetails(InferenceOperationType.Chat, "gpt-4o", "openai"); - var agent = new AgentDetails("agent-6"); - var start = DateTimeOffset.UtcNow.AddMinutes(-2); - var end = DateTimeOffset.UtcNow; - var conversationId = "conv-time"; - - // Act - var data = ExecuteInferenceDataBuilder.Build(details, agent, conversationId, startTime: start, endTime: end); - - // Assert - data.StartTime.Should().Be(start); - data.EndTime.Should().Be(end); - data.Duration.Should().BeCloseTo(TimeSpan.FromMinutes(2), TimeSpan.FromMilliseconds(100)); - } - - [TestMethod] - public void Build_SetsSpanIds_WhenProvided() - { - // Arrange - var details = new InferenceCallDetails(InferenceOperationType.Chat, "gpt-4o", "openai"); - var agent = new AgentDetails("agent-7"); - var spanId = "span-inf"; - var parentSpanId = "parent-inf"; - var conversationId = "conv-span"; - - // Act - var data = ExecuteInferenceDataBuilder.Build(details, agent, conversationId, spanId: spanId, parentSpanId: parentSpanId); - - // Assert - data.SpanId.Should().Be(spanId); - data.ParentSpanId.Should().Be(parentSpanId); - } - - [TestMethod] - public void Build_WithAllParameters_SetsAllExpectedAttributes() - { - // Arrange - var details = new InferenceCallDetails(InferenceOperationType.Chat, "gpt-4o", "openai", 33, 44, new[]{"length","stop"}, "resp-all"); - var agent = new AgentDetails("agent-8", "AgentEight", "Desc", agenticUserId: "auid8", agenticUserEmail: "upn8@example.com", agentBlueprintId: "bp-8"); - var conversationId = "conv-all"; - var input = new[]{"Hello"}; - var output = new[]{"World"}; - var start = DateTimeOffset.UtcNow.AddSeconds(-10); - var end = DateTimeOffset.UtcNow; - var spanId = "span-all-inf"; - var parentSpanId = "parent-all-inf"; - var thoughtProcess = "First, I analyzed the request. Then, I formulated a response."; - var callerDetails = new CallerDetails(userDetails: new UserDetails(userId: "caller-inf-123", userName: "Caller Inf Name", userEmail: "callerinf@example.com", userClientIP: System.Net.IPAddress.Parse("192.168.1.100"))); - - // Act - var data = ExecuteInferenceDataBuilder.Build( - details, - agent, - conversationId, - inputMessages: input, - outputMessages: output, - startTime: start, - endTime: end, - spanId: spanId, - parentSpanId: parentSpanId, - thoughtProcess: thoughtProcess, - callerDetails: callerDetails); - - // Assert - var attrs = data.Attributes; - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiUsageInputTokensKey); - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiUsageOutputTokensKey); - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiResponseFinishReasonsKey); - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiConversationIdKey); - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiInputMessagesKey); - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiOutputMessagesKey); - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiAgentThoughtProcessKey).WhoseValue.Should().Be(thoughtProcess); - attrs.Should().ContainKey(OpenTelemetryConstants.UserIdKey).WhoseValue.Should().Be("caller-inf-123"); - attrs.Should().ContainKey(OpenTelemetryConstants.UserNameKey).WhoseValue.Should().Be("Caller Inf Name"); - attrs.Should().ContainKey(OpenTelemetryConstants.UserEmailKey).WhoseValue.Should().Be("callerinf@example.com"); - attrs.Should().ContainKey(OpenTelemetryConstants.CallerClientIpKey).WhoseValue.Should().Be("192.168.1.100"); - data.StartTime.Should().Be(start); - data.EndTime.Should().Be(end); - data.Duration.Should().BeCloseTo(end - start, TimeSpan.FromMilliseconds(100)); - data.SpanId.Should().Be(spanId); - data.ParentSpanId.Should().Be(parentSpanId); - } - - [TestMethod] - public void Build_WithOnlyStartTime_DurationZero() - { - // Arrange - var details = new InferenceCallDetails(InferenceOperationType.Chat, "gpt-4o", "openai"); - var agent = new AgentDetails("agent-9"); - var start = DateTimeOffset.UtcNow; - var conversationId = "conv-zero"; - - // Act - var data = ExecuteInferenceDataBuilder.Build(details, agent, conversationId, startTime: start); - - // Assert - data.StartTime.Should().Be(start); - data.EndTime.Should().BeNull(); - data.Duration.Should().Be(TimeSpan.Zero); - } - - [TestMethod] - public void Build_AddsExtraAttributes_WhenNotReserved() - { - // Arrange - var details = new InferenceCallDetails(InferenceOperationType.Chat, "model-extra", "provider-extra"); - var agent = new AgentDetails("agent-extra", "ExtraInfAgent"); - var conversationId = "conv-extra-inf"; - var extras = new Dictionary - { - {"inf.attr1", "v1"}, - {"inf.attr2", 99} - }; - - // Act - var data = ExecuteInferenceDataBuilder.Build(details, agent, conversationId, extraAttributes: extras); - - // Assert - data.Attributes.Should().ContainKey("inf.attr1").WhoseValue.Should().Be("v1"); - data.Attributes.Should().ContainKey("inf.attr2").WhoseValue.Should().Be(99); - } - - [TestMethod] - public void Build_DoesNotOverrideReservedKeys_WithExtraAttributes() - { - // Arrange - var details = new InferenceCallDetails(InferenceOperationType.Chat, "model-resv", "provider-resv"); - var agent = new AgentDetails("agent-resv", "ReservedInfAgent"); - var conversationId = "conv-resv-inf"; - var extras = new Dictionary - { - {OpenTelemetryConstants.GenAiRequestModelKey, "fake-model"}, - {OpenTelemetryConstants.GenAiProviderNameKey, "fake-provider"}, - {"inf.other", "ok"} - }; - - // Act - var data = ExecuteInferenceDataBuilder.Build(details, agent, conversationId, extraAttributes: extras); - - // Assert - data.Attributes[OpenTelemetryConstants.GenAiRequestModelKey].Should().Be("model-resv"); - data.Attributes[OpenTelemetryConstants.GenAiProviderNameKey].Should().Be("provider-resv"); - data.Attributes.Should().ContainKey("inf.other").WhoseValue.Should().Be("ok"); - } - - [TestMethod] - public void Build_IgnoresNullValues_InExtraAttributes() - { - // Arrange - var details = new InferenceCallDetails(InferenceOperationType.Chat, "model-null", "provider-null"); - var agent = new AgentDetails("agent-null", "NullInfAgent"); - var conversationId = "conv-null-inf"; - var extras = new Dictionary - { - {"inf.null", null}, - {"inf.valid", "yes"} - }; - - // Act - var data = ExecuteInferenceDataBuilder.Build(details, agent, conversationId, extraAttributes: extras); - - // Assert - data.Attributes.Should().NotContainKey("inf.null"); - data.Attributes.Should().ContainKey("inf.valid").WhoseValue.Should().Be("yes"); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/Builders/ExecuteToolDataBuilderTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/Builders/ExecuteToolDataBuilderTests.cs deleted file mode 100644 index 6ce63765..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/Builders/ExecuteToolDataBuilderTests.cs +++ /dev/null @@ -1,356 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.DTOs; -using Microsoft.Agents.A365.Observability.Runtime.DTOs.Builders; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.DTOs.Builders -{ - [TestClass] - public class ExecuteToolDataBuilderTests - { - [TestMethod] - public void Build_WithMinimalParameters_SetsBasicAttributes() - { - // Arrange - var toolDetails = new ToolCallDetails("toolA", "{a:1}"); - var agent = new AgentDetails("agent-1", "AgentOne", tenantId: Guid.NewGuid().ToString()); - var conversationId = "conv-min"; - - // Act - var data = ExecuteToolDataBuilder.Build(toolDetails, agent, conversationId); - - // Assert - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiToolNameKey).WhoseValue.Should().Be("toolA"); - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiToolArgumentsKey); - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiAgentIdKey).WhoseValue.Should().Be("agent-1"); - data.Attributes.Should().ContainKey(OpenTelemetryConstants.TenantIdKey); - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiConversationIdKey).WhoseValue.Should().Be(conversationId); - } - - [TestMethod] - public void Build_WithChannel_IncludesChannelAttributes() - { - // Arrange - var toolDetails = new ToolCallDetails("toolSource", (string?)null); - var agent = new AgentDetails("agent-src"); - var conversationId = "conv-src-tool"; - var source = new Channel(name: "ChannelTool", link: "https://channel/tool"); - - // Act - var data = ExecuteToolDataBuilder.Build(toolDetails, agent, conversationId, channel: source); - - // Assert - data.Attributes.Should().ContainKey(OpenTelemetryConstants.ChannelNameKey).WhoseValue.Should().Be("ChannelTool"); - data.Attributes.Should().ContainKey(OpenTelemetryConstants.ChannelLinkKey).WhoseValue.Should().Be("https://channel/tool"); - } - - [TestMethod] - public void Build_WithFullToolDetails_IncludesAllToolAttributes() - { - // Arrange - var endpoint = new Uri("https://example.com:7071"); - var toolDetails = new ToolCallDetails("toolB", "{b:2}", "call-123", "Test tool", "function", endpoint, "my-tool-server"); - var agent = new AgentDetails("agent-2", "AgentTwo", "Desc", agenticUserId: "auid", agenticUserEmail: "upn@example.com", agentBlueprintId: "bp-1"); - var conversationId = "conv-full"; - - // Act - var data = ExecuteToolDataBuilder.Build(toolDetails, agent, conversationId); - - // Assert - var attrs = data.Attributes; - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiToolCallIdKey).WhoseValue.Should().Be("call-123"); - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiToolDescriptionKey).WhoseValue.Should().Be("Test tool"); - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiToolTypeKey).WhoseValue.Should().Be("function"); - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiToolServerNameKey).WhoseValue.Should().Be("my-tool-server"); - attrs.Should().ContainKey(OpenTelemetryConstants.ServerAddressKey).WhoseValue.Should().Be("example.com"); - attrs.Should().ContainKey(OpenTelemetryConstants.ServerPortKey).WhoseValue.Should().Be("7071"); - attrs.Should().ContainKey(OpenTelemetryConstants.AgentAUIDKey).WhoseValue.Should().Be("auid"); - attrs.Should().ContainKey(OpenTelemetryConstants.AgentEmailKey).WhoseValue.Should().Be("upn@example.com"); - attrs.Should().ContainKey(OpenTelemetryConstants.AgentBlueprintIdKey).WhoseValue.Should().Be("bp-1"); - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiConversationIdKey).WhoseValue.Should().Be(conversationId); - } - - [TestMethod] - public void Build_WithNonStandardPort_IncludesPort() - { - // Arrange - var toolDetails = new ToolCallDetails("toolC", (string?)null, endpoint: new Uri("https://example.com:8081")); - var agent = new AgentDetails("agent-3"); - var conversationId = "conv-port"; - - // Act - var data = ExecuteToolDataBuilder.Build(toolDetails, agent, conversationId); - - // Assert - data.Attributes.Should().ContainKey(OpenTelemetryConstants.ServerPortKey).WhoseValue.Should().Be("8081"); - } - - [TestMethod] - public void Build_WithStandardPort443_ExcludesPort() - { - // Arrange - var toolDetails = new ToolCallDetails("toolD", (string?)null, endpoint: new Uri("https://example.com:443")); - var agent = new AgentDetails("agent-4"); - var conversationId = "conv-443"; - - // Act - var data = ExecuteToolDataBuilder.Build(toolDetails, agent, conversationId); - - // Assert - data.Attributes.Should().ContainKey(OpenTelemetryConstants.ServerAddressKey); - data.Attributes.Should().NotContainKey(OpenTelemetryConstants.ServerPortKey); - } - - [TestMethod] - public void Build_WithResponseContent_IncludesEventContent() - { - // Arrange - var toolDetails = new ToolCallDetails("toolE", (string?)null); - var agent = new AgentDetails("agent-5"); - var conversationId = "conv-content"; - - // Act - var data = ExecuteToolDataBuilder.Build(toolDetails, agent, conversationId, responseContent: "result-value"); - - // Assert - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiToolCallResultKey).WhoseValue!.ToString()!.Should().Contain("result-value"); - } - - [TestMethod] - public void Build_WithConversationId_IncludesConversationId() - { - // Arrange - var toolDetails = new ToolCallDetails("toolF", (string?)null); - var agent = new AgentDetails("agent-6"); - var conversationId = "conv-123"; - - // Act - var data = ExecuteToolDataBuilder.Build(toolDetails, agent, conversationId); - - // Assert - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiConversationIdKey).WhoseValue.Should().Be("conv-123"); - } - - [TestMethod] - public void Build_WithNullOptionalParameters_OmitsThoseAttributes() - { - // Arrange - var toolDetails = new ToolCallDetails("toolG", (string?)null); // no optional fields, no endpoint - var agent = new AgentDetails("agent-7"); - var conversationId = "conv-null"; - - // Act - var data = ExecuteToolDataBuilder.Build(toolDetails, agent, conversationId); - - // Assert - data.Attributes.Should().NotContainKey(OpenTelemetryConstants.GenAiToolCallIdKey); - data.Attributes.Should().NotContainKey(OpenTelemetryConstants.GenAiToolDescriptionKey); - data.Attributes.Should().NotContainKey(OpenTelemetryConstants.GenAiToolTypeKey); - data.Attributes.Should().NotContainKey(OpenTelemetryConstants.GenAiToolServerNameKey); - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiConversationIdKey).WhoseValue.Should().Be(conversationId); - data.Attributes.Should().NotContainKey(OpenTelemetryConstants.GenAiToolCallResultKey); - } - - [TestMethod] - public void Build_SetsTimingInformation_WhenProvided() - { - // Arrange - var toolDetails = new ToolCallDetails("toolH", (string?)null); - var agent = new AgentDetails("agent-8"); - var start = DateTimeOffset.UtcNow.AddMinutes(-3); - var end = DateTimeOffset.UtcNow; - var conversationId = "conv-time"; - - // Act - var data = ExecuteToolDataBuilder.Build(toolDetails, agent, conversationId, startTime: start, endTime: end); - - // Assert - data.StartTime.Should().Be(start); - data.EndTime.Should().Be(end); - data.Duration.Should().BeCloseTo(TimeSpan.FromMinutes(3), TimeSpan.FromMilliseconds(100)); - } - - [TestMethod] - public void Build_SetsSpanIds_WhenProvided() - { - // Arrange - var toolDetails = new ToolCallDetails("toolI", (string?)null); - var agent = new AgentDetails("agent-9"); - var spanId = "span-tool"; - var parentSpanId = "parent-tool"; - var conversationId = "conv-span"; - - // Act - var data = ExecuteToolDataBuilder.Build(toolDetails, agent, conversationId, spanId: spanId, parentSpanId: parentSpanId); - - // Assert - data.SpanId.Should().Be(spanId); - data.ParentSpanId.Should().Be(parentSpanId); - } - - [TestMethod] - public void Build_WithAllParameters_SetsAllExpectedAttributes() - { - // Arrange - var endpoint = new Uri("https://example.org:6060"); - var toolDetails = new ToolCallDetails("toolJ", "{x:1}", "call-999", "Full tool", "extension", endpoint, "full-tool-server"); - var agent = new AgentDetails("agent-10", "AgentTen", "Desc", agenticUserId: "auid10", agenticUserEmail: "upn10@example.com", agentBlueprintId: "bp-10"); - var conversationId = "conv-all"; - var start = DateTimeOffset.UtcNow.AddSeconds(-30); - var end = DateTimeOffset.UtcNow; - var spanId = "span-all"; - var parentSpanId = "parent-all"; - var responseContent = "tool-response"; - var callerDetails = new CallerDetails(userDetails: new UserDetails(userId: "caller-tool-123", userName: "Caller Tool Name", userEmail: "callertool@example.com", userClientIP: System.Net.IPAddress.Parse("10.0.0.50"))); - - // Act - var data = ExecuteToolDataBuilder.Build( - toolDetails, - agent, - conversationId, - responseContent: responseContent, - startTime: start, - endTime: end, - spanId: spanId, - parentSpanId: parentSpanId, - callerDetails: callerDetails); - - // Assert - var attrs = data.Attributes; - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiToolCallIdKey); - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiToolDescriptionKey); - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiToolTypeKey); - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiToolServerNameKey).WhoseValue.Should().Be("full-tool-server"); - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiConversationIdKey); - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiToolCallResultKey); - attrs.Should().ContainKey(OpenTelemetryConstants.UserIdKey).WhoseValue.Should().Be("caller-tool-123"); - attrs.Should().ContainKey(OpenTelemetryConstants.UserNameKey).WhoseValue.Should().Be("Caller Tool Name"); - attrs.Should().ContainKey(OpenTelemetryConstants.UserEmailKey).WhoseValue.Should().Be("callertool@example.com"); - attrs.Should().ContainKey(OpenTelemetryConstants.CallerClientIpKey).WhoseValue.Should().Be("10.0.0.50"); - data.StartTime.Should().Be(start); - data.EndTime.Should().Be(end); - data.Duration.Should().BeCloseTo(end - start, TimeSpan.FromMilliseconds(100)); - data.SpanId.Should().Be(spanId); - data.ParentSpanId.Should().Be(parentSpanId); - } - - [TestMethod] - public void Build_WithOnlyStartTime_DurationZero() - { - // Arrange - var toolDetails = new ToolCallDetails("toolK", (string?)null); - var agent = new AgentDetails("agent-11"); - var start = DateTimeOffset.UtcNow; - var conversationId = "conv-zero"; - - // Act - var data = ExecuteToolDataBuilder.Build(toolDetails, agent, conversationId, startTime: start); - - // Assert - data.StartTime.Should().Be(start); - data.EndTime.Should().BeNull(); - data.Duration.Should().Be(TimeSpan.Zero); - } - - [TestMethod] - public void Build_AddsExtraAttributes_WhenNotReserved() - { - // Arrange - var tool = new ToolCallDetails("tool-extra", (string?)null); - var agent = new AgentDetails("agent-extra", "ExtraToolAgent"); - var conversationId = "conv-extra-tool"; - var extras = new Dictionary - { - {"tool.custom", "abc"}, - {"tool.number", 7} - }; - - // Act - var data = ExecuteToolDataBuilder.Build(tool, agent, conversationId, extraAttributes: extras); - - // Assert - data.Attributes.Should().ContainKey("tool.custom").WhoseValue.Should().Be("abc"); - data.Attributes.Should().ContainKey("tool.number").WhoseValue.Should().Be(7); - } - - [TestMethod] - public void Build_DoesNotOverrideReservedKeys_WithExtraAttributes() - { - // Arrange - var tool = new ToolCallDetails("tool-resv", (string?)null); - var agent = new AgentDetails("agent-resv", "ReservedToolAgent"); - var conversationId = "conv-resv-tool"; - var extras = new Dictionary - { - {OpenTelemetryConstants.GenAiToolNameKey, "fake-tool"}, - {OpenTelemetryConstants.GenAiAgentIdKey, "fake-agent"}, - {"tool.other", "ok"} - }; - - // Act - var data = ExecuteToolDataBuilder.Build(tool, agent, conversationId, extraAttributes: extras); - - // Assert - data.Attributes[OpenTelemetryConstants.GenAiToolNameKey].Should().Be("tool-resv"); - data.Attributes[OpenTelemetryConstants.GenAiAgentIdKey].Should().Be("agent-resv"); - data.Attributes.Should().ContainKey("tool.other").WhoseValue.Should().Be("ok"); - } - - [TestMethod] - public void Build_IgnoresNullValues_InExtraAttributes() - { - // Arrange - var tool = new ToolCallDetails("tool-null", (string?)null); - var agent = new AgentDetails("agent-null", "NullToolAgent"); - var conversationId = "conv-null-tool"; - var extras = new Dictionary - { - {"tool.null", null}, - {"tool.valid", "yes"} - }; - - // Act - var data = ExecuteToolDataBuilder.Build(tool, agent, conversationId, extraAttributes: extras); - - // Assert - data.Attributes.Should().NotContainKey("tool.null"); - data.Attributes.Should().ContainKey("tool.valid").WhoseValue.Should().Be("yes"); - } - - [TestMethod] - public void Build_SpanKind_DefaultsToNull() - { - // Arrange - var tool = new ToolCallDetails("toolSK", (string?)null); - var agent = new AgentDetails("agent-sk"); - var conversationId = "conv-sk-default"; - - // Act - var data = ExecuteToolDataBuilder.Build(tool, agent, conversationId); - - // Assert - data.SpanKind.Should().BeNull(); - } - - [TestMethod] - public void Build_SpanKind_PassesThroughProvidedValue() - { - // Arrange - var tool = new ToolCallDetails("toolSK", (string?)null); - var agent = new AgentDetails("agent-sk"); - var conversationId = "conv-sk-client"; - - // Act - var data = ExecuteToolDataBuilder.Build( - tool, agent, conversationId, - spanKind: SpanKindConstants.Client); - - // Assert - data.SpanKind.Should().Be(SpanKindConstants.Client); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/Builders/InvokeAgentDataBuilderTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/Builders/InvokeAgentDataBuilderTests.cs deleted file mode 100644 index ffe05dff..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/Builders/InvokeAgentDataBuilderTests.cs +++ /dev/null @@ -1,586 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.DTOs; -using Microsoft.Agents.A365.Observability.Runtime.DTOs.Builders; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.DTOs.Builders -{ - [TestClass] - public class InvokeAgentDataBuilderTests - { - [TestMethod] - public void Build_IncludesRequestDetails_WhenRequestProvided() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-123", "TestAgent"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var request = new Request( - "test content", - sessionId: "session-456", - channel: new Channel(name: "source-name", link: "source-description")); - var conversationId = "conv-123"; - - // Act - var telemetry = InvokeAgentDataBuilder.Build( - scopeDetails, - agentDetails, - conversationId, - request: request); - - // Assert - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.ChannelNameKey); - telemetry.Attributes[OpenTelemetryConstants.ChannelNameKey].Should().Be("source-name"); - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.ChannelLinkKey); - telemetry.Attributes[OpenTelemetryConstants.ChannelLinkKey].Should().Be("source-description"); - } - - [TestMethod] - public void Build_IncludesConversationId_WhenProvided() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-123", "TestAgent"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var conversationId = "conv-999"; - - // Act - var telemetry = InvokeAgentDataBuilder.Build( - scopeDetails, - agentDetails, - conversationId); - - // Assert - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiConversationIdKey); - telemetry.Attributes[OpenTelemetryConstants.GenAiConversationIdKey].Should().Be("conv-999"); - } - - [TestMethod] - public void Build_IncludesCallerDetails_WhenProvided() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-123", "TestAgent"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var callerDetails = new CallerDetails(userDetails: new UserDetails(userId: "caller-123", userName: "Caller Name", userEmail: "caller@example.com")); - var conversationId = "conv-123"; - - // Act - var telemetry = InvokeAgentDataBuilder.Build( - scopeDetails, - agentDetails, - conversationId, - callerDetails: callerDetails); - - // Assert - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.UserIdKey); - telemetry.Attributes[OpenTelemetryConstants.UserIdKey].Should().Be("caller-123"); - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.UserNameKey); - telemetry.Attributes[OpenTelemetryConstants.UserNameKey].Should().Be("Caller Name"); - } - - [TestMethod] - public void Build_IncludesCallerAgentDetails_WhenProvided() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-123", "TestAgent"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var callerAgentDetails = new AgentDetails("caller-agent-789", "CallerAgent"); - var conversationId = "conv-123"; - - // Act - var telemetry = InvokeAgentDataBuilder.Build( - scopeDetails, - agentDetails, - conversationId, - callerDetails: new CallerDetails(callerAgentDetails: callerAgentDetails)); - - // Assert - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.CallerAgentIdKey); - telemetry.Attributes[OpenTelemetryConstants.CallerAgentIdKey].Should().Be("caller-agent-789"); - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.CallerAgentNameKey); - telemetry.Attributes[OpenTelemetryConstants.CallerAgentNameKey].Should().Be("CallerAgent"); - } - - [TestMethod] - public void Build_IncludesInputMessages_WhenProvided() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-123", "TestAgent"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var inputMessages = new[] { "Hello", "How are you?" }; - var conversationId = "conv-123"; - - // Act - var telemetry = InvokeAgentDataBuilder.Build( - scopeDetails, - agentDetails, - conversationId, - inputMessages: inputMessages); - - // Assert - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiInputMessagesKey); - telemetry.Attributes[OpenTelemetryConstants.GenAiInputMessagesKey]!.ToString()!.Should().Contain("Hello").And.Contain("How are you?"); - } - - [TestMethod] - public void Build_IncludesOutputMessages_WhenProvided() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-123", "TestAgent"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var outputMessages = new[] { "Hi there!", "I'm fine." }; - var conversationId = "conv-123"; - - // Act - var telemetry = InvokeAgentDataBuilder.Build( - scopeDetails, - agentDetails, - conversationId, - outputMessages: outputMessages); - - // Assert - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiOutputMessagesKey); - telemetry.Attributes[OpenTelemetryConstants.GenAiOutputMessagesKey]!.ToString()!.Should().Contain("Hi there!").And.Contain("fine."); - } - - [TestMethod] - public void Build_IncludesBothInputAndOutputMessages_WhenProvided() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-123", "TestAgent"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var inputMessages = new[] { "Hello" }; - var outputMessages = new[] { "Hi" }; - var conversationId = "conv-123"; - - // Act - var telemetry = InvokeAgentDataBuilder.Build( - scopeDetails, - agentDetails, - conversationId, - inputMessages: inputMessages, - outputMessages: outputMessages); - - // Assert - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiInputMessagesKey); - telemetry.Attributes[OpenTelemetryConstants.GenAiInputMessagesKey]!.ToString()!.Should().Contain("Hello"); - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiOutputMessagesKey); - telemetry.Attributes[OpenTelemetryConstants.GenAiOutputMessagesKey]!.ToString()!.Should().Contain("Hi"); - } - - [TestMethod] - public void Build_OmitsInputMessages_WhenEmptyArray() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-123", "TestAgent"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var inputMessages = new string[] { }; - var conversationId = "conv-123"; - - // Act - var telemetry = InvokeAgentDataBuilder.Build( - scopeDetails, - agentDetails, - conversationId, - inputMessages: inputMessages); - - // Assert - telemetry.Attributes.Should().NotContainKey(OpenTelemetryConstants.GenAiInputMessagesKey); - } - - [TestMethod] - public void Build_OmitsMessages_WhenNull() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-123", "TestAgent"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var conversationId = "conv-123"; - - // Act - var telemetry = InvokeAgentDataBuilder.Build( - scopeDetails, - agentDetails, - conversationId, - inputMessages: null, - outputMessages: null); - - // Assert - telemetry.Attributes.Should().NotContainKey(OpenTelemetryConstants.GenAiInputMessagesKey); - telemetry.Attributes.Should().NotContainKey(OpenTelemetryConstants.GenAiOutputMessagesKey); - } - - [TestMethod] - public void Build_NoError_LeavesStatusUnset() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-123", "TestAgent"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - - // Act - var telemetry = InvokeAgentDataBuilder.Build(scopeDetails, agentDetails, "conv-1"); - - // Assert - telemetry.StatusCode.Should().Be(SpanStatusCode.Unset); - telemetry.StatusMessage.Should().BeNull(); - telemetry.Attributes.Should().NotContainKey(OpenTelemetryConstants.ErrorTypeKey); - } - - [TestMethod] - public void Build_WithError_SetsErrorStatusAndErrorType() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-123", "TestAgent"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var error = new InvalidOperationException("agent blew up"); - - // Act - var telemetry = InvokeAgentDataBuilder.Build(scopeDetails, agentDetails, "conv-1", error: error); - - // Assert - telemetry.StatusCode.Should().Be(SpanStatusCode.Error); - telemetry.StatusMessage.Should().Be("agent blew up"); - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.ErrorTypeKey); - telemetry.Attributes[OpenTelemetryConstants.ErrorTypeKey].Should().Be(typeof(InvalidOperationException).FullName); - } - - - [TestMethod] - public void Build_SetsTimingInformation_WhenProvided() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-123", "TestAgent"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var startTime = DateTimeOffset.UtcNow.AddMinutes(-5); - var endTime = DateTimeOffset.UtcNow; - var conversationId = "conv-123"; - - // Act - var telemetry = InvokeAgentDataBuilder.Build( - scopeDetails, - agentDetails, - conversationId, - startTime: startTime, - endTime: endTime); - - // Assert - telemetry.StartTime.Should().Be(startTime); - telemetry.EndTime.Should().Be(endTime); - telemetry.Duration.Should().BeCloseTo(TimeSpan.FromMinutes(5), TimeSpan.FromMilliseconds(100)); - } - - [TestMethod] - public void Build_SetsSpanIds_WhenProvided() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-123", "TestAgent"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var spanId = "abc123def456"; - var parentSpanId = "parent789ghi012"; - var conversationId = "conv-123"; - - // Act - var telemetry = InvokeAgentDataBuilder.Build( - scopeDetails, - agentDetails, - conversationId, - spanId: spanId, - parentSpanId: parentSpanId); - - // Assert - telemetry.SpanId.Should().Be(spanId); - telemetry.ParentSpanId.Should().Be(parentSpanId); - } - - [TestMethod] - public void Build_WithAllParameters_SetsAllExpectedAttributes() - { - // Arrange - var endpoint = new Uri("https://example.com:8080"); - var agentDetails = new AgentDetails( - "agent-123", - "TestAgent", - "Test Description", - agenticUserId: "auid-456", - agenticUserEmail: "agent@example.com", - agentBlueprintId: "blueprint-789", - tenantId: "tenant-999"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var request = new Request( - "test content", - sessionId: "session-456", - channel: new Channel(name: "source-name", link: "source-description")); - var callerAgentDetails = new AgentDetails("caller-agent-789", "CallerAgent"); - var fullCallerDetails = new CallerDetails( - userDetails: new UserDetails(userId: "caller-123", userName: "Caller Name", userEmail: "caller@example.com"), - callerAgentDetails: callerAgentDetails); - var conversationId = "conv-999"; - var inputMessages = new[] { "Hello" }; - var outputMessages = new[] { "Hi" }; - var startTime = DateTimeOffset.UtcNow.AddMinutes(-1); - var endTime = DateTimeOffset.UtcNow; - var spanId = "span123"; - var parentSpanId = "parent456"; - - // Act - var telemetry = InvokeAgentDataBuilder.Build( - scopeDetails, - agentDetails, - conversationId, - request: request, - callerDetails: fullCallerDetails, - inputMessages: inputMessages, - outputMessages: outputMessages, - startTime: startTime, - endTime: endTime, - spanId: spanId, - parentSpanId: parentSpanId); - - // Assert - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.ChannelNameKey); - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.UserIdKey); - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.CallerAgentIdKey); - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiInputMessagesKey); - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiOutputMessagesKey); - telemetry.StartTime.Should().Be(startTime); - telemetry.EndTime.Should().Be(endTime); - telemetry.Duration.Should().BeCloseTo(TimeSpan.FromMinutes(1), TimeSpan.FromMilliseconds(100)); - telemetry.SpanId.Should().Be(spanId); - telemetry.ParentSpanId.Should().Be(parentSpanId); - } - - [TestMethod] - public void Build_WithOnlyStartTime_DurationZero() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-123"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var startTime = DateTimeOffset.UtcNow; - var conversationId = "conv-123"; - - // Act - var telemetry = InvokeAgentDataBuilder.Build( - scopeDetails, - agentDetails, - conversationId, - startTime: startTime); - - // Assert - telemetry.StartTime.Should().Be(startTime); - telemetry.EndTime.Should().BeNull(); - telemetry.Duration.Should().Be(TimeSpan.Zero); - } - - [TestMethod] - public void Build_AddsExtraAttributes_WhenNotReserved() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-extra", "ExtraAgent"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var conversationId = "conv-extra"; - var extras = new Dictionary - { - {"custom.key1", "value1"}, - {"custom.key2", 42}, - }; - - // Act - var telemetry = InvokeAgentDataBuilder.Build( - scopeDetails, - agentDetails, - conversationId, - extraAttributes: extras); - - // Assert - telemetry.Attributes.Should().ContainKey("custom.key1").WhoseValue.Should().Be("value1"); - telemetry.Attributes.Should().ContainKey("custom.key2").WhoseValue.Should().Be(42); - } - - [TestMethod] - public void Build_DoesNotOverrideReservedKeys_WithExtraAttributes() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-resv", "ReservedAgent"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var conversationId = "conv-resv"; - var extras = new Dictionary - { - {OpenTelemetryConstants.GenAiAgentIdKey, "fake-id"}, // reserved - {OpenTelemetryConstants.GenAiAgentNameKey, "fake-name"}, // reserved - {"another.key", "present"} - }; - - // Act - var telemetry = InvokeAgentDataBuilder.Build( - scopeDetails, - agentDetails, - conversationId, - extraAttributes: extras); - - // Assert - reserved keys keep original values - telemetry.Attributes[OpenTelemetryConstants.GenAiAgentIdKey].Should().Be("agent-resv"); - telemetry.Attributes[OpenTelemetryConstants.GenAiAgentNameKey].Should().Be("ReservedAgent"); - telemetry.Attributes.Should().ContainKey("another.key").WhoseValue.Should().Be("present"); - } - - [TestMethod] - public void Build_IgnoresNullValues_InExtraAttributes() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-null", "NullAgent"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var conversationId = "conv-null-extra"; - var extras = new Dictionary - { - {"custom.null", null}, - {"custom.good", "ok"} - }; - - // Act - var telemetry = InvokeAgentDataBuilder.Build( - scopeDetails, - agentDetails, - conversationId, - extraAttributes: extras); - - // Assert - telemetry.Attributes.Should().NotContainKey("custom.null"); - telemetry.Attributes.Should().ContainKey("custom.good").WhoseValue.Should().Be("ok"); - } - - [TestMethod] - public void Build_AllowsRequestModel_ViaExtraAttributes_WhenNotSetByBuilder() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-model", "ModelAgent"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var conversationId = "conv-model"; - var extras = new Dictionary - { - {OpenTelemetryConstants.GenAiRequestModelKey, "gpt-4o"}, - }; - - // Act - var telemetry = InvokeAgentDataBuilder.Build( - scopeDetails, - agentDetails, - conversationId, - extraAttributes: extras); - - // Assert - gen_ai.request.model is allowed because InvokeAgentDataBuilder does not set it - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiRequestModelKey) - .WhoseValue.Should().Be("gpt-4o"); - } - - [TestMethod] - public void Build_WithAgentPlatformId_SetsExpectedAttributes() - { - // Arrange - var tenantId = Guid.NewGuid().ToString(); - var agentDetails = new AgentDetails( - agentPlatformId: "agent-123", - agentName: "TestAgent", - agentDescription: "Test Description", - tenantId: tenantId); - var scopeDetails = new InvokeAgentScopeDetails(); - var request = new Request( - "test content", - sessionId: "session-456", - channel: new Channel(name: "source-name", link: "source-description")); - var callerDetails = new CallerDetails(userDetails: new UserDetails(userId: "caller-123", userName: "Caller Name", userEmail: "caller@example.com")); - var conversationId = "conv-999"; - var inputMessages = new[] { "Hello" }; - var startTime = DateTimeOffset.UtcNow.AddMinutes(-1); - - // Act - var telemetry = InvokeAgentDataBuilder.Build( - scopeDetails, - agentDetails, - conversationId, - request: request, - callerDetails: callerDetails, - inputMessages: inputMessages, - startTime: startTime); - - // Assert - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.AgentPlatformIdKey); - telemetry.Attributes[OpenTelemetryConstants.AgentPlatformIdKey].Should().Be("agent-123"); - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiAgentNameKey); - telemetry.Attributes[OpenTelemetryConstants.GenAiAgentNameKey].Should().Be("TestAgent"); - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiAgentDescriptionKey); - telemetry.Attributes[OpenTelemetryConstants.GenAiAgentDescriptionKey].Should().Be("Test Description"); - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiConversationIdKey); - telemetry.Attributes[OpenTelemetryConstants.GenAiConversationIdKey].Should().Be("conv-999"); - telemetry.StartTime.Should().Be(startTime); - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.ChannelNameKey); - telemetry.Attributes[OpenTelemetryConstants.ChannelNameKey].Should().Be("source-name"); - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.ChannelLinkKey); - telemetry.Attributes[OpenTelemetryConstants.ChannelLinkKey].Should().Be("source-description"); - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.UserIdKey); - telemetry.Attributes[OpenTelemetryConstants.UserIdKey].Should().Be("caller-123"); - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.UserNameKey); - telemetry.Attributes[OpenTelemetryConstants.UserNameKey].Should().Be("Caller Name"); - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.UserEmailKey); - telemetry.Attributes[OpenTelemetryConstants.UserEmailKey].Should().Be("caller@example.com"); - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.TenantIdKey); - telemetry.Attributes[OpenTelemetryConstants.TenantIdKey].Should().Be(tenantId); - telemetry.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiInputMessagesKey); - telemetry.Attributes[OpenTelemetryConstants.GenAiInputMessagesKey]!.ToString()!.Should().Contain("Hello"); - } - - [TestMethod] - public void Build_SpanKind_DefaultsToNull() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-123", "TestAgent"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var conversationId = "conv-sk-default"; - - // Act - var data = InvokeAgentDataBuilder.Build(scopeDetails, agentDetails, conversationId); - - // Assert - data.SpanKind.Should().BeNull(); - } - - [TestMethod] - public void Build_SpanKind_PassesThroughProvidedValue() - { - // Arrange - var endpoint = new Uri("https://example.com"); - var agentDetails = new AgentDetails("agent-123", "TestAgent"); - var scopeDetails = new InvokeAgentScopeDetails(endpoint: endpoint); - var conversationId = "conv-sk-server"; - - // Act - var data = InvokeAgentDataBuilder.Build( - scopeDetails, - agentDetails, - conversationId, - spanKind: SpanKindConstants.Server); - - // Assert - data.SpanKind.Should().Be(SpanKindConstants.Server); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/Builders/OutputDataBuilderTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/Builders/OutputDataBuilderTests.cs deleted file mode 100644 index 92d77e3f..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/Builders/OutputDataBuilderTests.cs +++ /dev/null @@ -1,167 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.DTOs.Builders; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.DTOs.Builders -{ - [TestClass] - public class OutputDataBuilderTests - { - [TestMethod] - public void Build_WithMinimalParameters_SetsBasicAttributes() - { - var agent = new AgentDetails("agent-1", "AgentOne", tenantId: Guid.NewGuid().ToString()); - var response = new Response(new[] { "Hello" }); - - var data = OutputDataBuilder.Build(agent, response); - - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiOperationNameKey).WhoseValue.Should().Be("output_messages"); - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiAgentIdKey).WhoseValue.Should().Be("agent-1"); - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiOutputMessagesKey).WhoseValue!.ToString()!.Should().Contain("Hello"); - data.Name.Should().Be("OutputMessages"); - - // Null optional parameters should be omitted - data.Attributes.Should().NotContainKey(OpenTelemetryConstants.GenAiConversationIdKey); - data.Attributes.Should().NotContainKey(OpenTelemetryConstants.ChannelNameKey); - data.Attributes.Should().NotContainKey(OpenTelemetryConstants.ChannelLinkKey); - data.Attributes.Should().NotContainKey(OpenTelemetryConstants.UserIdKey); - } - - [TestMethod] - public void Build_WithMultipleOutputMessages_JoinsMessages() - { - var agent = new AgentDetails("agent-2"); - var response = new Response(new[] { "Hello", "World" }); - - var data = OutputDataBuilder.Build(agent, response); - - data.Attributes.Should().ContainKey(OpenTelemetryConstants.GenAiOutputMessagesKey).WhoseValue!.ToString()!.Should().Contain("Hello").And.Contain("World"); - } - - [TestMethod] - public void Build_WithFullAgentDetails_IncludesAllAgentAttributes() - { - var agent = new AgentDetails( - "agent-3", - "AgentThree", - "Description", - agenticUserId: "auid", - agenticUserEmail: "upn@example.com", - agentBlueprintId: "bp-1", - agentPlatformId: "platform-1", - agentType: AgentType.MicrosoftCopilot); - var response = new Response(new[] { "Test" }); - - var data = OutputDataBuilder.Build(agent, response); - - var attrs = data.Attributes; - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiAgentIdKey).WhoseValue.Should().Be("agent-3"); - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiAgentNameKey).WhoseValue.Should().Be("AgentThree"); - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiAgentDescriptionKey).WhoseValue.Should().Be("Description"); - attrs.Should().ContainKey(OpenTelemetryConstants.AgentAUIDKey).WhoseValue.Should().Be("auid"); - attrs.Should().ContainKey(OpenTelemetryConstants.AgentEmailKey).WhoseValue.Should().Be("upn@example.com"); - attrs.Should().ContainKey(OpenTelemetryConstants.AgentBlueprintIdKey).WhoseValue.Should().Be("bp-1"); - attrs.Should().ContainKey(OpenTelemetryConstants.AgentPlatformIdKey).WhoseValue.Should().Be("platform-1"); - - } - - [TestMethod] - public void Build_WithTimingAndSpanIds_SetsAllValues() - { - var agent = new AgentDetails("agent-4"); - var response = new Response(new[] { "Test" }); - var start = DateTimeOffset.UtcNow.AddMinutes(-3); - var end = DateTimeOffset.UtcNow; - - var data = OutputDataBuilder.Build(agent, response, startTime: start, endTime: end, spanId: "span-1", parentSpanId: "parent-1"); - - data.StartTime.Should().Be(start); - data.EndTime.Should().Be(end); - data.Duration.Should().BeCloseTo(TimeSpan.FromMinutes(3), TimeSpan.FromMilliseconds(100)); - data.SpanId.Should().Be("span-1"); - data.ParentSpanId.Should().Be("parent-1"); - } - - [TestMethod] - public void Build_WithEmptyResponseMessages_OmitsOutputMessagesAttribute() - { - var agent = new AgentDetails("agent-5"); - var response = new Response(Array.Empty()); - - var data = OutputDataBuilder.Build(agent, response); - - data.Attributes.Should().NotContainKey(OpenTelemetryConstants.GenAiOutputMessagesKey); - } - - [TestMethod] - public void Build_WithExtraAttributes_AddsNonReservedAndIgnoresReserved() - { - var agent = new AgentDetails("agent-6"); - var response = new Response(new[] { "real-output" }); - var extras = new Dictionary - { - { OpenTelemetryConstants.GenAiOutputMessagesKey, "fake-output" }, - { "output.custom", "abc" }, - { "output.null", null } - }; - - var data = OutputDataBuilder.Build(agent, response, extraAttributes: extras); - - data.Attributes[OpenTelemetryConstants.GenAiOutputMessagesKey]!.ToString()!.Should().Contain("real-output"); - data.Attributes.Should().ContainKey("output.custom").WhoseValue.Should().Be("abc"); - data.Attributes.Should().NotContainKey("output.null"); - } - - [TestMethod] - public void Build_WithAllParameters_SetsAllExpectedAttributes() - { - // Arrange - var agent = new AgentDetails("agent-all", "AgentAll", "Desc", agenticUserId: "auid-all", agenticUserEmail: "upn-all@example.com", agentBlueprintId: "bp-all", agentPlatformId: "platform-all"); - var response = new Response(new[] { "Hello", "World" }); - var conversationId = "conv-output-all"; - var source = new Channel(name: "ChannelOutput", link: "https://channel/output"); - var callerDetails = new CallerDetails( - userDetails: new UserDetails( - userId: "caller-output-123", - userName: "Output Caller Name", - userEmail: "calleroutput@example.com", - userClientIP: System.Net.IPAddress.Parse("192.168.1.50"))); - var start = DateTimeOffset.UtcNow.AddSeconds(-5); - var end = DateTimeOffset.UtcNow; - var spanId = "span-all-output"; - var parentSpanId = "parent-all-output"; - - // Act - var data = OutputDataBuilder.Build( - agent, - response, - conversationId: conversationId, - channel: source, - callerDetails: callerDetails, - startTime: start, - endTime: end, - spanId: spanId, - parentSpanId: parentSpanId); - - // Assert - var attrs = data.Attributes; - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiConversationIdKey).WhoseValue.Should().Be("conv-output-all"); - attrs.Should().ContainKey(OpenTelemetryConstants.ChannelNameKey).WhoseValue.Should().Be("ChannelOutput"); - attrs.Should().ContainKey(OpenTelemetryConstants.ChannelLinkKey).WhoseValue.Should().Be("https://channel/output"); - attrs.Should().ContainKey(OpenTelemetryConstants.UserIdKey).WhoseValue.Should().Be("caller-output-123"); - attrs.Should().ContainKey(OpenTelemetryConstants.UserNameKey).WhoseValue.Should().Be("Output Caller Name"); - attrs.Should().ContainKey(OpenTelemetryConstants.UserEmailKey).WhoseValue.Should().Be("calleroutput@example.com"); - attrs.Should().ContainKey(OpenTelemetryConstants.CallerClientIpKey).WhoseValue.Should().Be("192.168.1.50"); - attrs.Should().ContainKey(OpenTelemetryConstants.GenAiOutputMessagesKey).WhoseValue!.ToString()!.Should().Contain("Hello").And.Contain("World"); - data.StartTime.Should().Be(start); - data.EndTime.Should().Be(end); - data.Duration.Should().BeCloseTo(end - start, TimeSpan.FromMilliseconds(100)); - data.SpanId.Should().Be(spanId); - data.ParentSpanId.Should().Be(parentSpanId); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/Builders/SpanStatusBuilderTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/Builders/SpanStatusBuilderTests.cs deleted file mode 100644 index 54b77c42..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/Builders/SpanStatusBuilderTests.cs +++ /dev/null @@ -1,72 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using System; -using System.Collections.Generic; -using Azure; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.DTOs; -using Microsoft.Agents.A365.Observability.Runtime.DTOs.Builders; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.DTOs.Builders -{ - [TestClass] - public class SpanStatusBuilderTests - { - [TestMethod] - public void FromError_NullError_ReturnsUnset_AndDoesNotTouchAttributes() - { - var attributes = new Dictionary(); - - var status = SpanStatusBuilder.FromError(null, attributes); - - status.Code.Should().Be(SpanStatusCode.Unset); - status.Message.Should().BeNull(); - attributes.Should().NotContainKey(OpenTelemetryConstants.ErrorTypeKey); - } - - [TestMethod] - public void FromError_WithException_ReturnsError_WithMessage() - { - var error = new InvalidOperationException("boom"); - - var status = SpanStatusBuilder.FromError(error); - - status.Code.Should().Be(SpanStatusCode.Error); - status.Message.Should().Be("boom"); - } - - [TestMethod] - public void FromError_WithException_WritesErrorTypeAttribute_AsFullTypeName() - { - var attributes = new Dictionary(); - var error = new TimeoutException("timed out"); - - SpanStatusBuilder.FromError(error, attributes); - - attributes.Should().ContainKey(OpenTelemetryConstants.ErrorTypeKey); - attributes[OpenTelemetryConstants.ErrorTypeKey].Should().Be(typeof(TimeoutException).FullName); - } - - [TestMethod] - public void FromError_RequestFailedException_UsesHttpStatusAsErrorType() - { - var attributes = new Dictionary(); - var error = new RequestFailedException(404, "not found"); - - var status = SpanStatusBuilder.FromError(error, attributes); - - status.Code.Should().Be(SpanStatusCode.Error); - attributes[OpenTelemetryConstants.ErrorTypeKey].Should().Be("404"); - } - - [TestMethod] - public void FromError_NullAttributes_DoesNotThrow() - { - var act = () => SpanStatusBuilder.FromError(new Exception("x"), null); - - act.Should().NotThrow(); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/ExecuteInferenceDataTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/ExecuteInferenceDataTests.cs deleted file mode 100644 index 1dce8195..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/ExecuteInferenceDataTests.cs +++ /dev/null @@ -1,20 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.DTOs; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.DTOs -{ - [TestClass] - public class ExecuteInferenceDataTests - { - [TestMethod] - public void Name_ReturnsExecuteInference() - { - var data = new ExecuteInferenceData(); - data.Name.Should().Be(OpenTelemetryConstants.OperationNames.ExecuteInference.ToString()); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/ExecuteToolDataTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/ExecuteToolDataTests.cs deleted file mode 100644 index 56c8d623..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/ExecuteToolDataTests.cs +++ /dev/null @@ -1,34 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.DTOs; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.DTOs -{ - [TestClass] - public class ExecuteToolDataTests - { - [TestMethod] - public void Name_ReturnsExecuteTool() - { - var data = new ExecuteToolData(); - data.Name.Should().Be(OpenTelemetryConstants.OperationNames.ExecuteTool.ToString()); - } - - [TestMethod] - public void SpanKind_DefaultsToNull() - { - var data = new ExecuteToolData(); - data.SpanKind.Should().BeNull(); - } - - [TestMethod] - public void SpanKind_UsesProvidedValue() - { - var data = new ExecuteToolData(spanKind: SpanKindConstants.Client); - data.SpanKind.Should().Be(SpanKindConstants.Client); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/InvokeAgentDataTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/InvokeAgentDataTests.cs deleted file mode 100644 index f287d5ed..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/InvokeAgentDataTests.cs +++ /dev/null @@ -1,31 +0,0 @@ -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.DTOs; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.DTOs -{ - [TestClass] - public class InvokeAgentDataTests - { - [TestMethod] - public void Name_ReturnsInvokeAgent() - { - var data = new InvokeAgentData(); - data.Name.Should().Be(OpenTelemetryConstants.OperationNames.InvokeAgent.ToString()); - } - - [TestMethod] - public void SpanKind_DefaultsToNull() - { - var data = new InvokeAgentData(); - data.SpanKind.Should().BeNull(); - } - - [TestMethod] - public void SpanKind_UsesProvidedValue() - { - var data = new InvokeAgentData(spanKind: SpanKindConstants.Server); - data.SpanKind.Should().Be(SpanKindConstants.Server); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/OutputDataTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/OutputDataTests.cs deleted file mode 100644 index e82346bc..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/DTOs/OutputDataTests.cs +++ /dev/null @@ -1,20 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.DTOs; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.DTOs -{ - [TestClass] - public class OutputDataTests - { - [TestMethod] - public void Name_ReturnsOutputMessages() - { - var data = new OutputData(); - data.Name.Should().Be(OpenTelemetryConstants.OperationNames.OutputMessages.ToString()); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Etw/EtwEventSourceConfigureTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Etw/EtwEventSourceConfigureTests.cs deleted file mode 100644 index f8bccef6..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Etw/EtwEventSourceConfigureTests.cs +++ /dev/null @@ -1,27 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using Microsoft.Agents.A365.Observability.Runtime.Etw; -using System.Diagnostics.Tracing; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.Etw -{ - [TestClass] - public class EtwEventSourceConfigureTests - { - [TestMethod] - public void Log_HasThrowOnEventWriteErrors() - { - Assert.IsTrue(EtwEventSource.Log.Settings.HasFlag(EventSourceSettings.ThrowOnEventWriteErrors)); - } - - [TestMethod] - public void Log_ReturnsSameInstance() - { - var a = EtwEventSource.Log; - var b = EtwEventSource.Log; - - Assert.AreSame(a, b); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Etw/EtwEventSourceTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Etw/EtwEventSourceTests.cs deleted file mode 100644 index e2ae0f36..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Etw/EtwEventSourceTests.cs +++ /dev/null @@ -1,86 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using Microsoft.Agents.A365.Observability.Runtime.Etw; -using System.Diagnostics.Tracing; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.Etw -{ - [TestClass] - public class EtwEventSourceTests - { - private class TestEventListener : EventListener - { - public List Events { get; } = new List(); - - protected override void OnEventWritten(EventWrittenEventArgs eventData) - { - Events.Add(eventData); - } - } - - [TestMethod] - public void SpanStop_WritesExpectedEventData() - { - // Arrange - using var listener = new TestEventListener(); - listener.EnableEvents(EtwEventSource.Log, EventLevel.Informational); - - string name = "TestSpan"; - string spanId = "span123"; - string traceId = "trace456"; - string parentSpanId = "parent789"; - string content = $@"{{ - ""traceId"": ""{traceId}"", - ""spanId"": ""{spanId}"", - ""parentSpanId"": ""{parentSpanId}"", - ""name"": ""{name}"", - ""kind"": 1, - ""startTimeUnixNano"": 1710000000000000000, - ""endTimeUnixNano"": 1710000000000000123, - ""attributes"": {{ ""http.method"": ""GET"", ""http.url"": ""https://example.com"" }}, - ""status"": {{ ""code"": 1, ""message"": ""OK"" }} - }}"; - - // Act - EtwEventSource.Log.SpanStop(name, spanId, traceId, parentSpanId, content); - - // Assert - var evt = listener.Events.Find(e => e.EventId == 1000); - - Assert.IsNotNull(evt); - Assert.AreEqual(EventLevel.Informational, evt.Level); - Assert.AreEqual(EventOpcode.Stop, evt.Opcode); - Assert.AreEqual("A365 Otel span: Name={0} Id={1} Body={4}", evt.Message); - - Assert.IsNotNull(evt.Payload); - Assert.AreEqual(5, evt.Payload.Count); - Assert.AreEqual(name, evt.Payload[0]); - Assert.AreEqual(spanId, evt.Payload[1]); - Assert.AreEqual(traceId, evt.Payload[2]); - Assert.AreEqual(parentSpanId, evt.Payload[3]); - Assert.AreEqual(content, evt.Payload[4]); - } - - [TestMethod] - public void LogJson_WritesExpectedEventData() - { - // Arrange - using var listener = new TestEventListener(); - listener.EnableEvents(EtwEventSource.Log, EventLevel.Informational); - string message = @"{ ""event"": ""test_event"", ""value"": 42 }"; - - // Act - EtwEventSource.Log.LogJson(message); - - // Assert - var evt = listener.Events.Find(e => e.EventId == 2000); - Assert.IsNotNull(evt); - Assert.AreEqual(EventLevel.Informational, evt.Level); - Assert.AreEqual("{0}", evt.Message); - Assert.IsNotNull(evt.Payload); - Assert.AreEqual(1, evt.Payload.Count); - Assert.AreEqual(message, evt.Payload[0]); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Etw/EtwLoggerTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Etw/EtwLoggerTests.cs deleted file mode 100644 index 26bf5aca..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Etw/EtwLoggerTests.cs +++ /dev/null @@ -1,122 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. -using Microsoft.Agents.A365.Observability.Runtime.Etw; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Extensions.DependencyInjection; -using System.Diagnostics.Tracing; -using System.Text.Json; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.Etw -{ - [TestClass] - public class EtwLoggerTests - { - private class TestEventListener : EventListener - { - public List Events { get; } = new List(); - protected override void OnEventWritten(EventWrittenEventArgs eventData) => Events.Add(eventData); - } - - private ServiceProvider BuildProvider() => new ServiceCollection().AddLoggingWithEtw().BuildServiceProvider(); - - [TestMethod] - public void Logs_InvokeAgent_Event() - { - // Arrange - using var listener = new TestEventListener(); - listener.EnableEvents(EtwEventSource.Log, EventLevel.Informational); - using var provider = BuildProvider(); - var etwLogger = provider.GetRequiredService>(); - var agentDetails = new AgentDetails("agent-id", agentName: "agent-name", agentPlatformId: "platform-123"); - var invokeAgentScopeDetails = new InvokeAgentScopeDetails(endpoint: new Uri("https://example.com/agent")); - string conversationId = "conv-123"; - - // Act - etwLogger.LogInvokeAgent(invokeAgentScopeDetails, agentDetails, conversationId, request: new Request(sessionId: "session-1")); - - // Assert - var evt = listener.Events.FirstOrDefault(e => e.EventId == 2000); - Assert.IsNotNull(evt); - var payloadStr = evt!.Payload![0] as string; - Assert.IsNotNull(payloadStr); - var root = JsonDocument.Parse(payloadStr!).RootElement; - Assert.AreEqual(OpenTelemetryConstants.OperationNames.InvokeAgent.ToString(), root.GetProperty("Name").GetString()); - } - - [TestMethod] - public void Logs_InferenceCall_Event() - { - // Arrange - using var listener = new TestEventListener(); - listener.EnableEvents(EtwEventSource.Log, EventLevel.Informational); - using var provider = BuildProvider(); - var etwLogger = provider.GetRequiredService>(); - var agentDetails = new AgentDetails("agent-id", agentName: "agent-name"); - var inferenceDetails = new InferenceCallDetails(InferenceOperationType.Chat, "model-x", "provider-y"); - string conversationId = "conv-inf-1"; - - // Act - etwLogger.LogInferenceCall(inferenceDetails, agentDetails, conversationId, inputMessages: new[] { "hello" }, outputMessages: new[] { "world" }); - - // Assert - var evt = listener.Events.FirstOrDefault(e => e.EventId == 2000); - Assert.IsNotNull(evt); - var payloadStr = evt!.Payload![0] as string; - Assert.IsNotNull(payloadStr); - var root = JsonDocument.Parse(payloadStr!).RootElement; - Assert.AreEqual(OpenTelemetryConstants.OperationNames.ExecuteInference.ToString(), root.GetProperty("Name").GetString()); - } - - [TestMethod] - public void Logs_ToolCall_Event() - { - // Arrange - using var listener = new TestEventListener(); - listener.EnableEvents(EtwEventSource.Log, EventLevel.Informational); - using var provider = BuildProvider(); - var etwLogger = provider.GetRequiredService>(); - var agentDetails = new AgentDetails("agent-id", agentName: "agent-name"); - var toolDetails = new ToolCallDetails("tool-a", arguments: @"{ ""arg"": 1 }", toolCallId: "tool-call-1", description: "desc", toolType: "function"); - string conversationId = "conv-tool-1"; - string responseContent = @"{ ""value"": ""result"" }"; - - // Act - etwLogger.LogToolCall(toolDetails, agentDetails, conversationId, responseContent: responseContent); - - // Assert - var evt = listener.Events.FirstOrDefault(e => e.EventId == 2000); - Assert.IsNotNull(evt); - var payloadStr = evt!.Payload![0] as string; - Assert.IsNotNull(payloadStr); - var root = JsonDocument.Parse(payloadStr!).RootElement; - Assert.AreEqual(OpenTelemetryConstants.OperationNames.ExecuteTool.ToString(), root.GetProperty("Name").GetString()); - } - - [TestMethod] - public void Logs_Output_Event() - { - // Arrange - using var listener = new TestEventListener(); - listener.EnableEvents(EtwEventSource.Log, EventLevel.Informational); - using var provider = BuildProvider(); - var etwLogger = provider.GetRequiredService>(); - var agentDetails = new AgentDetails("agent-id", agentName: "agent-name"); - var response = new Response(new[] { "Hello", "World" }); - var conversationId = "conv-output-etw"; - var sourceMetadata = new Channel(name: "EtwChannel", link: "https://channel/etw"); - var callerDetails = new CallerDetails(userDetails: new UserDetails(userId: "caller-etw-123", userName: "Etw Caller", userEmail: "etw-caller@example.com")); - - // Act - etwLogger.LogOutput(agentDetails, response, conversationId: conversationId, channel: sourceMetadata, callerDetails: callerDetails); - - // Assert - var evt = listener.Events.FirstOrDefault(e => e.EventId == 2000); - Assert.IsNotNull(evt); - var payloadStr = evt!.Payload![0] as string; - Assert.IsNotNull(payloadStr); - var root = JsonDocument.Parse(payloadStr!).RootElement; - Assert.AreEqual(OpenTelemetryConstants.OperationNames.OutputMessages.ToString(), root.GetProperty("Name").GetString()); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Etw/EtwLoggingBuilderTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Etw/EtwLoggingBuilderTests.cs deleted file mode 100644 index 87e0249c..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Etw/EtwLoggingBuilderTests.cs +++ /dev/null @@ -1,258 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using Microsoft.Agents.A365.Observability.Runtime.Etw; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.Logging; -using System.Diagnostics.Tracing; -using System.Net; -using System.Text.Json; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.Etw -{ - [TestClass] - public class EtwLoggingBuilderTests - { - private class TestEventListener : EventListener - { - public List Events { get; } = new List(); - - protected override void OnEventWritten(EventWrittenEventArgs eventData) - { - Events.Add(eventData); - } - } - - private ServiceProvider BuildProvider() => new ServiceCollection().AddLoggingWithEtw().BuildServiceProvider(); - - [TestMethod] - public void Build_AddsEtwLogProcessor_AndWritesExpectedAttributes_FromInvokeAgent() - { - // Arrange - using var listener = new TestEventListener(); - listener.EnableEvents(EtwEventSource.Log, EventLevel.Informational); - using var provider = BuildProvider(); - var logger = provider.GetRequiredService>(); - var agentDetails = new AgentDetails("agent-id", agentName: "agent-name", agentType: AgentType.MicrosoftCopilot, tenantId: Guid.NewGuid().ToString()); - var invokeAgentScopeDetails = new InvokeAgentScopeDetails(endpoint: new Uri("https://example.com/agent")); - var callerDetails = new CallerDetails(userDetails: new UserDetails(userId: "caller-id-1", userName: "Caller Name", userEmail: "caller@example.com", userClientIP: IPAddress.Parse("192.168.1.100"))); - string conversationId = "conv-123"; - - // Act - logger.LogInvokeAgent(invokeAgentScopeDetails, agentDetails, conversationId, request: new Request(sessionId: "session-1"), callerDetails: callerDetails); - - // Assert - var evt = listener.Events.Find(e => e.EventId == 2000); - Assert.IsNotNull(evt); - Assert.IsNotNull(evt.Payload); - var payloadStr = evt.Payload[0] as string; - Assert.IsNotNull(payloadStr); - var root = JsonDocument.Parse(payloadStr).RootElement; - Assert.IsTrue(root.TryGetProperty("Name", out var nameProp)); - Assert.AreEqual(OpenTelemetryConstants.OperationNames.InvokeAgent.ToString(), nameProp.GetString()); - Assert.IsTrue(root.TryGetProperty("SpanId", out var spanIdProp)); - Assert.IsFalse(string.IsNullOrWhiteSpace(spanIdProp.GetString())); - Assert.IsTrue(root.TryGetProperty("Attributes", out var attrsElement)); - Assert.AreEqual(JsonValueKind.Object, attrsElement.ValueKind, "Attributes JSON element should be an object."); - Assert.AreEqual("agent-id", attrsElement.GetProperty(OpenTelemetryConstants.GenAiAgentIdKey).GetString()); - Assert.AreEqual("agent-name", attrsElement.GetProperty(OpenTelemetryConstants.GenAiAgentNameKey).GetString()); - Assert.AreEqual("example.com", attrsElement.GetProperty(OpenTelemetryConstants.ServerAddressKey).GetString()); - Assert.AreEqual(conversationId, attrsElement.GetProperty(OpenTelemetryConstants.GenAiConversationIdKey).GetString()); - Assert.AreEqual("invoke_agent", attrsElement.GetProperty(OpenTelemetryConstants.GenAiOperationNameKey).GetString()); - Assert.AreEqual(agentDetails.TenantId, attrsElement.GetProperty(OpenTelemetryConstants.TenantIdKey).GetString()); - Assert.AreEqual("caller-id-1", attrsElement.GetProperty(OpenTelemetryConstants.UserIdKey).GetString()); - Assert.AreEqual("Caller Name", attrsElement.GetProperty(OpenTelemetryConstants.UserNameKey).GetString()); - Assert.AreEqual("caller@example.com", attrsElement.GetProperty(OpenTelemetryConstants.UserEmailKey).GetString()); - Assert.AreEqual("192.168.1.100", attrsElement.GetProperty(OpenTelemetryConstants.CallerClientIpKey).GetString()); - } - - [TestMethod] - public void Build_AddsEtwLogProcessor_AndWritesExpectedAttributes_FromInferenceCall() - { - // Arrange - using var listener = new TestEventListener(); - listener.EnableEvents(EtwEventSource.Log, EventLevel.Informational); - using var provider = BuildProvider(); - var logger = provider.GetRequiredService>(); - var agentDetails = new AgentDetails("agent-id", agentName: "agent-name", tenantId: Guid.NewGuid().ToString()); - var inferenceDetails = new InferenceCallDetails(InferenceOperationType.Chat, "model-x", "provider-y"); - string conversationId = "conv-inf-1"; - var source = new Channel(name: "ChannelInf", link: "https://channel/inf"); - var callerDetails = new CallerDetails(userDetails: new UserDetails(userId: "inf-caller-id", userName: "Inference Caller", userEmail: "infcaller@example.com", userClientIP: IPAddress.Parse("10.0.0.50"))); - - // Act - logger.LogInferenceCall(inferenceDetails, agentDetails, conversationId, inputMessages: new[] { "hello" }, outputMessages: new[] { "world" }, channel: source, callerDetails: callerDetails); - - // Assert - var evt = listener.Events.Find(e => e.EventId == 2000); - Assert.IsNotNull(evt); - Assert.IsNotNull(evt.Payload); - var payloadStr = evt.Payload[0] as string; - Assert.IsNotNull(payloadStr); - var root = JsonDocument.Parse(payloadStr).RootElement; - Assert.IsTrue(root.TryGetProperty("Name", out var nameProp)); - Assert.AreEqual(OpenTelemetryConstants.OperationNames.ExecuteInference.ToString(), nameProp.GetString()); - Assert.IsTrue(root.TryGetProperty("SpanId", out var spanIdProp)); - Assert.IsFalse(string.IsNullOrWhiteSpace(spanIdProp.GetString())); - Assert.IsTrue(root.TryGetProperty("Attributes", out var attrsElement)); - Assert.AreEqual(JsonValueKind.Object, attrsElement.ValueKind, "Attributes JSON element should be an object."); - Assert.AreEqual("agent-id", attrsElement.GetProperty(OpenTelemetryConstants.GenAiAgentIdKey).GetString()); - Assert.AreEqual("agent-name", attrsElement.GetProperty(OpenTelemetryConstants.GenAiAgentNameKey).GetString()); - Assert.AreEqual(conversationId, attrsElement.GetProperty(OpenTelemetryConstants.GenAiConversationIdKey).GetString()); - Assert.AreEqual("chat", attrsElement.GetProperty(OpenTelemetryConstants.GenAiOperationNameKey).GetString()); - Assert.AreEqual("model-x", attrsElement.GetProperty(OpenTelemetryConstants.GenAiRequestModelKey).GetString()); - Assert.AreEqual("provider-y", attrsElement.GetProperty(OpenTelemetryConstants.GenAiProviderNameKey).GetString()); - Assert.IsTrue(attrsElement.GetProperty(OpenTelemetryConstants.GenAiInputMessagesKey).GetString()!.Contains("hello")); - Assert.IsTrue(attrsElement.GetProperty(OpenTelemetryConstants.GenAiOutputMessagesKey).GetString()!.Contains("world")); - Assert.AreEqual("ChannelInf", attrsElement.GetProperty(OpenTelemetryConstants.ChannelNameKey).GetString()); - Assert.AreEqual("https://channel/inf", attrsElement.GetProperty(OpenTelemetryConstants.ChannelLinkKey).GetString()); - var tenantIdString = attrsElement.GetProperty(OpenTelemetryConstants.TenantIdKey).GetString(); - Assert.AreEqual(agentDetails.TenantId, tenantIdString); - Assert.AreEqual("inf-caller-id", attrsElement.GetProperty(OpenTelemetryConstants.UserIdKey).GetString()); - Assert.AreEqual("Inference Caller", attrsElement.GetProperty(OpenTelemetryConstants.UserNameKey).GetString()); - Assert.AreEqual("infcaller@example.com", attrsElement.GetProperty(OpenTelemetryConstants.UserEmailKey).GetString()); - Assert.AreEqual("10.0.0.50", attrsElement.GetProperty(OpenTelemetryConstants.CallerClientIpKey).GetString()); - } - - [TestMethod] - public void Build_AddsEtwLogProcessor_AndWritesExpectedAttributes_FromToolCall() - { - // Arrange - using var listener = new TestEventListener(); - listener.EnableEvents(EtwEventSource.Log, EventLevel.Informational); - using var provider = BuildProvider(); - var logger = provider.GetRequiredService>(); - var agentDetails = new AgentDetails("agent-id", agentName: "agent-name", tenantId: Guid.NewGuid().ToString()); - var toolDetails = new ToolCallDetails("tool-a", arguments: @"{ ""arg"": 1 }", toolCallId: "tool-call-1", description: "desc", toolType: "function"); - string conversationId = "conv-tool-1"; - string responseContent = @"{ ""value"": ""result"" }"; - var source = new Channel(name: "ChannelInf", link: "https://channel/inf"); - var callerDetails = new CallerDetails(userDetails: new UserDetails(userId: "tool-caller-id", userName: "Tool Caller", userEmail: "toolcaller@example.com")); - - // Act - logger.LogToolCall(toolDetails, agentDetails, conversationId, responseContent: responseContent, channel: source, callerDetails: callerDetails); - - // Assert - var evt = listener.Events.Find(e => e.EventId == 2000); - Assert.IsNotNull(evt); - Assert.IsNotNull(evt.Payload); - var payloadStr = evt.Payload[0] as string; - Assert.IsNotNull(payloadStr); - var root = JsonDocument.Parse(payloadStr).RootElement; - Assert.IsTrue(root.TryGetProperty("Name", out var nameProp)); - Assert.AreEqual(OpenTelemetryConstants.OperationNames.ExecuteTool.ToString(), nameProp.GetString()); - Assert.IsTrue(root.TryGetProperty("SpanId", out var spanIdProp)); - Assert.IsFalse(string.IsNullOrWhiteSpace(spanIdProp.GetString())); - Assert.IsTrue(root.TryGetProperty("Attributes", out var attrsElement)); - Assert.AreEqual(JsonValueKind.Object, attrsElement.ValueKind, "Attributes JSON element should be an object."); - Assert.AreEqual("agent-id", attrsElement.GetProperty(OpenTelemetryConstants.GenAiAgentIdKey).GetString()); - Assert.AreEqual("agent-name", attrsElement.GetProperty(OpenTelemetryConstants.GenAiAgentNameKey).GetString()); - Assert.AreEqual(conversationId, attrsElement.GetProperty(OpenTelemetryConstants.GenAiConversationIdKey).GetString()); - Assert.AreEqual("tool-a", attrsElement.GetProperty(OpenTelemetryConstants.GenAiToolNameKey).GetString()); - Assert.AreEqual(@"{ ""arg"": 1 }", attrsElement.GetProperty(OpenTelemetryConstants.GenAiToolArgumentsKey).GetString()); - Assert.AreEqual("tool-call-1", attrsElement.GetProperty(OpenTelemetryConstants.GenAiToolCallIdKey).GetString()); - Assert.AreEqual("desc", attrsElement.GetProperty(OpenTelemetryConstants.GenAiToolDescriptionKey).GetString()); - Assert.AreEqual("function", attrsElement.GetProperty(OpenTelemetryConstants.GenAiToolTypeKey).GetString()); - Assert.AreEqual(responseContent, attrsElement.GetProperty(OpenTelemetryConstants.GenAiToolCallResultKey).GetString()); - Assert.AreEqual("execute_tool", attrsElement.GetProperty(OpenTelemetryConstants.GenAiOperationNameKey).GetString()); - Assert.AreEqual("ChannelInf", attrsElement.GetProperty(OpenTelemetryConstants.ChannelNameKey).GetString()); - Assert.AreEqual("https://channel/inf", attrsElement.GetProperty(OpenTelemetryConstants.ChannelLinkKey).GetString()); - var tenantIdString = attrsElement.GetProperty(OpenTelemetryConstants.TenantIdKey).GetString(); - Assert.AreEqual(agentDetails.TenantId, tenantIdString); - Assert.AreEqual("tool-caller-id", attrsElement.GetProperty(OpenTelemetryConstants.UserIdKey).GetString()); - Assert.AreEqual("Tool Caller", attrsElement.GetProperty(OpenTelemetryConstants.UserNameKey).GetString()); - Assert.AreEqual("toolcaller@example.com", attrsElement.GetProperty(OpenTelemetryConstants.UserEmailKey).GetString()); - } - - [TestMethod] - public void Build_AddsEtwLogProcessor_AndWritesExpectedAttributes_FromOutputMessages() - { - // Arrange - using var listener = new TestEventListener(); - listener.EnableEvents(EtwEventSource.Log, EventLevel.Informational); - using var provider = BuildProvider(); - var logger = provider.GetRequiredService>(); - var agentDetails = new AgentDetails("agent-id", agentName: "agent-name", agentType: AgentType.MicrosoftCopilot, tenantId: Guid.NewGuid().ToString()); - var response = new Response(new[] { "Hello", "World" }); - - // Act - logger.LogOutput(agentDetails, response); - - // Assert - var evt = listener.Events.Find(e => e.EventId == 2000); - Assert.IsNotNull(evt); - Assert.IsNotNull(evt.Payload); - var payloadStr = evt.Payload[0] as string; - Assert.IsNotNull(payloadStr); - var root = JsonDocument.Parse(payloadStr).RootElement; - Assert.IsTrue(root.TryGetProperty("Name", out var nameProp)); - Assert.AreEqual(OpenTelemetryConstants.OperationNames.OutputMessages.ToString(), nameProp.GetString()); - Assert.IsTrue(root.TryGetProperty("SpanId", out var spanIdProp)); - Assert.IsFalse(string.IsNullOrWhiteSpace(spanIdProp.GetString())); - Assert.IsTrue(root.TryGetProperty("Attributes", out var attrsElement)); - Assert.AreEqual(JsonValueKind.Object, attrsElement.ValueKind, "Attributes JSON element should be an object."); - Assert.AreEqual("agent-id", attrsElement.GetProperty(OpenTelemetryConstants.GenAiAgentIdKey).GetString()); - Assert.AreEqual("agent-name", attrsElement.GetProperty(OpenTelemetryConstants.GenAiAgentNameKey).GetString()); - Assert.AreEqual("output_messages", attrsElement.GetProperty(OpenTelemetryConstants.GenAiOperationNameKey).GetString()); - var outputValue = attrsElement.GetProperty(OpenTelemetryConstants.GenAiOutputMessagesKey).GetString()!; - Assert.IsTrue(outputValue.Contains("Hello") && outputValue.Contains("World") && outputValue.StartsWith("[")); - var tenantIdString = attrsElement.GetProperty(OpenTelemetryConstants.TenantIdKey).GetString(); - Assert.AreEqual(agentDetails.TenantId, tenantIdString); - } - - [TestMethod] - public void LoggerFilter_Allows_EtwCategoryPrefix_Only() - { - // Arrange - using var listener = new TestEventListener(); - listener.EnableEvents(EtwEventSource.Log, EventLevel.Informational); - using var provider = BuildProvider(); - var etwLogger = provider.GetRequiredService>(); - var factory = provider.GetRequiredService(); - var blockedLogger = factory.CreateLogger("Custom.Blocked"); - var agentDetails = new AgentDetails("agent-id", agentName: "agent-name"); - var invokeAgentScopeDetails = new InvokeAgentScopeDetails(endpoint: new Uri("https://example.com/agent")); - string conversationId = "conv-123"; - - // Act - etwLogger.LogInvokeAgent(invokeAgentScopeDetails, agentDetails, conversationId, request: new Request(sessionId: "session-1")); - blockedLogger.LogInformation("Blocked log"); - - // Assert - var payloads = listener.Events.Where(e => e.EventId == 2000).Select(e => e.Payload?[0] as string).Where(p => p != null).ToList(); - Assert.IsTrue(payloads.Any(p => p!.Contains("InvokeAgent")), "Expected at least one InvokeAgent log to be exported"); - Assert.IsFalse(payloads.Any(p => p!.Contains("Blocked")), "Blocked log without prefix should not be exported"); - } - - [TestMethod] - public void LoggerFilter_Allows_All_Our_CustomLogs() - { - // Arrange - using var listener = new TestEventListener(); - listener.EnableEvents(EtwEventSource.Log, EventLevel.Informational); - using var provider = BuildProvider(); - var etwLogger = provider.GetRequiredService>(); - var agentDetails = new AgentDetails("agent-id", agentName: "agent-name"); - var invokeAgentScopeDetails = new InvokeAgentScopeDetails(endpoint: new Uri("https://example.com/agent")); - var inferenceDetails = new InferenceCallDetails(InferenceOperationType.Chat, "model-x", "provider-y"); - var toolDetails = new ToolCallDetails("tool-a", arguments: @"{ ""arg"": 1 }", toolCallId: "tool-call-1", description: "desc", toolType: "function"); - var response = new Response(new[] { "output" }); - string conversationId = "conv-123"; - - // Act - etwLogger.LogInvokeAgent(invokeAgentScopeDetails, agentDetails, conversationId, request: new Request(sessionId: "session-1")); - etwLogger.LogInferenceCall(inferenceDetails, agentDetails, conversationId, inputMessages: new[] { "hello" }, outputMessages: new[] { "world" }); - etwLogger.LogToolCall(toolDetails, agentDetails, conversationId, responseContent: @"{ ""value"": ""result"" }"); - etwLogger.LogOutput(agentDetails, response); - - // Assert - var payloads = listener.Events.Where(e => e.EventId == 2000).Select(e => e.Payload?[0] as string).Where(p => p != null).ToList(); - Assert.IsTrue(payloads.Any(p => p!.Contains("InvokeAgent")), "Expected at least one InvokeAgent log to be exported"); - Assert.IsTrue(payloads.Any(p => p!.Contains("ExecuteInference")), "Expected at least one ExecuteInference log to be exported"); - Assert.IsTrue(payloads.Any(p => p!.Contains("ExecuteTool")), "Expected at least one ExecuteTool log to be exported"); - Assert.IsTrue(payloads.Any(p => p!.Contains("OutputMessages")), "Expected at least one OutputMessages log to be exported"); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Etw/EtwTracingBuilderTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Etw/EtwTracingBuilderTests.cs deleted file mode 100644 index e7cdb528..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Etw/EtwTracingBuilderTests.cs +++ /dev/null @@ -1,173 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using Microsoft.Agents.A365.Observability.Runtime.Etw; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Extensions.DependencyInjection; -using System.Diagnostics; -using System.Diagnostics.Tracing; -using System.Text.Json; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.Etw -{ - [TestClass] - public class EtwTracingBuilderTests - { - private class TestEventListener : EventListener - { - public List Events { get; } = new List(); - - protected override void OnEventWritten(EventWrittenEventArgs eventData) - { - Events.Add(eventData); - } - } - - private ServiceProvider BuildProvider() => new ServiceCollection().AddTracingWithEtw().BuildServiceProvider(); - - [TestMethod] - public void Build_AddsEtwScopeEventProcessor_AndWritesExpectedAttributes_ForInvokeAgentSpan() - { - // Arrange - using var listener = new TestEventListener(); - listener.EnableEvents(EtwEventSource.Log, EventLevel.Informational); - using var provider = BuildProvider(); - using var tracerProvider = provider.GetRequiredService(); - using var source = new ActivitySource(OpenTelemetryConstants.SourceName); - - var activityName = OpenTelemetryConstants.OperationNames.InvokeAgent.ToString(); - using var activity = source.StartActivity(activityName, ActivityKind.Server); - Assert.IsNotNull(activity); - - activity?.SetTag(OpenTelemetryConstants.GenAiAgentIdKey, "agent-id"); - activity?.SetTag(OpenTelemetryConstants.GenAiAgentNameKey, "agent-name"); - activity?.SetTag(OpenTelemetryConstants.GenAiOperationNameKey, "invoke_agent"); - activity?.SetTag(OpenTelemetryConstants.GenAiConversationIdKey, "conv-123"); - activity?.SetTag(OpenTelemetryConstants.ServerAddressKey, "example.com"); - - // Act - activity?.Stop(); - - // Assert - var evt = listener.Events.Find(e => e.EventId == 1000); - Assert.IsNotNull(evt); - Assert.IsNotNull(evt.Payload); - Assert.AreEqual(activityName, evt.Payload![0] as string); - - var payloadStr = evt.Payload![4] as string; - Assert.IsNotNull(payloadStr); - var root = JsonDocument.Parse(payloadStr!).RootElement; - Assert.IsTrue(root.TryGetProperty("resourceSpan", out var resourceSpan)); - var span = resourceSpan.GetProperty("scopeSpan").GetProperty("span"); - Assert.AreEqual(activityName, span.GetProperty("name").GetString()); - Assert.IsTrue(span.TryGetProperty("attributes", out var attrsElement)); - Assert.AreEqual("agent-id", attrsElement.GetProperty(OpenTelemetryConstants.GenAiAgentIdKey).GetString()); - Assert.AreEqual("agent-name", attrsElement.GetProperty(OpenTelemetryConstants.GenAiAgentNameKey).GetString()); - Assert.AreEqual("invoke_agent", attrsElement.GetProperty(OpenTelemetryConstants.GenAiOperationNameKey).GetString()); - Assert.AreEqual("conv-123", attrsElement.GetProperty(OpenTelemetryConstants.GenAiConversationIdKey).GetString()); - Assert.AreEqual("example.com", attrsElement.GetProperty(OpenTelemetryConstants.ServerAddressKey).GetString()); - } - - [TestMethod] - public void Build_AddsEtwScopeEventProcessor_AndWritesExpectedAttributes_ForInferenceSpan() - { - // Arrange - using var listener = new TestEventListener(); - listener.EnableEvents(EtwEventSource.Log, EventLevel.Informational); - using var provider = BuildProvider(); - using var tracerProvider = provider.GetRequiredService(); - using var source = new ActivitySource(OpenTelemetryConstants.SourceName); - - var activityName = OpenTelemetryConstants.OperationNames.ExecuteInference.ToString(); - using var activity = source.StartActivity(activityName, ActivityKind.Server); - Assert.IsNotNull(activity); - - activity?.SetTag(OpenTelemetryConstants.GenAiAgentIdKey, "agent-id"); - activity?.SetTag(OpenTelemetryConstants.GenAiAgentNameKey, "agent-name"); - activity?.SetTag(OpenTelemetryConstants.GenAiOperationNameKey, "chat"); - activity?.SetTag(OpenTelemetryConstants.GenAiConversationIdKey, "conv-inf-1"); - activity?.SetTag(OpenTelemetryConstants.GenAiRequestModelKey, "model-x"); - activity?.SetTag(OpenTelemetryConstants.GenAiProviderNameKey, "provider-y"); - activity?.SetTag(OpenTelemetryConstants.GenAiInputMessagesKey, "hello"); - activity?.SetTag(OpenTelemetryConstants.GenAiOutputMessagesKey, "world"); - - // Act - activity?.Stop(); - - // Assert - var evt = listener.Events.Find(e => e.EventId == 1000); - Assert.IsNotNull(evt); - Assert.IsNotNull(evt.Payload); - Assert.AreEqual(activityName, evt.Payload![0] as string); - - var payloadStr = evt.Payload![4] as string; - Assert.IsNotNull(payloadStr); - var root = JsonDocument.Parse(payloadStr!).RootElement; - Assert.IsTrue(root.TryGetProperty("resourceSpan", out var resourceSpan)); - var span = resourceSpan.GetProperty("scopeSpan").GetProperty("span"); - Assert.AreEqual(activityName, span.GetProperty("name").GetString()); - var attrsElement = span.GetProperty("attributes"); - Assert.AreEqual("agent-id", attrsElement.GetProperty(OpenTelemetryConstants.GenAiAgentIdKey).GetString()); - Assert.AreEqual("agent-name", attrsElement.GetProperty(OpenTelemetryConstants.GenAiAgentNameKey).GetString()); - Assert.AreEqual("conv-inf-1", attrsElement.GetProperty(OpenTelemetryConstants.GenAiConversationIdKey).GetString()); - Assert.AreEqual("chat", attrsElement.GetProperty(OpenTelemetryConstants.GenAiOperationNameKey).GetString()); - Assert.AreEqual("model-x", attrsElement.GetProperty(OpenTelemetryConstants.GenAiRequestModelKey).GetString()); - Assert.AreEqual("provider-y", attrsElement.GetProperty(OpenTelemetryConstants.GenAiProviderNameKey).GetString()); - Assert.AreEqual("hello", attrsElement.GetProperty(OpenTelemetryConstants.GenAiInputMessagesKey).GetString()); - Assert.AreEqual("world", attrsElement.GetProperty(OpenTelemetryConstants.GenAiOutputMessagesKey).GetString()); - } - - [TestMethod] - public void Build_AddsEtwScopeEventProcessor_AndWritesExpectedAttributes_ForToolCallSpan() - { - // Arrange - using var listener = new TestEventListener(); - listener.EnableEvents(EtwEventSource.Log, EventLevel.Informational); - using var provider = BuildProvider(); - using var tracerProvider = provider.GetRequiredService(); - using var source = new ActivitySource(OpenTelemetryConstants.SourceName); - - var activityName = OpenTelemetryConstants.OperationNames.ExecuteTool.ToString(); - using var activity = source.StartActivity(activityName, ActivityKind.Server); - Assert.IsNotNull(activity); - - activity?.SetTag(OpenTelemetryConstants.GenAiAgentIdKey, "agent-id"); - activity?.SetTag(OpenTelemetryConstants.GenAiAgentNameKey, "agent-name"); - activity?.SetTag(OpenTelemetryConstants.GenAiOperationNameKey, "execute_tool"); - activity?.SetTag(OpenTelemetryConstants.GenAiConversationIdKey, "conv-tool-1"); - activity?.SetTag(OpenTelemetryConstants.GenAiToolNameKey, "tool-a"); - activity?.SetTag(OpenTelemetryConstants.GenAiToolArgumentsKey, "{ \"arg\": 1 }"); - activity?.SetTag(OpenTelemetryConstants.GenAiToolCallIdKey, "tool-call-1"); - activity?.SetTag(OpenTelemetryConstants.GenAiToolDescriptionKey, "desc"); - activity?.SetTag(OpenTelemetryConstants.GenAiToolTypeKey, "function"); - activity?.SetTag(OpenTelemetryConstants.GenAiToolCallResultKey, "{ \"value\": \"result\" }"); - - // Act - activity?.Stop(); - - // Assert - var evt = listener.Events.Find(e => e.EventId == 1000); - Assert.IsNotNull(evt); - Assert.IsNotNull(evt.Payload); - Assert.AreEqual(activityName, evt.Payload![0] as string); - - var payloadStr = evt.Payload![4] as string; - Assert.IsNotNull(payloadStr); - var root = JsonDocument.Parse(payloadStr!).RootElement; - Assert.IsTrue(root.TryGetProperty("resourceSpan", out var resourceSpan)); - var span = resourceSpan.GetProperty("scopeSpan").GetProperty("span"); - Assert.AreEqual(activityName, span.GetProperty("name").GetString()); - var attrsElement = span.GetProperty("attributes"); - Assert.AreEqual("agent-id", attrsElement.GetProperty(OpenTelemetryConstants.GenAiAgentIdKey).GetString()); - Assert.AreEqual("agent-name", attrsElement.GetProperty(OpenTelemetryConstants.GenAiAgentNameKey).GetString()); - Assert.AreEqual("conv-tool-1", attrsElement.GetProperty(OpenTelemetryConstants.GenAiConversationIdKey).GetString()); - Assert.AreEqual("tool-a", attrsElement.GetProperty(OpenTelemetryConstants.GenAiToolNameKey).GetString()); - Assert.AreEqual("{ \"arg\": 1 }", attrsElement.GetProperty(OpenTelemetryConstants.GenAiToolArgumentsKey).GetString()); - Assert.AreEqual("tool-call-1", attrsElement.GetProperty(OpenTelemetryConstants.GenAiToolCallIdKey).GetString()); - Assert.AreEqual("desc", attrsElement.GetProperty(OpenTelemetryConstants.GenAiToolDescriptionKey).GetString()); - Assert.AreEqual("function", attrsElement.GetProperty(OpenTelemetryConstants.GenAiToolTypeKey).GetString()); - Assert.AreEqual("{ \"value\": \"result\" }", attrsElement.GetProperty(OpenTelemetryConstants.GenAiToolCallResultKey).GetString()); - Assert.AreEqual("execute_tool", attrsElement.GetProperty(OpenTelemetryConstants.GenAiOperationNameKey).GetString()); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Microsoft.Agents.A365.Observability.Runtime.Tests.csproj b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Microsoft.Agents.A365.Observability.Runtime.Tests.csproj deleted file mode 100644 index c97e5a88..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Microsoft.Agents.A365.Observability.Runtime.Tests.csproj +++ /dev/null @@ -1,26 +0,0 @@ - - - net8.0 - false - false - true - - - - - - - - - - - - - - - - - - - - diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Middleware/BaggageBuilderTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Middleware/BaggageBuilderTests.cs deleted file mode 100644 index facc0dfb..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Middleware/BaggageBuilderTests.cs +++ /dev/null @@ -1,121 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Tests.Middleware; - -using System.Net; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.Common; -using Microsoft.VisualStudio.TestTools.UnitTesting; -using OpenTelemetry; - -using static Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes.OpenTelemetryConstants; - -[TestClass] -public sealed class BaggageBuilderTest -{ - [TestInitialize] - public void EnableTelemetry() - { - AppContext.SetSwitch(EnableOpenTelemetrySwitch, true); - } - - [TestMethod] - public void Apply_SetsAndRestores_BaggageValues() - { - // Arrange - Baggage.Current = default; // clear prior test pollution - var tenant = "tenant-1"; - var agent = "agent-1"; - var session = "session-1"; - var sessionDescription = "Test Session"; - var callerClientIp = IPAddress.Parse("203.0.113.42"); - var platformId = "platform-123"; - - // Act - using (new BaggageBuilder() - .TenantId(tenant) - .AgentId(agent) - .SessionId(session) - .SessionDescription(sessionDescription) - .UserClientIp(callerClientIp) - .AgentPlatformId(platformId) - .Build()) - { - // Assert inside scope - Baggage.Current.GetBaggage(TenantIdKey).Should().Be(tenant); - Baggage.Current.GetBaggage(GenAiAgentIdKey).Should().Be(agent); - Baggage.Current.GetBaggage(SessionIdKey).Should().Be(session); - Baggage.Current.GetBaggage(SessionDescriptionKey).Should().Be(sessionDescription); - Baggage.Current.GetBaggage(CallerClientIpKey).Should().Be(callerClientIp.ToString()); - Baggage.Current.GetBaggage(AgentPlatformIdKey).Should().Be(platformId); - } - - // Assert after dispose (restored -> no values) - Baggage.Current.GetBaggage(TenantIdKey).Should().BeNull(); - Baggage.Current.GetBaggage(GenAiAgentIdKey).Should().BeNull(); - Baggage.Current.GetBaggage(SessionIdKey).Should().BeNull(); - Baggage.Current.GetBaggage(SessionDescriptionKey).Should().BeNull(); - Baggage.Current.GetBaggage(CallerClientIpKey).Should().BeNull(); - Baggage.Current.GetBaggage(AgentPlatformIdKey).Should().BeNull(); - } - - [TestMethod] - public void InvokeAgentServer_SetsAddressAndPort() - { - // Arrange - Baggage.Current = default; - var address = "app.azurewebsites.net"; - var port = 8080; - - // Act - using (new BaggageBuilder() - .InvokeAgentServer(address, port) - .Build()) - { - // Assert inside scope - Baggage.Current.GetBaggage(ServerAddressKey).Should().Be(address); - Baggage.Current.GetBaggage(ServerPortKey).Should().Be(port.ToString()); - } - - // Assert after dispose - Baggage.Current.GetBaggage(ServerAddressKey).Should().BeNull(); - Baggage.Current.GetBaggage(ServerPortKey).Should().BeNull(); - } - - [TestMethod] - public void InvokeAgentServer_OmitsPort_WhenDefault443() - { - // Arrange - Baggage.Current = default; - var address = "app.azurewebsites.net"; - - // Act - using (new BaggageBuilder() - .InvokeAgentServer(address, 443) - .Build()) - { - // Assert - address set, port omitted for 443 - Baggage.Current.GetBaggage(ServerAddressKey).Should().Be(address); - Baggage.Current.GetBaggage(ServerPortKey).Should().BeNull(); - } - } - - [TestMethod] - public void InvokeAgentServer_SetsAddressOnly_WhenPortNull() - { - // Arrange - Baggage.Current = default; - var address = "app.azurewebsites.net"; - - // Act - using (new BaggageBuilder() - .InvokeAgentServer(address) - .Build()) - { - // Assert - Baggage.Current.GetBaggage(ServerAddressKey).Should().Be(address); - Baggage.Current.GetBaggage(ServerPortKey).Should().BeNull(); - } - } -} \ No newline at end of file diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/ActivityAsserts.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/ActivityAsserts.cs deleted file mode 100644 index 57dc04db..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/ActivityAsserts.cs +++ /dev/null @@ -1,29 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Tests.Tracing; - -using System.Diagnostics; -using FluentAssertions; -using FluentAssertions.Primitives; - -public static class ActivityAsserts -{ - public static AndConstraint ShouldHaveTag(this Activity activity, string key, string value) - { - return activity.Tags.Should().ContainKey(key, $"Activity should have tag '{key}'") - .WhoseValue.Should().Be(value, $"Activity tag '{key}' should be '{value}'"); - } - - public static AndConstraint ShouldHaveTagContaining(this Activity activity, string key, string substring) - { - return activity.Tags.Should().ContainKey(key, $"Activity should have tag '{key}'") - .WhoseValue.Should().Contain(substring, $"Activity tag '{key}' should contain '{substring}'"); - } - - public static void ShouldBeError(this Activity activity, string expectedMessage) - { - activity.Status.Should().Be(ActivityStatusCode.Error); - activity.StatusDescription.Should().Be(expectedMessage); - } -} \ No newline at end of file diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Exporters/Agent365ExporterTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Exporters/Agent365ExporterTests.cs deleted file mode 100644 index 94f96f78..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Exporters/Agent365ExporterTests.cs +++ /dev/null @@ -1,1470 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using FluentAssertions; -using Microsoft.Extensions.Logging.Abstractions; -using Microsoft.Agents.A365.Observability.Runtime.Common; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using OpenTelemetry; -using OpenTelemetry.Resources; -using System.Diagnostics; -using System.Reflection; -using System.Net; - -#pragma warning disable CS0618 // Tests intentionally cover ignored legacy S2S compatibility switches. - -namespace Microsoft.Agents.A365.Observability.Tests.Tracing.Exporters; - -[TestClass] -public sealed class Agent365ExporterTests -{ - /// - /// All valid cluster categories from PowerPlatformApiDiscovery.GetEnvironmentApiHostNameSuffix() - /// - private static readonly string[] ValidClusterCategories = new[] - { - "firstrelease", "prod", "production", "gov", "high", "dod", "mooncake", "ex", "rx" - }; - - private static Activity CreateActivity(string? tenantId = null, string? agentId = null, string? operationName = "invoke_agent") - { - using var listener = new ActivityListener - { - ShouldListenTo = source => source.Name == "Agent365Sdk", - Sample = (ref ActivityCreationOptions _) => ActivitySamplingResult.AllDataAndRecorded, - ActivityStarted = _ => { }, - ActivityStopped = _ => { } - }; - ActivitySource.AddActivityListener(listener); - - var source = new ActivitySource("Agent365Sdk"); - var activity = source.StartActivity("test-span", ActivityKind.Client); - if (activity == null) - throw new InvalidOperationException("Failed to start activity. Ensure an ActivityListener is registered."); - - if (operationName != null) - { - activity.SetTag(OpenTelemetryConstants.GenAiOperationNameKey, operationName); - } - if (tenantId != null) - { - activity.SetTag(OpenTelemetryConstants.TenantIdKey, tenantId); - } - if (agentId != null) - { - activity.SetTag(OpenTelemetryConstants.GenAiAgentIdKey, agentId); - } - activity.Stop(); - return activity; - } - - private static Batch CreateBatch(params Activity[] activities) - { - // Batch has an internal ctor; use reflection - var batchType = typeof(Batch); - var ctor = batchType - .GetConstructors(BindingFlags.NonPublic | BindingFlags.Instance) - .FirstOrDefault(); - - if (ctor == null) - { - Assert.Inconclusive("Could not locate internal Batch constructor - OpenTelemetry version changed."); - } - - // Create a CircularBuffer from the activities array - var circularBufferType = batchType.Assembly.GetType("OpenTelemetry.Internal.CircularBuffer`1")! - .MakeGenericType(typeof(Activity)); - var buffer = Activator.CreateInstance(circularBufferType, activities.Length); - - if (buffer == null) - { - Assert.Inconclusive("Could not create CircularBuffer - Activator.CreateInstance returned null."); - } - - var addMethod = circularBufferType.GetMethod("Add"); - foreach (var act in activities) - { - addMethod!.Invoke(buffer, new object[] { act }); - } - - object? batchObj; - try - { - batchObj = ctor.Invoke(new object[] { buffer, activities.Length }); - } - catch (TargetParameterCountException) - { - Assert.Inconclusive("Unexpected Batch constructor shape - adjust test helper."); - throw; - } - - return (Batch)batchObj!; - } - - private static readonly Agent365ExporterCore _agent365ExporterCore = new Agent365ExporterCore(new ExportFormatter(NullLogger.Instance), NullLogger.Instance); - - private static Agent365Exporter CreateExporter(Func? tokenResolver) - { - var options = new Agent365ExporterOptions - { - TokenResolver = tokenResolver == null - ? (_, _) => Task.FromResult("token") - : (agentId, tenantId) => Task.FromResult(tokenResolver(agentId, tenantId)) - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - return new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - } - - [TestMethod] - public void Constructor_NullLogger_Throws() - { - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("token") - }; - Action act = () => _ = new Agent365Exporter(Agent365ExporterTests._agent365ExporterCore, null!, options, resource: null); - act.Should().Throw().WithParameterName("logger"); - } - - [TestMethod] - public void Constructor_NullOptions_Throws() - { - Action act = () => _ = new Agent365Exporter(Agent365ExporterTests._agent365ExporterCore, NullLogger.Instance, null!, null); - act.Should().Throw().WithParameterName("options"); - } - - [TestMethod] - public void Constructor_NullTokenResolver_Throws() - { - var options = new Agent365ExporterOptions - { - TokenResolver = null - }; - Action act = () => _ = new Agent365Exporter(Agent365ExporterTests._agent365ExporterCore, NullLogger.Instance, options, null); - act.Should().Throw() - .WithParameterName("TokenResolver"); - } - - [TestMethod] - public void Export_NoIdentityActivities_ReturnsSuccess() - { - var exporter = CreateExporter((_, _) => "token"); - using var act1 = CreateActivity(); // No tenant / agent tags - using var act2 = CreateActivity(); // No tenant / agent tags - - var batch = CreateBatch(act1, act2); - var result = exporter.Export(in batch); - - result.Should().Be(ExportResult.Success); - } - - [TestMethod] - public void Export_WithIdentity_TokenResolverThrows_ReturnsFailure() - { - var exporter = CreateExporter((_, _) => throw new InvalidOperationException("Resolver failed")); - using var act = CreateActivity(tenantId: "tenant-123", agentId: "agent-456"); - - var batch = CreateBatch(act); - var result = exporter.Export(in batch); - - result.Should().Be(ExportResult.Failure); - } - - [TestMethod] - public void PartitionByIdentity_GroupsActivitiesByTenantAndAgent() - { - // Arrange - using var a1 = CreateActivity("tenant-1", "agent-1"); - using var a2 = CreateActivity("tenant-1", "agent-1"); - using var a3 = CreateActivity("tenant-1", "agent-2"); - using var a4 = CreateActivity("tenant-1", null); // ignored (missing agent) - using var a5 = CreateActivity(null, "agent-3"); // ignored (missing tenant) - using var a6 = CreateActivity(); // ignored (missing both) - - var batch = CreateBatch(a1, a2, a3, a4, a5, a6); - - // Act - var groups = Agent365ExporterTests._agent365ExporterCore.PartitionByIdentity(in batch); - - // Assert - groups.Should().HaveCount(2); - groups.Should().Contain(g => g.TenantId == "tenant-1" && g.AgentId == "agent-1" && g.Activities.Count == 2); - groups.Should().Contain(g => g.TenantId == "tenant-1" && g.AgentId == "agent-2" && g.Activities.Count == 1); - } - - [TestMethod] - public void PartitionByIdentity_FiltersOutNonGenAISpans() - { - // Arrange - using var genaiSpan = CreateActivity("tenant-1", "agent-1", operationName: "invoke_agent"); - using var httpSpan = CreateActivity("tenant-1", "agent-1", operationName: null); - using var dbSpan = CreateActivity("tenant-1", "agent-1", operationName: "some_random_op"); - using var chatSpan = CreateActivity("tenant-1", "agent-1", operationName: "Chat"); - - var batch = CreateBatch(genaiSpan, httpSpan, dbSpan, chatSpan); - - // Act - var groups = Agent365ExporterTests._agent365ExporterCore.PartitionByIdentity(in batch); - - // Assert - groups.Should().HaveCount(1); - groups.Should().Contain(g => g.TenantId == "tenant-1" && g.AgentId == "agent-1" && g.Activities.Count == 2); - } - - [TestMethod] - public void Agent365ExporterOptions_DefaultBatchingParameters_AreSet() - { - // Arrange & Act - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("token") - }; - - // Assert - options.MaxQueueSize.Should().Be(2048); - options.ScheduledDelayMilliseconds.Should().Be(5000); - options.ExporterTimeoutMilliseconds.Should().Be(30000); - options.MaxExportBatchSize.Should().Be(512); - } - - [TestMethod] - public void Agent365ExporterOptions_CustomBatchingParameters_CanBeSet() - { - // Arrange & Act - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("token"), - MaxQueueSize = 4096, - ScheduledDelayMilliseconds = 10000, - ExporterTimeoutMilliseconds = 60000, - MaxExportBatchSize = 1024 - }; - - // Assert - options.MaxQueueSize.Should().Be(4096); - options.ScheduledDelayMilliseconds.Should().Be(10000); - options.ExporterTimeoutMilliseconds.Should().Be(60000); - options.MaxExportBatchSize.Should().Be(1024); - } - - [TestMethod] - public void Agent365ExporterOptions_UseS2SEndpoint_DefaultsToFalse() - { - // Arrange & Act - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("token") - }; - - // Assert - options.UseS2SEndpoint.Should().BeFalse(); - } - - [TestMethod] - public void Agent365ExporterOptions_UseS2SEndpoint_CanBeSetToTrue() - { - // Arrange & Act - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("token"), - UseS2SEndpoint = true - }; - - // Assert - options.UseS2SEndpoint.Should().BeTrue(); - } - - [TestMethod] - public void Agent365ExporterOptions_UseS2SEndpoint_CanBeSetToFalse() - { - // Arrange & Act - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("token"), - UseS2SEndpoint = false - }; - - // Assert - options.UseS2SEndpoint.Should().BeFalse(); - } - - #region S2S Endpoint Functional Tests - - [TestMethod] - public void UseS2SEndpoint_WhenFalse_IsIgnoredAndUsesS2SEndpoint() - { - // Arrange - string? observedUri = null; - var handler = new TestHttpMessageHandler(req => - { - observedUri = req.RequestUri?.AbsoluteUri; - return new HttpResponseMessage(HttpStatusCode.OK); - }); - var httpClient = new HttpClient(handler); - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = false - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource, - httpClient); - - using var activity = CreateActivity(tenantId: "tenant-123", agentId: "agent-456"); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - options.UseS2SEndpoint.Should().BeFalse(); - result.Should().Be(ExportResult.Success); - observedUri.Should().Contain("/observabilityService/tenants/tenant-123/otlp/agents/agent-456/traces"); - } - - [TestMethod] - public void UseS2SEndpoint_WhenTrue_UsesS2SEndpoint() - { - // Arrange - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = true - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - - using var activity = CreateActivity(tenantId: "tenant-123", agentId: "agent-456"); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - options.UseS2SEndpoint.Should().BeTrue(); - result.Should().Be(ExportResult.Failure); // Expected to fail as there's no real endpoint - } - - [TestMethod] - public void UseS2SEndpoint_CanBeToggled_FromFalseToTrue() - { - // Arrange - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = false - }; - - // Act - options.UseS2SEndpoint = true; - - // Assert - options.UseS2SEndpoint.Should().BeTrue(); - } - - [TestMethod] - public void UseS2SEndpoint_CanBeToggled_FromTrueToFalse() - { - // Arrange - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = true - }; - - // Act - options.UseS2SEndpoint = false; - - // Assert - options.UseS2SEndpoint.Should().BeFalse(); - } - - [TestMethod] - public void Export_WithMultipleActivities_StandardEndpoint_GroupsByIdentity() - { - // Arrange - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = false - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - - // Create activities for different tenant/agent combinations - using var activity1 = CreateActivity("tenant-1", "agent-1"); - using var activity2 = CreateActivity("tenant-1", "agent-1"); - using var activity3 = CreateActivity("tenant-2", "agent-2"); - - var batch = CreateBatch(activity1, activity2, activity3); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Failure); // Expected to fail due to no real endpoint - } - - [TestMethod] - public void Export_WithMultipleActivities_S2SEndpoint_GroupsByIdentity() - { - // Arrange - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = true - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - - // Create activities for different tenant/agent combinations - using var activity1 = CreateActivity("tenant-1", "agent-1"); - using var activity2 = CreateActivity("tenant-1", "agent-1"); - using var activity3 = CreateActivity("tenant-2", "agent-2"); - - var batch = CreateBatch(activity1, activity2, activity3); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Failure); // Expected to fail due to no real endpoint - } - - [TestMethod] - public void Export_S2SEndpoint_TokenResolverCalled_WithCorrectParameters() - { - // Arrange - string? capturedAgentId = null; - string? capturedTenantId = null; - - var options = new Agent365ExporterOptions - { - TokenResolver = (agentId, tenantId) => - { - capturedAgentId = agentId; - capturedTenantId = tenantId; - return Task.FromResult("test-token"); - }, - UseS2SEndpoint = true - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - - using var activity = CreateActivity(tenantId: "tenant-123", agentId: "agent-456"); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - capturedAgentId.Should().Be("agent-456"); - capturedTenantId.Should().Be("tenant-123"); - } - - [TestMethod] - public void Export_StandardEndpoint_TokenResolverCalled_WithCorrectParameters() - { - // Arrange - string? capturedAgentId = null; - string? capturedTenantId = null; - - var options = new Agent365ExporterOptions - { - TokenResolver = (agentId, tenantId) => - { - capturedAgentId = agentId; - capturedTenantId = tenantId; - return Task.FromResult("test-token"); - }, - UseS2SEndpoint = false - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - - using var activity = CreateActivity(tenantId: "tenant-123", agentId: "agent-456"); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - capturedAgentId.Should().Be("agent-456"); - capturedTenantId.Should().Be("tenant-123"); - } - - [DataTestMethod] - [DataRow(null)] - [DataRow("")] - public void Export_S2SEndpoint_NullOrEmptyToken_FailsWithoutSendingRequest(string? token) - { - // Arrange - var sendCount = 0; - var handler = new TestHttpMessageHandler(req => - { - Interlocked.Increment(ref sendCount); - return new HttpResponseMessage(HttpStatusCode.OK); - }); - var httpClient = new HttpClient(handler); - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult(token), - UseS2SEndpoint = true - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource, - httpClient); - - using var activity = CreateActivity(tenantId: "tenant-123", agentId: "agent-456"); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Failure); - sendCount.Should().Be(0, "empty resolver results must fail before sending a request"); - } - - [TestMethod] - public void Export_S2SEndpoint_TokenResolverThrows_ReturnsFailure() - { - // Arrange - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromException(new InvalidOperationException("Token resolver failed")), - UseS2SEndpoint = true - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - - using var activity = CreateActivity(tenantId: "tenant-123", agentId: "agent-456"); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Failure); - } - - [TestMethod] - public void Export_StandardEndpoint_TokenResolverThrows_ReturnsFailure() - { - // Arrange - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromException(new InvalidOperationException("Token resolver failed")), - UseS2SEndpoint = false - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - - using var activity = CreateActivity(tenantId: "tenant-123", agentId: "agent-456"); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Failure); - } - - [TestMethod] - public void Export_TokenResolver_IsInvokedForEachExportBatch() - { - // Arrange - var resolverCalls = 0; - var handler = new TestHttpMessageHandler(req => new HttpResponseMessage(HttpStatusCode.OK)); - var httpClient = new HttpClient(handler); - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => - { - Interlocked.Increment(ref resolverCalls); - return Task.FromResult("test-token"); - } - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource, - httpClient); - - using var firstActivity = CreateActivity("tenant-123", "agent-456"); - using var secondActivity = CreateActivity("tenant-123", "agent-456"); - var firstBatch = CreateBatch(firstActivity); - var secondBatch = CreateBatch(secondActivity); - - // Act - exporter.Export(in firstBatch).Should().Be(ExportResult.Success); - exporter.Export(in secondBatch).Should().Be(ExportResult.Success); - - // Assert - resolverCalls.Should().Be(2); - } - - [DataTestMethod] - [DataRow(HttpStatusCode.Unauthorized)] - [DataRow(HttpStatusCode.Forbidden)] - [DataRow(HttpStatusCode.NotFound)] - public void Export_HttpAuthOrNotFoundFailure_DoesNotRetryOrFallback(HttpStatusCode statusCode) - { - // Arrange - var resolverCalls = 0; - var sendCount = 0; - var handler = new TestHttpMessageHandler(req => - { - Interlocked.Increment(ref sendCount); - return new HttpResponseMessage(statusCode); - }); - var httpClient = new HttpClient(handler); - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => - { - Interlocked.Increment(ref resolverCalls); - return Task.FromResult("test-token"); - }, - UseS2SEndpoint = false - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource, - httpClient); - - using var activity = CreateActivity("tenant-123", "agent-456"); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Failure); - resolverCalls.Should().Be(1); - sendCount.Should().Be(1, "401/403/404 must not trigger a delegated retry or alternate endpoint"); - } - - [TestMethod] - public async Task Agent365ExporterAsync_UsesS2SEndpoint_WhenLegacyFlagIsFalse() - { - // Arrange - string? observedUri = null; - var handler = new TestHttpMessageHandler(req => - { - observedUri = req.RequestUri?.AbsoluteUri; - return new HttpResponseMessage(HttpStatusCode.OK); - }); - var httpClient = new HttpClient(handler); - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = false - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365ExporterAsync( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource, - httpClient); - - using var activity = CreateActivity("tenant-async", "agent-async"); - - // Act - await exporter.ExportAsync(new[] { activity }, CancellationToken.None).ConfigureAwait(false); - - // Assert - observedUri.Should().Contain("/observabilityService/tenants/tenant-async/otlp/agents/agent-async/traces"); - } - - [TestMethod] - public void Export_S2SEndpoint_ActivitiesWithoutIdentity_ReturnsSuccess() - { - // Arrange - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = true - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - - using var activity1 = CreateActivity(); // No tenant/agent - using var activity2 = CreateActivity(); // No tenant/agent - - var batch = CreateBatch(activity1, activity2); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Success); - } - - [TestMethod] - public void Export_StandardEndpoint_ActivitiesWithoutIdentity_ReturnsSuccess() - { - // Arrange - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = false - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - - using var activity1 = CreateActivity(); // No tenant/agent - using var activity2 = CreateActivity(); // No tenant/agent - - var batch = CreateBatch(activity1, activity2); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Success); - } - - [TestMethod] - public void Export_S2SEndpoint_ActivityWithOnlyTenantId_IsSkipped() - { - // Arrange - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = true - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - - using var activity = CreateActivity(tenantId: "tenant-123", agentId: null); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Success); - } - - [TestMethod] - public void Export_S2SEndpoint_ActivityWithOnlyAgentId_IsSkipped() - { - // Arrange - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = true - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - - using var activity = CreateActivity(tenantId: null, agentId: "agent-456"); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Success); - } - - [TestMethod] - public void Export_StandardEndpoint_ActivityWithOnlyTenantId_IsSkipped() - { - // Arrange - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = false - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - - using var activity = CreateActivity(tenantId: "tenant-123", agentId: null); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Success); - } - - [TestMethod] - public void Export_StandardEndpoint_ActivityWithOnlyAgentId_IsSkipped() - { - // Arrange - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = false - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - - using var activity = CreateActivity(tenantId: null, agentId: "agent-456"); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Success); - } - - [TestMethod] - public void Export_S2SEndpoint_MixedBatch_ProcessesOnlyValidActivities() - { - // Arrange - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = true - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - - using var validActivity = CreateActivity("tenant-123", "agent-456"); - using var noTenant = CreateActivity(null, "agent-789"); - using var noAgent = CreateActivity("tenant-456", null); - using var noIdentity = CreateActivity(); - - var batch = CreateBatch(validActivity, noTenant, noAgent, noIdentity); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Failure); - } - - [TestMethod] - public void Export_StandardEndpoint_MixedBatch_ProcessesOnlyValidActivities() - { - // Arrange - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = false - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - - using var validActivity = CreateActivity("tenant-123", "agent-456"); - using var noTenant = CreateActivity(null, "agent-789"); - using var noAgent = CreateActivity("tenant-456", null); - using var noIdentity = CreateActivity(); - - var batch = CreateBatch(validActivity, noTenant, noAgent, noIdentity); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Failure); - } - - [TestMethod] - public void Export_S2SEndpoint_WithCustomResource_ProcessesCorrectly() - { - // Arrange - var resource = ResourceBuilder.CreateEmpty() - .AddService("custom-service", serviceVersion: "2.0.0") - .AddAttributes(new Dictionary - { - ["custom.attribute"] = "custom-value", - ["environment"] = "test" - }) - .Build(); - - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = true - }; - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - - using var activity = CreateActivity("tenant-123", "agent-456"); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Failure); // Expected to fail due to no real endpoint - } - - [TestMethod] - public void Export_StandardEndpoint_WithCustomResource_ProcessesCorrectly() - { - // Arrange - var resource = ResourceBuilder.CreateEmpty() - .AddService("custom-service", serviceVersion: "2.0.0") - .AddAttributes(new Dictionary - { - ["custom.attribute"] = "custom-value", - ["environment"] = "test" - }) - .Build(); - - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = false - }; - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - - using var activity = CreateActivity("tenant-123", "agent-456"); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Failure); // Expected to fail due to no real endpoint - } - - [TestMethod] - public void Export_S2SEndpoint_WithDifferentClusterCategories_ProcessesCorrectly() - { - // Test with all valid cluster categories from PowerPlatformApiDiscovery.GetEnvironmentApiHostNameSuffix - foreach (var category in ValidClusterCategories) - { - // Arrange - var options = new Agent365ExporterOptions - { - ClusterCategory = category, - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = true - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - - using var activity = CreateActivity("tenant-123", "agent-456"); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Failure, $"cluster category '{category}' should be processed"); // Expected to fail due to no real endpoint - } - } - - [TestMethod] - public void Export_StandardEndpoint_WithDifferentClusterCategories_ProcessesCorrectly() - { - // Test with all valid cluster categories from PowerPlatformApiDiscovery.GetEnvironmentApiHostNameSuffix - foreach (var category in ValidClusterCategories) - { - // Arrange - var options = new Agent365ExporterOptions - { - ClusterCategory = category, - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = false - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource); - - using var activity = CreateActivity("tenant-123", "agent-456"); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Failure, $"cluster category '{category}' should be processed"); // Expected to fail due to no real endpoint - } - } - - #endregion - - #region Domain Override Functional Tests - - [TestMethod] - public void Export_RequestUri_EnvVar_Overrides_CustomDomainResolver_WhenBothSet() - { - // Arrange - var overrideDomain = "override.example.com"; - Environment.SetEnvironmentVariable("A365_OBSERVABILITY_DOMAIN_OVERRIDE", overrideDomain); - var resolverDomain = "resolver.example.com"; // should be ignored due to env var - - string? observedUri = null; - var handler = new TestHttpMessageHandler(req => - { - observedUri = req.RequestUri?.AbsoluteUri; - return new HttpResponseMessage(HttpStatusCode.OK); - }); - var httpClient = new HttpClient(handler); - - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = false, - DomainResolver = tenantId => resolverDomain - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource, - httpClient); - - using var activity = CreateActivity(tenantId: "tenant-env-overrides", agentId: "agent-xyz"); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Success); - observedUri.Should().NotBeNull(); - observedUri!.Should().StartWith($"https://{overrideDomain}"); - observedUri!.Should().Contain($"/observabilityService/tenants/tenant-env-overrides/otlp/agents/agent-xyz/traces"); - observedUri!.Should().Contain("api-version=1"); - - // Cleanup - Environment.SetEnvironmentVariable("A365_OBSERVABILITY_DOMAIN_OVERRIDE", null); - } - - [TestMethod] - public void Export_RequestUri_UsesEnvVar_WhenNoResolverSet() - { - // Arrange - var overrideDomain = "override.example.com"; - Environment.SetEnvironmentVariable("A365_OBSERVABILITY_DOMAIN_OVERRIDE", overrideDomain); - - string? observedUri = null; - var handler = new TestHttpMessageHandler(req => - { - observedUri = req.RequestUri?.AbsoluteUri; - return new HttpResponseMessage(HttpStatusCode.OK); - }); - var httpClient = new HttpClient(handler); - - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = false, - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource, - httpClient); - - using var activity = CreateActivity(tenantId: "tenant-env", agentId: "agent-xyz"); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Success); - observedUri.Should().NotBeNull(); - observedUri!.Should().StartWith($"https://{overrideDomain}"); - observedUri!.Should().Contain($"/observabilityService/tenants/tenant-env/otlp/agents/agent-xyz/traces"); - observedUri!.Should().Contain("api-version=1"); - - // Cleanup - Environment.SetEnvironmentVariable("A365_OBSERVABILITY_DOMAIN_OVERRIDE", null); - } - - [TestMethod] - public void Export_RequestUri_UsesCustomDomainResolver_WhenProvided() - { - // Arrange - var resolverDomain = "resolver.example.com"; - // Ensure env variable is not set so resolver is used - Environment.SetEnvironmentVariable("A365_OBSERVABILITY_DOMAIN_OVERRIDE", null); - - string? observedUri = null; - var handler = new TestHttpMessageHandler(req => - { - observedUri = req.RequestUri?.AbsoluteUri; - return new HttpResponseMessage(HttpStatusCode.OK); - }); - var httpClient = new HttpClient(handler); - - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = false, - DomainResolver = tenantId => resolverDomain - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource, - httpClient); - - using var activity = CreateActivity(tenantId: "tenant-resolver", agentId: "agent-xyz"); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Success); - observedUri.Should().NotBeNull(); - observedUri!.Should().StartWith($"https://{resolverDomain}"); - observedUri!.Should().Contain($"/observabilityService/tenants/tenant-resolver/otlp/agents/agent-xyz/traces"); - observedUri!.Should().Contain("api-version=1"); - // Cleanup - Environment.SetEnvironmentVariable("A365_OBSERVABILITY_DOMAIN_OVERRIDE", null); - } - - [TestMethod] - public void Export_RequestUri_UsesDefaultEndpoint_WhenNoResolverAndNoEnvVarSet() - { - // Arrange - Environment.SetEnvironmentVariable("A365_OBSERVABILITY_DOMAIN_OVERRIDE", null); - - string? observedUri = null; - var handler = new TestHttpMessageHandler(req => - { - observedUri = req.RequestUri?.AbsoluteUri; - return new HttpResponseMessage(HttpStatusCode.OK); - }); - var httpClient = new HttpClient(handler); - - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = false - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource, - httpClient); - - var tenantId = "tenant-discovery-noset"; - using var activity = CreateActivity(tenantId: tenantId, agentId: "agent-xyz"); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Success); - observedUri.Should().NotBeNull(); - observedUri!.Should().StartWith($"https://{Agent365ExporterOptions.DefaultEndpointHost}"); - observedUri!.Should().Contain($"/observabilityService/tenants/{tenantId}/otlp/agents/agent-xyz/traces"); - observedUri!.Should().Contain("api-version=1"); - } - - [TestMethod] - public void Export_RequestUri_S2SEndpoint_ContainsOtlpSegment() - { - // Arrange - Environment.SetEnvironmentVariable("A365_OBSERVABILITY_DOMAIN_OVERRIDE", null); - - string? observedUri = null; - var handler = new TestHttpMessageHandler(req => - { - observedUri = req.RequestUri?.AbsoluteUri; - return new HttpResponseMessage(HttpStatusCode.OK); - }); - var httpClient = new HttpClient(handler); - - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - UseS2SEndpoint = true - }; - - var resource = ResourceBuilder.CreateEmpty() - .AddService("unit-test-service", serviceVersion: "1.0.0") - .Build(); - - var exporter = new Agent365Exporter( - Agent365ExporterTests._agent365ExporterCore, - NullLogger.Instance, - options, - resource, - httpClient); - - var tenantId = "tenant-s2s"; - using var activity = CreateActivity(tenantId: tenantId, agentId: "agent-xyz"); - var batch = CreateBatch(activity); - - // Act - var result = exporter.Export(in batch); - - // Assert - result.Should().Be(ExportResult.Success); - observedUri.Should().NotBeNull(); - observedUri!.Should().Contain($"/observabilityService/tenants/{tenantId}/otlp/agents/agent-xyz/traces"); - observedUri!.Should().Contain("api-version=1"); - } - - #endregion - - #region BuildRequestUri Tests - - [TestMethod] - public void BuildRequestUri_BareHost_PrependsHttps() - { - var uri = _agent365ExporterCore.BuildRequestUri("agent365.svc.cloud.microsoft", "/observability/tenants/t1/otlp/agents/a1/traces"); - uri.Should().Be("https://agent365.svc.cloud.microsoft/observability/tenants/t1/otlp/agents/a1/traces?api-version=1"); - } - - [TestMethod] - public void BuildRequestUri_HttpsEndpoint_DoesNotDoublePrependScheme() - { - var uri = _agent365ExporterCore.BuildRequestUri("https://custom.example.com", "/observability/tenants/t1/otlp/agents/a1/traces"); - uri.Should().Be("https://custom.example.com/observability/tenants/t1/otlp/agents/a1/traces?api-version=1"); - } - - [TestMethod] - public void BuildRequestUri_TrailingSlash_IsNormalized() - { - var uri = _agent365ExporterCore.BuildRequestUri("https://custom.example.com/", "/observability/tenants/t1/otlp/agents/a1/traces"); - uri.Should().Be("https://custom.example.com/observability/tenants/t1/otlp/agents/a1/traces?api-version=1"); - } - - [TestMethod] - public void BuildRequestUri_BareHostWithTrailingSlash_IsNormalized() - { - var uri = _agent365ExporterCore.BuildRequestUri("agent365.svc.cloud.microsoft/", "/observability/tenants/t1/otlp/agents/a1/traces"); - uri.Should().Be("https://agent365.svc.cloud.microsoft/observability/tenants/t1/otlp/agents/a1/traces?api-version=1"); - } - - [TestMethod] - public void BuildRequestUri_HttpEndpoint_ThrowsArgumentException() - { - Action act = () => _agent365ExporterCore.BuildRequestUri("http://insecure.example.com", "/observability/tenants/t1/otlp/agents/a1/traces"); - act.Should().Throw().WithParameterName("endpoint"); - } - - #endregion - - private class TestHttpMessageHandler : HttpMessageHandler - { - private readonly Func _handler; - public TestHttpMessageHandler(Func handler) - { - _handler = handler; - } - protected override Task SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) - { - return Task.FromResult(_handler(request)); - } - } - -} \ No newline at end of file diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Exporters/ContextualTokenResolverTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Exporters/ContextualTokenResolverTests.cs deleted file mode 100644 index 5cbd4893..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Exporters/ContextualTokenResolverTests.cs +++ /dev/null @@ -1,345 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using FluentAssertions; -using Microsoft.Extensions.Logging.Abstractions; -using Microsoft.Agents.A365.Observability.Runtime.Common; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using OpenTelemetry; -using OpenTelemetry.Resources; -using System.Diagnostics; -using System.Net; - -namespace Microsoft.Agents.A365.Observability.Tests.Tracing.Exporters; - -[TestClass] -public sealed class ContextualTokenResolverTests -{ - private static readonly Agent365ExporterCore Core = new( - new ExportFormatter(NullLogger.Instance), - NullLogger.Instance); - - #region AgentIdentity Tests - - [TestMethod] - public void AgentIdentity_Constructor_SetsProperties() - { - var identity = new AgentIdentity("agent-1", "user-oid-123"); - - identity.AgentId.Should().Be("agent-1"); - identity.AgenticUserId.Should().Be("user-oid-123"); - } - - [TestMethod] - public void AgentIdentity_Constructor_NullAgenticUserId_IsValid() - { - var identity = new AgentIdentity("agent-1"); - - identity.AgentId.Should().Be("agent-1"); - identity.AgenticUserId.Should().BeNull(); - } - - [TestMethod] - public void AgentIdentity_Constructor_ExplicitNull_IsValid() - { - var identity = new AgentIdentity("agent-1", null); - - identity.AgentId.Should().Be("agent-1"); - identity.AgenticUserId.Should().BeNull(); - } - - #endregion - - #region TokenResolverContext Tests - - [TestMethod] - public void TokenResolverContext_Constructor_SetsProperties() - { - var identity = new AgentIdentity("agent-1", "user-oid-123"); - var context = new TokenResolverContext(identity, "tenant-abc"); - - context.Identity.Should().BeSameAs(identity); - context.TenantId.Should().Be("tenant-abc"); - context.Identity.AgentId.Should().Be("agent-1"); - context.Identity.AgenticUserId.Should().Be("user-oid-123"); - } - - [TestMethod] - public void TokenResolverContext_S2SScenario_NullAgenticUserId() - { - var identity = new AgentIdentity("agent-1"); - var context = new TokenResolverContext(identity, "tenant-abc"); - - context.Identity.AgenticUserId.Should().BeNull(); - } - - #endregion - - #region Exporter Constructor Validation Tests - - [TestMethod] - public void Constructor_NeitherResolver_Throws() - { - var options = new Agent365ExporterOptions - { - TokenResolver = null, - ContextualTokenResolver = null, - }; - - Action act = () => _ = new Agent365Exporter(Core, NullLogger.Instance, options, null); - - act.Should().Throw(); - } - - [TestMethod] - public void Constructor_OnlyTokenResolver_Succeeds() - { - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("token"), - }; - - var exporter = new Agent365Exporter(Core, NullLogger.Instance, options, null); - exporter.Should().NotBeNull(); - } - - [TestMethod] - public void Constructor_OnlyContextualTokenResolver_Succeeds() - { - var options = new Agent365ExporterOptions - { - ContextualTokenResolver = ctx => Task.FromResult("token"), - }; - - var exporter = new Agent365Exporter(Core, NullLogger.Instance, options, null); - exporter.Should().NotBeNull(); - } - - [TestMethod] - public void Constructor_BothResolvers_Succeeds() - { - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("legacy-token"), - ContextualTokenResolver = ctx => Task.FromResult("contextual-token"), - }; - - var exporter = new Agent365Exporter(Core, NullLogger.Instance, options, null); - exporter.Should().NotBeNull(); - } - - #endregion - - #region ContextualTokenResolver Precedence Tests - - [TestMethod] - public async Task ExportBatchCoreAsync_ContextualResolver_TakesPrecedence() - { - // Arrange - string? capturedAgentId = null; - string? capturedTenantId = null; - string? capturedAuid = null; - bool legacyResolverCalled = false; - - var options = new Agent365ExporterOptions - { - TokenResolver = (_, _) => - { - legacyResolverCalled = true; - return Task.FromResult("legacy-token"); - }, - ContextualTokenResolver = ctx => - { - capturedAgentId = ctx.Identity.AgentId; - capturedTenantId = ctx.TenantId; - capturedAuid = ctx.Identity.AgenticUserId; - return Task.FromResult("contextual-token"); - }, - }; - - using var activity = CreateActivityWithAuid("tenant-1", "agent-1", "user-oid-123"); - var groups = new List<(string TenantId, string AgentId, List Activities)> - { - ("tenant-1", "agent-1", new List { activity }) - }; - - // Act - var result = await Core.ExportBatchCoreAsync( - groups, - ResourceBuilder.CreateEmpty().Build(), - options, - (agentId, tenantId) => options.TokenResolver!(agentId, tenantId), - request => Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK))); - - // Assert - result.Should().Be(ExportResult.Success); - legacyResolverCalled.Should().BeFalse("ContextualTokenResolver should take precedence"); - capturedAgentId.Should().Be("agent-1"); - capturedTenantId.Should().Be("tenant-1"); - capturedAuid.Should().Be("user-oid-123"); - } - - [TestMethod] - public async Task ExportBatchCoreAsync_LegacyResolver_UsedWhenNoContextual() - { - // Arrange - bool legacyResolverCalled = false; - string? capturedAgentId = null; - string? capturedTenantId = null; - - var options = new Agent365ExporterOptions - { - TokenResolver = (agentId, tenantId) => - { - legacyResolverCalled = true; - capturedAgentId = agentId; - capturedTenantId = tenantId; - return Task.FromResult("legacy-token"); - }, - ContextualTokenResolver = null, - }; - - using var activity = CreateActivityWithAuid("tenant-1", "agent-1", "user-oid-123"); - var groups = new List<(string TenantId, string AgentId, List Activities)> - { - ("tenant-1", "agent-1", new List { activity }) - }; - - // Act - var result = await Core.ExportBatchCoreAsync( - groups, - ResourceBuilder.CreateEmpty().Build(), - options, - (agentId, tenantId) => options.TokenResolver!(agentId, tenantId), - request => Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK))); - - // Assert - result.Should().Be(ExportResult.Success); - legacyResolverCalled.Should().BeTrue(); - capturedAgentId.Should().Be("agent-1"); - capturedTenantId.Should().Be("tenant-1"); - } - - [TestMethod] - public async Task ExportBatchCoreAsync_ContextualResolver_NullAuid_InS2SScenario() - { - // Arrange - string? capturedAuid = "sentinel"; - - var options = new Agent365ExporterOptions - { - ContextualTokenResolver = ctx => - { - capturedAuid = ctx.Identity.AgenticUserId; - return Task.FromResult("token"); - }, - }; - - // Activity without AUID tag - using var activity = CreateActivityWithAuid("tenant-1", "agent-1", null); - var groups = new List<(string TenantId, string AgentId, List Activities)> - { - ("tenant-1", "agent-1", new List { activity }) - }; - - // Act - var result = await Core.ExportBatchCoreAsync( - groups, - ResourceBuilder.CreateEmpty().Build(), - options, - (_, _) => Task.FromResult(null), - request => Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK))); - - // Assert - result.Should().Be(ExportResult.Success); - capturedAuid.Should().BeNull("S2S scenario should pass null AgenticUserId"); - } - - [TestMethod] - public async Task ExportBatchCoreAsync_ContextualResolver_ReturnsNull_Fails() - { - // Arrange - var options = new Agent365ExporterOptions - { - ContextualTokenResolver = ctx => Task.FromResult(null), - }; - - using var activity = CreateActivityWithAuid("tenant-1", "agent-1", "user-oid-123"); - var groups = new List<(string TenantId, string AgentId, List Activities)> - { - ("tenant-1", "agent-1", new List { activity }) - }; - - // Act - var result = await Core.ExportBatchCoreAsync( - groups, - ResourceBuilder.CreateEmpty().Build(), - options, - (_, _) => Task.FromResult(null), - request => Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK))); - - // Assert - result.Should().Be(ExportResult.Failure, "null token should result in failure"); - } - - [TestMethod] - public async Task ExportBatchCoreAsync_ContextualResolver_Throws_Fails() - { - // Arrange - var options = new Agent365ExporterOptions - { - ContextualTokenResolver = ctx => throw new InvalidOperationException("auth error"), - }; - - using var activity = CreateActivityWithAuid("tenant-1", "agent-1", "user-oid-123"); - var groups = new List<(string TenantId, string AgentId, List Activities)> - { - ("tenant-1", "agent-1", new List { activity }) - }; - - // Act - var result = await Core.ExportBatchCoreAsync( - groups, - ResourceBuilder.CreateEmpty().Build(), - options, - (_, _) => Task.FromResult(null), - request => Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK))); - - // Assert - result.Should().Be(ExportResult.Failure, "exception in resolver should not crash, but fail export"); - } - - #endregion - - #region Helpers - - private static Activity CreateActivityWithAuid(string? tenantId, string? agentId, string? agenticUserId) - { - using var listener = new ActivityListener - { - ShouldListenTo = source => source.Name == "Agent365Sdk.ContextualTest", - Sample = (ref ActivityCreationOptions _) => ActivitySamplingResult.AllDataAndRecorded, - ActivityStarted = _ => { }, - ActivityStopped = _ => { } - }; - ActivitySource.AddActivityListener(listener); - - var source = new ActivitySource("Agent365Sdk.ContextualTest"); - var activity = source.StartActivity("test-span", ActivityKind.Client)!; - - activity.SetTag(OpenTelemetryConstants.GenAiOperationNameKey, "invoke_agent"); - - if (tenantId != null) - activity.SetTag(OpenTelemetryConstants.TenantIdKey, tenantId); - if (agentId != null) - activity.SetTag(OpenTelemetryConstants.GenAiAgentIdKey, agentId); - if (agenticUserId != null) - activity.SetTag(OpenTelemetryConstants.AgentAUIDKey, agenticUserId); - - activity.Stop(); - return activity; - } - - #endregion -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Exporters/ExportConfigConsistencyTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Exporters/ExportConfigConsistencyTests.cs deleted file mode 100644 index 8ed7e3e6..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Exporters/ExportConfigConsistencyTests.cs +++ /dev/null @@ -1,62 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.Common; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters; -using Microsoft.Extensions.Logging.Abstractions; - -namespace Microsoft.Agents.A365.Observability.Runtime.Tests.Tracing.Exporters; - -/// -/// Cross-constant consistency test for observability export configuration. -/// -/// Three production constants must stay in sync: the auth scope -/// (ProdObservabilityScope), the default endpoint host -/// (DefaultEndpointHost), and the export URL path pattern -/// (BuildEndpointPath). This test pins all three together so that -/// changing any one forces a review of the others. Individual value tests -/// already exist in EnvironmentUtilsTests and Agent365ExporterTests; -/// this test adds the cross-cutting invariant they don't cover. -/// -[TestClass] -public sealed class ExportConfigConsistencyTests -{ - private const string ScopeOverrideEnvVar = "A365_OBSERVABILITY_SCOPE_OVERRIDE"; - - // Pinned production values — update ALL of these together when any one changes. - private const string ExpectedScope = "api://9b975845-388f-4429-889e-eab1ef63949c/.default"; - private const string ExpectedS2SUri = "https://agent365.svc.cloud.microsoft/observabilityService/tenants/t1/otlp/agents/a1/traces?api-version=1"; - - [TestInitialize] - public void TestInitialize() => Environment.SetEnvironmentVariable(ScopeOverrideEnvVar, null); - - [TestCleanup] - public void TestCleanup() => Environment.SetEnvironmentVariable(ScopeOverrideEnvVar, null); - - [TestMethod] - public void ExportConfig_Scope_Endpoint_And_Paths_AreConsistent() - { - // Auth scope - var scopes = EnvironmentUtils.GetObservabilityAuthenticationScope(); - scopes.Should().ContainSingle() - .Which.Should().Be(ExpectedScope, - "ProdObservabilityScope changed — also review DefaultEndpointHost and BuildEndpointPath."); - - // Full URI combining DefaultEndpointHost + BuildEndpointPath + BuildRequestUri. - var core = new Agent365ExporterCore( - new ExportFormatter(NullLogger.Instance), - NullLogger.Instance); - - var s2sUri = core.BuildRequestUri( - Agent365ExporterOptions.DefaultEndpointHost, - core.BuildEndpointPath("t1", "a1")); - - s2sUri.Should().Be(ExpectedS2SUri, - "S2S export URI changed — also review ProdObservabilityScope and DefaultEndpointHost."); - - // Coarse sanity: scope targets Agent365 Observability, endpoint targets agent365 service - scopes[0].Should().EndWith("/.default"); - Agent365ExporterOptions.DefaultEndpointHost.Should().Contain("agent365"); - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Exporters/ObservabilityBuilderExtensionsTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Exporters/ObservabilityBuilderExtensionsTests.cs deleted file mode 100644 index 67ca662d..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Exporters/ObservabilityBuilderExtensionsTests.cs +++ /dev/null @@ -1,92 +0,0 @@ -using FluentAssertions; -using Microsoft.Extensions.DependencyInjection; -using Microsoft.Extensions.Logging; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters; -using OpenTelemetry.Trace; - -namespace Microsoft.Agents.A365.Observability.Tests.Tracing.Exporters; - -[TestClass] -public sealed class ObservabilityBuilderExtensionsTests -{ - [TestMethod] - public void AddAgent365Exporter_WithCustomBatchingParameters_ShouldConfigureProcessor() - { - // Arrange - var services = new ServiceCollection(); - services.AddLogging(); - - // Configure custom batching parameters - services.AddSingleton(new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token"), - MaxQueueSize = 4096, - ScheduledDelayMilliseconds = 10000, - ExporterTimeoutMilliseconds = 60000, - MaxExportBatchSize = 1024 - }); - - var tracerProviderBuilder = services.AddOpenTelemetry().WithTracing(builder => - { - builder.AddAgent365Exporter(); - }); - - // Act - var serviceProvider = services.BuildServiceProvider(); - - // Assert - Service provider should be built successfully with custom parameters - serviceProvider.Should().NotBeNull(); - - // Verify the options were registered correctly - var options = serviceProvider.GetRequiredService(); - options.MaxQueueSize.Should().Be(4096); - options.ScheduledDelayMilliseconds.Should().Be(10000); - options.ExporterTimeoutMilliseconds.Should().Be(60000); - options.MaxExportBatchSize.Should().Be(1024); - } - - [TestMethod] - public void AddAgent365Exporter_WithDefaultBatchingParameters_ShouldUseDefaults() - { - // Arrange - var services = new ServiceCollection(); - services.AddLogging(); - - // Configure with default batching parameters - services.AddSingleton(new Agent365ExporterOptions - { - TokenResolver = (_, _) => Task.FromResult("test-token") - }); - - var tracerProviderBuilder = services.AddOpenTelemetry().WithTracing(builder => - { - builder.AddAgent365Exporter(); - }); - - // Act - var serviceProvider = services.BuildServiceProvider(); - - // Assert - Service provider should be built successfully with default parameters - serviceProvider.Should().NotBeNull(); - - // Verify the default options - var options = serviceProvider.GetRequiredService(); - options.MaxQueueSize.Should().Be(2048); - options.ScheduledDelayMilliseconds.Should().Be(5000); - options.ExporterTimeoutMilliseconds.Should().Be(30000); - options.MaxExportBatchSize.Should().Be(512); - } - - [TestMethod] - public void AddAgent365Exporter_ThrowsWhenTracerProviderBuilderIsNull() - { - // Arrange - TracerProviderBuilder? builder = null; - - // Act - Action act = () => builder!.AddAgent365Exporter(); - - // Assert - act.Should().Throw().WithParameterName("builder"); - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Exporters/PayloadChunkingTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Exporters/PayloadChunkingTests.cs deleted file mode 100644 index 39ed0667..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Exporters/PayloadChunkingTests.cs +++ /dev/null @@ -1,188 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Exporters; -using System.Diagnostics; - -namespace Microsoft.Agents.A365.Observability.Tests.Tracing.Exporters; - -[TestClass] -public sealed class PayloadChunkingTests -{ - private static Activity CreateActivity(string displayName, IDictionary? tags = null) - { - using var listener = new ActivityListener - { - ShouldListenTo = source => source.Name == "Agent365Sdk.Test.Chunking", - Sample = (ref ActivityCreationOptions _) => ActivitySamplingResult.AllDataAndRecorded, - ActivityStarted = _ => { }, - ActivityStopped = _ => { } - }; - ActivitySource.AddActivityListener(listener); - - using var source = new ActivitySource("Agent365Sdk.Test.Chunking"); - var activity = source.StartActivity(displayName, ActivityKind.Client) - ?? throw new InvalidOperationException("Failed to start activity."); - if (tags != null) - { - foreach (var kvp in tags) - { - activity.SetTag(kvp.Key, kvp.Value); - } - } - activity.Stop(); - return activity; - } - - // ----------------------------------------------------------------------- - // EstimateValueBytes - // ----------------------------------------------------------------------- - - [TestMethod] - public void EstimateValueBytes_String_ReturnsHeaderPlusUtf8ByteCount() - { - PayloadChunking.EstimateValueBytes("hello").Should().Be(40 + 5); - } - - [TestMethod] - public void EstimateValueBytes_EmptyArray_ReturnsBaseOverhead() - { - PayloadChunking.EstimateValueBytes(Array.Empty()).Should().Be(60); - } - - [TestMethod] - public void EstimateValueBytes_StringArray_SumsPerElementOverhead() - { - PayloadChunking.EstimateValueBytes(new[] { "a", "bb" }) - .Should().Be(60 + (40 + 1) + (40 + 2)); - } - - [TestMethod] - public void EstimateValueBytes_NumericArray_UsesFlatPerElementCost() - { - PayloadChunking.EstimateValueBytes(new[] { 1, 2 }).Should().Be(60 + 50 * 2); - } - - [TestMethod] - public void EstimateValueBytes_Scalars_ReturnsScalarOverhead() - { - PayloadChunking.EstimateValueBytes(true).Should().Be(40); - PayloadChunking.EstimateValueBytes(42).Should().Be(40); - PayloadChunking.EstimateValueBytes(null).Should().Be(40); - } - - // ----------------------------------------------------------------------- - // EstimateActivityBytes - // ----------------------------------------------------------------------- - - [TestMethod] - public void EstimateActivityBytes_GrowsWithAttributeContent() - { - var small = CreateActivity("test", new Dictionary { ["key"] = "val" }); - var large = CreateActivity("test", new Dictionary { ["key"] = new string('x', 10_000) }); - - PayloadChunking.EstimateActivityBytes(large) - .Should().BeGreaterThan(PayloadChunking.EstimateActivityBytes(small)); - } - - [TestMethod] - public void EstimateActivityBytes_OverEstimatesActualJsonSize() - { - // A representative span: many attributes including some large strings. - var attrs = new Dictionary - { - ["gen_ai.system"] = "openai", - ["gen_ai.tool.call.arguments"] = new string('x', 1000), - ["gen_ai.tool.call.result"] = new string('y', 1000), - }; - var activity = CreateActivity("test", attrs); - - // The estimator targets the eventual OTLP JSON span shape (with traceId, spanId, - // status, scope wrapper, etc.). A simple Dictionary serialization is a strict - // lower bound, so the estimate should comfortably exceed it. - var dictJson = System.Text.Json.JsonSerializer.Serialize(attrs); - var actualLowerBound = System.Text.Encoding.UTF8.GetByteCount(dictJson); - - PayloadChunking.EstimateActivityBytes(activity) - .Should().BeGreaterThan(actualLowerBound); - } - - [TestMethod] - public void EstimateActivityBytes_NullActivity_Throws() - { - var act = () => PayloadChunking.EstimateActivityBytes(null!); - act.Should().Throw(); - } - - // ----------------------------------------------------------------------- - // ChunkBySize - // ----------------------------------------------------------------------- - - [TestMethod] - public void ChunkBySize_EmptyInput_ReturnsEmptyOutput() - { - var result = PayloadChunking.ChunkBySize(Array.Empty(), x => x, 900_000); - result.Should().BeEmpty(); - } - - [TestMethod] - public void ChunkBySize_SmallItemsFitInOneChunk() - { - var items = Enumerable.Range(0, 10).Select(i => 100).ToList(); - var chunks = PayloadChunking.ChunkBySize(items, x => x, 900_000); - chunks.Should().HaveCount(1); - chunks[0].Should().HaveCount(10); - } - - [TestMethod] - public void ChunkBySize_SplitsWhenCumulativeExceedsLimitAndPreservesOrder() - { - var items = Enumerable.Range(0, 5).Select(i => (Id: $"s{i}", Size: 300_000L)).ToList(); - var chunks = PayloadChunking.ChunkBySize(items, x => x.Size, 900_000); - chunks.Count.Should().BeGreaterOrEqualTo(2); - chunks.SelectMany(c => c).Select(x => x.Id) - .Should().Equal("s0", "s1", "s2", "s3", "s4"); - } - - [TestMethod] - public void ChunkBySize_OversizeSingleItemGetsItsOwnChunk() - { - var chunks = PayloadChunking.ChunkBySize( - new[] { (Id: "big", Size: 2_000_000L) }, - x => x.Size, - 900_000); - chunks.Should().HaveCount(1); - chunks[0][0].Id.Should().Be("big"); - } - - [TestMethod] - public void ChunkBySize_MultiItemChunksRespectLimitAndAreNonEmpty() - { - var items = Enumerable.Range(0, 5).Select(i => (Id: $"s{i}", Size: 200_000L)).ToList(); - var chunks = PayloadChunking.ChunkBySize(items, x => x.Size, 500_000); - - foreach (var chunk in chunks) - { - chunk.Count.Should().BeGreaterThan(0); - if (chunk.Count > 1) - { - chunk.Sum(x => x.Size).Should().BeLessOrEqualTo(500_000); - } - } - } - - [TestMethod] - public void ChunkBySize_NullItems_Throws() - { - var act = () => PayloadChunking.ChunkBySize(null!, x => x, 100); - act.Should().Throw(); - } - - [TestMethod] - public void ChunkBySize_NullGetSize_Throws() - { - var act = () => PayloadChunking.ChunkBySize(new[] { 1 }, null!, 100); - act.Should().Throw(); - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/MessageUtilsTest.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/MessageUtilsTest.cs deleted file mode 100644 index ede1c6d4..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/MessageUtilsTest.cs +++ /dev/null @@ -1,134 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Tests.Tracing; - -using System.Text.Json; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.Tracing; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts.Messages; - -[TestClass] -public sealed class MessageUtilsTest -{ - [TestMethod] - public void NormalizeInputMessages_FromStrings_WrapsAsUserRole() - { - var result = MessageUtils.NormalizeInputMessages(new[] { "Hello" }); - result.Messages.Should().HaveCount(1); - result.Messages[0].Role.Should().Be(MessageRole.User); - result.Messages[0].Parts.Should().HaveCount(1); - result.Messages[0].Parts[0].Should().BeOfType().Which.Content.Should().Be("Hello"); - } - - [TestMethod] - public void NormalizeOutputMessages_FromStrings_WrapsAsAssistantRole() - { - var result = MessageUtils.NormalizeOutputMessages(new[] { "Response" }); - result.Messages.Should().HaveCount(1); - result.Messages[0].Role.Should().Be(MessageRole.Assistant); - result.Messages[0].Parts[0].Should().BeOfType().Which.Content.Should().Be("Response"); - result.Messages[0].FinishReason.Should().Be("stop"); - } - - [TestMethod] - public void SerializeMessages_InputMessages_ProducesValidJsonArray() - { - var wrapper = new InputMessages(new[] - { - new ChatMessage(MessageRole.User, new IMessagePart[] { new TextPart("Hello") }) - }); - var json = MessageUtils.Serialize(wrapper); - var doc = JsonDocument.Parse(json); - doc.RootElement.ValueKind.Should().Be(JsonValueKind.Array); - doc.RootElement.GetArrayLength().Should().Be(1); - doc.RootElement[0].GetProperty("role").GetString().Should().Be("user"); - doc.RootElement[0].GetProperty("parts")[0].GetProperty("type").GetString().Should().Be("text"); - doc.RootElement[0].GetProperty("parts")[0].GetProperty("content").GetString().Should().Be("Hello"); - } - - [TestMethod] - public void SerializeMessages_OutputMessages_IncludesFinishReason() - { - var wrapper = new OutputMessages(new[] - { - new OutputMessage(MessageRole.Assistant, new IMessagePart[] { new TextPart("Answer") }, finishReason: "stop") - }); - var json = MessageUtils.Serialize(wrapper); - var doc = JsonDocument.Parse(json); - doc.RootElement[0].GetProperty("finish_reason").GetString().Should().Be("stop"); - } - - [TestMethod] - public void SerializeMessages_OutputMessages_DefaultsFinishReasonToStop() - { - var wrapper = new OutputMessages(new[] - { - new OutputMessage(MessageRole.Assistant, new IMessagePart[] { new TextPart("Answer") }) - }); - var json = MessageUtils.Serialize(wrapper); - var doc = JsonDocument.Parse(json); - doc.RootElement[0].GetProperty("finish_reason").GetString().Should().Be("stop"); - } - - [TestMethod] - public void SerializeMessages_OmitsNullValues() - { - var wrapper = new InputMessages(new[] - { - new ChatMessage(MessageRole.User, new IMessagePart[] { new TextPart("Hi") }) - }); - var json = MessageUtils.Serialize(wrapper); - json.Should().NotContain("\"name\""); - } - - [TestMethod] - public void SerializeMessages_EnumSerializesAsSnakeCaseString() - { - var wrapper = new InputMessages(new[] - { - new ChatMessage(MessageRole.User, new IMessagePart[] { new TextPart("test") }) - }); - var json = MessageUtils.Serialize(wrapper); - json.Should().Contain("\"role\":\"user\""); - } - - [TestMethod] - public void SerializeMessages_PreservesUnicode() - { - var content = "日本語テスト 🚀"; - var wrapper = new InputMessages(new[] - { - new ChatMessage(MessageRole.User, new IMessagePart[] { new TextPart(content) }) - }); - var json = MessageUtils.Serialize(wrapper); - var doc = JsonDocument.Parse(json); - doc.RootElement[0].GetProperty("parts")[0] - .GetProperty("content").GetString().Should().Be(content); - } - - [TestMethod] - public void SerializeMessages_EmptyMessages_ProducesEmptyArray() - { - var wrapper = new InputMessages(Array.Empty()); - var json = MessageUtils.Serialize(wrapper); - var doc = JsonDocument.Parse(json); - doc.RootElement.ValueKind.Should().Be(JsonValueKind.Array); - doc.RootElement.GetArrayLength().Should().Be(0); - } - - [TestMethod] - public void SerializeMessages_MultipleMessages_ProducesArray() - { - var wrapper = new InputMessages(new[] - { - new ChatMessage(MessageRole.System, new IMessagePart[] { new TextPart("You are helpful.") }), - new ChatMessage(MessageRole.User, new IMessagePart[] { new TextPart("Question?") }) - }); - var json = MessageUtils.Serialize(wrapper); - var doc = JsonDocument.Parse(json); - doc.RootElement.GetArrayLength().Should().Be(2); - doc.RootElement[0].GetProperty("role").GetString().Should().Be("system"); - doc.RootElement[1].GetProperty("role").GetString().Should().Be("user"); - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Processors/ActivityTest.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Processors/ActivityTest.cs deleted file mode 100644 index c4097579..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Processors/ActivityTest.cs +++ /dev/null @@ -1,119 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Tests.Tracing; - -using System.Diagnostics; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Processors; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using OpenTelemetry; -using OpenTelemetry.Trace; -using static Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes.OpenTelemetryConstants; - -public abstract class ActivityTest -{ - protected const string AgentId = "agentId"; - - protected readonly InvokeAgentScopeDetails ScopeDetails = new( - endpoint: new Uri("https://microsoft.com")); - - protected readonly AgentDetails TestAgentDetails = new(AgentId); - - protected ActivityTest() - { - AppContext.SetSwitch(EnableOpenTelemetrySwitch, true); - } - - protected TracerProvider ConstructTracerProvider() - { - return Sdk.CreateTracerProviderBuilder() - .AddSource(SourceName) - .AddProcessor(new ActivityProcessor()) - .Build(); - } - - protected Activity ListenForActivity(Action action) - { - Activity? startedActivity = null; - using var activityListener = new ActivityListener(); - activityListener.ShouldListenTo = source => source.Name == SourceName; - activityListener.Sample = (ref ActivityCreationOptions _) => ActivitySamplingResult.AllData; - activityListener.ActivityStarted = activity => startedActivity = activity; - ActivitySource.AddActivityListener(activityListener); - action(); - startedActivity.Should().NotBeNull(); - return startedActivity!; - } - - protected Activity CreateActivity( - string sourceName = SourceName, - string displayName = "test-span", - ActivityKind kind = ActivityKind.Server, - DateTime? startTimeUtc = null, - TimeSpan? duration = null, - Dictionary? tags = null, - List? events = null, - List? links = null, - ActivitySpanId? parentSpanId = null, - ActivityStatusCode status = ActivityStatusCode.Ok, - string? statusDescription = null) - { - var source = new ActivitySource(sourceName); - - using var listener = new ActivityListener - { - ShouldListenTo = s => s.Name == sourceName, - Sample = (ref ActivityCreationOptions _) => ActivitySamplingResult.AllDataAndRecorded, - ActivityStarted = _ => { }, - ActivityStopped = _ => { } - }; - ActivitySource.AddActivityListener(listener); - - Activity? activity; - if (parentSpanId.HasValue) - { - var parentContext = new ActivityContext( - ActivityTraceId.CreateRandom(), - parentSpanId.Value, - ActivityTraceFlags.Recorded); - activity = source.StartActivity(displayName, kind, parentContext); - } - else - { - activity = source.StartActivity(displayName, kind); - } - - if (activity == null) - throw new InvalidOperationException("Failed to start activity."); - - if (startTimeUtc.HasValue) - activity.SetStartTime(startTimeUtc.Value); - - if (duration.HasValue) - activity.SetEndTime(activity.StartTimeUtc + duration.Value); - - if (tags != null) - { - foreach (var tag in tags) - activity.SetTag(tag.Key, tag.Value); - } - - if (events != null) - { - foreach (var ev in events) - activity.AddEvent(ev); - } - - if (links != null) - { - foreach (var link in links) - activity.AddLink(link); - } - - activity.SetStatus(status, statusDescription); - - activity.Stop(); - return activity; - } -} \ No newline at end of file diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/ApplyGuardrailScopeTest.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/ApplyGuardrailScopeTest.cs deleted file mode 100644 index f429369e..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/ApplyGuardrailScopeTest.cs +++ /dev/null @@ -1,286 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Tests.Tracing.Scopes; - -using System; -using System.Linq; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; - -[TestClass] -public sealed class ApplyGuardrailScopeTest : ActivityTest -{ - [TestMethod] - public void Start_SetsRequiredAttributes() - { - var details = new GuardrailDetails( - targetType: GuardrailTargetType.LlmInput, - decisionType: GuardrailDecisionType.Deny); - - var activity = ListenForActivity(() => - { - using var scope = ApplyGuardrailScope.Start(details, Util.GetAgentDetails()); - }); - - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiOperationNameKey, OpenTelemetryConstants.ApplyGuardrailOperationName); - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiSecurityDecisionTypeKey, "deny"); - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiSecurityTargetTypeKey, GuardrailTargetType.LlmInput); - } - - [TestMethod] - public void Start_SetsGuardianAttributes_WhenProvided() - { - var details = new GuardrailDetails( - targetType: GuardrailTargetType.LlmOutput, - decisionType: GuardrailDecisionType.Allow, - guardianName: "PII Filter", - guardianId: "guard_abc123", - guardianProviderName: "azure.ai.content_safety", - guardianVersion: "2.1.0"); - - var activity = ListenForActivity(() => - { - using var scope = ApplyGuardrailScope.Start(details, Util.GetAgentDetails()); - }); - - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiGuardianNameKey, "PII Filter"); - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiGuardianIdKey, "guard_abc123"); - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiGuardianProviderNameKey, "azure.ai.content_safety"); - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiGuardianVersionKey, "2.1.0"); - } - - [TestMethod] - public void Start_SetsPolicyAttributes_WhenProvided() - { - var details = new GuardrailDetails( - targetType: GuardrailTargetType.ToolCall, - decisionType: GuardrailDecisionType.Modify, - policyId: "policy_pii_v2", - policyName: "PII Protection Policy", - policyVersion: "1.0"); - - var activity = ListenForActivity(() => - { - using var scope = ApplyGuardrailScope.Start(details, Util.GetAgentDetails()); - }); - - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiSecurityPolicyIdKey, "policy_pii_v2"); - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiSecurityPolicyNameKey, "PII Protection Policy"); - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiSecurityPolicyVersionKey, "1.0"); - } - - [TestMethod] - public void Start_SpanName_IncludesGuardianNameAndTargetType() - { - var details = new GuardrailDetails( - targetType: GuardrailTargetType.LlmInput, - decisionType: GuardrailDecisionType.Allow, - guardianName: "Content Safety"); - - var activity = ListenForActivity(() => - { - using var scope = ApplyGuardrailScope.Start(details, Util.GetAgentDetails()); - }); - - activity.DisplayName.Should().Be("apply_guardrail Content Safety llm_input"); - } - - [TestMethod] - public void Start_SpanName_OmitsGuardianName_WhenNull() - { - var details = new GuardrailDetails( - targetType: GuardrailTargetType.LlmOutput, - decisionType: GuardrailDecisionType.Deny); - - var activity = ListenForActivity(() => - { - using var scope = ApplyGuardrailScope.Start(details, Util.GetAgentDetails()); - }); - - activity.DisplayName.Should().Be("apply_guardrail llm_output"); - } - - [TestMethod] - public void Start_SetsContentAttributes_WhenProvided() - { - var details = new GuardrailDetails( - targetType: GuardrailTargetType.LlmOutput, - decisionType: GuardrailDecisionType.Modify, - contentInputHash: "sha256:a3f2b8c9", - contentModified: true); - - var activity = ListenForActivity(() => - { - using var scope = ApplyGuardrailScope.Start(details, Util.GetAgentDetails()); - }); - - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiSecurityContentInputHashKey, "sha256:a3f2b8c9"); - var modifiedTag = activity.TagObjects.First(t => t.Key == OpenTelemetryConstants.GenAiSecurityContentModifiedKey); - modifiedTag.Value.Should().Be(true); - } - - [TestMethod] - public void RecordDecision_UpdatesDecisionType() - { - var details = new GuardrailDetails( - targetType: GuardrailTargetType.LlmInput, - decisionType: GuardrailDecisionType.Allow); - - var activity = ListenForActivity(() => - { - using var scope = ApplyGuardrailScope.Start(details, Util.GetAgentDetails()); - scope.RecordDecision(GuardrailDecisionType.Deny, "Prompt injection detected"); - }); - - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiSecurityDecisionTypeKey, "deny"); - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiSecurityDecisionReasonKey, "Prompt injection detected"); - } - - [TestMethod] - public void RecordContentOutput_SetsOutputValue() - { - var details = new GuardrailDetails( - targetType: GuardrailTargetType.LlmOutput, - decisionType: GuardrailDecisionType.Modify); - - var activity = ListenForActivity(() => - { - using var scope = ApplyGuardrailScope.Start(details, Util.GetAgentDetails()); - scope.RecordContentOutput("Redacted content here"); - }); - - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiSecurityContentOutputValueKey, "Redacted content here"); - } - - [TestMethod] - public void RecordFinding_EmitsSingleEvent() - { - var details = new GuardrailDetails( - targetType: GuardrailTargetType.LlmInput, - decisionType: GuardrailDecisionType.Deny); - - var activity = ListenForActivity(() => - { - using var scope = ApplyGuardrailScope.Start(details, Util.GetAgentDetails()); - scope.RecordFinding(new GuardrailFinding( - riskCategory: "prompt_injection", - riskSeverity: GuardrailRiskSeverity.High, - policyDecisionType: "deny")); - }); - - activity.Events.Should().HaveCount(1); - var evt = activity.Events.First(); - evt.Name.Should().Be(OpenTelemetryConstants.GenAiSecurityFindingEventName); - evt.Tags.First(t => t.Key == OpenTelemetryConstants.GenAiSecurityRiskCategoryKey).Value.Should().Be("prompt_injection"); - evt.Tags.First(t => t.Key == OpenTelemetryConstants.GenAiSecurityRiskSeverityKey).Value.Should().Be("high"); - evt.Tags.First(t => t.Key == OpenTelemetryConstants.GenAiSecurityPolicyDecisionTypeKey).Value.Should().Be("deny"); - } - - [TestMethod] - public void RecordFinding_EmitsMultipleEvents() - { - var details = new GuardrailDetails( - targetType: GuardrailTargetType.LlmOutput, - decisionType: GuardrailDecisionType.Modify); - - var activity = ListenForActivity(() => - { - using var scope = ApplyGuardrailScope.Start(details, Util.GetAgentDetails()); - scope.RecordFinding(new GuardrailFinding( - riskCategory: "pii", - riskSeverity: GuardrailRiskSeverity.Medium, - policyDecisionType: "modify")); - scope.RecordFinding(new GuardrailFinding( - riskCategory: "toxicity", - riskSeverity: GuardrailRiskSeverity.Low)); - }); - - activity.Events.Should().HaveCount(2); - } - - [TestMethod] - public void RecordFinding_IncludesAllAttributes() - { - var details = new GuardrailDetails( - targetType: GuardrailTargetType.LlmInput, - decisionType: GuardrailDecisionType.Deny); - - var activity = ListenForActivity(() => - { - using var scope = ApplyGuardrailScope.Start(details, Util.GetAgentDetails()); - scope.RecordFinding(new GuardrailFinding( - riskCategory: "sensitive_info_disclosure", - riskSeverity: GuardrailRiskSeverity.High, - policyDecisionType: "deny", - policyId: "policy_pii_v2", - policyName: "PII Policy", - policyVersion: "2.0", - riskScore: 0.92, - riskMetadata: new[] { "pattern:ssn", "count:2" })); - }); - - var evt = activity.Events.First(); - evt.Tags.First(t => t.Key == OpenTelemetryConstants.GenAiSecurityRiskCategoryKey).Value.Should().Be("sensitive_info_disclosure"); - evt.Tags.First(t => t.Key == OpenTelemetryConstants.GenAiSecurityRiskSeverityKey).Value.Should().Be("high"); - evt.Tags.First(t => t.Key == OpenTelemetryConstants.GenAiSecurityPolicyDecisionTypeKey).Value.Should().Be("deny"); - evt.Tags.First(t => t.Key == OpenTelemetryConstants.GenAiSecurityPolicyIdKey).Value.Should().Be("policy_pii_v2"); - evt.Tags.First(t => t.Key == OpenTelemetryConstants.GenAiSecurityPolicyNameKey).Value.Should().Be("PII Policy"); - evt.Tags.First(t => t.Key == OpenTelemetryConstants.GenAiSecurityPolicyVersionKey).Value.Should().Be("2.0"); - evt.Tags.First(t => t.Key == OpenTelemetryConstants.GenAiSecurityRiskScoreKey).Value.Should().Be(0.92); - evt.Tags.First(t => t.Key == OpenTelemetryConstants.GenAiSecurityRiskMetadataKey).Value.Should().BeEquivalentTo(new[] { "pattern:ssn", "count:2" }); - } - - [TestMethod] - public void RecordError_SetsExpectedFields() - { - const string expected = "Guardian service unavailable"; - var details = new GuardrailDetails( - targetType: GuardrailTargetType.LlmInput, - decisionType: GuardrailDecisionType.Allow); - - var activity = ListenForActivity(() => - { - using var scope = ApplyGuardrailScope.Start(details, Util.GetAgentDetails()); - scope.RecordError(new Exception(expected)); - }); - - activity.ShouldBeError(expected); - } - - [TestMethod] - public void Start_SetsConversationId_WhenProvided() - { - var conversationId = "conv-guardrail-123"; - var details = new GuardrailDetails( - targetType: GuardrailTargetType.LlmInput, - decisionType: GuardrailDecisionType.Allow); - - var activity = ListenForActivity(() => - { - using var scope = ApplyGuardrailScope.Start( - details, - Util.GetAgentDetails(), - request: new Request(conversationId: conversationId)); - }); - - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiConversationIdKey, conversationId); - } - - [TestMethod] - public void Start_SetsExternalEventId_WhenProvided() - { - var details = new GuardrailDetails( - targetType: GuardrailTargetType.Message, - decisionType: GuardrailDecisionType.Audit, - externalEventId: "evt_abc123"); - - var activity = ListenForActivity(() => - { - using var scope = ApplyGuardrailScope.Start(details, Util.GetAgentDetails()); - }); - - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiSecurityExternalEventIdKey, "evt_abc123"); - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/ExecuteToolScopeTest.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/ExecuteToolScopeTest.cs deleted file mode 100644 index 2c86d4ff..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/ExecuteToolScopeTest.cs +++ /dev/null @@ -1,325 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Tests.Tracing.Scopes; - -using System; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; - -[TestClass] -public sealed class ExecuteToolScopeTest : ActivityTest -{ - [TestMethod] - public void Start_Arguments_Set() - { - const string expected = "Input: 42"; - var activity = ListenForActivity(() => - { - using var scope = ExecuteToolScope.Start(Util.GetDefaultRequest(), new ToolCallDetails("TestTool", expected), Util.GetAgentDetails()); - }); - - activity.ShouldHaveTagContaining(OpenTelemetryConstants.GenAiToolArgumentsKey, expected); - } - - [TestMethod] - public void RecordResponse_Response_Set() - { - const string expected = "Output: 42"; - var activity = ListenForActivity(() => - { - using var scope = ExecuteToolScope.Start(Util.GetDefaultRequest(), new ToolCallDetails("TestTool", "x"), Util.GetAgentDetails()); - scope.RecordResponse(expected); - }); - - var tagValue = activity.Tags.First(t => t.Key == OpenTelemetryConstants.GenAiToolCallResultKey).Value; - tagValue.Should().Contain("\"result\""); - tagValue.Should().Contain(expected); - } - - [TestMethod] - public void RecordError_SetsExpectedFields() - { - const string expected = "Test error"; - var activity = ListenForActivity(() => - { - using var scope = ExecuteToolScope.Start(Util.GetDefaultRequest(), new ToolCallDetails("TestTool", "x"), Util.GetAgentDetails()); - scope?.RecordError(new Exception(expected)); - }); - - activity.ShouldBeError(expected); - } - - [TestMethod] - public void SetStartTime_SetsActivityStartTime() - { - var customStartTime = DateTimeOffset.UtcNow.AddMinutes(-5); - var activity = ListenForActivity(() => - { - using var scope = ExecuteToolScope.Start( - Util.GetDefaultRequest(), - new ToolCallDetails("TestTool", "args"), - Util.GetAgentDetails()); - scope.SetStartTime(customStartTime); - }); - - // Activity start time should be close to the custom start time - var startTime = new DateTimeOffset(activity.StartTimeUtc); - startTime.Should().BeCloseTo(customStartTime, TimeSpan.FromMilliseconds(100)); - } - - [TestMethod] - public void Start_SetsConversationId_WhenProvided() - { - var conversationId = "conv-tool-123"; - var activity = ListenForActivity(() => - { - using var scope = ExecuteToolScope.Start( - new Request(conversationId: conversationId), - new ToolCallDetails("TestTool", "args"), - Util.GetAgentDetails()); - }); - - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiConversationIdKey, conversationId); - } - - [TestMethod] - public void Start_SetsChannel_Tags() - { - var metadata = new Channel(name: "ChannelY", link: "https://channel/link/y"); - - var activity = ListenForActivity(() => - { - using var scope = ExecuteToolScope.Start( - new Request(channel: metadata), - new ToolCallDetails("TestTool", "args"), - Util.GetAgentDetails()); - }); - - activity.ShouldHaveTag(OpenTelemetryConstants.ChannelNameKey, metadata.Name!); - activity.ShouldHaveTag(OpenTelemetryConstants.ChannelLinkKey, metadata.Link!); - } - - [TestMethod] - public void ThreatDiagnosticsSummary_IsSetCorrectly_WhenProvided() - { - // Arrange - var threatSummary = new ThreatDiagnosticsSummary( - blockAction: false, - reasonCode: 0, - reason: "No threat detected.", - diagnostics: null); - var toolCallDetails = new ToolCallDetails("SecurityTool", "scan args"); - var agentDetails = Util.GetAgentDetails(); - - // Act - var activity = ListenForActivity(() => - { - using var scope = ExecuteToolScope.Start( - Util.GetDefaultRequest(), - toolCallDetails, - agentDetails, - threatDiagnosticsSummary: threatSummary); - }); - - // Assert - activity.ShouldHaveTag(OpenTelemetryConstants.ThreatDiagnosticsSummaryKey, "{\"blockAction\":false,\"reasonCode\":0,\"reason\":\"No threat detected.\",\"diagnostics\":null}"); - } - - [TestMethod] - public void ThreatDiagnosticsSummary_IsNotSet_WhenNull() - { - // Arrange - var toolCallDetails = new ToolCallDetails("TestTool", "args"); - var agentDetails = Util.GetAgentDetails(); - - // Act - var activity = ListenForActivity(() => - { - using var scope = ExecuteToolScope.Start( - Util.GetDefaultRequest(), - toolCallDetails, - agentDetails, - threatDiagnosticsSummary: null); - }); - - // Assert - activity.Tags.Should().NotContainKey(OpenTelemetryConstants.ThreatDiagnosticsSummaryKey); - } - - [TestMethod] - public void RecordThreatDiagnosticsSummary_SetsTagCorrectly() - { - // Arrange - var threatSummary = new ThreatDiagnosticsSummary( - blockAction: true, - reasonCode: 200, - reason: "Blocked due to policy violation.", - diagnostics: "{\"policy\":\"data-loss-prevention\"}"); - var toolCallDetails = new ToolCallDetails("TestTool", "args"); - var agentDetails = Util.GetAgentDetails(); - - // Act - var activity = ListenForActivity(() => - { - using var scope = ExecuteToolScope.Start(Util.GetDefaultRequest(), toolCallDetails, agentDetails); - scope.RecordThreatDiagnosticsSummary(threatSummary); - }); - - // Assert - var tagValue = activity.Tags.First(t => t.Key == OpenTelemetryConstants.ThreatDiagnosticsSummaryKey).Value; - tagValue.Should().Contain("\"blockAction\":true"); - tagValue.Should().Contain("\"reasonCode\":200"); - tagValue.Should().Contain("\"reason\":\"Blocked due to policy violation.\""); - tagValue.Should().Contain("data-loss-prevention"); - } - - [TestMethod] - public void Start_ToolServerName_IsSetCorrectly() - { - // Arrange - const string expectedToolServerName = "test-tool-server"; - var toolCallDetails = new ToolCallDetails( - toolName: "TestTool", - arguments: "args", - toolServerName: expectedToolServerName); - var agentDetails = Util.GetAgentDetails(); - - // Act - var activity = ListenForActivity(() => - { - using var scope = ExecuteToolScope.Start(Util.GetDefaultRequest(), toolCallDetails, agentDetails); - }); - - // Assert - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiToolServerNameKey, expectedToolServerName); - } - - [TestMethod] - public void Start_SetsCallerDetails_WhenProvided() - { - // Arrange - var userDetails = new UserDetails( - userId: "caller-123", - userName: "Test Caller", - userEmail: "caller@example.com", - userClientIP: System.Net.IPAddress.Parse("192.168.1.1")); - - // Act - var activity = ListenForActivity(() => - { - using var scope = ExecuteToolScope.Start( - Util.GetDefaultRequest(), - new ToolCallDetails("TestTool", "args"), - Util.GetAgentDetails(), - userDetails: userDetails); - }); - - // Assert - activity.ShouldHaveTag(OpenTelemetryConstants.UserIdKey, userDetails.UserId!); - activity.ShouldHaveTag(OpenTelemetryConstants.UserNameKey, userDetails.UserName!); - activity.ShouldHaveTag(OpenTelemetryConstants.UserEmailKey, userDetails.UserEmail!); - activity.ShouldHaveTag(OpenTelemetryConstants.CallerClientIpKey, userDetails.UserClientIP!.ToString()); - } - - [TestMethod] - public void Start_WithCustomStartTime_SetsActivityStartTime() - { - // Arrange - var customStartTime = new DateTimeOffset(2023, 11, 14, 22, 13, 20, TimeSpan.Zero); - - // Act - var activity = ListenForActivity(() => - { - using var scope = ExecuteToolScope.Start( - Util.GetDefaultRequest(), - new ToolCallDetails("TestTool", "args"), - Util.GetAgentDetails(), - spanDetails: new SpanDetails(startTime: customStartTime)); - }); - - // Assert - var startTime = new DateTimeOffset(activity.StartTimeUtc); - startTime.Should().BeCloseTo(customStartTime, TimeSpan.FromMilliseconds(100)); - } - - [TestMethod] - public void Start_WithCustomStartAndEndTime_SetsActivityTimes() - { - // Arrange - var customStartTime = new DateTimeOffset(2023, 11, 14, 22, 13, 20, TimeSpan.Zero); - var customEndTime = new DateTimeOffset(2023, 11, 14, 22, 13, 25, TimeSpan.Zero); // 5 seconds later - - // Act - var activity = ListenForActivity(() => - { - using var scope = ExecuteToolScope.Start( - Util.GetDefaultRequest(), - new ToolCallDetails("TestTool", "args"), - Util.GetAgentDetails(), - spanDetails: new SpanDetails(startTime: customStartTime, endTime: customEndTime)); - }); - - // Assert - Start time should be set to custom time - var startTime = new DateTimeOffset(activity.StartTimeUtc); - startTime.Should().BeCloseTo(customStartTime, TimeSpan.FromMilliseconds(100)); - } - - [TestMethod] - public void SetEndTime_OverridesEndTime() - { - // Arrange - var customStartTime = new DateTimeOffset(2023, 11, 14, 22, 13, 40, TimeSpan.Zero); - var initialEndTime = new DateTimeOffset(2023, 11, 14, 22, 13, 45, TimeSpan.Zero); - var laterEndTime = new DateTimeOffset(2023, 11, 14, 22, 13, 48, TimeSpan.Zero); - - // Act - var activity = ListenForActivity(() => - { - using var scope = ExecuteToolScope.Start( - Util.GetDefaultRequest(), - new ToolCallDetails("TestTool", "args"), - Util.GetAgentDetails(), - spanDetails: new SpanDetails(startTime: customStartTime, endTime: initialEndTime)); - scope.SetEndTime(laterEndTime); - }); - - // Assert - The start time should be set - var startTime = new DateTimeOffset(activity.StartTimeUtc); - startTime.Should().BeCloseTo(customStartTime, TimeSpan.FromMilliseconds(100)); - } - - [TestMethod] - public void SpanKind_DefaultsToInternal() - { - // Act - var activity = ListenForActivity(() => - { - using var scope = ExecuteToolScope.Start( - Util.GetDefaultRequest(), - new ToolCallDetails("TestTool", "args"), - Util.GetAgentDetails()); - }); - - // Assert - activity.Kind.Should().Be(System.Diagnostics.ActivityKind.Internal); - } - - [TestMethod] - public void SpanKind_OverrideToClient() - { - // Act - var activity = ListenForActivity(() => - { - using var scope = ExecuteToolScope.Start( - Util.GetDefaultRequest(), - new ToolCallDetails("TestTool", "args"), - Util.GetAgentDetails(), - spanDetails: new SpanDetails(spanKind: System.Diagnostics.ActivityKind.Client)); - }); - - // Assert - activity.Kind.Should().Be(System.Diagnostics.ActivityKind.Client); - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/InferenceScopeTest.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/InferenceScopeTest.cs deleted file mode 100644 index df2df74d..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/InferenceScopeTest.cs +++ /dev/null @@ -1,321 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Tests.Tracing.Scopes; - -using System; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using static Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes.OpenTelemetryConstants; - -[TestClass] -public sealed class InferenceScopeTest : ActivityTest -{ - [TestMethod] - public void Start_SetsExpectedTags() - { - var details = new InferenceCallDetails( - InferenceOperationType.Chat, - "gpt-4o", - "openai", - 123, - 456, - new[] { "stop", "length" }, - "response-123"); - - var activity = ListenForActivity(() => - { - using var scope = InferenceScope.Start(Util.GetDefaultRequest(), details, Util.GetAgentDetails()); - }); - - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiOperationNameKey, details.OperationName.ToString()); - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiRequestModelKey, details.Model); - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiProviderNameKey, details.ProviderName); - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiUsageInputTokensKey, details.InputTokens!.Value.ToString()); - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiUsageOutputTokensKey, details.OutputTokens!.Value.ToString()); - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiResponseFinishReasonsKey, string.Join(",", details.FinishReasons!)); - } - - [TestMethod] - public void RecordInputTokens_SetsTag() - { - var inputTokens = 789; - var details = new InferenceCallDetails( - InferenceOperationType.Chat, - "gpt-4o", - "openai"); - var activity = ListenForActivity(() => - { - using var scope = InferenceScope.Start(Util.GetDefaultRequest(), details, Util.GetAgentDetails())!; - scope.RecordInputTokens(inputTokens); - }); - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiUsageInputTokensKey, inputTokens.ToString()); - } - - [TestMethod] - public void RecordOutputTokens_SetsTag() - { - var outputTokens = 321; - var details = new InferenceCallDetails( - InferenceOperationType.Chat, - "gpt-4o", - "openai"); - var activity = ListenForActivity(() => - { - using var scope = InferenceScope.Start(Util.GetDefaultRequest(), details, Util.GetAgentDetails())!; - scope.RecordOutputTokens(outputTokens); - }); - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiUsageOutputTokensKey, outputTokens.ToString()); - } - - [TestMethod] - public void RecordFinishReasons_SetsTag() - { - var finishReasons = new[] { "tool_calls", "stop" }; - var details = new InferenceCallDetails( - InferenceOperationType.Chat, - "gpt-4o", - "openai"); - - var activity = ListenForActivity(() => - { - using var scope = InferenceScope.Start(Util.GetDefaultRequest(), details, Util.GetAgentDetails())!; - scope.RecordFinishReasons(finishReasons); - }); - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiResponseFinishReasonsKey, string.Join(",", finishReasons)); - } - - [TestMethod] - public void RecordInputMessages_SetsTag() - { - var messages = new[] { "Hello", "How are you?" }; - var details = new InferenceCallDetails( - InferenceOperationType.Chat, - "gpt-4o", - "openai"); - - var activity = ListenForActivity(() => - { - using var scope = InferenceScope.Start(Util.GetDefaultRequest(), details, Util.GetAgentDetails())!; - scope.RecordInputMessages(messages); - }); - - var tagValue = activity.Tags.First(t => t.Key == "gen_ai.input.messages").Value; - tagValue.Should().StartWith("["); - tagValue.Should().Contain("\"role\":\"user\""); - tagValue.Should().Contain("Hello"); - tagValue.Should().Contain("How are you?"); - } - - [TestMethod] - public void RecordOutputMessages_SetsTag() - { - var messages = new[] { "Hi there!", "I'm fine." }; - var details = new InferenceCallDetails( - InferenceOperationType.Chat, - "gpt-4o", - "openai"); - - var activity = ListenForActivity(() => - { - using var scope = InferenceScope.Start(Util.GetDefaultRequest(), details, Util.GetAgentDetails())!; - scope.RecordOutputMessages(messages); - }); - - var tagValue = activity.Tags.First(t => t.Key == "gen_ai.output.messages").Value; - tagValue.Should().StartWith("["); - tagValue.Should().Contain("\"role\":\"assistant\""); - tagValue.Should().Contain("Hi there!"); - tagValue.Should().Contain("I\\u0027m fine."); - } - - [TestMethod] - public void SetStartTime_SetsActivityStartTime() - { - var customStartTime = DateTimeOffset.UtcNow.AddMinutes(-5); - var details = new InferenceCallDetails( - InferenceOperationType.Chat, - "gpt-4o", - "openai"); - - var activity = ListenForActivity(() => - { - using var scope = InferenceScope.Start(Util.GetDefaultRequest(), details, Util.GetAgentDetails()); - scope.SetStartTime(customStartTime); - }); - - // Activity start time should be close to the custom start time - var startTime = new DateTimeOffset(activity.StartTimeUtc); - startTime.Should().BeCloseTo(customStartTime, TimeSpan.FromMilliseconds(100)); - } - - [TestMethod] - public void Start_SetsConversationId_WhenProvided() - { - var conversationId = "conv-inf-123"; - var details = new InferenceCallDetails( - InferenceOperationType.Chat, - "gpt-4o", - "openai"); - - var activity = ListenForActivity(() => - { - using var scope = InferenceScope.Start( - new Request(conversationId: conversationId), - details, - Util.GetAgentDetails()); - }); - - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiConversationIdKey, conversationId); - } - - [TestMethod] - public void Start_SetsChannel_Tags() - { - var details = new InferenceCallDetails( - InferenceOperationType.Chat, - "gpt-4o", - "openai"); - var metadata = new Channel(name: "ChannelZ", link: "https://channel/link/z"); - - var activity = ListenForActivity(() => - { - using var scope = InferenceScope.Start( - new Request(channel: metadata), - details, - Util.GetAgentDetails()); - }); - - activity.ShouldHaveTag(OpenTelemetryConstants.ChannelNameKey, metadata.Name!); - activity.ShouldHaveTag(OpenTelemetryConstants.ChannelLinkKey, metadata.Link!); - } - - [TestMethod] - public void RecordThoughtProcess_SetsTag() - { - var thoughtProcess = "First, I analyzed the user's request. Then, I determined the best approach would be to provide a structured response. Finally, I composed the answer."; - var details = new InferenceCallDetails( - InferenceOperationType.Chat, - "gpt-4o", - "openai"); - - var activity = ListenForActivity(() => - { - using var scope = InferenceScope.Start(Util.GetDefaultRequest(), details, Util.GetAgentDetails())!; - scope.RecordThoughtProcess(thoughtProcess); - }); - - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiAgentThoughtProcessKey, thoughtProcess); - } - - [TestMethod] - public void Start_SetsCallerDetails_WhenProvided() - { - // Arrange - var userDetails = new UserDetails( - userId: "caller-inf-123", - userName: "Inference Caller", - userEmail: "caller-inf@example.com", - userClientIP: System.Net.IPAddress.Parse("10.0.0.1")); - var details = new InferenceCallDetails( - InferenceOperationType.Chat, - "gpt-4o", - "openai"); - - // Act - var activity = ListenForActivity(() => - { - using var scope = InferenceScope.Start( - Util.GetDefaultRequest(), - details, - Util.GetAgentDetails(), - userDetails: userDetails); - }); - - // Assert - activity.ShouldHaveTag(OpenTelemetryConstants.UserIdKey, userDetails.UserId!); - activity.ShouldHaveTag(OpenTelemetryConstants.UserNameKey, userDetails.UserName!); - activity.ShouldHaveTag(OpenTelemetryConstants.UserEmailKey, userDetails.UserEmail!); - activity.ShouldHaveTag(OpenTelemetryConstants.CallerClientIpKey, userDetails.UserClientIP!.ToString()); - } - - [TestMethod] - public void Start_WithCustomStartTime_SetsActivityStartTime() - { - // Arrange - var customStartTime = new DateTimeOffset(2023, 11, 14, 22, 13, 20, TimeSpan.Zero); - var details = new InferenceCallDetails( - InferenceOperationType.Chat, - "gpt-4o", - "openai"); - - // Act - var activity = ListenForActivity(() => - { - using var scope = InferenceScope.Start( - Util.GetDefaultRequest(), - details, - Util.GetAgentDetails(), - spanDetails: new SpanDetails(startTime: customStartTime)); - }); - - // Assert - var startTime = new DateTimeOffset(activity.StartTimeUtc); - startTime.Should().BeCloseTo(customStartTime, TimeSpan.FromMilliseconds(100)); - } - - [TestMethod] - public void Start_WithCustomStartAndEndTime_SetsActivityTimes() - { - // Arrange - var customStartTime = new DateTimeOffset(2023, 11, 14, 22, 13, 20, TimeSpan.Zero); - var customEndTime = new DateTimeOffset(2023, 11, 14, 22, 13, 25, TimeSpan.Zero); // 5 seconds later - var details = new InferenceCallDetails( - InferenceOperationType.Chat, - "gpt-4o", - "openai"); - - // Act - var activity = ListenForActivity(() => - { - using var scope = InferenceScope.Start( - Util.GetDefaultRequest(), - details, - Util.GetAgentDetails(), - spanDetails: new SpanDetails(startTime: customStartTime, endTime: customEndTime)); - }); - - // Assert - Start time should be set to custom time - var startTime = new DateTimeOffset(activity.StartTimeUtc); - startTime.Should().BeCloseTo(customStartTime, TimeSpan.FromMilliseconds(100)); - } - - [TestMethod] - public void SetEndTime_OverridesEndTime() - { - // Arrange - var customStartTime = new DateTimeOffset(2023, 11, 14, 22, 13, 40, TimeSpan.Zero); - var initialEndTime = new DateTimeOffset(2023, 11, 14, 22, 13, 45, TimeSpan.Zero); - var laterEndTime = new DateTimeOffset(2023, 11, 14, 22, 13, 48, TimeSpan.Zero); - var details = new InferenceCallDetails( - InferenceOperationType.Chat, - "gpt-4o", - "openai"); - - // Act - var activity = ListenForActivity(() => - { - using var scope = InferenceScope.Start( - Util.GetDefaultRequest(), - details, - Util.GetAgentDetails(), - spanDetails: new SpanDetails(startTime: customStartTime, endTime: initialEndTime)); - scope.SetEndTime(laterEndTime); - }); - - // Assert - The start time should be set - var startTime = new DateTimeOffset(activity.StartTimeUtc); - startTime.Should().BeCloseTo(customStartTime, TimeSpan.FromMilliseconds(100)); - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/InvokeAgentScopeTest.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/InvokeAgentScopeTest.cs deleted file mode 100644 index 1287fd35..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/InvokeAgentScopeTest.cs +++ /dev/null @@ -1,382 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Tests.Tracing.Scopes; - -using System; -using System.Diagnostics; -using System.Net; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using static Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes.OpenTelemetryConstants; - -[TestClass] -public sealed class InvokeAgentScopeTest : ActivityTest -{ - [TestMethod] - public void RecordResponse_ActivityTagSet() - { - const string expected = "response"; - - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start(Util.GetDefaultRequest(), ScopeDetails, TestAgentDetails); - scope.RecordResponse(expected); - }); - - var tagValue = activity.Tags.First(t => t.Key == "gen_ai.output.messages").Value; - tagValue.Should().StartWith("["); - tagValue.Should().Contain("\"role\":\"assistant\""); - tagValue.Should().Contain(expected); - } - - [TestMethod] - public void RecordError_SetsExpectedFields() - { - const string expected = "Test error"; - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start(Util.GetDefaultRequest(), ScopeDetails, TestAgentDetails); - scope?.RecordError(new Exception(expected)); - }); - - activity.ShouldBeError(expected); - } - - [TestMethod] - public void RecordInputMessages_ActivityTagSet() - { - var messages = new[] { "Hello", "How are you?" }; - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start(Util.GetDefaultRequest(), ScopeDetails, TestAgentDetails); - scope.RecordInputMessages(messages); - }); - - var tagValue = activity.Tags.First(t => t.Key == "gen_ai.input.messages").Value; - tagValue.Should().StartWith("["); - tagValue.Should().Contain("\"role\":\"user\""); - tagValue.Should().Contain("Hello"); - tagValue.Should().Contain("How are you?"); - } - - [TestMethod] - public void RecordOutputMessages_ActivityTagSet() - { - var messages = new[] { "Hi there!", "I'm fine." }; - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start(Util.GetDefaultRequest(), ScopeDetails, TestAgentDetails); - scope.RecordOutputMessages(messages); - }); - - var tagValue = activity.Tags.First(t => t.Key == "gen_ai.output.messages").Value; - tagValue.Should().StartWith("["); - tagValue.Should().Contain("\"role\":\"assistant\""); - tagValue.Should().Contain("Hi there!"); - tagValue.Should().Contain("I\\u0027m fine."); - } - - [TestMethod] - public void SetStartTime_SetsActivityStartTime() - { - var customStartTime = DateTimeOffset.UtcNow.AddMinutes(-5); - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start(Util.GetDefaultRequest(), ScopeDetails, TestAgentDetails); - scope.SetStartTime(customStartTime); - }); - - // Activity start time should be close to the custom start time - var startTime = new DateTimeOffset(activity.StartTimeUtc); - startTime.Should().BeCloseTo(customStartTime, TimeSpan.FromMilliseconds(100)); - } - - [TestMethod] - public void RequestContent_PopulatesInputMessagesAttribute() - { - const string requestContent = "This is the input message content"; - var request = new Request(content: requestContent); - - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start(request, ScopeDetails, TestAgentDetails); - }); - - var tagValue = activity.Tags.First(t => t.Key == GenAiInputMessagesKey).Value; - tagValue.Should().StartWith("["); - tagValue.Should().Contain("\"role\":\"user\""); - tagValue.Should().Contain(requestContent); - } - - [TestMethod] - public void CallerClientIpTag_IsSetCorrectly() - { - var callerIp = IPAddress.Parse("203.0.113.42"); - var userDetails = new UserDetails( - userId: "caller-001", - userName: "Test Caller", - userEmail: "test.caller@contoso.com", - userClientIP: callerIp); - var callerDetails = new CallerDetails(userDetails: userDetails); - - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start( - request: Util.GetDefaultRequest(), - scopeDetails: ScopeDetails, - agentDetails: TestAgentDetails, - callerDetails: callerDetails); - }); - - // Assert - activity.ShouldHaveTag(CallerClientIpKey, callerIp.ToString()); - } - - [TestMethod] - public void AgentPlatformIdTag_IsSetCorrectly() - { - // Arrange - var platformId = "platform-001"; - var agentDetails = new AgentDetails( - agentId: "agent-789", - agentName: "PlatformAgent", - agentPlatformId: platformId); - - var scopeDetails = new InvokeAgentScopeDetails(endpoint: new Uri("https://example.com")); - - // Act - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start( - Util.GetDefaultRequest(), - scopeDetails, - agentDetails); - }); - - // Assert - activity.ShouldHaveTag(AgentPlatformIdKey, platformId); - } - - [TestMethod] - public void ThreatDiagnosticsSummary_IsSetCorrectly_WhenProvided() - { - // Arrange - var threatSummary = new ThreatDiagnosticsSummary( - blockAction: true, - reasonCode: 112, - reason: "The action was blocked because there is a noncompliant email address in the BCC field.", - diagnostics: "{\"flaggedField\":\"bcc\",\"flaggedValue\":\"hacker@evil.com\"}"); - - // Act - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start( - request: Util.GetDefaultRequest(), - scopeDetails: ScopeDetails, - agentDetails: TestAgentDetails, - threatDiagnosticsSummary: threatSummary); - }); - - // Assert - use Contains checks to handle JSON Unicode encoding variations - var tagValue = activity.Tags.First(t => t.Key == ThreatDiagnosticsSummaryKey).Value; - tagValue.Should().Contain("\"blockAction\":true"); - tagValue.Should().Contain("\"reasonCode\":112"); - tagValue.Should().Contain("\"reason\":\"The action was blocked because there is a noncompliant email address in the BCC field.\""); - tagValue.Should().Contain("flaggedField"); - tagValue.Should().Contain("bcc"); - tagValue.Should().Contain("hacker@evil.com"); - } - - [TestMethod] - public void ThreatDiagnosticsSummary_IsNotSet_WhenNull() - { - // Act - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start( - request: Util.GetDefaultRequest(), - scopeDetails: ScopeDetails, - agentDetails: TestAgentDetails, - threatDiagnosticsSummary: null); - }); - - // Assert - activity.Tags.Should().NotContainKey(ThreatDiagnosticsSummaryKey); - } - - [TestMethod] - public void RecordThreatDiagnosticsSummary_SetsTagCorrectly() - { - // Arrange - var threatSummary = new ThreatDiagnosticsSummary( - blockAction: true, - reasonCode: 200, - reason: "Blocked due to policy violation.", - diagnostics: "{\"policy\":\"data-loss-prevention\"}"); - - // Act - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start(Util.GetDefaultRequest(), ScopeDetails, TestAgentDetails); - scope.RecordThreatDiagnosticsSummary(threatSummary); - }); - - // Assert - var tagValue = activity.Tags.First(t => t.Key == ThreatDiagnosticsSummaryKey).Value; - tagValue.Should().Contain("\"blockAction\":true"); - tagValue.Should().Contain("\"reasonCode\":200"); - tagValue.Should().Contain("\"reason\":\"Blocked due to policy violation.\""); - tagValue.Should().Contain("data-loss-prevention"); - } - - [TestMethod] - public void Start_WithParentContext_SetsParentOnActivity() - { - // Arrange - ActivityContext? parentContext = null; - ListenForActivity(() => - { - using var parentScope = InvokeAgentScope.Start(Util.GetDefaultRequest(), ScopeDetails, TestAgentDetails); - parentContext = parentScope.GetActivityContext(); - }); - - // Act - var childActivity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start( - Util.GetDefaultRequest(), - ScopeDetails, - TestAgentDetails, - spanDetails: new SpanDetails(parentContext: parentContext)); - }); - - // Assert - childActivity.ParentSpanId.ToString().Should().NotBeNullOrEmpty(); - } - - [TestMethod] - public void Start_WithCustomStartTime_SetsActivityStartTime() - { - // Arrange - var customStartTime = new DateTimeOffset(2023, 11, 14, 22, 13, 20, TimeSpan.Zero); - - // Act - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start( - Util.GetDefaultRequest(), - ScopeDetails, - TestAgentDetails, - spanDetails: new SpanDetails(startTime: customStartTime)); - }); - - // Assert - var startTime = new DateTimeOffset(activity.StartTimeUtc); - startTime.Should().BeCloseTo(customStartTime, TimeSpan.FromMilliseconds(100)); - } - - [TestMethod] - public void Start_WithCustomStartAndEndTime_SetsActivityTimes() - { - // Arrange - var customStartTime = new DateTimeOffset(2023, 11, 14, 22, 13, 20, TimeSpan.Zero); - var customEndTime = new DateTimeOffset(2023, 11, 14, 22, 13, 25, TimeSpan.Zero); // 5 seconds later - - // Act - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start( - Util.GetDefaultRequest(), - ScopeDetails, - TestAgentDetails, - spanDetails: new SpanDetails(startTime: customStartTime, endTime: customEndTime)); - }); - - // Assert - Start time should be set to custom time - var startTime = new DateTimeOffset(activity.StartTimeUtc); - startTime.Should().BeCloseTo(customStartTime, TimeSpan.FromMilliseconds(100)); - } - - [TestMethod] - public void SetEndTime_OverridesEndTime() - { - // Arrange - var customStartTime = new DateTimeOffset(2023, 11, 14, 22, 13, 40, TimeSpan.Zero); - var initialEndTime = new DateTimeOffset(2023, 11, 14, 22, 13, 45, TimeSpan.Zero); - var laterEndTime = new DateTimeOffset(2023, 11, 14, 22, 13, 48, TimeSpan.Zero); - - // Act - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start( - Util.GetDefaultRequest(), - ScopeDetails, - TestAgentDetails, - spanDetails: new SpanDetails(startTime: customStartTime, endTime: initialEndTime)); - scope.SetEndTime(laterEndTime); - }); - - // Assert - The start time should be set - var startTime = new DateTimeOffset(activity.StartTimeUtc); - startTime.Should().BeCloseTo(customStartTime, TimeSpan.FromMilliseconds(100)); - } - - [TestMethod] - public void SpanKind_DefaultsToClient() - { - // Act - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start(Util.GetDefaultRequest(), ScopeDetails, TestAgentDetails); - }); - - // Assert - activity.Kind.Should().Be(System.Diagnostics.ActivityKind.Client); - } - - [TestMethod] - public void SpanKind_OverrideToServer() - { - // Act - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start( - Util.GetDefaultRequest(), - ScopeDetails, - TestAgentDetails, - spanDetails: new SpanDetails(spanKind: System.Diagnostics.ActivityKind.Server)); - }); - - // Assert - activity.Kind.Should().Be(System.Diagnostics.ActivityKind.Server); - } - - [TestMethod] - public void ActivityProcessor_PropagatesServerBaggage_ForInvokeAgentSpan() - { - // Arrange - using var tracerProvider = ConstructTracerProvider(); - var serverAddress = "myagent.azurewebsites.net"; - var serverPort = "8443"; - - // Act - set server address/port in baggage, then start an invoke_agent span - using (new Runtime.Common.BaggageBuilder() - .InvokeAgentServer(serverAddress, 8443) - .Build()) - { - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start( - Util.GetDefaultRequest(), - new InvokeAgentScopeDetails(endpoint: null), - new AgentDetails("agent-1")); - }); - - // Assert - processor should coalesce server baggage onto the span - activity.ShouldHaveTag(ServerAddressKey, serverAddress); - activity.ShouldHaveTag(ServerPortKey, serverPort); - } - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/OutputScopeTest.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/OutputScopeTest.cs deleted file mode 100644 index c2858c77..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/OutputScopeTest.cs +++ /dev/null @@ -1,212 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Tests.Tracing.Scopes; - -using System.Diagnostics; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; - -[TestClass] -public sealed class OutputScopeTest : ActivityTest -{ - [TestMethod] - public void Start_SetsExpectedTags() - { - // Arrange - var initialMessages = new[] { "Hello", "World" }; - var response = new Response(initialMessages); - var agentDetails = new AgentDetails( - agentId: "agent-output-123", - agentName: "OutputAgent", - agentType: AgentType.MicrosoftCopilot); - - // Act - var activity = ListenForActivity(() => - { - using var scope = OutputScope.Start(Util.GetDefaultRequest(), response, agentDetails); - }); - - // Assert - operation name and activity name - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiOperationNameKey, OutputScope.OperationName); - activity.DisplayName.Should().Be($"{OutputScope.OperationName} {agentDetails.AgentId}"); - - // Assert - agent details - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiAgentIdKey, agentDetails.AgentId!); - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiAgentNameKey, agentDetails.AgentName!); - - // Assert - output messages (structured JSON format) - var tagValue = activity.Tags.First(t => t.Key == OpenTelemetryConstants.GenAiOutputMessagesKey).Value; - tagValue.Should().StartWith("["); - tagValue.Should().Contain("\"role\":\"assistant\""); - tagValue.Should().Contain("Hello"); - tagValue.Should().Contain("World"); - } - - [TestMethod] - public void RecordOutputMessages_OverwritesMessages() - { - // Arrange - var initialMessages = new[] { "Hello", "World" }; - var additionalMessages = new[] { "Goodbye", "Moon" }; - var response = new Response(initialMessages); - var agentDetails = Util.GetAgentDetails(); - - // Act - var activity = ListenForActivity(() => - { - using var scope = OutputScope.Start(Util.GetDefaultRequest(), response, agentDetails); - scope.RecordOutputMessages(additionalMessages); - }); - - // Assert - output messages are overwritten (set-once), only additional messages remain - var tagValue = activity.Tags.First(t => t.Key == OpenTelemetryConstants.GenAiOutputMessagesKey).Value; - tagValue.Should().StartWith("["); - tagValue.Should().Contain("\"role\":\"assistant\""); - tagValue.Should().Contain("Goodbye"); - tagValue.Should().Contain("Moon"); - } - - [TestMethod] - public void Start_WithParentContext_SetsParentCorrectly() - { - // Arrange - var response = new Response(new[] { "Test message" }); - var agentDetails = Util.GetAgentDetails(); - - // Create a parent activity to get a valid parent context - ActivityContext? parentContext = null; - ListenForActivity(() => - { - using var parentScope = InvokeAgentScope.Start(Util.GetDefaultRequest(), ScopeDetails, TestAgentDetails); - parentContext = parentScope.GetActivityContext(); - }); - - // Act - var childActivity = ListenForActivity(() => - { - using var scope = OutputScope.Start( - Util.GetDefaultRequest(), - response, - agentDetails, - spanDetails: new SpanDetails(parentContext: parentContext)); - }); - - // Assert - child activity should have the parent set - childActivity.ParentSpanId.ToString().Should().NotBeNullOrEmpty(); - childActivity.ShouldHaveTag(OpenTelemetryConstants.GenAiOperationNameKey, OutputScope.OperationName); - childActivity.ShouldHaveTagContaining(OpenTelemetryConstants.GenAiOutputMessagesKey, "Test message"); - } - - [TestMethod] - public void Start_WithCustomStartTime_SetsActivityStartTime() - { - // Arrange - var customStartTime = new DateTimeOffset(2023, 11, 14, 22, 13, 20, TimeSpan.Zero); - var response = new Response(new[] { "Test message" }); - var agentDetails = Util.GetAgentDetails(); - - // Act - var activity = ListenForActivity(() => - { - using var scope = OutputScope.Start( - Util.GetDefaultRequest(), - response, - agentDetails, - spanDetails: new SpanDetails(startTime: customStartTime)); - }); - - // Assert - var startTime = new DateTimeOffset(activity.StartTimeUtc); - startTime.Should().BeCloseTo(customStartTime, TimeSpan.FromMilliseconds(100)); - } - - [TestMethod] - public void Start_WithCustomStartAndEndTime_SetsActivityTimes() - { - // Arrange - var customStartTime = new DateTimeOffset(2023, 11, 14, 22, 13, 20, TimeSpan.Zero); - var customEndTime = new DateTimeOffset(2023, 11, 14, 22, 13, 25, TimeSpan.Zero); // 5 seconds later - var response = new Response(new[] { "Test message" }); - var agentDetails = Util.GetAgentDetails(); - - // Act - var activity = ListenForActivity(() => - { - using var scope = OutputScope.Start( - Util.GetDefaultRequest(), - response, - agentDetails, - spanDetails: new SpanDetails(startTime: customStartTime, endTime: customEndTime)); - }); - - // Assert - Start time should be set to custom time - var startTime = new DateTimeOffset(activity.StartTimeUtc); - startTime.Should().BeCloseTo(customStartTime, TimeSpan.FromMilliseconds(100)); - } - - [TestMethod] - public void Start_SetsConversationIdChannelAndCallerDetails_WhenProvided() - { - // Arrange - var conversationId = "conv-output-123"; - var metadata = new Channel(name: "ChannelOutput", link: "https://channel/output"); - var userDetails = new UserDetails( - userId: "caller-output-123", - userName: "Output Caller", - userEmail: "caller-output@example.com", - userClientIP: System.Net.IPAddress.Parse("10.0.0.2")); - var response = new Response(new[] { "Test message" }); - var agentDetails = Util.GetAgentDetails(); - - // Act - var activity = ListenForActivity(() => - { - using var scope = OutputScope.Start( - new Request(conversationId: conversationId, channel: metadata), - response, - agentDetails, - userDetails: userDetails); - }); - - // Assert - conversation ID - activity.ShouldHaveTag(OpenTelemetryConstants.GenAiConversationIdKey, conversationId); - - // Assert - source metadata - activity.ShouldHaveTag(OpenTelemetryConstants.ChannelNameKey, metadata.Name!); - activity.ShouldHaveTag(OpenTelemetryConstants.ChannelLinkKey, metadata.Link!); - - // Assert - caller details - activity.ShouldHaveTag(OpenTelemetryConstants.UserIdKey, userDetails.UserId!); - activity.ShouldHaveTag(OpenTelemetryConstants.UserNameKey, userDetails.UserName!); - activity.ShouldHaveTag(OpenTelemetryConstants.UserEmailKey, userDetails.UserEmail!); - activity.ShouldHaveTag(OpenTelemetryConstants.CallerClientIpKey, userDetails.UserClientIP!.ToString()); - } - - [TestMethod] - public void SetEndTime_OverridesEndTime() - { - // Arrange - var customStartTime = new DateTimeOffset(2023, 11, 14, 22, 13, 40, TimeSpan.Zero); - var initialEndTime = new DateTimeOffset(2023, 11, 14, 22, 13, 45, TimeSpan.Zero); - var laterEndTime = new DateTimeOffset(2023, 11, 14, 22, 13, 48, TimeSpan.Zero); - var response = new Response(new[] { "Test message" }); - var agentDetails = Util.GetAgentDetails(); - - // Act - var activity = ListenForActivity(() => - { - using var scope = OutputScope.Start( - Util.GetDefaultRequest(), - response, - agentDetails, - spanDetails: new SpanDetails(startTime: customStartTime, endTime: initialEndTime)); - scope.SetEndTime(laterEndTime); - }); - - // Assert - The start time should be set - var startTime = new DateTimeOffset(activity.StartTimeUtc); - startTime.Should().BeCloseTo(customStartTime, TimeSpan.FromMilliseconds(100)); - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/ScopeTests.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/ScopeTests.cs deleted file mode 100644 index 001fba25..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/ScopeTests.cs +++ /dev/null @@ -1,84 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Tests.Tracing.Scopes; - -using System.Diagnostics; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using static Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes.OpenTelemetryConstants; - -[TestClass] -public sealed class ScopeTests : ActivityTest -{ - private class TestScope : OpenTelemetryScope - { - public TestScope(string operationName, string activityName, AgentDetails agentDetails, SpanDetails? spanDetails = null) - : base(operationName, activityName, agentDetails, spanDetails) { } - } - - [TestMethod] - public void NestedScope_PropagatesAgentId() - { - // Arrange - using var tracerProvider = ConstructTracerProvider(); - - // Act - var activity = ListenForActivity(() => - { - using var invokeAgentScope = InvokeAgentScope.Start(new Request(), ScopeDetails, Util.GetAgentDetails()); - using var toolScope = ExecuteToolScope.Start(new Request(), new ToolCallDetails("TestTool", "Input: 42"), Util.GetAgentDetails()); - }); - - // Assert - activity.Should().NotBeNull(); - activity.Kind.Should().Be(ActivityKind.Internal); - activity.TagObjects.Should().ContainKey(GenAiOperationNameKey) - .WhoseValue.Should().Be(ExecuteToolScope.OperationName); - activity.TagObjects.Should().ContainKey(GenAiAgentIdKey) - .WhoseValue.Should().BeOfType() - .Which.Should().Be(AgentId); - } - - [TestMethod] - public void Id_ReturnsActivityId() - { - // Arrange - using var listener = new ActivityListener - { - ShouldListenTo = s => s.Name == SourceName, - Sample = (ref ActivityCreationOptions _) => ActivitySamplingResult.AllDataAndRecorded, - ActivityStarted = _ => { }, - ActivityStopped = _ => { } - }; - ActivitySource.AddActivityListener(listener); - - using var scope = new TestScope("TestOperation", "TestActivity", Util.GetAgentDetails(), new SpanDetails(ActivityKind.Internal)); - - // Act - var expectedId = scope.Id; - - // Assert - expectedId.Should().NotBeNullOrEmpty(); - } - - [TestMethod] - public void SetParentContext_SetsActivityParentId() - { - // Arrange - var manualParentActivity = CreateActivity(); - var parentContext = manualParentActivity.Context; - var parentSpanId = manualParentActivity.SpanId.ToString() ?? string.Empty; - - // Act - var activity = ListenForActivity(() => - { - using var toolScope = ExecuteToolScope.Start(new Request(), new ToolCallDetails("TestTool", "Input: 42"), Util.GetAgentDetails(), spanDetails: new SpanDetails(parentContext: parentContext)); - }); - - // Assert - activity.Should().NotBeNull(); - activity!.ParentSpanId.ToString().Should().Be(parentSpanId); - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/SpanLinksTest.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/SpanLinksTest.cs deleted file mode 100644 index 207360b3..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/SpanLinksTest.cs +++ /dev/null @@ -1,160 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Tests.Tracing.Scopes; - -using System.Diagnostics; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; - -[TestClass] -public sealed class SpanLinksTest : ActivityTest -{ - private static readonly ActivityLink[] SampleLinks = new[] - { - new ActivityLink( - new ActivityContext( - ActivityTraceId.CreateFromString("0aa4621e5ae09963a3de354f3d18aa65"), - ActivitySpanId.CreateFromString("c1aaa519600b1bf0"), - ActivityTraceFlags.Recorded)), - new ActivityLink( - new ActivityContext( - ActivityTraceId.CreateFromString("bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"), - ActivitySpanId.CreateFromString("aaaaaaaaaaaaaaaa"), - ActivityTraceFlags.None), - new ActivityTagsCollection(new[] { new KeyValuePair("link.reason", "retry") })), - }; - - [TestMethod] - public void ExecuteToolScope_RecordsSpanLinks_WithFullContextAndAttributes() - { - var activity = ListenForActivity(() => - { - using var scope = ExecuteToolScope.Start( - Util.GetDefaultRequest(), - new ToolCallDetails("my-tool", "args"), - Util.GetAgentDetails(), - spanDetails: new SpanDetails(spanLinks: SampleLinks)); - }); - - var links = activity.Links.ToList(); - links.Should().HaveCount(2); - links[0].Context.TraceId.ToHexString().Should().Be("0aa4621e5ae09963a3de354f3d18aa65"); - links[0].Context.SpanId.ToHexString().Should().Be("c1aaa519600b1bf0"); - links[0].Context.TraceFlags.Should().Be(ActivityTraceFlags.Recorded); - links[1].Context.TraceId.ToHexString().Should().Be("bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"); - links[1].Context.SpanId.ToHexString().Should().Be("aaaaaaaaaaaaaaaa"); - links[1].Tags.Should().Contain(new KeyValuePair("link.reason", "retry")); - } - - [TestMethod] - public void ExecuteToolScope_HasEmptyLinks_WhenSpanLinksOmitted() - { - var activity = ListenForActivity(() => - { - using var scope = ExecuteToolScope.Start( - Util.GetDefaultRequest(), - new ToolCallDetails("my-tool", "args"), - Util.GetAgentDetails()); - }); - - activity.Links.Should().BeEmpty(); - } - - [TestMethod] - public void InvokeAgentScope_ForwardsSpanLinks() - { - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start( - Util.GetDefaultRequest(), - ScopeDetails, - TestAgentDetails, - spanDetails: new SpanDetails(spanLinks: SampleLinks)); - }); - - var links = activity.Links.ToList(); - links.Should().HaveCount(2); - links[0].Context.TraceId.ToHexString().Should().Be("0aa4621e5ae09963a3de354f3d18aa65"); - links[0].Context.SpanId.ToHexString().Should().Be("c1aaa519600b1bf0"); - links[0].Context.TraceFlags.Should().Be(ActivityTraceFlags.Recorded); - links[1].Context.TraceId.ToHexString().Should().Be("bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"); - links[1].Context.SpanId.ToHexString().Should().Be("aaaaaaaaaaaaaaaa"); - links[1].Context.TraceFlags.Should().Be(ActivityTraceFlags.None); - links[1].Tags.Should().Contain(new KeyValuePair("link.reason", "retry")); - } - - [TestMethod] - public void InferenceScope_ForwardsSpanLinks() - { - var details = new InferenceCallDetails( - InferenceOperationType.Chat, "gpt-4", "openai"); - - var activity = ListenForActivity(() => - { - using var scope = InferenceScope.Start( - Util.GetDefaultRequest(), - details, - Util.GetAgentDetails(), - spanDetails: new SpanDetails(spanLinks: SampleLinks)); - }); - - var links = activity.Links.ToList(); - links.Should().HaveCount(2); - links[0].Context.TraceId.ToHexString().Should().Be("0aa4621e5ae09963a3de354f3d18aa65"); - links[0].Context.SpanId.ToHexString().Should().Be("c1aaa519600b1bf0"); - } - - [TestMethod] - public void OutputScope_ForwardsSpanLinks() - { - var response = new Response(new[] { "hello" }); - - var activity = ListenForActivity(() => - { - using var scope = OutputScope.Start( - Util.GetDefaultRequest(), - response, - Util.GetAgentDetails(), - spanDetails: new SpanDetails(spanLinks: SampleLinks)); - }); - - var links = activity.Links.ToList(); - links.Should().HaveCount(2); - links[0].Context.TraceId.ToHexString().Should().Be("0aa4621e5ae09963a3de354f3d18aa65"); - links[0].Context.SpanId.ToHexString().Should().Be("c1aaa519600b1bf0"); - } - - [TestMethod] - public void SpanLinks_PreservesTypedAttributes() - { - var linksWithAttrs = new[] - { - new ActivityLink( - new ActivityContext( - ActivityTraceId.CreateFromString("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"), - ActivitySpanId.CreateFromString("bbbbbbbbbbbbbbbb"), - ActivityTraceFlags.Recorded), - new ActivityTagsCollection(new[] - { - new KeyValuePair("link.type", "causal"), - new KeyValuePair("link.index", 0), - })), - }; - - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start( - Util.GetDefaultRequest(), - ScopeDetails, - new AgentDetails("attr-agent"), - spanDetails: new SpanDetails(spanLinks: linksWithAttrs)); - }); - - var links = activity.Links.ToList(); - links.Should().HaveCount(1); - links[0].Tags.Should().Contain(new KeyValuePair("link.type", "causal")); - links[0].Tags.Should().Contain(new KeyValuePair("link.index", 0)); - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/TraceContextPropagationTest.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/TraceContextPropagationTest.cs deleted file mode 100644 index f8eacbef..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/TraceContextPropagationTest.cs +++ /dev/null @@ -1,234 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Tests.Tracing.Scopes; - -using System; -using System.Collections.Generic; -using System.Diagnostics; -using FluentAssertions; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes; -using static Microsoft.Agents.A365.Observability.Runtime.Tracing.Scopes.OpenTelemetryConstants; - -[TestClass] -public sealed class TraceContextPropagationTest : ActivityTest -{ - [TestMethod] - public void InjectTraceContext_ReturnsTraceparentHeader() - { - // Arrange & Act - Dictionary? headers = null; - ListenForActivity(() => - { - using var scope = InferenceScope.Start( - Util.GetDefaultRequest(), - new InferenceCallDetails(InferenceOperationType.Chat, "gpt-4", "openai"), - Util.GetAgentDetails()); - headers = scope.InjectTraceContext(); - }); - - // Assert - headers.Should().NotBeNull(); - headers!.Should().ContainKey("traceparent"); - - var traceparent = headers!["traceparent"]; - var parts = traceparent.Split('-'); - parts.Should().HaveCount(4, "traceparent should have 4 parts (version-trace_id-span_id-flags)"); - parts[0].Should().Be("00", "version should be 00"); - parts[1].Should().HaveLength(32, "trace_id should be 32 hex chars"); - parts[2].Should().HaveLength(16, "span_id should be 16 hex chars"); - parts[3].Should().HaveLength(2, "flags should be 2 hex chars"); - } - - [TestMethod] - public void GetActivityContext_ReturnsValidContext() - { - // Arrange & Act - ActivityContext? ctx = null; - ListenForActivity(() => - { - using var scope = InferenceScope.Start( - Util.GetDefaultRequest(), - new InferenceCallDetails(InferenceOperationType.Chat, "gpt-4", "openai"), - Util.GetAgentDetails()); - ctx = scope.GetActivityContext(); - }); - - // Assert - ctx.Should().NotBeNull(); - ctx!.Value.TraceId.Should().NotBe(default(ActivityTraceId)); - ctx!.Value.SpanId.Should().NotBe(default(ActivitySpanId)); - } - - [TestMethod] - public void ExtractContextFromHeaders_ParsesTraceparent() - { - // Arrange - var traceId = "1234567890abcdef1234567890abcdef"; - var spanId = "abcdefabcdef1234"; - var traceparent = $"00-{traceId}-{spanId}-01"; - var headers = new Dictionary { { "traceparent", traceparent } }; - - // Act - var ctx = TraceContextHelper.ExtractContextFromHeaders(headers); - - // Assert - ctx.TraceId.ToHexString().Should().Be(traceId); - ctx.SpanId.ToHexString().Should().Be(spanId); - ctx.TraceFlags.Should().Be(ActivityTraceFlags.Recorded); - } - - [TestMethod] - public void ExtractContextFromHeaders_ReturnsDefault_WhenNoTraceparent() - { - // Arrange - var headers = new Dictionary(); - - // Act - var ctx = TraceContextHelper.ExtractContextFromHeaders(headers); - - // Assert - ctx.TraceId.Should().Be(default(ActivityTraceId)); - } - - [TestMethod] - public void GetTraceparent_ReturnsValue_WhenPresent() - { - // Arrange - var expected = "00-1234567890abcdef1234567890abcdef-abcdefabcdef1234-01"; - var headers = new Dictionary { { "traceparent", expected } }; - - // Act - var result = TraceContextHelper.GetTraceparent(headers); - - // Assert - result.Should().Be(expected); - } - - [TestMethod] - public void GetTraceparent_ReturnsNull_WhenMissing() - { - // Arrange - var headers = new Dictionary(); - - // Act - var result = TraceContextHelper.GetTraceparent(headers); - - // Assert - result.Should().BeNull(); - } - - [TestMethod] - public void InjectExtract_RoundTrip_PreservesTraceContext() - { - // Arrange - create a parent scope and inject its context - Dictionary? injectedHeaders = null; - ListenForActivity(() => - { - using var parentScope = InferenceScope.Start( - Util.GetDefaultRequest(), - new InferenceCallDetails(InferenceOperationType.Chat, "gpt-4", "openai"), - Util.GetAgentDetails()); - injectedHeaders = parentScope.InjectTraceContext(); - }); - - // Act - extract context from headers and create child scope - var parentContext = TraceContextHelper.ExtractContextFromHeaders(injectedHeaders!); - var childActivity = ListenForActivity(() => - { - using var childScope = ExecuteToolScope.Start( - Util.GetDefaultRequest(), - new ToolCallDetails("search_tool", "{\"query\": \"test\"}"), - Util.GetAgentDetails(), - spanDetails: new SpanDetails(parentContext: parentContext)); - }); - - // Assert - child should share parent's trace ID - var parentTraceparent = injectedHeaders!["traceparent"]; - var parentTraceId = parentTraceparent.Split('-')[1]; - var parentSpanId = parentTraceparent.Split('-')[2]; - childActivity.TraceId.ToHexString().Should().Be(parentTraceId, "child should have same trace_id as parent"); - childActivity.ParentSpanId.ToHexString().Should().Be(parentSpanId, "child's parent span should be the parent's span"); - } - - [TestMethod] - public void InvokeAgentScope_WithParentContext_InheritsTraceId() - { - // Arrange - var traceId = "1234567890abcdef1234567890abcdef"; - var spanId = "abcdefabcdef1234"; - var traceparent = $"00-{traceId}-{spanId}-01"; - var parentContext = TraceContextHelper.ExtractContextFromHeaders( - new Dictionary { { "traceparent", traceparent } }); - - // Act - Dictionary? childHeaders = null; - var activity = ListenForActivity(() => - { - using var scope = InvokeAgentScope.Start( - Util.GetDefaultRequest(), - ScopeDetails, - TestAgentDetails, - spanDetails: new SpanDetails(parentContext: parentContext)); - childHeaders = scope.InjectTraceContext(); - }); - - // Assert - span inherits parent's trace ID - activity.TraceId.ToHexString().Should().Be(traceId); - activity.ParentSpanId.ToHexString().Should().Be(spanId); - - // Assert - injected headers contain valid traceparent - childHeaders.Should().ContainKey("traceparent"); - } - - [TestMethod] - public void ExecuteToolScope_WithExtractedParentContext_SetsParent() - { - // Arrange - var traceId = "abcdef1234567890abcdef1234567890"; - var spanId = "1234567890abcdef"; - var traceparent = $"00-{traceId}-{spanId}-01"; - var parentContext = TraceContextHelper.ExtractContextFromHeaders( - new Dictionary { { "traceparent", traceparent } }); - - // Act - var activity = ListenForActivity(() => - { - using var scope = ExecuteToolScope.Start( - Util.GetDefaultRequest(), - new ToolCallDetails("test-tool", "args"), - Util.GetAgentDetails(), - spanDetails: new SpanDetails(parentContext: parentContext)); - }); - - // Assert - activity.TraceId.ToHexString().Should().Be(traceId); - activity.ParentSpanId.ToHexString().Should().Be(spanId); - } - - [TestMethod] - public void OutputScope_WithExtractedParentContext_SetsParent() - { - // Arrange - var traceId = "abcdef1234567890abcdef1234567890"; - var spanId = "1234567890abcdef"; - var traceparent = $"00-{traceId}-{spanId}-01"; - var parentContext = TraceContextHelper.ExtractContextFromHeaders( - new Dictionary { { "traceparent", traceparent } }); - - // Act - var activity = ListenForActivity(() => - { - using var scope = OutputScope.Start( - Util.GetDefaultRequest(), - new Response(new[] { "test" }), - Util.GetAgentDetails(), - spanDetails: new SpanDetails(parentContext: parentContext)); - }); - - // Assert - activity.TraceId.ToHexString().Should().Be(traceId); - activity.ParentSpanId.ToHexString().Should().Be(spanId); - } -} diff --git a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/Util.cs b/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/Util.cs deleted file mode 100644 index 02dc216b..00000000 --- a/src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/Util.cs +++ /dev/null @@ -1,15 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -namespace Microsoft.Agents.A365.Observability.Tests.Tracing.Scopes; - -using Microsoft.Agents.A365.Observability.Runtime.Tracing.Contracts; - -public static class Util -{ - public static AgentDetails GetAgentDetails() => - new AgentDetails("agentId", "Test Agent", "A test agent for unit testing."); - - public static Request GetDefaultRequest() => - new Request(content: "test"); -} diff --git a/src/Tooling/Extensions/AgentFramework/Microsoft.Agents.A365.Tooling.Extensions.AgentFramework.csproj b/src/Tooling/Extensions/AgentFramework/Microsoft.Agents.A365.Tooling.Extensions.AgentFramework.csproj index c59cafc6..c4970e76 100644 --- a/src/Tooling/Extensions/AgentFramework/Microsoft.Agents.A365.Tooling.Extensions.AgentFramework.csproj +++ b/src/Tooling/Extensions/AgentFramework/Microsoft.Agents.A365.Tooling.Extensions.AgentFramework.csproj @@ -11,7 +11,7 @@ A .NET SDK that integrates Microsoft Agent 365 with Agent Framework, providing tooling functionalities. Microsoft Agent 365 Agent Framework Tooling Integration - Provides a way to add Agent 365 MCP Servers Microsoft Agent 365 Agent Framework Tooling SDK - Microsoft;A365AgentFramework;OpenTelemetry;AI;Agents;Tooling + Microsoft;A365AgentFramework;AI;Agents;Tooling $(NoWarn);NU5104 diff --git a/src/Tooling/Extensions/AzureAIFoundry/Microsoft.Agents.A365.Tooling.Extensions.AzureAIFoundry.csproj b/src/Tooling/Extensions/AzureAIFoundry/Microsoft.Agents.A365.Tooling.Extensions.AzureAIFoundry.csproj index 6393be15..696bfab0 100644 --- a/src/Tooling/Extensions/AzureAIFoundry/Microsoft.Agents.A365.Tooling.Extensions.AzureAIFoundry.csproj +++ b/src/Tooling/Extensions/AzureAIFoundry/Microsoft.Agents.A365.Tooling.Extensions.AzureAIFoundry.csproj @@ -11,7 +11,7 @@ A .NET SDK that integrates Microsoft Agent 365 with Azure AI Foundry, providing tooling functionalities. Microsoft Agent 365 Azure AI Foundry Tooling Integration - Provides a way to add Agent 365 MCP Servers Microsoft Agent 365 Azure AI Foundry Tooling SDK - Microsoft;A365AzureAI;OpenTelemetry;AI;Agents;Tooling + Microsoft;A365AzureAI;AI;Agents;Tooling $(NoWarn);NU5104 diff --git a/src/Tooling/Extensions/SemanticKernel/Microsoft.Agents.A365.Tooling.Extensions.SemanticKernel.csproj b/src/Tooling/Extensions/SemanticKernel/Microsoft.Agents.A365.Tooling.Extensions.SemanticKernel.csproj index 0e2b8594..363086ef 100644 --- a/src/Tooling/Extensions/SemanticKernel/Microsoft.Agents.A365.Tooling.Extensions.SemanticKernel.csproj +++ b/src/Tooling/Extensions/SemanticKernel/Microsoft.Agents.A365.Tooling.Extensions.SemanticKernel.csproj @@ -4,14 +4,14 @@ Microsoft.Agents.A365.Tooling.Extensions.SemanticKernel net8.0 latest - + Microsoft.Agents.A365.Tooling.Extensions.SemanticKernel Microsoft Agent 365 Semantic Kernel Tooling SDK A .NET SDK that integrates Microsoft Agent 365 with Semantic Kernel, providing tooling functionalities. Microsoft Agent 365 Semantic Kernel Tooling Integration - Provides a way to add Agent 365 MCP Servers Microsoft Agent 365 Semantic Kernel Tooling SDK - Microsoft;A365SemanticKernel;OpenTelemetry;AI;Agents;Tooling + Microsoft;A365SemanticKernel;AI;Agents;Tooling $(NoWarn);NU5104 @@ -30,5 +30,3 @@ - - diff --git a/src/dirs.proj b/src/dirs.proj index cf7fa627..806f0c4b 100644 --- a/src/dirs.proj +++ b/src/dirs.proj @@ -5,10 +5,8 @@ - - diff --git a/src/version.json b/src/version.json index f6612f4a..5332a642 100644 --- a/src/version.json +++ b/src/version.json @@ -1,6 +1,6 @@ { "$schema": "https://raw.githubusercontent.com/dotnet/Nerdbank.GitVersioning/main/src/NerdBank.GitVersioning/version.schema.json", - "version": "2.1-preview", + "version": "3.0-preview", "assemblyVersion": { "precision": "revision" },