Skip to content

Commit 4565a80

Browse files
authored
Merge branch 'main' into danfiedler/pin-actions
2 parents 204acea + 4cb1042 commit 4565a80

30 files changed

Lines changed: 910 additions & 57 deletions

File tree

‎docs/integrating-with-existing-opentelemetry.md‎

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -93,10 +93,15 @@ The SDK produces three core span kinds. Your backend should show them in this ty
9393
| `gen_ai.operation.name` | Produced by | Typical parent | Span name (default) | Notes |
9494
|-------------------------|---------------------------------------------------|----------------|---------------------|-------|
9595
| `invoke_agent` | `InvokeAgentScope` (one per user turn) | (root or app) | `invoke_agent <agent_name>` when set, else `invoke_agent` | |
96-
| (varies — see notes) | `InferenceScope` (one per LLM call) | `invoke_agent` | `<operation> <model>` | **Manual instrumentation** uses `InferenceOperationType.value` (currently `Chat` / `TextCompletion` / `GenerateContent`, capitalized). **Auto-instrumentation** (e.g. `OpenAIAgentsTraceInstrumentor`) uses lowercase per the [OTel GenAI semconv](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-spans/) (e.g. `chat`). The two are inconsistent today. |
96+
| `chat` / `text_completion` / `generate_content` | `InferenceScope` (one per LLM call) | `invoke_agent` | `<operation> <model>` | Manual inference scopes use canonical lowercase [OTel GenAI operation names](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-spans/), matching auto-instrumentation for the corresponding operation. |
9797
| `execute_tool` | `ExecuteToolScope` (one per tool invocation) | `invoke_agent` | `execute_tool <tool_name>` (always includes the tool name) | Records tool name, args, and result. |
9898

99-
Filter your backend by the `gen_ai.operation.name` attribute or by span name. Note that `inference` is *not* the literal attribute value — manual instrumentation produces `Chat` / `TextCompletion` / `GenerateContent` (the `InferenceOperationType.value`), while auto-instrumentation extension packages produce the lowercase OTel-spec form (e.g. `chat`). This casing discrepancy is tracked as an SDK issue.
99+
Filter your backend by the `gen_ai.operation.name` attribute or by span name.
100+
`InferenceScope` emits `chat`, `text_completion`, or `generate_content` according
101+
to the selected `InferenceOperationType`, with span names such as `chat gpt-4`.
102+
`inference` is not a literal operation value. When upgrading, update case-sensitive
103+
filters that matched the previous `Chat`, `TextCompletion`, or `GenerateContent`
104+
values and span-name prefixes, or compare case-insensitively.
100105

101106
## Verifying the integration
102107

@@ -110,7 +115,11 @@ from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExport
110115
trace.get_tracer_provider().add_span_processor(BatchSpanProcessor(ConsoleSpanExporter()))
111116
```
112117

113-
Run a single turn. If you see `invoke_agent` / `Chat` (or `chat`) / `execute_tool` JSON dumps on stdout, the SDK is producing spans correctly — the issue is in your backend exporter (network, auth, sampling). If you don't see them, the integration itself is wrong; check the pitfalls below.
118+
Run a single turn. If you see `invoke_agent`, the inference operation
119+
(`chat`, `text_completion`, or `generate_content`), and any invoked `execute_tool`
120+
spans as JSON dumps on stdout, the SDK is producing spans correctly - investigate
121+
your backend exporter (network, auth, sampling). If expected spans are missing
122+
from stdout too, check the integration pitfalls below.
114123

115124
## Common pitfalls
116125

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Changelog — microsoft-agents-a365-notifications
2+
3+
All notable changes to this package will be documented in this file.
4+
5+
## [2.0.0] - 2026-10-20
6+
7+
### Breaking Changes
8+
9+
- **`AgentLifecycleEvent.USERDELETED` value changed** — it is now
10+
`"agenticuserdeleted"` (previously `"agenticuseridentitydeleted"`), so
11+
`on_user_deleted()` handlers match the new event name. Update any code that
12+
compares against the raw string value.
13+
14+
### Added
15+
16+
- New lifecycle events and handler decorators on `AgentNotification`:
17+
- `USERUNDELETED` — `on_user_undeleted()`
18+
- `USERUPDATED` — `on_user_updated()`
19+
- `USERMANAGERUPDATED` — `on_user_manager_updated()`
20+
- `USERENABLED` — `on_user_enabled()`
21+
- `USERDISABLED` — `on_user_disabled()`
22+
23+
### Security
24+
25+
- Raised the minimum `PyJWT` version to `>= 2.14.0`.

‎libraries/microsoft-agents-a365-observability-core/CHANGELOG.md‎

Lines changed: 65 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
All notable changes to this package will be documented in this file.
44

5-
## [Unreleased]
5+
## [2.0.0] - 2026-10-20
66

77
### Breaking Changes
88

@@ -17,6 +17,59 @@ All notable changes to this package will be documented in this file.
1717
failures fail export without an HTTP request or delegated fallback. The
1818
exporter invokes the resolver on every export batch/identity group, so
1919
resolvers must cache the acquired token and refresh only near expiry.
20+
- **Manual inference operation names and span-name prefixes now use canonical
21+
OpenTelemetry values** — `Chat`, `TextCompletion`, and `GenerateContent`
22+
become `chat`, `text_completion`, and `generate_content`, respectively.
23+
For example, `Chat gpt-4` becomes `chat gpt-4`. The Agent 365 exporter
24+
recognizes the canonical names (the legacy `Chat` value is still accepted),
25+
so text-completion and generate-content spans that were previously filtered
26+
out are now exported to Agent 365.
27+
- **`InferenceCallDetails` now validates model identity during construction** —
28+
`model` and `responseModel` are stripped and whitespace-only values are
29+
treated as `None`. If both are missing, empty, or whitespace-only,
30+
construction raises `ValueError`. Previously these values were accepted
31+
without validation. This happens before an `InferenceScope` is created.
32+
- **Message payloads serialize as a plain JSON array** — `gen_ai.input.messages`
33+
and `gen_ai.output.messages` are now serialized per the OpenTelemetry GenAI
34+
semantic conventions as `[{"role": ..., "parts": [...]}]` instead of the
35+
`{"version": "0.1.0", "messages": [...]}` wrapper. The
36+
`A365_MESSAGE_SCHEMA_VERSION` constant and the `version` field on
37+
`InputMessages`/`OutputMessages` are removed, and `OutputMessage.finish_reason`
38+
now defaults to `"stop"` instead of `None`.
39+
40+
### Added
41+
42+
- **`ApplyGuardrailScope`** for tracing security guardrail evaluations
43+
(`apply_guardrail` operation), with supporting `GuardrailDetails`,
44+
`GuardrailFinding`, `GuardrailDecisionType`, `GuardrailRiskSeverity`, and
45+
`GuardrailTargetType` types and `microsoft.guardian.*` /
46+
`microsoft.security.*` attribute constants. `apply_guardrail` spans are
47+
exported to Agent 365.
48+
- **`AgentDetails.agent_type`** and **`BaggageBuilder.agent_type()`** emit
49+
`gen_ai.agent.type`. Both accept an `AgentType` enum member or a string;
50+
enum members are emitted as their string `.value`. The new dataclass field is
51+
appended after existing fields, preserving positional-argument compatibility.
52+
- **`InferenceCallDetails.responseModel`**, **`cacheReadInputTokens`**, and
53+
**`cacheWriteInputTokens`** emit `gen_ai.response.model`,
54+
`gen_ai.usage.cache_read.input_tokens`, and
55+
`gen_ai.usage.cache_write.input_tokens`. `model` is now optional when
56+
`responseModel` is supplied. The new fields are appended after existing
57+
fields, preserving positional-argument compatibility.
58+
- **`InferenceScope.record_cache_read_input_tokens()`** and
59+
**`record_cache_write_input_tokens()`**.
60+
- **`InferenceOperationType.canonical_name`** returns the canonical lowercase
61+
OpenTelemetry operation name.
62+
63+
### Changed
64+
65+
- `InferenceScope` emits `server.port` as an integer instead of a string.
66+
67+
### Fixed
68+
69+
- Scopes and the Agent 365 exporter no longer raise `AttributeError` when the
70+
tracer returns a `NonRecordingSpan` (for example, when the token resolver
71+
returns `None` on the first turn); span context is read via
72+
`get_span_context()`.
2073

2174
### Migration
2275

@@ -28,8 +81,17 @@ All notable changes to this package will be documented in this file.
2881
`idtyp` other than `app`; when `idtyp` is absent, accept only a non-empty
2982
`roles` array or a non-empty `oid` equal to `sub`; verify `aud` is the OBS
3083
resource and the token is not expired.
31-
32-
## [0.3.0]
84+
- Update case-sensitive operation-name and span-name filters, dashboards, and
85+
alerts to the canonical values above, or compare case-insensitively.
86+
- Supply a non-blank `model` or `responseModel` when constructing
87+
`InferenceCallDetails`; validate optional configuration before constructing
88+
the telemetry object so missing model identity does not interrupt a request.
89+
- Update consumers that parse `gen_ai.input.messages` /
90+
`gen_ai.output.messages` to expect a JSON array rather than a
91+
`{version, messages}` object, and remove references to
92+
`A365_MESSAGE_SCHEMA_VERSION`.
93+
94+
## [1.0.0] - 2026-05-01
3395

3496
### Breaking Changes
3597

‎libraries/microsoft-agents-a365-observability-core/docs/design.md‎

Lines changed: 33 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -121,13 +121,23 @@ with InvokeAgentScope.start(
121121
Traces LLM/AI model inference calls:
122122

123123
```python
124-
from microsoft_agents_a365.observability.core import InferenceScope, InferenceCallDetails, Request
124+
from microsoft_agents_a365.observability.core import (
125+
InferenceCallDetails,
126+
InferenceOperationType,
127+
InferenceScope,
128+
Request,
129+
ServiceEndpoint,
130+
)
125131

126132
with InferenceScope.start(
127133
request=Request(content="Hello"),
128134
details=InferenceCallDetails(
129-
model_name="gpt-4",
130-
provider="openai"
135+
operationName=InferenceOperationType.CHAT,
136+
model="gpt-4",
137+
providerName="openai",
138+
responseModel="gpt-4-2025-04-14",
139+
cacheReadInputTokens=20,
140+
endpoint=ServiceEndpoint(hostname="models.example.com", port=443),
131141
),
132142
agent_details=agent_details,
133143
) as scope:
@@ -276,9 +286,9 @@ class AgentDetails:
276286
agent_auid: str | None # Agent unique identifier
277287
agent_email: str | None # Agent email address
278288
agent_blueprint_id: str | None
279-
agent_type: AgentType | None
280289
tenant_id: str | None
281290
icon_uri: str | None
291+
agent_type: AgentType | str | None # appended last for positional compat
282292
```
283293

284294
### UserDetails
@@ -317,11 +327,27 @@ class SpanDetails:
317327
```python
318328
@dataclass
319329
class InferenceCallDetails:
320-
model_name: str | None
321-
provider: str | None
322-
# Additional inference metadata
330+
operationName: InferenceOperationType
331+
model: str | None
332+
providerName: str
333+
inputTokens: int | None
334+
outputTokens: int | None
335+
finishReasons: list[str] | None
336+
thoughtProcess: str | None
337+
endpoint: ServiceEndpoint | None
338+
responseModel: str | None # appended last for positional compat
339+
cacheReadInputTokens: int | None # appended last for positional compat
340+
cacheWriteInputTokens: int | None # appended last for positional compat
323341
```
324342

343+
At least one of `model` or `responseModel` must be present (whitespace-only
344+
values are normalized to `None`). The `gen_ai.operation.name` attribute and the
345+
span name prefix are derived from `operationName` via its canonical lowercase
346+
form (`InferenceOperationType.CHAT` -> `chat`, `TEXT_COMPLETION` ->
347+
`text_completion`, `GENERATE_CONTENT` -> `generate_content`), matching the
348+
OpenTelemetry GenAI semantic conventions and the Agent365 model observation
349+
pipeline. Endpoint ports and token counts are emitted as integers.
350+
325351
### ToolCallDetails
326352

327353
```python

‎libraries/microsoft-agents-a365-observability-core/microsoft_agents_a365/observability/core/agent_details.py‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,9 @@
22
# Licensed under the MIT License.
33

44
from dataclasses import dataclass
5-
from typing import Optional
5+
from typing import Optional, Union
6+
7+
from .models.agent_type import AgentType
68

79

810
@dataclass
@@ -41,3 +43,10 @@ class AgentDetails:
4143

4244
agent_version: Optional[str] = None
4345
"""Optional version of the agent (e.g., "1.0.0", "2025-05-01")."""
46+
47+
# NOTE: Appended after all pre-existing fields (rather than inserted in the
48+
# middle) to preserve positional-argument compatibility for existing callers.
49+
agent_type: Optional[Union[AgentType, str]] = None
50+
"""Type classification for the agent. Accepts an `AgentType` enum member or
51+
a raw string; enum members are normalized to their string `.value` when
52+
emitted as a span attribute or baggage entry."""

‎libraries/microsoft-agents-a365-observability-core/microsoft_agents_a365/observability/core/constants.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@
3737
GEN_AI_RESPONSE_MODEL_KEY = "gen_ai.response.model"
3838
GEN_AI_RESPONSE_ID_KEY = "gen_ai.response.id"
3939
GEN_AI_AGENT_ID_KEY = "gen_ai.agent.id"
40+
GEN_AI_AGENT_TYPE_KEY = "gen_ai.agent.type"
4041
GEN_AI_AGENT_NAME_KEY = "gen_ai.agent.name"
4142
GEN_AI_AGENT_DESCRIPTION_KEY = "gen_ai.agent.description"
4243
GEN_AI_AGENT_VERSION_KEY = "gen_ai.agent.version"
@@ -47,6 +48,8 @@
4748
GEN_AI_TOKEN_TYPE_KEY = "gen_ai.token.type"
4849
GEN_AI_USAGE_INPUT_TOKENS_KEY = "gen_ai.usage.input_tokens"
4950
GEN_AI_USAGE_OUTPUT_TOKENS_KEY = "gen_ai.usage.output_tokens"
51+
GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS_KEY = "gen_ai.usage.cache_read.input_tokens"
52+
GEN_AI_USAGE_CACHE_WRITE_INPUT_TOKENS_KEY = "gen_ai.usage.cache_write.input_tokens"
5053
GEN_AI_CHOICE = "gen_ai.choice"
5154
GEN_AI_PROVIDER_NAME_KEY = "gen_ai.provider.name"
5255

‎libraries/microsoft-agents-a365-observability-core/microsoft_agents_a365/observability/core/exporters/utils.py‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,12 @@
3434
# Operation names that identify a span as eligible for export to the Agent 365
3535
# observability ingest service. Only spans whose gen_ai.operation.name matches
3636
# one of these values are included; all other spans are filtered out.
37+
#
38+
# Inference spans now emit the canonical lowercase operation name for their
39+
# InferenceOperationType (e.g. "chat", "text_completion", "generate_content" -
40+
# see InferenceOperationType.canonical_name), so all three are recognized here.
41+
# The legacy PascalCase "Chat" value is kept for back-compat with any
42+
# previously-emitted/non-Python spans that still use the raw enum value.
3743
GEN_AI_OPERATION_NAMES: frozenset[str] = frozenset(
3844
{
3945
INVOKE_AGENT_OPERATION_NAME,
@@ -42,6 +48,9 @@
4248
APPLY_GUARDRAIL_OPERATION_NAME,
4349
CHAT_OPERATION_NAME,
4450
InferenceOperationType.CHAT.value,
51+
InferenceOperationType.CHAT.canonical_name,
52+
InferenceOperationType.TEXT_COMPLETION.canonical_name,
53+
InferenceOperationType.GENERATE_CONTENT.canonical_name,
4554
}
4655
)
4756

‎libraries/microsoft-agents-a365-observability-core/microsoft_agents_a365/observability/core/inference_call_details.py‎

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,30 @@ class InferenceCallDetails:
1212
"""Details of an inference call for generative AI operations."""
1313

1414
operationName: InferenceOperationType
15-
model: str
15+
model: str | None
1616
providerName: str
1717
inputTokens: int | None = None
1818
outputTokens: int | None = None
1919
finishReasons: list[str] | None = None
2020
thoughtProcess: str | None = None
2121
endpoint: ServiceEndpoint | None = None
22+
# NOTE: Appended after all pre-existing fields (rather than inserted in the
23+
# middle) to preserve positional-argument compatibility for existing callers.
24+
responseModel: str | None = None
25+
cacheReadInputTokens: int | None = None
26+
cacheWriteInputTokens: int | None = None
27+
28+
def __post_init__(self) -> None:
29+
"""Normalize and validate required model identity.
30+
31+
Whitespace-only `model`/`responseModel` values are normalized to
32+
`None` so that validation and downstream consumers (e.g.
33+
`InferenceScope`) never treat a blank string as a present model.
34+
"""
35+
if self.model is not None:
36+
self.model = self.model.strip() or None
37+
if self.responseModel is not None:
38+
self.responseModel = self.responseModel.strip() or None
39+
40+
if not self.model and not self.responseModel:
41+
raise ValueError("At least one of model or responseModel must be provided")

‎libraries/microsoft-agents-a365-observability-core/microsoft_agents_a365/observability/core/inference_operation_type.py‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,3 +10,12 @@ class InferenceOperationType(Enum):
1010
CHAT = "Chat"
1111
TEXT_COMPLETION = "TextCompletion"
1212
GENERATE_CONTENT = "GenerateContent"
13+
14+
@property
15+
def canonical_name(self) -> str:
16+
"""The canonical lowercase operation name per the OpenTelemetry GenAI
17+
semantic conventions (e.g. ``chat``, ``text_completion``,
18+
``generate_content``). Used for both the span name prefix and the
19+
``gen_ai.operation.name`` attribute.
20+
"""
21+
return self.name.lower()

0 commit comments

Comments
 (0)