Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions A365_DOCUMENTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,68 @@ Provided agent ID, name, blueprint ID, platform ID, and version are recorded und
`version`). Other target identity fields are not emitted, and the source agent and tenant
attributes remain unchanged.

### Tool skill and command refinements

Supply explicit `ToolCallDetails.refinementDetails` from the tool API or runtime state,
including on failed attempts. The exported `ExecuteToolRefinementDetails` union accepts
`SkillRefinementDetails` (`kind: "load_skill"` or `"read_skill_resource"`) and
`ExecuteCommandRefinementDetails` (`kind: "execute_command"`). Skill refinements do not
accept process fields. An invalid `kind` throws before creating a span.

All refinements accept `skillName`, `description`, `sourceUri`, and `resourceName`,
emitted as `gen_ai.skill.name`, `gen_ai.skill.description`, `gen_ai.skill.source.uri`,
and `gen_ai.skill.resource.name`. Command refinements additionally accept
`executableName`, `executablePath`, and numeric `exitCode`, emitted as
`process.executable.name`, `process.executable.path`, and `process.exit.code`.
Only supply executable name/path when the tool directly executes a single process,
not for an opaque shell expression or pipeline. String fields that are absent, empty,
or whitespace-only are omitted; nonblank values are recorded unchanged.

```typescript
import { ExecuteToolScope } from "@microsoft/opentelemetry";
import type { ToolCallDetails } from "@microsoft/opentelemetry";

const details: ToolCallDetails = {
toolName: "shell",
refinementDetails: {
kind: "execute_command",
skillName: "code-review",
sourceUri: "file:///opt/skills/code-review",
resourceName: "scripts/check.py",
executableName: "python",
executablePath: "/usr/bin/python",
},
};
const scope = ExecuteToolScope.start(
{ conversationId: "conv-123" },
details,
{ agentId: "agent-1", tenantId: "tenant-1" },
);
// Record the actual exit code reported by the runtime, including zero for success.
scope.recordProcessExitCode(0);
scope.dispose();
```

For `load_skill`, the span name is `execute_tool {toolName} {skillName}`, falling back
to `execute_tool {toolName}` without a nonblank skill name. For resource readers and
commands, a nonblank resource name takes precedence:
`execute_tool {toolName} {skillName} {resourceName}` (omit the skill name if unavailable).
Without a resource name, commands append the executable name if available; resource
readers retain the ordinary tool span name, even if a skill name is present.

The operation remains `execute_tool`; no extra spans are created. No refinement is
inferred from a tool name or opaque arguments. Existing tool, endpoint, transfer,
source-agent, and tenant attributes are unchanged. `recordProcessExitCode()` records
the reported numeric value, including zero, and can replace an initially supplied
exit code. It does not infer failure or clear an existing error; use `recordError()`
when the tool reports an error.

Skill descriptions and source URIs may contain sensitive information. Supply the
skill's source URI, not a temporary materialization path, and remove credentials,
access tokens, and other sensitive components before recording it. Refinement metadata
is explicitly supplied and is not automatically scrubbed or gated by `enableSensitiveData`.
Skill identity/version and skill metrics proposals are not part of this API.

### Request and response attributes

`InvokeAgentScope`, `InferenceScope`, and `ExecuteToolScope` accept `request.sessionId`.
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
- **Breaking change on Unix (Linux/macOS):** Agent365 offline buffering/replay now requires an explicitly configured, existing, absolute `durableDelivery.storageDirectory`. No automatic Unix temporary-directory fallback or root creation. Missing/invalid roots are logged and disable only disk buffering/replay; online export continues. Customers provision and maintain ownership, permissions (normally a private service-account-owned `0700` root), ACLs, and path-replacement protection; the SDK does not inspect or repair Unix access controls. Windows behavior, explicit-root cache identities, record compatibility, and retention limits are unchanged. Old temporary backlogs are not automatically migrated; see [storage provisioning and migration](A365_DOCUMENTATION.md#storage-provisioning-and-migration).

### Features Added
- Add explicit skill-loading, skill-resource-reading, and command refinements to `ExecuteToolScope` through typed `refinementDetails`. Emit skill/process attributes and semantic-convention span names, and support recording process exit codes after execution without changing error status. Ordinary tool calls remain unchanged.
- Add optional full tool endpoint URLs (`endpoint.url` -> `url.full`) and typed execute-tool transfer metadata for agent, human, and workflow targets. Export transfer enums and contracts, validate incompatible targets before creating spans, and preserve source agent identity.
- Add privacy-safe Model Observation attributes to `InferenceScope`, including the canonical served response model and cache read/write input-token counts. Attributes can be supplied when the scope starts or recorded after the response is available; inference operations accept enum values or canonical strings and use canonical OpenTelemetry names for both operation attributes and span names. Cache-token semantic-convention constants are exported. [#261](https://github.com/microsoft/opentelemetry-distro-javascript/pull/261)

Expand Down
15 changes: 15 additions & 0 deletions src/a365/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,21 @@ export class OpenTelemetryConstants {
/** Attribute key for the tool type (`gen_ai.tool.type`). */
public static readonly GEN_AI_TOOL_TYPE_KEY = "gen_ai.tool.type";

/** Skill name supplied by the tool runtime. */
public static readonly GEN_AI_SKILL_NAME_KEY = "gen_ai.skill.name";
/** Skill description supplied by the tool runtime. */
public static readonly GEN_AI_SKILL_DESCRIPTION_KEY = "gen_ai.skill.description";
/** Caller-scrubbed source URI of the skill. */
public static readonly GEN_AI_SKILL_SOURCE_URI_KEY = "gen_ai.skill.source.uri";
/** Skill-relative resource or script name. */
public static readonly GEN_AI_SKILL_RESOURCE_NAME_KEY = "gen_ai.skill.resource.name";
/** Name of a directly executed process. */
public static readonly PROCESS_EXECUTABLE_NAME_KEY = "process.executable.name";
/** Path of a directly executed process. */
public static readonly PROCESS_EXECUTABLE_PATH_KEY = "process.executable.path";
/** Reported exit code of a command or process. */
public static readonly PROCESS_EXIT_CODE_KEY = "process.exit.code";

// ── Tenant ───────────────────────────────────────────────────────

/** Attribute key for the tenant identifier (`microsoft.tenant.id`). */
Expand Down
42 changes: 42 additions & 0 deletions src/a365/contracts.ts
Original file line number Diff line number Diff line change
Expand Up @@ -480,6 +480,46 @@ export interface InvokeAgentScopeDetails {
responseParameters?: GenAiResponseParameters;
}

/** Explicit metadata for skill loading or resource reading. Process fields are command-only. */
export interface SkillRefinementDetails {
/** The skill operation performed by the tool. */
kind: "load_skill" | "read_skill_resource";
/** Skill name, when available. */
skillName?: string;
/** Skill description. May contain sensitive information. */
description?: string;
/** Skill source URI, not a temporary path. Callers must remove credentials and tokens. */
sourceUri?: string;
/** Skill-relative resource name, such as `references/policy.md`. */
resourceName?: string;
executableName?: never;
executablePath?: never;
exitCode?: never;
}

/** Explicit command metadata, optionally associated with a skill's bundled script. */
export interface ExecuteCommandRefinementDetails {
/** Identifies a command-executing tool. */
kind: "execute_command";
/** Executable name, only when directly executing a single process. */
executableName?: string;
/** Executable path, only when directly executing a single process. */
executablePath?: string;
/** Reported exit code for a single command or process, including zero. */
exitCode?: number;
/** Skill name, when executing a skill's script. */
skillName?: string;
/** Skill description. May contain sensitive information. */
description?: string;
/** Skill source URI, not a temporary path. Callers must remove credentials and tokens. */
sourceUri?: string;
/** Skill-relative script name, such as `scripts/check.py`. */
resourceName?: string;
}

/** Explicit skill or command metadata; never inferred from tool names or arguments. */
export type ExecuteToolRefinementDetails = SkillRefinementDetails | ExecuteCommandRefinementDetails;

/** Details of a tool call made by an agent. */
export interface ToolCallDetails {
/** Name of the tool being called (required). */
Expand All @@ -496,6 +536,8 @@ export interface ToolCallDetails {
endpoint?: ServiceEndpoint;
/** Optional explicit transfer performed by this tool call. */
transferDetails?: TransferDetails;
/** Optional explicit skill or command refinement, including on failed attempts. */
refinementDetails?: ExecuteToolRefinementDetails;
}

/** Details for an inference call. */
Expand Down
3 changes: 3 additions & 0 deletions src/a365/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,9 @@ export type {
GenAiResponseParameters,
InvokeAgentScopeDetails,
ToolCallDetails,
ExecuteToolRefinementDetails,
SkillRefinementDetails,
ExecuteCommandRefinementDetails,
TransferDetails,
InferenceDetails,
InferenceResponse,
Expand Down
53 changes: 52 additions & 1 deletion src/a365/scopes/ExecuteToolScope.ts
Original file line number Diff line number Diff line change
Expand Up @@ -63,14 +63,39 @@ export class ExecuteToolScope extends OpenTelemetryScope {
}
}

const refinement = details.refinementDetails;
if (
refinement &&
refinement.kind !== "load_skill" &&
refinement.kind !== "read_skill_resource" &&
refinement.kind !== "execute_command"
) {
throw new RangeError("ExecuteToolScope: invalid refinement kind");
}

let spanName = `${OpenTelemetryConstants.EXECUTE_TOOL_OPERATION_NAME} ${details.toolName}`;
if (refinement?.kind === "load_skill") {
if (refinement.skillName?.trim()) {
spanName += ` ${refinement.skillName}`;
}
} else if (refinement) {
if (refinement.resourceName?.trim()) {
spanName += refinement.skillName?.trim()
? ` ${refinement.skillName} ${refinement.resourceName}`
: ` ${refinement.resourceName}`;
} else if (refinement.kind === "execute_command" && refinement.executableName?.trim()) {
spanName += ` ${refinement.executableName}`;
}
}

const resolvedSpanDetails: SpanDetails = {
spanKind: SpanKind.INTERNAL,
...spanDetails,
};

super(
OpenTelemetryConstants.EXECUTE_TOOL_OPERATION_NAME,
`${OpenTelemetryConstants.EXECUTE_TOOL_OPERATION_NAME} ${details.toolName}`,
spanName,
agentDetails,
resolvedSpanDetails,
userDetails,
Expand All @@ -90,6 +115,27 @@ export class ExecuteToolScope extends OpenTelemetryScope {
this.setTagMaybe(OpenTelemetryConstants.GEN_AI_TOOL_CALL_ID_KEY, toolCallId);
this.setTagMaybe(OpenTelemetryConstants.GEN_AI_TOOL_DESCRIPTION_KEY, description);

if (refinement) {
const textAttributes: [string, string | undefined][] = [
[OpenTelemetryConstants.GEN_AI_SKILL_NAME_KEY, refinement.skillName],
[OpenTelemetryConstants.GEN_AI_SKILL_DESCRIPTION_KEY, refinement.description],
[OpenTelemetryConstants.GEN_AI_SKILL_SOURCE_URI_KEY, refinement.sourceUri],
[OpenTelemetryConstants.GEN_AI_SKILL_RESOURCE_NAME_KEY, refinement.resourceName],
];
if (refinement.kind === "execute_command") {
textAttributes.push(
[OpenTelemetryConstants.PROCESS_EXECUTABLE_NAME_KEY, refinement.executableName],
[OpenTelemetryConstants.PROCESS_EXECUTABLE_PATH_KEY, refinement.executablePath],
);
this.setTagMaybe(OpenTelemetryConstants.PROCESS_EXIT_CODE_KEY, refinement.exitCode);
}
for (const [key, value] of textAttributes) {
if (value?.trim()) {
this.setTagMaybe(key, value);
}
}
}

if (transfer) {
this.setTagMaybe(OpenTelemetryConstants.TRANSFER_MODE_KEY, transfer.mode);
this.setTagMaybe(OpenTelemetryConstants.TRANSFER_TARGET_TYPE_KEY, targetType);
Expand Down Expand Up @@ -119,6 +165,11 @@ export class ExecuteToolScope extends OpenTelemetryScope {
}
}

/** Records a reported command or process exit code without changing the span's error status. */
public recordProcessExitCode(exitCode: number): void {
this.setTagMaybe(OpenTelemetryConstants.PROCESS_EXIT_CODE_KEY, exitCode);
}

/**
* Records response information for telemetry tracking.
* Objects are serialized to JSON automatically.
Expand Down
3 changes: 3 additions & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,9 @@ export type {
GenAiResponseParameters,
InvokeAgentScopeDetails,
ToolCallDetails,
ExecuteToolRefinementDetails,
SkillRefinementDetails,
ExecuteCommandRefinementDetails,
TransferDetails,
InferenceDetails,
InferenceResponse,
Expand Down
Loading
Loading