Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .changeset/span-export-hooks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
"braintrust": minor
---

feat: add span export hooks

Support synchronous `onSpanExport` customizers for incremental instrumentation
span records. Customizers can add, modify, delete, or replace fields before export,
with callbacks applied once per record rather than once per transport retry.
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,8 @@ pnpm run test # Run all workspace tests via turbo

Run from the repo root. **Always run `fix:formatting` before committing** — there is a pre-commit hook that will reject unformatted code.

Agents MUST run Prettier on every file they create or edit before handing work back, even when no commit is requested. Include Markdown, changelogs, config files, and generated files supported by Prettier—not just source code. From the repo root, run `pnpm exec prettier --write <edited-files>` followed by `pnpm exec prettier --check <edited-files>`. If further edits are made, repeat formatting and verification after the final edit. Do not rely on tests, typechecks, CI, or the pre-commit hook to catch formatting issues.

```bash
pnpm run formatting # Check formatting (prettier)
pnpm run lint # Run eslint checks
Expand Down
6 changes: 5 additions & 1 deletion js/src/exports.ts
Original file line number Diff line number Diff line change
Expand Up @@ -374,6 +374,10 @@ export {
braintrustFlueObserver,
braintrustFlueInstrumentation,
} from "./instrumentation";
export type { InstrumentationConfig } from "./instrumentation";
export type {
InstrumentationConfig,
SpanCustomizer,
SpanExportData,
} from "./instrumentation";

export { wrapElevenLabs } from "./wrappers/elevenlabs";
62 changes: 62 additions & 0 deletions js/src/instrumentation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,68 @@ termination, and async context.
- Use narrow vendored provider interfaces shared by wrappers and plugins.
- Keep enable, disable, subscription, and patching behavior idempotent.

## Export Customizers

Configure `spanCustomizers` through the standalone instrumentation entrypoint
before importing the main SDK, which enables instrumentation during platform
initialization. Use a bootstrap module before any auto-instrumentation preload
that initializes the SDK. Static imports of the main SDK are hoisted; use a
dynamic import after configuration:

```ts
import { configureInstrumentation } from "braintrust/instrumentation";

configureInstrumentation({
spanCustomizers: [
{
onSpanExport(data) {
data.tags = ["reviewed"];
if ("output" in data) data.output = "[redacted]";
delete data.error;
return data;
},
},
],
});

const { initLogger } = await import("braintrust");
initLogger({ projectName: "my-project" });
// Import and use instrumented provider SDKs here.
```

`onSpanExport` receives each incremental record from an instrumentation-created
span after lazy values resolve, before attachment processing, merging, masking,
and JSON serialization. It can run before the span ends; fields may be absent.
Ordinary manually created spans, dataset rows, and feedback are not customized.

Callbacks run synchronously in registration order. Mutate and return the record,
or return a replacement plain object for the next callback. Exceptions and invalid
return values are ignored while synchronous payload mutations remain; promises
are not awaited and their rejections are swallowed. Do not mutate the record after
returning. Export retries reuse the transformed record without invoking callbacks
again. Configuration is shared across SDK bundles.

The SDK restores these fields after every callback, including removing injected
fields that were absent from the original record:

- Identity: `id`, `span_id`, `root_span_id`, `span_parents`.
- Routing: `org_id`, `project_id`, `experiment_id`, `dataset_id`,
`prompt_session_id`, `log_id`, `function_data`.
- Transport controls: `_is_merge`, `_merge_paths`, `_parent_id`, `_object_delete`,
`_array_delete`, `_xact_id`.

Payload values must remain supported by the SDK logging pipeline. They can still
include `Attachment` objects at this point; attachment processing and JSON
serialization happen after customization.

This is an export-only hook, not a fail-closed privacy boundary. The local
experiment/scorer cache is populated before export and may retain unredacted
values. Applications requiring secrets to stay off local disk must disable the
span cache separately; export customization alone does not provide that guarantee.

Customizers receive only the outgoing record, not a live span or provider
instrumentation context.

## Testing

Test at the narrowest useful layers:
Expand Down
32 changes: 32 additions & 0 deletions js/src/instrumentation/config.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,29 @@
export type SpanExportData = Record<string, unknown>;

export interface SpanCustomizer {
/**
* Customize an outgoing span record after lazy values resolve, before JSON
* serialization. Records are incremental and may not contain every span field.
*
* Callbacks are synchronous. Add, change, or delete payload fields, then return
* the record or a replacement plain object. Payloads may still contain SDK
* Attachment objects; attachment processing and serialization happen later.
*
* The SDK restores identity and routing fields (id, span_id, root_span_id,
* span_parents, org_id, project_id, experiment_id, dataset_id, prompt_session_id,
* log_id, function_data) and transport controls (_is_merge, _merge_paths,
* _parent_id, _object_delete, _array_delete, _xact_id) after every callback.
*
* Exceptions and invalid return values are ignored; synchronous payload
* mutations remain. Promises are not awaited and their rejections are swallowed.
* Do not mutate the record after returning.
*
* This hook does not guarantee redaction of the local experiment/scorer cache,
* which is populated before export, and is not a fail-closed privacy boundary.
*/
onSpanExport?(data: SpanExportData): SpanExportData;
}

export interface InstrumentationIntegrationsConfig {
openai?: boolean;
anthropic?: boolean;
Expand Down Expand Up @@ -46,6 +72,12 @@ export interface InstrumentationConfig {
* Set to false to disable instrumentation for that SDK.
*/
integrations?: InstrumentationIntegrationsConfig;

/**
* Instrumentation-wide customizers, in callback execution order.
* Configure before instrumentation is enabled.
*/
spanCustomizers?: readonly SpanCustomizer[];
}

const envIntegrationAliases: Record<
Expand Down
1 change: 1 addition & 0 deletions js/src/instrumentation/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,4 @@ export {
// Configuration API
export { configureInstrumentation } from "./registry";
export type { InstrumentationConfig } from "./registry";
export type { SpanCustomizer, SpanExportData } from "./config";
4 changes: 4 additions & 0 deletions js/src/instrumentation/registry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import {
type InstrumentationConfig,
} from "./config";
import { GLOBAL_INSTRUMENTATION_HOOKS_PROTOCOL_VERSION } from "../global-instrumentation-hooks";
import { setSpanCustomizers } from "../span-customizer";

export type { InstrumentationConfig } from "./config";

Expand Down Expand Up @@ -62,6 +63,9 @@ class PluginRegistry {
return;
}
this.config = { ...this.config, ...config };
if ("spanCustomizers" in config) {
setSpanCustomizers(config.spanCustomizers);
}
}

/**
Expand Down
Loading
Loading