feat(telemetry): emit llm.prompt_template.* on backend spans

[ajbozarth]

Issue

Fixes #1067

Description

Emits the OpenInference llm.prompt_template.template and llm.prompt_template.variables attributes on backend generation spans, so traces surface the prompt template and the variables substituted into it rather than only the fully-rendered prompt string.

The template and variables are derived at the formatter layer (TemplateFormatter.prompt_template_for) and recorded on mot.generation (GenerationMetadata) by each formatter backend, alongside the existing model/provider fields — so they are first-class generation data, not a telemetry-only artifact. The backend tracing plugin reads them off mot.generation and emits llm.prompt_template.template always (static structure) and llm.prompt_template.variables only when content capture is enabled (MELLEA_TRACES_CONTENT), since variables may contain user data.

Scope is the chat path (generate_from_context); generate_from_raw is intentionally excluded because its batched actions have no single unified template. These are the first OpenInference (llm.*) attributes in the codebase, complementing the existing OTel Gen-AI (gen_ai.*) and Mellea (mellea.*) attributes.

Testing

• Tests added to the respective file if code was changed
• New code has 100% coverage if code was added
• Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

• AI coding assistants used

---

Adding a new component, requirement, sampling strategy, or tool?

If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.

• Component
• Requirement
• Sampling Strategy
• Tool

NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.

[AngeloDanducci]

Given the changes to ChatFormatter and TemplateFormatter we may need some other changes - every formatter has prompt_template_for seems to be the assumption but I don't think that's the case in mellea/core/formatter.py for the base class.

I think the test_huggingface_unit.py test failures are related to the same thing, in SimpleNamespace there's no defining of prompt_template_for at the moment.

[ajbozarth]

@AngeloDanducci the bug was actually that I accidentally was setting the var on the unsupported intrinsics path when only the chat path is supported, I removed it in f5017c2

[planetf1]

Two things worth addressing, one blocking. Verified against f5017c2b; CI is green and I reran test/telemetry/ test/formatters/ locally (194 passed).

Confirming first that the SimpleNamespace failure Angelo hit is properly fixed. Root cause was _generate_from_intrinsic (huggingface.py:1066) also calling prompt_template_for, and dropping it there is right on both counts: the stub has no such attribute, and intrinsics have no unified template anyway. The base-Formatter worry doesn't bite either, since Backend.formatter is typed ChatFormatter (backend.py:52), which now supplies the (None, None) default.

Blocking: prompt_template_for mutates the caller's TemplateRepresentation.

mellea/formatters/template_formatter.py:212

if representation.obj is None:
    representation.obj = action

That's a write to an object the component owns, inside a method the docstring describes as best-effort and read-only. A component that returns a cached or shared representation with obj=None has it silently reassigned as a side effect of telemetry collection:

obj before: None
obj after : Shared
MUTATED SHARED STATE: True

Two things make it worse than it first looks. The same mutation previously only happened in _stringify (:122), where a render was actually about to occur, and it logged a warning when it did. This copy is silent. And because set_prompt_template_attrs is also wired into finish_backend_span_error, a failed generation can leave the mutation behind too.

Cheap fix, keeps the accessor side-effect-free:

if representation.obj is None:
    representation = dataclasses.replace(representation, obj=action)

Non-blocking: the work runs even with tracing disabled.

All five backends call prompt_template_for(action) in _generate_from_context* without checking is_tracing_enabled(). With MELLEA_TRACES_ENABLED unset, format_for_llm is still invoked and every template arg still recursively stringified, to fill fields nothing reads.

I measured it before raising it, and I want to be clear this is not a performance problem, so please don't spend time optimising it:

prompt_template_for: 0.053 ms/call
formatter.print()  : 1.123 ms/call   (for scale)

That's ~5% of a single render against hundreds of ms of network latency. The reason it's still worth noting is that it means a side effect (see above) fires when the feature is switched off, and that format_for_llm, a user extension point, is now invoked twice per generation. That quietly assumes it's cheap and idempotent for every component anyone writes. A guard on is_tracing_enabled() handles both; documenting the assumption would do.

Minor, no action needed here: the identical 4-line block now appears at six call sites (ollama:1205, watsonx:578, litellm:525, openai:1506, huggingface:1726 and :1922). The sixth is the one that caused the failure above, and it surfaced through a broken test rather than review. A parametrised cross-backend assertion would be cheap insurance for whenever a seventh backend appears.

[ajbozarth]

@planetf1 I've addressed you review in 6aa0086

Blocking: prompt_template_for mutates the caller's TemplateRepresentation.

updated to use replace as suggested

Non-blocking: the work runs even with tracing disabled.

We don't gate on telemetry flags outside the telemetry dir (I assume this item was just the AI seeing a user API is_tracing_enabled() and wondering why we're not using it internally)

[planetf1]

We don't gate on telemetry flags outside the telemetry dir (I assume this item was just the AI seeing a user API is_tracing_enabled() and wondering why we're not using it internally)

I think the point here was that format_for_llm() is a user extension point, and this now runs it a second time per generation to populate telemetry fields. If someone's override has any side effect that isn't idempotent, it now happens twice for every request. The mutation bug above is one concrete example of that. Other examples might include it being used for custom counters etc

So it’s more the behaviour / side effect.

Maybe we can cover in docs as it doesn’t actually cause an issue right now , being more likely if someone is writing a custom component. So perhaps where we document the method, or in custom components md?

[planetf1]

Going to approve - there’s a possible minor issue but will leave it to you to decide what to do. just wanted to ensure my point was clear. therefore I’m going to approve.

[ajbozarth]

@planetf1 I added a couple short docs/docstring updates in 8751b02 about format_for_llm

If you could take a final look I'll merge this once you give another approve