Repository navigation
Callback Enhancement: Include intermediate responses #1187
Description
Is your feature request related to a problem?
The current callback response only contains the final provider response, omitting important intermediate inputs and validation results. This limits visibility into the API's processing steps and effectiveness.
Describe the solution you'd like
Enhance the callback response to include:
- User’s input (post-input guardrail)
- Input to LLM (after input guardrail)
- Output from LLM
- response to user(after output guardrail)
Add a flag to enable detaled metadata
Original issue
Describe the current behavior
Consider a simple llm_call with a guardrail config.
POST api/v1/llm/call
{
"query": {
"input": "Amit Gupta phone number is 919611188278"
},
"config": {
"blob": {
"completion": {
"provider": "openai",
"type": "text",
"params": { "model": "gpt-5.6-sol" }
},
"input_guardrails": [
{ "validator_config_id": "d9e3c944-5718-44d2-b1b6-b52bfba7ee33" } //PII_remover
],
"output_guardrails": []
}
},
"callback_url": "https://webhooksite.net/ffda71fe-ece9-4871-a47e-4949753a0748",
"include_provider_raw_response": false
}
the final response sent to the user via the callback is
"success": true,
"data": {
"response": {
"provider_response_id": "resp_0860bf7f27cab00b006a9fcc7fd3b087d1a5c844f3d614176a",
"conversation_id": null,
"provider": "openai-native",
"model": "gpt-5.6-sol",
"output": {
"type": "text",
"content": {
"format": "text",
**"value": "Got it. What would you like me to do with that phone number?",**
"language_code": null
}
}
},
"usage": {
"input_tokens": 16,
"output_tokens": 68,
"total_tokens": 84,
"reasoning_tokens": null
},
"provider_raw_response": null
},
"error": null,
"errors": null,
"metadata": {
"warnings": []
}
}
The response sent back to the caller only contains final provider response. No intermediate input/output is sent across to the api caller. e.g
What gets input to lllm call post stripping of PII information is
<PERSON> phone number is <PHONE_NUMBER>
In a similar manner, for chained validator setup, we store intermediate responses in database but do not send them with the response payload. Ditto for output guardrails. We need to add four fields in callback response
- User’s input (post-input guardrail)
- Input to LLM (after input guardrail)
- Output from LLM
- response to user(after output guardrail)
Metadata
Metadata
Assignees
Type
Projects
- Status
Activity
[-]Guardrails: Callback response body enhancement[/-][+]Callback Enhancement: Include intermediate responses[/+]github-actions commentedon Sep 11, 2026
🎉 This issue has been resolved in version 1.7.0-main.4 🎉
The release is available on GitHub release
Your semantic-release bot 📦🚀