feat(mcp): use fixed Cloud error message as Datadog span error.message by calebevans-ab · Pull Request #1356 · airbytehq/PyAirbyte

Caleb Evans (calebevans-ab) · 2026-10-10T07:31:15Z

Summary

TL;DR: failed Cloud tool calls now show in Datadog the same fixed message the agent gets, plus the Cloud endpoint that failed, with no IDs or query strings. initialize and tools/list stop going to LLM Obs and stay plain APM spans.

before: tool resulted in an error (user_error, HTTP 403, forbidden)
after:  error.message = The requested resource was not found, or these credentials can't access it (HTTP 403).
        airbyte.mcp.upstream.method = GET
        airbyte.mcp.upstream.route  = /v1/connections/{id}

1. Fixed Cloud error.message

sdk_error_message(error: SDKError) -> str in api_util picks the message for an SDKError: actor-not-ready, then 403 (FORBIDDEN_RESOURCE_MESSAGE), then describe_cloud_error. _wrap_sdk_error now uses it, with no change in behavior.
cloud_error_trace_message(error) walks the exception chain. It returns the fixed message for the first of these it finds, and None if there is none:
- an AirbyteCloudApiError;
- an SDKError;
- a Config API wrapper error, meaning an AirbyteError raised from requests.HTTPError by _make_config_api_request, whose message is already the fixed catalog text.
_exception_attributes sets error.message while the exception is still available, in the intent middleware. FastMCP then turns the exception into a returned CallToolResult(isError=True). The outer request middleware sets the tool resulted in an error (...) summary only when no error.message is set yet.

2. Failing Cloud endpoint (new, native Datadog backend only)

cloud_error_route(error) -> tuple[str, str] | None reads the first SDKError's raw_response.request and keeps only the URL path, so host, credentials, query and fragment are dropped.
- Methods outside GET/POST/PUT/PATCH/DELETE give None.
- Every segment before the last v<N> version segment becomes {id}, so a custom api_root prefix never shows. With no version segment, every segment becomes {id}.
- After the version, segments are kept only if they are in _CLOUD_API_ROUTE_SEGMENTS (the literal segments of the airbyte_api path templates). Everything else becomes {id}.
- Routes over 200 characters give None.
It is set as the APM tags airbyte.mcp.upstream.method/.route, and as LLM Obs metadata.

3. Lifecycle requests leave LLM Obs (new)

initialize and tools/list now use tracer.trace("mcp.<method>", resource="server_request") instead of LLMObs.task.
- They keep their APM tags, the _dd.measured metric and the error/outcome attributes.
- They no longer call LLMObs.annotate, so no lifecycle input or output is recorded.
Why: these were about 64% of cloud-mcp's LLM Obs traces (~3,200/day). Each tools/list stored the whole tool list, about 105 KB.
Tool calls are unchanged.

Safety: error.message is only fixed text from cloud_errors.yaml, or a constant. It never contains Cloud's detail, data.message, raw title, the guidance half (which can include Cloud's error ID), or anything else from the response body. error.message and error.stack stay out of LLM Obs metadata.

Tests: tests/unit_tests/test_mcp_datadog.py:

route masking: UUIDs, job IDs, lowercase-only IDs, custom api_root prefixes (including ones that contain allowlisted words) and paths with no version all become {id}; URLs with credentials or a query string are cleaned; a missing request, a non-SDK error or an unsupported method gives None;
Config API 403/404 messages; a wrapper error with no HTTP cause gives None;
an end-to-end FastMCP run of a failing 403 Cloud call checks the fixed message and the route, and that the UUID appears in no tag or metadata;
a returned error result with no exception still gets the summary;
lifecycle spans are plain APM spans with resource server_request and no LLM Obs event.

Unchanged: agent-facing text, tool-filter rejections and non-Cloud exceptions. The OTel/datadog-otlp backend sets no error text or route today, since its error attributes come from fastmcp-extensions.

Link to Devin session: https://app.devin.ai/sessions/85c491e543a74ab89151fdf7a0ba5d20
Open in Devin Desktop: https://app.devin.ai/desktop/session/85c491e543a74ab89151fdf7a0ba5d20?variant=devin
Requested by: Caleb Evans (@calebevans-ab)

Important

Auto-merge enabled.

This PR is set to merge automatically when all requirements are met.

devin-ai-integration · 2026-10-10T07:31:19Z

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

Disable automatic comment, CI, and merge conflict monitoring

octavia-bot · 2026-10-10T07:31:25Z

Note

📝 PR Converted to Draft

More info...

Thank you for creating this PR. As a policy to protect our engineers' time, Airbyte requires all PRs to be created first in draft status. Your PR has been automatically converted to draft status in respect for this policy.

As soon as your PR is ready for formal review, you can proceed to convert the PR to "ready for review" status by clicking the "Ready for review" button at the bottom of the PR page.

To skip draft status in future PRs, please include [ready] in your PR title or add the skip-draft-status label when creating your PR.

chatgpt-codex-connector · 2026-10-10T07:31:28Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-10T09:22:10.123206Z	`b4e7dd1`	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "Codex (@codex) review" or "Codex (@codex) security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

github-actions · 2026-10-10T07:31:43Z

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This PyAirbyte Version

You can test this version of PyAirbyte using the following:

# Run PyAirbyte CLI from this branch:
uvx --from 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1791617171-span-cloud-error-message' pyairbyte --help

# Install PyAirbyte from this branch for development:
pip install 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1791617171-span-cloud-error-message'

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

/fix-pr - Fixes most formatting and linting issues
/uv-lock - Updates uv.lock file
/test-pr - Runs tests with the updated PyAirbyte
/prerelease - Builds and publishes a prerelease version to PyPI

📚 Show Repo Guidance

Helpful Resources

Community Support

Questions? Join the #pyairbyte channel in our Slack workspace.

📝 Edit this welcome message.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c17102ca06

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".

github-code-quality · 2026-10-10T07:35:49Z

Code Coverage Overview

Languages: Python

Python / code-coverage/pytest-no-creds

The overall line coverage in commit b4e7dd1 in the devin/1791617171-spa... branch is 79%. The line coverage in commit d9f652f in the release-candidate/v1 branch is 65%.

Show a line coverage summary of the most impacted files.

File	`release-candidate/v1` d9f652f	`devin/1791617171-spa...` b4e7dd1	+/-
`airbyte/mcp/cloud.py`	52%	83%	+31%
`airbyte/_util/api_util.py`	36%	68%	+32%
`airbyte/cloud/connectors.py`	34%	75%	+41%
`airbyte/mcp/_datadog.py`	0%	50%	+50%
`airbyte/mcp/_error_handling.py`	0%	89%	+89%
`airbyte/_direct...ctors/models.py`	0%	94%	+94%
`airbyte/mcp/_telemetry.py`	0%	94%	+94%
`airbyte/mcp/_sso_auth.py`	0%	96%	+96%
`airbyte/cloud/models.py`	0%	97%	+97%
`airbyte/mcp/int...nc_status_ui.py`	0%	97%	+97%

Python / code-coverage/pytest

The overall line coverage in commit b4e7dd1 in the devin/1791617171-spa... branch is 81%. The line coverage in commit d9f652f in the release-candidate/v1 branch is 71%.

Show a line coverage summary of the most impacted files.

File	`release-candidate/v1` d9f652f	`devin/1791617171-spa...` b4e7dd1	+/-
`airbyte/mcp/cloud.py`	52%	83%	+31%
`airbyte/mcp/_datadog.py`	0%	50%	+50%
`airbyte/mcp/_error_handling.py`	0%	89%	+89%
`airbyte/mcp/int..._registry_ui.py`	0%	92%	+92%
`airbyte/mcp/_user_identity.py`	0%	93%	+93%
`airbyte/_direct...ctors/models.py`	0%	94%	+94%
`airbyte/mcp/_telemetry.py`	0%	94%	+94%
`airbyte/mcp/_sso_auth.py`	0%	96%	+96%
`airbyte/cloud/models.py`	0%	97%	+97%
`airbyte/mcp/int...nc_status_ui.py`	0%	97%	+97%

Python / code-coverage/pytest-unit

The overall line coverage in commit b4e7dd1 in the devin/1791617171-spa... branch is 75%. Line coverage data for the release-candidate/v1 branch is not yet available.

Show a line coverage summary of the most covered files.

File	`release-candidate/v1`	`devin/1791617171-spa...` b4e7dd1	+/-
`airbyte/cloud/models.py`	—	97%	—
`airbyte/mcp/_sso_auth.py`	—	96%	—
`airbyte/_direct...ctors/models.py`	—	94%	—
`airbyte/cloud/client.py`	—	90%	—
`airbyte/exceptions.py`	—	89%	—
`airbyte/mcp/cloud.py`	—	83%	—
`airbyte/progress.py`	—	80%	—
`airbyte/cloud/connectors.py`	—	75%	—
`airbyte/_util/api_util.py`	—	68%	—
`airbyte/shared/...ql_processor.py`	—	68%	—

_{Updated October 10, 2026 09:27 UTC}

Caleb Evans (calebevans-ab) · 2026-10-10T08:53:22Z

Codex (@codex) review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dc2090ea90

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".

Caleb Evans (calebevans-ab) · 2026-10-10T09:04:56Z

Codex (@codex) review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 366bbf93b0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

Caleb Evans (calebevans-ab) · 2026-10-10T09:19:14Z

Codex (@codex) review

chatgpt-codex-connector · 2026-10-10T09:22:08Z

Codex Review: Didn't find any major issues. What shall we delve into next?

Reviewed commit: b4e7dd1df8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".

feat(mcp): use fixed Cloud error message as Datadog span error.message

c17102c

devin-ai-integration Bot assigned Caleb Evans (calebevans-ab) Oct 10, 2026

octavia-bot Bot marked this pull request as draft October 10, 2026 07:31

chatgpt-codex-connector Bot reviewed Oct 10, 2026

View reviewed changes

Comment thread airbyte/mcp/_datadog.py

Comment thread airbyte/mcp/_datadog.py

Caleb Evans (calebevans-ab) marked this pull request as ready for review October 10, 2026 08:42

devin-ai-integration Bot added 3 commits October 10, 2026 08:44

feat(mcp): tag sanitized upstream Cloud routes

7cc3d2a

feat(mcp): separate lifecycle APM spans from LLM Obs

e32b3f7

fix(mcp): preserve safe Cloud error messages on Datadog spans

dc2090e

chatgpt-codex-connector Bot reviewed Oct 10, 2026

View reviewed changes

Comment thread airbyte/mcp/_error_handling.py

Comment thread airbyte/mcp/_error_handling.py Outdated

Comment thread airbyte/mcp/_otel.py Outdated

fix(mcp): tighten Cloud route allowlist and cover Config API errors

366bbf9

chatgpt-codex-connector Bot reviewed Oct 10, 2026

View reviewed changes

Comment thread airbyte/mcp/_error_handling.py Outdated

devin-ai-integration Bot and others added 2 commits October 10, 2026 09:12

fix(mcp): mask Cloud route segments before the API version

dc946f0

fix(mcp): annotate Cloud route segments for type checks

b4e7dd1

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

Caleb Evans (calebevans-ab) enabled auto-merge (squash) October 10, 2026 09:22

Caleb Evans (calebevans-ab) merged commit 9a8cc2c into release-candidate/v1 Oct 10, 2026
29 checks passed

Caleb Evans (calebevans-ab) deleted the devin/1791617171-span-cloud-error-message branch October 10, 2026 09:27

Caleb Evans (calebevans-ab) mentioned this pull request Oct 11, 2026

feat(mcp): give non-Cloud tool errors fixed Datadog span messages from exception docstrings #1358

Merged

Conversation

Caleb Evans (calebevans-ab) commented Oct 10, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

devin-ai-integration Bot commented Oct 10, 2026

Uh oh!

octavia-bot Bot commented Oct 10, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

github-actions Bot commented Oct 10, 2026

👋 Greetings, Airbyte Team Member!

Testing This PyAirbyte Version

PR Slash Commands

Helpful Resources

Community Support

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

github-code-quality Bot commented Oct 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage Overview

Python / code-coverage/pytest-no-creds

Python / code-coverage/pytest

Python / code-coverage/pytest-unit

Uh oh!

Caleb Evans (calebevans-ab) commented Oct 10, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Caleb Evans (calebevans-ab) commented Oct 10, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Caleb Evans (calebevans-ab) commented Oct 10, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Caleb Evans (calebevans-ab) commented Oct 10, 2026 •

edited by github-actions Bot

Loading

chatgpt-codex-connector Bot commented Oct 10, 2026 •

edited

Loading

github-code-quality Bot commented Oct 10, 2026 •

edited

Loading