feat(mcp): give non-Cloud tool errors fixed Datadog span messages from exception docstrings by calebevans-ab · Pull Request #1358 · airbytehq/PyAirbyte

Caleb Evans (calebevans-ab) · 2026-10-11T22:59:47Z

TL;DR: Failed tool calls that aren't Cloud API errors (local-demo-mode refusals, a missing workspace, bad input, an unknown tool, ...) now get a fixed, class-specific Datadog error.message: the first line of the exception class's docstring. Before, they got tool resulted in an error (user_error). There's no separate list to maintain, and what the agent sees doesn't change.

Summary

#1356 put fixed text on the span only for Cloud API errors. Every other failure still exported only the classification summary, so trace consumers (e.g. an eval pipeline judging whether a refusal explained itself) had nothing to go on.

tool_error_trace_message(error):
    cloud_error_trace_message(error)              # Cloud catalog keeps precedence
    or, for exc in _chain(error) (outermost first), for cls in type(exc).__mro__:
         AirbyteLibError subclass (not the base) with a docstring -> cls.__doc__ first line
         cls in _NON_AIRBYTE_TOOL_ERROR_MESSAGES                 -> fixed text
    or None                                       # -> existing `tool resulted in an error (...)` summary

The message comes from the exception class docstring, never from the instance's message or args, so no IDs, argument values or exception text reach Datadog. It's the same first-line rule AirbyteLibError.get_message() already uses as its default.
_NON_AIRBYTE_TOOL_ERROR_MESSAGES covers the 5 types we don't own: FastMCP ValidationError/NotFoundError/DisabledError, asyncio.CancelledError and NotImplementedError. It's keyed by class, so a pydantic ValidationError raised inside tool code doesn't match FastMCP's argument-validation entry.
_datadog._exception_attributes and _raised_tool_error_message now call tool_error_trace_message instead of cloud_error_trace_message. The OTel backend is untouched: span messages stay Datadog-only, as in feat(mcp): use fixed Cloud error message as Datadog span error.message #1356.
Two docstring first lines are fixed (AirbyteMissingResourceError, AirbyteConnectionSyncTimeoutError). This also changes get_message()'s default text for those two classes when they're raised without a message.
test_airbyte_exception_docstrings_have_safe_first_lines requires every airbyte.exceptions class to have a docstring first line with no {, } or ://. The real-FastMCP end-to-end test covers a local-demo-mode refusal, bad arguments and an unknown tool. It also checks three more things: the bare ValueError and Cloud cases are unchanged, no exception text appears in span tags, and the agent-facing text is unchanged.

Example, the local-demo-mode refusal span:

- error.message: tool resulted in an error (user_error)
+ error.message: A change was blocked by MCP local demo mode.

Link to Devin session: https://app.devin.ai/sessions/85c491e543a74ab89151fdf7a0ba5d20
Open in Devin Desktop: https://app.devin.ai/desktop/session/85c491e543a74ab89151fdf7a0ba5d20?variant=devin
Requested by: Caleb Evans (@calebevans-ab)

Important

Auto-merge enabled.

This PR is set to merge automatically when all requirements are met.

…m a manifest

devin-ai-integration · 2026-10-11T22:59:51Z

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

Disable automatic comment, CI, and merge conflict monitoring

github-actions · 2026-10-11T23:00:13Z

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This PyAirbyte Version

You can test this version of PyAirbyte using the following:

# Run PyAirbyte CLI from this branch:
uvx --from 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1791758951-mcp-tool-error-trace-messages' pyairbyte --help

# Install PyAirbyte from this branch for development:
pip install 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1791758951-mcp-tool-error-trace-messages'

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

/fix-pr - Fixes most formatting and linting issues
/uv-lock - Updates uv.lock file
/test-pr - Runs tests with the updated PyAirbyte
/prerelease - Builds and publishes a prerelease version to PyPI

📚 Show Repo Guidance

Helpful Resources

Community Support

Questions? Join the #pyairbyte channel in our Slack workspace.

📝 Edit this welcome message.

github-code-quality · 2026-10-11T23:04:26Z

Code Coverage Overview

Languages: Python

Python / code-coverage/pytest-no-creds

The overall line coverage in commit d6a598d in the devin/1791758951-mcp... branch is 79%. The line coverage in commit d9f652f in the release-candidate/v1 branch is 65%.

Show a line coverage summary of the most impacted files.

File	`release-candidate/v1` d9f652f	`devin/1791758951-mcp...` d6a598d	+/-
`airbyte/mcp/cloud.py`	52%	83%	+31%
`airbyte/_util/api_util.py`	36%	68%	+32%
`airbyte/cloud/connectors.py`	34%	75%	+41%
`airbyte/mcp/_datadog.py`	0%	50%	+50%
`airbyte/mcp/_error_handling.py`	0%	89%	+89%
`airbyte/_direct...ctors/models.py`	0%	94%	+94%
`airbyte/mcp/_telemetry.py`	0%	94%	+94%
`airbyte/mcp/_sso_auth.py`	0%	96%	+96%
`airbyte/cloud/models.py`	0%	97%	+97%
`airbyte/mcp/int...nc_status_ui.py`	0%	97%	+97%

Python / code-coverage/pytest

The overall line coverage in commit d6a598d in the devin/1791758951-mcp... branch is 81%. The line coverage in commit d9f652f in the release-candidate/v1 branch is 71%.

Show a line coverage summary of the most impacted files.

File	`release-candidate/v1` d9f652f	`devin/1791758951-mcp...` d6a598d	+/-
`airbyte/mcp/cloud.py`	52%	83%	+31%
`airbyte/mcp/_datadog.py`	0%	50%	+50%
`airbyte/mcp/_error_handling.py`	0%	89%	+89%
`airbyte/mcp/int..._registry_ui.py`	0%	92%	+92%
`airbyte/mcp/_user_identity.py`	0%	93%	+93%
`airbyte/_direct...ctors/models.py`	0%	94%	+94%
`airbyte/mcp/_telemetry.py`	0%	94%	+94%
`airbyte/mcp/_sso_auth.py`	0%	96%	+96%
`airbyte/cloud/models.py`	0%	97%	+97%
`airbyte/mcp/int...nc_status_ui.py`	0%	97%	+97%

Python / code-coverage/pytest-unit

The overall line coverage in commit d6a598d in the devin/1791758951-mcp... branch is 76%. Line coverage data for the release-candidate/v1 branch is not yet available.

Show a line coverage summary of the most covered files.

File	`release-candidate/v1`	`devin/1791758951-mcp...` d6a598d	+/-
`airbyte/cloud/models.py`	—	97%	—
`airbyte/mcp/_sso_auth.py`	—	96%	—
`airbyte/_direct...ctors/models.py`	—	94%	—
`airbyte/cloud/client.py`	—	90%	—
`airbyte/exceptions.py`	—	89%	—
`airbyte/mcp/cloud.py`	—	83%	—
`airbyte/progress.py`	—	80%	—
`airbyte/cloud/connectors.py`	—	75%	—
`airbyte/_util/api_util.py`	—	68%	—
`airbyte/shared/...ql_processor.py`	—	68%	—

_{Updated October 12, 2026 19:41 UTC}

Caleb Evans (calebevans-ab) · 2026-10-12T18:55:51Z

Codex (@codex) review

chatgpt-codex-connector · 2026-10-12T18:56:08Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-12T19:23:22.413284Z	`e36f997`	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "Codex (@codex) review" or "Codex (@codex) security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

chatgpt-codex-connector · 2026-10-12T18:59:15Z

Codex Review: Didn't find any major issues. 👍

Reviewed commit: e36f997f7e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e36f997f7e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".

Aaron ("AJ") Steers (aaronsteers)

Approved. Great work integrating with the existing exception class declarations. This is a nice reuse of those existing date structures.

…r-trace-messages

devin-ai-integration Bot and others added 2 commits October 11, 2026 22:57

feat(mcp): give non-Cloud tool errors fixed Datadog span messages fro…

e67fd98

…m a manifest

test(mcp): drop fixed entry count from tool error manifest test

6e6892f

devin-ai-integration Bot assigned Caleb Evans (calebevans-ab) Oct 11, 2026

refactor(mcp): derive non-Cloud trace messages from exception docstrings

e36f997

devin-ai-integration Bot changed the title ~~feat(mcp): give non-Cloud tool errors fixed Datadog span messages from a manifest~~ feat(mcp): give non-Cloud tool errors fixed Datadog span messages from exception docstrings Oct 12, 2026

Caleb Evans (calebevans-ab) marked this pull request as ready for review October 12, 2026 19:17

Caleb Evans (calebevans-ab) requested review from Aaron ("AJ") Steers (aaronsteers) and Alexandre Girard (girarda) October 12, 2026 19:17

chatgpt-codex-connector Bot reviewed Oct 12, 2026

View reviewed changes

Comment thread airbyte/mcp/_error_handling.py

Comment thread tests/unit_tests/test_mcp_datadog.py

Aaron ("AJ") Steers (aaronsteers) approved these changes Oct 12, 2026

View reviewed changes

Merge origin/release-candidate/v1 into devin/1791758951-mcp-tool-erro…

d6a598d

…r-trace-messages

Caleb Evans (calebevans-ab) enabled auto-merge (squash) October 12, 2026 19:38

Caleb Evans (calebevans-ab) merged commit bce0b0e into release-candidate/v1 Oct 12, 2026
30 checks passed

Caleb Evans (calebevans-ab) deleted the devin/1791758951-mcp-tool-error-trace-messages branch October 12, 2026 20:00

Navigation Menu

Repository navigation

feat(mcp): give non-Cloud tool errors fixed Datadog span messages from exception docstrings - #1358

feat(mcp): give non-Cloud tool errors fixed Datadog span messages from exception docstrings#1358
Caleb Evans (calebevans-ab) merged 4 commits into
release-candidate/v1from
devin/1791758951-mcp-tool-error-trace-messages

Caleb Evans (calebevans-ab) commented Oct 11, 2026 •

edited by github-actions Bot

Loading

Uh oh!

devin-ai-integration Bot commented Oct 11, 2026

Uh oh!

github-actions Bot commented Oct 11, 2026

Testing This PyAirbyte Version

PR Slash Commands

Helpful Resources

Community Support

Uh oh!

github-code-quality Bot commented Oct 11, 2026 •

edited

Loading

Uh oh!

Caleb Evans (calebevans-ab) commented Oct 12, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 12, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot commented Oct 12, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Aaron ("AJ") Steers (aaronsteers) left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Caleb Evans (calebevans-ab) commented Oct 11, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

devin-ai-integration Bot commented Oct 11, 2026

Uh oh!

github-actions Bot commented Oct 11, 2026

👋 Greetings, Airbyte Team Member!

Testing This PyAirbyte Version

PR Slash Commands

Helpful Resources

Community Support

Uh oh!

github-code-quality Bot commented Oct 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage Overview

Python / code-coverage/pytest-no-creds

Python / code-coverage/pytest

Python / code-coverage/pytest-unit

Uh oh!

Caleb Evans (calebevans-ab) commented Oct 12, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

chatgpt-codex-connector Bot commented Oct 12, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Aaron ("AJ") Steers (aaronsteers) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Caleb Evans (calebevans-ab) commented Oct 11, 2026 •

edited by github-actions Bot

Loading

github-code-quality Bot commented Oct 11, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Oct 12, 2026 •

edited

Loading