fix(cloud): raise `AirbyteCloudError` on malformed skill docs responses by aaronsteers · Pull Request #1367 · airbytehq/PyAirbyte

Aaron ("AJ") Steers (aaronsteers) · 2026-10-13T01:32:55Z

Overview

👉 TL;DR: A badly shaped connector-docs response from Airbyte Cloud now raises the same Cloud error type as other Cloud failures, so describing a connector reports a warning instead of failing outright. Specifically, `read_cloud_skill_docs` now validates the response as `DirectAccessGuidance` and wraps a pydantic `ValidationError` in `AirbyteCloudError("Malformed Airbyte Cloud skill docs response.")`, matching how `get_cloud_connector_enablement` handles a malformed enablement response.

Replaces the remaining malformed-docs part of #1240.

Changes

airbyte/_direct_connectors/api_util.py::read_cloud_skill_docs: returns DirectAccessGuidance instead of dict[str, Any]; a validation failure raises AirbyteCloudError with path, skill_id, and the pydantic errors in context.
airbyte/cloud/connectors.py (_context_layer_inspect, read_agent_skill_docs, get_direct_access_guidance) and airbyte/cloud/workspaces.py::get_agent_skill_docs: drop the now-redundant DirectAccessGuidance.model_validate(...) wrappers.
Tests: new test_read_cloud_skill_docs_rejects_malformed_response; existing fakes of read_cloud_skill_docs in test_cloud_describe.py / test_cloud_guidance.py return DirectAccessGuidance to match the new contract.

Before, a malformed docs payload raised a raw ValidationError from the call sites above, which _describe_cloud_connector does not catch (it catches AirbyteCloudError / PyAirbyteError / RequestException), so one bad optional payload failed the whole describe call. Now the non-403/404 AirbyteCloudError follows the existing paths: describe records a warning, _context_layer_inspect still re-raises it (not treated as "not enabled").

Review Spotlight

Reviewers with limited time, please review first:

api_util.py#L348-L381: the new validation boundary

Test plan

pytest tests/unit_tests/test_cloud_describe.py test_cloud_credentials.py test_cloud_direct_execute.py test_cloud_guidance.py test_mcp_cloud.py test_cloud_api_util.py -> 660 passed
ruff check, ruff format, pyrefly check -> clean

Link to Devin session: https://app.devin.ai/sessions/ee9a2ac6748746439ad543c3b12bc37c
Open in Devin Desktop: https://app.devin.ai/desktop/session/ee9a2ac6748746439ad543c3b12bc37c?variant=devin
Requested by: Aaron ("AJ") Steers (@aaronsteers)

Co-Authored-By: AJ Steers <aj@airbyte.io>

devin-ai-integration · 2026-10-13T01:32:58Z

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

Disable automatic comment, CI, and merge conflict monitoring

github-actions · 2026-10-13T01:33:34Z

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This PyAirbyte Version

You can test this version of PyAirbyte using the following:

# Run PyAirbyte CLI from this branch:
uvx --from 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1791854828-malformed-skill-docs' pyairbyte --help

# Install PyAirbyte from this branch for development:
pip install 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1791854828-malformed-skill-docs'

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

/fix-pr - Fixes most formatting and linting issues
/uv-lock - Updates uv.lock file
/test-pr - Runs tests with the updated PyAirbyte
/prerelease - Builds and publishes a prerelease version to PyPI

📚 Show Repo Guidance

Helpful Resources

Community Support

Questions? Join the #pyairbyte channel in our Slack workspace.

📝 Edit this welcome message.

Copilot

🟡 Changes recommended

Validation diagnostics can expose malformed Cloud response content through agent-facing warnings.

1 open finding

Prevent malformed response errors from leaking payload contents · New

What changed in this PR

Validates Cloud skill-doc responses at the API boundary so malformed payloads become recoverable Cloud errors.

Changes:

Return typed DirectAccessGuidance objects from read_cloud_skill_docs.
Remove redundant downstream validation.
Update mocks and add malformed-response coverage.

File	Description
`airbyte/_direct_connectors/api_util.py`	Adds response validation and error wrapping.
`airbyte/cloud/connectors.py`	Uses the typed response directly.
`airbyte/cloud/workspaces.py`	Removes redundant validation.
`tests/unit_tests/test_cloud_credentials.py`	Tests malformed response handling.
`tests/unit_tests/test_cloud_describe.py`	Updates guidance mocks to typed models.
`tests/unit_tests/test_cloud_guidance.py`	Updates workspace guidance mocks.

🧠 Review effort: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Co-Authored-By: AJ Steers <aj@airbyte.io>

github-code-quality · 2026-10-13T01:40:14Z

Code Coverage Overview

Languages: Python

Python / code-coverage/pytest-no-creds

The overall line coverage in commit 2157bf3 in the devin/1791854828-mal... branch is 79%. The line coverage in commit d9f652f in the release-candidate/v1 branch is 65%.

Show a line coverage summary of the most impacted files.

File	`release-candidate/v1` d9f652f	`devin/1791854828-mal...` 2157bf3	+/-
`airbyte/mcp/cloud.py`	52%	83%	+31%
`airbyte/_util/api_util.py`	36%	68%	+32%
`airbyte/cloud/connectors.py`	34%	75%	+41%
`airbyte/mcp/_datadog.py`	0%	50%	+50%
`airbyte/mcp/_user_identity.py`	0%	93%	+93%
`airbyte/_direct...ctors/models.py`	0%	94%	+94%
`airbyte/mcp/_telemetry.py`	0%	94%	+94%
`airbyte/mcp/_sso_auth.py`	0%	96%	+96%
`airbyte/cloud/models.py`	0%	97%	+97%
`airbyte/mcp/int...nc_status_ui.py`	0%	97%	+97%

Python / code-coverage/pytest

The overall line coverage in commit 2157bf3 in the devin/1791854828-mal... branch is 81%. The line coverage in commit d9f652f in the release-candidate/v1 branch is 71%.

Show a line coverage summary of the most impacted files.

File	`release-candidate/v1` d9f652f	`devin/1791854828-mal...` 2157bf3	+/-
`airbyte/cloud/connectors.py`	51%	80%	+29%
`airbyte/mcp/cloud.py`	52%	83%	+31%
`airbyte/mcp/_datadog.py`	0%	50%	+50%
`airbyte/mcp/int..._registry_ui.py`	0%	92%	+92%
`airbyte/mcp/_user_identity.py`	0%	93%	+93%
`airbyte/_direct...ctors/models.py`	0%	94%	+94%
`airbyte/mcp/_telemetry.py`	0%	94%	+94%
`airbyte/mcp/_sso_auth.py`	0%	96%	+96%
`airbyte/cloud/models.py`	0%	97%	+97%
`airbyte/mcp/int...nc_status_ui.py`	0%	97%	+97%

Python / code-coverage/pytest-unit

The overall line coverage in commit 2157bf3 in the devin/1791854828-mal... branch is 76%. Line coverage data for the release-candidate/v1 branch is not yet available.

Show a line coverage summary of the most covered files.

File	`release-candidate/v1`	`devin/1791854828-mal...` 2157bf3	+/-
`airbyte/cloud/models.py`	—	97%	—
`airbyte/mcp/_sso_auth.py`	—	96%	—
`airbyte/_direct...ctors/models.py`	—	94%	—
`airbyte/cloud/client.py`	—	90%	—
`airbyte/exceptions.py`	—	89%	—
`airbyte/mcp/cloud.py`	—	83%	—
`airbyte/progress.py`	—	80%	—
`airbyte/cloud/connectors.py`	—	75%	—
`airbyte/_util/api_util.py`	—	68%	—
`airbyte/shared/...ql_processor.py`	—	68%	—

_{Updated October 13, 2026 02:10 UTC}

Aaron ("AJ") Steers (aaronsteers) · 2026-10-13T01:57:08Z

+        raise AirbyteCloudError(
+            message="Malformed Airbyte Cloud skill docs response.",
+            context={
+                "path": path,
+                "skill_id": skill_id,
+                "errors": ex.errors(include_url=False, include_input=False),
+            },
+        ) from ex


No. Too generic. Define a new exception for this.

Also - why does this even exist? Does it help anything?

Added AirbyteMalformedSkillDocsError(AirbyteCloudError) with a skill_id field in 2157bf3.

On "does it help": only in a hypothetical case. If Cloud ever returns a skills/docs body without the required metadata, then on RC today DirectAccessGuidance.model_validate raises a raw pydantic ValidationError, which describe_cloud_connector doesn't catch, so the whole describe fails. With this PR, describe returns a warning instead. We haven't seen Cloud return a payload like that; the only source for the scenario is #1240. The cost: it turns a server-contract break into a soft warning, and it adds a new public exception. No further changes until you decide.

Written by Devin

…cs responses Co-Authored-By: AJ Steers <aj@airbyte.io>

Aaron ("AJ") Steers (aaronsteers) · 2026-10-13T02:11:54Z

Closing. Won't do, pending repro or in-the-wild reports.

We haven't seen Cloud return a malformed skills/docs payload, to my knowledge, and if it ever does, we'd first inspect as a server contract bug that should fail loudly instead of turning into a soft warning.

Declining adding a new public exception for what appears as of now as only a hypothetical case.

Note: I couldn't get this PR to pass the "foot gun" test - which is to ask: is this more likely to help or hurt? Since chance or helping hurting is 60/40% and I don't know which is more likely, I'm electing to close until I have a report of an issue I can work with.

fix(cloud): raise AirbyteCloudError on malformed skill docs responses

3fe52b7

Co-Authored-By: AJ Steers <aj@airbyte.io>

devin-ai-integration Bot assigned Aaron ("AJ") Steers (aaronsteers) Oct 13, 2026

Airbyte Support Bot (airbyte-support-bot) requested a balanced review from Copilot October 13, 2026 01:33

Copilot started reviewing on behalf of Airbyte Support Bot (airbyte-support-bot) October 13, 2026 01:33

View session

Copilot AI reviewed Oct 13, 2026

View reviewed changes

Comment thread airbyte/_direct_connectors/api_util.py Outdated

fix(cloud): omit response input from skill docs validation errors

f9f163f

Co-Authored-By: AJ Steers <aj@airbyte.io>

Aaron ("AJ") Steers (aaronsteers) mentioned this pull request Oct 13, 2026

fix(cloud): preserve guidance warnings and tolerate malformed docs #1240

Closed

Aaron ("AJ") Steers (aaronsteers) commented Oct 13, 2026

View reviewed changes

fix(cloud): add AirbyteMalformedSkillDocsError for malformed skill do…

2157bf3

…cs responses Co-Authored-By: AJ Steers <aj@airbyte.io>

Aaron ("AJ") Steers (aaronsteers) closed this Oct 13, 2026

Navigation Menu

Repository navigation

fix(cloud): raise `AirbyteCloudError` on malformed skill docs responses - #1367

fix(cloud): raise `AirbyteCloudError` on malformed skill docs responses#1367
Aaron ("AJ") Steers (aaronsteers) wants to merge 3 commits into
release-candidate/v1from
devin/1791854828-malformed-skill-docs

Aaron ("AJ") Steers (aaronsteers) commented Oct 13, 2026

Uh oh!

devin-ai-integration Bot commented Oct 13, 2026

Uh oh!

github-actions Bot commented Oct 13, 2026

Testing This PyAirbyte Version

PR Slash Commands

Helpful Resources

Community Support

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

github-code-quality Bot commented Oct 13, 2026 •

edited

Loading

Uh oh!

Aaron ("AJ") Steers (aaronsteers) Oct 13, 2026

Uh oh!

Aaron ("AJ") Steers (aaronsteers) Oct 13, 2026

Uh oh!

devin-ai-integration Bot Oct 13, 2026

Uh oh!

Aaron ("AJ") Steers (aaronsteers) commented Oct 13, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Aaron ("AJ") Steers (aaronsteers) commented Oct 13, 2026

Overview

Changes

Review Spotlight

Test plan

Uh oh!

devin-ai-integration Bot commented Oct 13, 2026

Uh oh!

github-actions Bot commented Oct 13, 2026

👋 Greetings, Airbyte Team Member!

Testing This PyAirbyte Version

PR Slash Commands

Helpful Resources

Community Support

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

github-code-quality Bot commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage Overview

Python / code-coverage/pytest-no-creds

Python / code-coverage/pytest

Python / code-coverage/pytest-unit

Uh oh!

Aaron ("AJ") Steers (aaronsteers) Oct 13, 2026

Choose a reason for hiding this comment

Uh oh!

Aaron ("AJ") Steers (aaronsteers) Oct 13, 2026

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Oct 13, 2026

Choose a reason for hiding this comment

Uh oh!

Aaron ("AJ") Steers (aaronsteers) commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-code-quality Bot commented Oct 13, 2026 •

edited

Loading

Aaron ("AJ") Steers (aaronsteers) commented Oct 13, 2026 •

edited

Loading