Skip to content

docs(mcp): clarify that get_agent_skill_docs is for a deployed connector's direct-access docs - #1389

Merged
Aaron ("AJ") Steers (aaronsteers) merged 1 commit into
release-candidate/v1from
devin/1792092702-skill-docs-description
Oct 16, 2026
Merged

Aaron ("AJ") Steers (aaronsteers) merged 1 commit into
release-candidate/v1from
devin/1792092702-skill-docs-description

Conversation

@girarda

Copy link
Copy Markdown
Contributor

Overview

👉 TL;DR: Rewords the tool description so agents stop calling the connector-docs tool for general Airbyte questions or with the wrong kind of ID.

Specifically, this rewrites the get_agent_skill_docs docstring and connector_id description, and fixes the "no section" wording in SKILL_DOCS_SECTION_HINT (shared with execute_external_api_query).

Summary

Requested by Alexandre Girard (@girarda) after triaging production get_agent_skill_docs errors (BigQuery mcp_events + Datadog service:cloud-mcp). Two of the external failures came from caller confusion rather than server bugs:

  • An agent called it with no arguments with the intent "Load Airbyte agent guidance about setting up a connection" → Provide exactly one of docs_skill_id or connector_id. The old hint said "Call get_agent_skill_docs with no section input to list all available sections", which reads like the tool is a general guidance index.
  • An agent passed the source-google-sheets definition ID as connector_id → 403 on both /v1/sources/{id} and /v1/destinations/{id}.

Changes

  • get_agent_skill_docs docstring: describes it as a deployed connector's direct-access docs (entity types/actions/args for execute_external_api_query), says it's not general Airbyte guidance and points to search_airbyte_knowledge_sources, and states connector_id must be a deployed workspace connector ID (e.g. from list_cloud_connectors), not a registry definition ID.
  • connector_id field description: adds "(not a connector definition ID)".
  • SKILL_DOCS_SECTION_HINT: "Call get_agent_skill_docs with the deployed connector_id (or a docs_skill_id) and no section to get the overview and list of available sections, then retrieve the relevant section(s)…".

Text-only; no behavior change.

Follow-ups

Not in this PR (discussed, not requested): returning the skill index when called with no args, detecting registry definition IDs on 403, and returning a result instead of raising when direct access isn't enabled.

Test plan

  • ruff format / ruff check and pyrefly check on the two files: pass.
  • pytest tests/unit_tests -k "guidance or skill or mcp": 1078 passed.
  • Listed tools via an in-memory fastmcp.Client and confirmed the new get_agent_skill_docs and execute_external_api_query descriptions are what MCP clients receive.

Link to Devin session: https://app.devin.ai/sessions/8fc02d415e774a5cbf1d9c07c0ddb97b
Open in Devin Desktop: https://app.devin.ai/desktop/session/8fc02d415e774a5cbf1d9c07c0ddb97b?variant=devin

…tor's direct-access docs

Co-Authored-By: alexandre@airbyte.io <alexandre@airbyte.io>
@devin-ai-integration

Copy link
Copy Markdown
Contributor

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@octavia-bot

octavia-bot Bot commented Oct 15, 2026

Copy link
Copy Markdown

Note

📝 PR Converted to Draft

More info...

Thank you for creating this PR. As a policy to protect our engineers' time, Airbyte requires all PRs to be created first in draft status. Your PR has been automatically converted to draft status in respect for this policy.

As soon as your PR is ready for formal review, you can proceed to convert the PR to "ready for review" status by clicking the "Ready for review" button at the bottom of the PR page.

To skip draft status in future PRs, please include [ready] in your PR title or add the skip-draft-status label when creating your PR.

@octavia-bot
octavia-bot Bot marked this pull request as draft October 15, 2026 19:38
@github-actions

Copy link
Copy Markdown

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This PyAirbyte Version

You can test this version of PyAirbyte using the following:

# Run PyAirbyte CLI from this branch:
uvx --from 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1792092702-skill-docs-description' pyairbyte --help

# Install PyAirbyte from this branch for development:
pip install 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1792092702-skill-docs-description'

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /fix-pr - Fixes most formatting and linting issues
  • /uv-lock - Updates uv.lock file
  • /test-pr - Runs tests with the updated PyAirbyte
  • /prerelease - Builds and publishes a prerelease version to PyPI
📚 Show Repo Guidance

Helpful Resources

Community Support

Questions? Join the #pyairbyte channel in our Slack workspace.

📝 Edit this welcome message.

@github-code-quality

github-code-quality Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Code coverage overview

Languages: Python

Python / code-coverage/pytest-no-creds

release-candidate/v1 d9f652f devin/1792092702-ski... 9711b38 +/-
65% 79% +14%
View coverage of the 10 most impacted files.
File release-candidate/v1 d9f652f devin/1792092702-ski... 9711b38 +/-
airbyte/mcp/cloud.py 52% 84% +32%
airbyte/_util/api_util.py 36% 68% +32%
airbyte/cloud/connectors.py 34% 76% +42%
airbyte/mcp/_datadog.py 0% 50% +50%
airbyte/mcp/_user_identity.py 0% 93% +93%
airbyte/mcp/_telemetry.py 0% 94% +94%
airbyte/_direct...ctors/models.py 0% 95% +95%
airbyte/mcp/_sso_auth.py 0% 96% +96%
airbyte/cloud/models.py 0% 97% +97%
airbyte/mcp/int...nc_status_ui.py 0% 97% +97%

Python / code-coverage/pytest

release-candidate/v1 d9f652f devin/1792092702-ski... 9711b38 +/-
71% 81% +10%
View coverage of the 10 most impacted files.
File release-candidate/v1 d9f652f devin/1792092702-ski... 9711b38 +/-
airbyte/cloud/connectors.py 51% 80% +29%
airbyte/mcp/cloud.py 52% 84% +32%
airbyte/mcp/_datadog.py 0% 50% +50%
airbyte/mcp/int..._registry_ui.py 0% 92% +92%
airbyte/mcp/_user_identity.py 0% 93% +93%
airbyte/mcp/_telemetry.py 0% 94% +94%
airbyte/_direct...ctors/models.py 0% 95% +95%
airbyte/mcp/_sso_auth.py 0% 96% +96%
airbyte/cloud/models.py 0% 97% +97%
airbyte/mcp/int...nc_status_ui.py 0% 97% +97%

Python / code-coverage/pytest-unit

release-candidate/v1 devin/1792092702-ski... 9711b38 +/-
— 76% —

Line coverage data for the release-candidate/v1 branch is not yet available.

View coverage of the 10 most covered files.
File release-candidate/v1 devin/1792092702-ski... 9711b38 +/-
airbyte/cloud/models.py — 97% —
airbyte/mcp/_sso_auth.py — 96% —
airbyte/_direct...ctors/models.py — 95% —
airbyte/cloud/client.py — 90% —
airbyte/exceptions.py — 89% —
airbyte/mcp/cloud.py — 84% —
airbyte/progress.py — 80% —
airbyte/cloud/connectors.py — 76% —
airbyte/_util/api_util.py — 68% —
airbyte/shared/...ql_processor.py — 68% —

ℹ️ Calculations represent line coverage. Results may change as calculations are updated.


Updated October 15, 2026 19:50 UTC

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Eval evidence. I added two regression scenarios to cloud-mcp-evals, based on the rc.28 public session where both failures happened. I ran them in the local sandbox on gpt-6.1-sol, 5 repeats each, comparing the RC base baaf0be with this PR at 9711b38.

Scenario base this PR
General "agent guidance on setting up a connection" question; the goal is no get_agent_skill_docs call 0/5 5/5
Registry definition ID presented as "our catalog's connector ID"; the goal is no get_agent_skill_docs error 0/5 4/5
  • General question, base: every run called get_agent_skill_docs(docs_skill_id="airbyte") and got an error.
  • General question, PR: no run called the tool.
  • Definition ID, base: every run passed the definition ID as connector_id and got an error.
  • Definition ID, PR: in 4 runs the agent checked list_cloud_connectors first and explained that it was a catalog ID. The failing run passed the definition ID anyway, so this PR reduces that mistake but doesn't eliminate it.

What's simulated:

  • The prompts paraphrase intents recorded in telemetry; the real prompt isn't recorded.
  • search_airbyte_knowledge_sources isn't available in the sandbox.
  • One model only.
  • The Cloud API is the saas-sandbox twin, not real Cloud.

Written by Devin

@aaronsteers
Aaron ("AJ") Steers (aaronsteers) marked this pull request as ready for review October 16, 2026 02:28
Copilot AI balanced review requested due to automatic review settings October 16, 2026 02:28
@aaronsteers
Aaron ("AJ") Steers (aaronsteers) merged commit 0360edc into release-candidate/v1 Oct 16, 2026
33 checks passed
@aaronsteers
Aaron ("AJ") Steers (aaronsteers) deleted the devin/1792092702-skill-docs-description branch October 16, 2026 02:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The documentation-only changes align with the existing interfaces, with no unresolved issues.

0 open findings

What changed in this PR

Clarifies MCP tool guidance so agents request connector-specific direct-access docs with the correct identifier. No runtime behavior changes.

Changes:

  • Distinguishes deployed connector IDs from definition IDs and redirects general Airbyte questions.
  • Clarifies the required identifier when requesting the documentation overview without a section.
File Description
airbyte/​mcp/​guidance.py Clarifies tool purpose and connector ID requirements.
airbyte/​mcp/​cloud.py Corrects shared instructions for requesting documentation sections.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants