Repository navigation
ci(hydra): add error_classification_change and error_handler_fail_or_retry automerge scopes - #79660
Open
Pedro S. Lopez (pedroslopez) wants to merge 2 commits into
Open
Pedro S. Lopez (pedroslopez) wants to merge 2 commits into
Pedro S. Lopez (pedroslopez) wants to merge 2 commits into
Conversation
… automerge scopes Add two new change scopes to the Hydra auto-merge evaluator: - error_classification_change: PRs that only annotate existing response filters with failure_type/error_message fields (zero behavior change) - error_handler_addition: PRs that add new response filters with safe actions only (FAIL, RETRY, RATE_LIMITED). Explicitly blocks IGNORE and SUCCESS actions to prevent silent data loss. Also adds a shared 'accompanying files' rule to the change_scope description so version bumps, changelog entries, and test files don't need repeating per scope definition. Co-Authored-By: Pedro Lopez <pedroslopez@me.com>
Contributor
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Contributor
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. 💡 Show Tips and TricksPR Slash CommandsAirbyte Maintainers (that's you!) can execute the following slash commands on your PR:
📚 Show Repo GuidanceHelpful Resources
|
Makes the scope name self-documenting: only FAIL/RETRY/RATE_LIMITED actions are in scope, IGNORE is explicitly excluded. Co-Authored-By: Pedro Lopez <pedroslopez@me.com>
devin-ai-integration
Bot
changed the title
Pedro S. Lopez (pedroslopez)
marked this pull request as ready for review
June 10, 2026 22:46
NexusMBD
approved these changes
Jun 13, 2026
NexusMBD
left a comment
NexusMBD
left a comment
There was a problem hiding this comment.
✅ Approved
Clear bug fix / improvement. CI all passed.
Safe to merge.
— Code Reviewer (L - Code Reviewer)
Contributor
|
❌ Cannot revive Devin session - the session is too old. Please start a new session instead. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds two new change scopes to the Hydra auto-merge evaluator, derived from analysis of Phase 1 merged results. These scopes cover the most common pattern in successful Hydra PRs (5/7 merged PRs) while explicitly blocking the dangerous
action: IGNOREpattern that can cause silent data loss (e.g. #79171).Requested by Pedro S. Lopez (@pedroslopez).
How
Three changes to
.github/workflows/hydra-automerge.yml:Schema — Added two new scope definitions to the structured-output-schema:
error_classification_change: PRs that ONLY addfailure_type/error_messageannotations to existing response filters. Zero behavior change.error_handler_fail_or_retry: PRs that add new response filters but ONLY with safe actions (FAIL,RETRY,RATE_LIMITED). Explicitly blocksIGNOREandSUCCESS— the name makes this self-documenting.Evaluate step — Added both new scopes to the ANY-must-pass gate logic.
Report builder — Added both new keys to the scope display loop.
Also added a shared "accompanying files" rule to the
change_scopedescription preamble so version bumps, changelog entries, and test files don't need repeating in each scope definition.Evaluation Evidence
Ran the exact prompt + structured output schema against 11 real PRs using child Devin sessions. All 11 classified correctly (100% match rate).
Phase 1 merged PRs (expected: new scopes PASS for 5, FAIL for 2)
error_classification_changeerror_handler_fail_or_retryfailure_typeto 6 existing filtersfailure_typeto 1 existing filteraction: IGNORE(correctly blocked)Additional test PRs (expected: correctly classified)
error_classification_changeerror_handler_fail_or_retrydependency_version_bump= PASSaction: IGNOREcorrectly blockedKey safety validations
action: IGNOREto skip pagination-limit records →error_handler_fail_or_retrycorrectly returns FAIL with reasoning: "The PR adds a new HttpResponseFilter with action: IGNORE. The scope requires that every new filter uses ONLY action FAIL, action RETRY, or action RATE_LIMITED."action: IGNOREfor 404 deleted subscriptions → both new scopes correctly FAILChild session links
Evaluation sessions (Phase 1 batch)
Evaluation sessions (additional batch)
Review guide
.github/workflows/hydra-automerge.yml— single file, three logical sections modifiedUser Impact
Hydra PRs that modify only declarative manifest error handling (the most common Hydra fix pattern) will now be eligible for auto-merge, provided they don't use
action: IGNOREoraction: SUCCESS.Expected impact from Phase 1 data:
Can this PR be safely reverted and rolled back?
Link to Devin session: https://app.devin.ai/sessions/5c2831dd9e3f41d0a52011656442e285