Skip to content

Honor DEVIN_COMPANION_DEFAULT_MODEL and explain Pro-only model failures - #19

Open
hessabi wants to merge 1 commit into
TheOnlyFusionCube:mainfrom
hessabi:fix/default-model-tier-v2
Open

hessabi wants to merge 1 commit into
TheOnlyFusionCube:mainfrom
hessabi:fix/default-model-tier-v2

Conversation

@hessabi

@hessabi hessabi commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

Re-file of the former #6, as requested in #4, rebased onto current main.

The default model is pinned to swe-2-max, which Devin Free accounts can't use. Every run fails with only Error: Upgrade to Pro to access this model (https://devin.ai/pricing) and no guidance.

Changes

  • DEFAULT_MODEL honors DEVIN_COMPANION_DEFAULT_MODEL. The pinned default is unchanged, and the override flows into the job's recorded model, so status shows it.
  • When a run fails with "Upgrade to Pro" and --model wasn't passed, the output gets one hint line: pass --model <id> (see devin models list) or set the env var. It renders next to the sandbox hint through renderDevinOutput, so sandbox output is unchanged.
  • No retry, and exit codes are untouched, so this stays out of the exit-code discussion.
  • One README line documents the env var.

Tests: 3 new tests, reusing the fixture's stderrText option:

  • the hint shows without --model;
  • it's absent with an explicit --model;
  • the env var changes the --model passed and the model recorded in status.

Full suite: 112/112.

Note: this PR and the welcome-screen re-file both append tests to the end of tests/companion.test.mjs. Whichever merges second needs a trivial rebase that keeps both blocks; I'm happy to do it.

The pinned default swe-2-max is not available on Devin Free, so every
run failed with only "Upgrade to Pro to access this model".

DEFAULT_MODEL now honors DEVIN_COMPANION_DEFAULT_MODEL (pinned default
unchanged). When a run fails with that error and --model was not passed,
the output gets a hint to pass --model or set the env var, rendered next
to the sandbox hint via renderDevinOutput. No retry, exit codes unchanged.

@TheOnlyFusionCube TheOnlyFusionCube left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed against main and issue #4. The override flows into buildDevinPrintArgs and persisted job.model; explicit --model takes precedence, and the three tests exercise these paths. Existing ordinary-failure exit policy is unchanged. One error-reporting issue to address before approval: renderDevinOutput uses stdout as the base when nonempty, and only appends devinFailureMessage(result) when a hint exists. With an explicit --model, modelHint is suppressed, so a failed run with stdout and the Pro-access error only on stderr hides the actual error. Please ensure failed runs with nonempty stdout still show stderr in the rendered result even when no hint is emitted; add a regression test for this case. Also consider matching a more specific model-entitlement error rather than any upgrade to pro text. HOL logs show the same one HIGH hardcoded-secret finding present before this PR; #17 fixes it and has now been merged. Please refresh this branch and rerun both required workflows before merge.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants