Skip to content

Add custom provider model selection and capability options - #112

Open
sambitcreate wants to merge 4 commits into
mainfrom
feature/custom-model-options
Open

Add custom provider model selection and capability options#112
sambitcreate wants to merge 4 commits into
mainfrom
feature/custom-model-options

Conversation

@sambitcreate

@sambitcreate sambitcreate commented Sep 12, 2026

Copy link
Copy Markdown
Owner

Custom and Tailscale endpoints can now expose More options for model visibility, manual model IDs, and per-model capabilities and limits. Users can override vision, reasoning, tool calling, open weights, server video support, context length, output tokens, and images per message, or reset to detected capabilities.

User-authored model settings persist in portable configuration separately from discovery metadata. Explicit rediscovery preserves overrides and manual IDs; manual setup also works without a discovery endpoint. Runtime limits, tool availability, and native image capabilities respect the overrides. Video records server support; Aiden does not add video uploads.

The existing Git push-cancellation test now waits for an explicit completion marker and drains cancellation before fixture cleanup, fixing a timing race exposed by CI. Its upstream-safety assertions are unchanged.

Validation:

  • Three independent sub-agent reviews completed; all reported findings fixed and re-reviewed.
  • Pullfrog findings fixed with regressions for assistant image history, Bot/Telegram overrides, preserved manual IDs, numeric reset, actual harness tool removal, and deferred browser discovery.
  • TypeScript and E2E type checks, production build, focused service/storage tests, onboarding tests, and Electron save/reopen/rediscovery/reset/manual-entry flow passed.
  • Android AidenChatTest passed; generic iOS hardware build-for-testing compiled the app and test bundle. Physical-device XCTest acceptance was not performed.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

The override plumbing is sound on the desktop runtime path, but three gaps should be addressed before merge: the per-message image limit counts assistant-displayed images (a chat can become permanently unable to generate), the Bot catalog and Telegram picker ignore the new overrides, and the stale manual-add path drops previously configured manual models and overrides.

Reviewed changes

  • Portable model intent: customModelOptions is split out of modelMetadata on save and re-merged on read, with explicit-reset semantics in saveProvider and validation in isPortableProvider.
  • Runtime enforcement: resolveProviderRuntimeLimits applies context/output/reasoning/vision overrides (maxImages: 0 forces text-only), and generation enforces the image limit and removes tools when toolCall === false.
  • Display and native projection: withProviderFallback overlays overrides and exposes detectedCapabilities; AidenRemoteModelService now honors overrides for supportsImages.
  • Settings UI: new CustomModelOptionsEditor under a "More options" disclosure with manual model IDs, per-model switches, numeric limits, and reset; model visibility moved into the same disclosure.
  • Tests: focused custom-model-options.test.ts, config-store restart/reset test, Android/iOS decode tests, and an Electron e2e spec covering save/rediscovery/reset/manual entry.

ℹ️ Nitpicks

  • The new toolCall === false runtime gate (llm-client.ts:2068-2071) has no test; only its persistence is covered. A focused test that tools are actually removed would pin the behavior.

Pullfrog  | Fix all ➔Fix 👍s ➔View workflow run | Using DeepSeek Flash𝕏

Comment thread main/services/llm-client.ts
Comment thread main/services/aiden-remote-models.ts
Comment thread renderer/components/settings/provider-editor.tsx
Comment thread renderer/components/settings/custom-model-options.tsx

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Reviewed 96958601 since the prior pullfrog review at 7351057e — the four inline concerns are addressed and pinned by new regressions.

  • Role-aware image limit: The limit now counts only user-authored attachments, so assistant-displayed images can no longer block generation.
  • Bot and Telegram projection: Bot inventory applies overrides.vision ?? vision with maxImages !== 0, and Telegram model choices route through the new override-aware telegramModelChoice.
  • Numeric override clearing: Clearing a numeric field deletes the key instead of storing explicit undefined, restoring the detected placeholder before save.
  • Stale manual-add: Adding a manual ID while discovery is stale now retains prior manual IDs and their overrides.
  • Tool policy extraction: applyCustomModelToolPolicy/prepareCustomModelToolContext centralize the toolCall: false gate, with a harness test showing base and extension tools are removed from a frozen snapshot without mutating it.
  • Regression coverage: Added focused tests for assistant images, Telegram reasoning overrides, Bot image flags, deferred browser discovery, and the Electron save/rediscovery/reset/manual flows.

Verified with npm run type-check, the focused custom-model-options, telegram-controls, and bot-capability-inventory-ports suites, and the new harness test — all pass.

Pullfrog  | View workflow run | Using DeepSeek Flash𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

Reviewed 2c9e90b1 since the prior pullfrog review at 9695860 — a test-only follow-up that removes the CI race in the Git push cancellation fixture.

  • Deterministic cancellation handshake: the wrapper now writes its completion marker as soon as git push exits successfully and stays alive, so the test aborts into the cancellation-reconciliation path instead of racing a two-second process-exit window.
  • Bounded wait and cleanup drain: waitForFile(marker, 15_000) replaces the fixed 150-attempt poll, and a finally block aborts and awaits the operation so fixture cleanup never races a live push; pushTimeoutMs rises to 30_000 as a backstop above the marker wait.
  • Papercuts note: records the pre-existing fixture race for future reference.

Verified locally by running the updated test four times and the full push-related subset of git.test.ts (19/19 pass). The head-SHA E2E, Android, and verify checks were still in progress at review time.

Pullfrog  | View workflow run | Using DeepSeek Flash𝕏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant