Repository navigation
feat: add NVIDIA NIM as an LLM provider - #2923
Draft
Jon Patterson (freshyjmp) wants to merge 1 commit into
Draft
Jon Patterson (freshyjmp) wants to merge 1 commit into
Jon Patterson (freshyjmp) wants to merge 1 commit into
Conversation
Contributor
|
Thank you for your contribution! Before we can merge this PR, we need you to sign our Contributor License Agreement. To sign the CLA, please add a comment to this PR with the following text: You only need to sign once. After signing, this check will pass automatically. Troubleshooting
You can retrigger this bot by commenting **recheck** in this Pull Request. Posted by the **CLA Assistant Lite bot**. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Adds NVIDIA NIM as a built-in LLM provider (
nvidia). Users can pick an NVIDIA NIM card in AI settings, enter an API key, and run the BrowserOS agent on NVIDIA-hosted models through NIM's OpenAI-compatible API. The base URL is pre-filled withhttps://integrate.api.nvidia.com/v1and can be changed to point at a self-hosted NIM.packages/browseros-agent/packages/shared/src/schemas/llm.ts: addsLLM_PROVIDERS.NVIDIAand adds it toLLMProviderSchema.packages/browseros-agent/apps/server/src/agent/provider-factory.ts,packages/browseros-agent/apps/server/src/lib/clients/llm/provider.ts: addnvidiafactories built oncreateOpenAICompatible, following the Moonshot provider (base URL and API key are required, custom headers are forwarded).packages/browseros-agent/apps/app/lib/llm-providers/{types.ts,providerTemplates.ts,providerIcons.tsx},packages/browseros-agent/apps/app/screens/ai-settings/{add-provider.helpers.ts,provider-form-schema.ts}: add the NVIDIA NIM template (default modelnvidia/nemotron-3-super-120b-a12b, text only), its icon, and form support.packages/browseros-agent/apps/server/tests/agent/provider-factory-nvidia.test.ts: new tests for the factory (base URL, API key, missing credentials, headers).docs/browseros/features/bring-your-own-llm.mdx: adds an NVIDIA NIM section.No dependencies were added.
Why
NIM serves NVIDIA-hosted models over an OpenAI-compatible API. Before this change, users had to set it up by hand as a generic OpenAI-compatible provider. A dedicated provider gives them a ready-made template with the correct endpoint and a default model that supports tool calls, which the agent loop needs.
How to check it
Path: BrowserOS (side panel chat / agent loop).
chrome://browseros/settings.nvidia/nemotron-3-super-120b-a12band the base URL tohttps://integrate.api.nvidia.com/v1.Checks run:
cd packages/browseros-agent/apps/server && bun --env-file=../../.env.development test ./tests/agent/provider-factory-nvidia.test.ts: 8 pass, 0 failcd packages/browseros-agent/apps/server && bun --env-file=../../.env.development test ./tests/agent/provider-factory-baseurl.test.ts: 18 pass, 0 failcd packages/browseros-agent/apps/app && bun run ../../scripts/run-bun-test.ts --cwd=apps/app ./apps/app/lib/llm-providers/providerTemplates.test.ts ./apps/app/screens/ai-settings/add-provider.helpers.test.ts: passbunx @biomejs/biome checkon the changed TypeScript files: no issuesbun run typecheckinapps/serverandapps/app: passgit diff --check -- docs/browseros/features/bring-your-own-llm.mdx: cleanhttps://integrate.api.nvidia.com/v1withnvidia/nemotron-3-super-120b-a12b, using the server'screateLLMProvider: a chat reply and a tool call both succeededBefore you request review
Validation
Independent validation: passed on a clean checkout of
b768a3f68ba9with this patch applied.tool-setup-1: passedsetup-install: passedsetup-env: passedtest-nvidia-factory: passedtest-peer-factory: passedtest-app-providers: passeddocs-diff-check: passedlint: passedtypecheck-server: passedtypecheck-app: passednim-smoke: passednim-smoke-result: passed