Skip to content

feat: add NVIDIA NIM as an LLM provider - #2923

Draft
Jon Patterson (freshyjmp) wants to merge 1 commit into
browseros-ai:mainfrom
freshyjmp:nim-oss/browseros-job-28986
Draft

Jon Patterson (freshyjmp) wants to merge 1 commit into
browseros-ai:mainfrom
freshyjmp:nim-oss/browseros-job-28986

Conversation

@freshyjmp

Copy link
Copy Markdown

What this changes

Adds NVIDIA NIM as a built-in LLM provider (nvidia). Users can pick an NVIDIA NIM card in AI settings, enter an API key, and run the BrowserOS agent on NVIDIA-hosted models through NIM's OpenAI-compatible API. The base URL is pre-filled with https://integrate.api.nvidia.com/v1 and can be changed to point at a self-hosted NIM.

  • packages/browseros-agent/packages/shared/src/schemas/llm.ts: adds LLM_PROVIDERS.NVIDIA and adds it to LLMProviderSchema.
  • packages/browseros-agent/apps/server/src/agent/provider-factory.ts, packages/browseros-agent/apps/server/src/lib/clients/llm/provider.ts: add nvidia factories built on createOpenAICompatible, following the Moonshot provider (base URL and API key are required, custom headers are forwarded).
  • packages/browseros-agent/apps/app/lib/llm-providers/{types.ts,providerTemplates.ts,providerIcons.tsx}, packages/browseros-agent/apps/app/screens/ai-settings/{add-provider.helpers.ts,provider-form-schema.ts}: add the NVIDIA NIM template (default model nvidia/nemotron-3-super-120b-a12b, text only), its icon, and form support.
  • packages/browseros-agent/apps/server/tests/agent/provider-factory-nvidia.test.ts: new tests for the factory (base URL, API key, missing credentials, headers).
  • docs/browseros/features/bring-your-own-llm.mdx: adds an NVIDIA NIM section.

No dependencies were added.

Why

NIM serves NVIDIA-hosted models over an OpenAI-compatible API. Before this change, users had to set it up by hand as a generic OpenAI-compatible provider. A dedicated provider gives them a ready-made template with the correct endpoint and a default model that supports tool calls, which the agent loop needs.

How to check it

Path: BrowserOS (side panel chat / agent loop).

  1. Go to chrome://browseros/settings.
  2. Click USE on the NVIDIA NIM card.
  3. Enter an NVIDIA API key from build.nvidia.com. The model is pre-set to nvidia/nemotron-3-super-120b-a12b and the base URL to https://integrate.api.nvidia.com/v1.
  4. Set Context Window, click Save, then chat with the agent and run a task that uses tools.

Checks run:

  • cd packages/browseros-agent/apps/server && bun --env-file=../../.env.development test ./tests/agent/provider-factory-nvidia.test.ts: 8 pass, 0 fail
  • cd packages/browseros-agent/apps/server && bun --env-file=../../.env.development test ./tests/agent/provider-factory-baseurl.test.ts: 18 pass, 0 fail
  • cd packages/browseros-agent/apps/app && bun run ../../scripts/run-bun-test.ts --cwd=apps/app ./apps/app/lib/llm-providers/providerTemplates.test.ts ./apps/app/screens/ai-settings/add-provider.helpers.test.ts: pass
  • bunx @biomejs/biome check on the changed TypeScript files: no issues
  • bun run typecheck in apps/server and apps/app: pass
  • git diff --check -- docs/browseros/features/bring-your-own-llm.mdx: clean
  • Live check against https://integrate.api.nvidia.com/v1 with nvidia/nemotron-3-super-120b-a12b, using the server's createLLMProvider: a chat reply and a tool call both succeeded

Before you request review

  • Ran the checks for the path you touched (see CONTRIBUTING.md)
  • Added or updated tests where the behaviour changed
  • Checked this on the platforms it affects

Validation

Independent validation: passed on a clean checkout of b768a3f68ba9 with this patch applied.

  • tool-setup-1: passed
  • setup-install: passed
  • setup-env: passed
  • test-nvidia-factory: passed
  • test-peer-factory: passed
  • test-app-providers: passed
  • docs-diff-check: passed
  • lint: passed
  • typecheck-server: passed
  • typecheck-app: passed
  • nim-smoke: passed
  • nim-smoke-result: passed

@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Before we can merge this PR, we need you to sign our Contributor License Agreement.

To sign the CLA, please add a comment to this PR with the following text:

I have read the CLA Document and I hereby sign the CLA

You only need to sign once. After signing, this check will pass automatically.


Troubleshooting
  • Already signed but still failing? Comment recheck to trigger a re-verification.
  • Signed with a different email? Make sure your commit email matches your GitHub account email, or add your commit email to your GitHub account.
- - - I have read the CLA Document and I hereby sign the CLA - - - **NIM OSS Manager** seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please [add the email address used for this commit to your account](https://help.github.com/articles/why-are-my-commits-linked-to-the-wrong-user/#commits-are-not-linked-to-any-user).
You can retrigger this bot by commenting **recheck** in this Pull Request. Posted by the **CLA Assistant Lite bot**.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant