[earakely-scale]

Makes a bundle's tasks trainable from NeMo Gym, and so with NeMo RL, the trainer that takes its environments. Gym is also the environment layer for verl, Unsloth and Miles.

What it adds

python -m agent_env.training.nemo_gym SOURCE ROOT. SOURCE is a bundle, or the URL of an OpenEnv server serving one. The command writes the bundle's resources server into a Gym root: the root NEMO_GYM_EXTRA_ROOTS names, or a Gym checkout's own, as NeMo RL's is. Under resources_servers/<name>/ it writes:

• app.py;
• pyproject.toml;
• configs/<name>.yaml, which runs the server, Gym's simple_agent and the legacy_agent environment server NeMo RL dispatches through;
• data/<split>.jsonl, a row per task: the prompt, every tool the task's turns offer, split, index and task_id.

It finds a toolset's tools by playing one of its tasks to the last turn, unscored, as the TRL helper does.

agent_env.training.nemo_gym.server.AgentEnvResourcesServer is that server. It runs in the venv Gym builds for it.

Gym route	What the server does
/seed_session (the row)	Starts the row's episode in the background and returns, so the deploy overlaps the model's first turn. It checks the task id, and that the episode offers only tools the row shows the model.
POST /<tool>	Calls the tool and answers as text: the tool's text or error, or for end_turn the next turn's prompt.
/verify	Ends the turns the model left open, its last reply ending the first and any others empty, and returns the episode's reward. A rollout the environment failed comes back with mask_sample and a failure_reason, so it's left out of training.
/close_session	Ends the episode.

The bundle is served by agent_env.openenv.build_app. The config names an OpenEnv server's url, or the bundle itself, which the server then serves on loopback for its lifetime.

training/_openenv.py. The TRL helper's session code moves here so both helpers share it: Session, the Task API reads, the toolset probe and the in-process Server. trl.py keeps only what's TRL's own: the sync wrappers, its event-loop thread, the environment classes and their pickling, and the exit hook.

Why a pyproject.toml, not a requirements.txt

A Gym checkout's [tool.uv] exclude-dependencies drops packages from every server venv under it, cryptography among them, which agent-env imports. uv applies the [tool.uv] settings of the nearest pyproject.toml, so the server's own puts them back. --override doesn't (uv 0.13.6). Outside a checkout, Gym installs nemo-gym itself and ignores the ../.. source.

Packaging and CI

• A nemo-gym extra: openenv, plus nemo-gym>=0.7.0rc0 on Python 3.13.14 or later, the Python nemo-gym needs. The rc0 floor is the version NeMo RL's Gym checkout reports.
  • It's left out of dev, as smol is, so the 3.12 jobs install nothing new.
  • It adds 68 packages to uv.lock (Ray, MLflow and others), all under that Python marker.
• .github/workflows/nemo-gym.yml runs the new tier tst/nemo_gym on Python 3.13 with that extra, on the same triggers as trl.yml. It allows no skips and isn't a required check.
• make nemo-gym-test runs the tier locally.
• Guard tests: the import-guard tests learn the extra the way they know smol's.

THIRD_PARTY_NOTICES.md is unchanged. It lists the extras its intro names, and smol and vercel aren't there either. Whether to add nemo-gym's tree is the maintainers' call.

Tests

• Unit, Python 3.14 on macOS: tst/unit/training/nemo_gym_test.py covers the server's folder, config and rows, from a URL and from a bundle; one probe per toolset; and a tool named after a Gym route being refused. The TRL and OpenEnv suites pass unchanged on the shared module, as do the import guards.

• tst/nemo_gym, Linux, Python 3.13.14, nemo-gym 0.7.0 from uv.lock:

  • seed, tool, and verify scoring with the last reply;
  • end_turn answering with the next turn's prompt;
  • masking when a deploy fails, the task isn't the row's, or the episode shows the model other tools;
  • a closed session ending its episode;
  • a bundle served for the server's lifetime.

  37 pass, with the training unit tests.

• End to end, on a Linux box with Docker: the GridSim v1 training bundle's first 32 eval days, collected with gym env start and gym eval run. A scripted OpenAI-compatible model submits one fixed schedule per day. Gym's rewards equal the Episode API's for the same schedule on 32 of 32 days, with none masked; they take 12 distinct values between 0 and 0.49. That holds on both setups:

  • nemo-gym 0.7.0 from PyPI, out-of-tree root;
  • the Gym commit NeMo RL main pins (3f6f205, 0.7.0rc0), server written into the checkout.

  32 rollouts take 36 s.

NeMo RL GRPO through this server on GPU follows.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge, with the multi-score startup issue fixed.

Summary

Adds a NeMo Gym resources server for training on a bundle’s tasks.

• A bundle’s tasks can now be loaded into NeMo Gym.
• NeMo Gym rollouts now run bundle episodes and return their scores.
• TRL and NeMo Gym now share the OpenEnv episode session code.
• NeMo Gym stays optional and gets its own test setup.

Diagram

sequenceDiagram
    participant Gym as NeMo Gym
    participant Resources as AgentEnvResourcesServer
    participant OpenEnv as OpenEnv bundle server
    Gym->>Resources: seed_session with task row
    Resources->>OpenEnv: Start episode in background
    Gym->>Resources: Call tool
    Resources->>OpenEnv: Call episode tool
    OpenEnv-->>Gym: Tool text through resources server
    Gym->>Resources: verify with last reply
    Resources->>OpenEnv: End remaining turns
    OpenEnv-->>Resources: Reward and scores
    Resources-->>Gym: Reward, optional components, or masked sample

Loading

_{Reviews (8) · Last reviewed commit: "fix(training): serve a multi-score bundl..." · Reviewed by Greptile}

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	pypi/​nemo-gym@​0.7.0
	pypi/​packaging@​26.3

View full report

[socket-security]

Warning

Review the following alerts detected in dependencies.

According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.

Priority	Alert  (click "▶" to expand/collapse)	Action
	Potential vulnerability: pypi ray with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/ray@2.59.0 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/ray@2.59.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi ray with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/ray@2.59.0 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/ray@2.59.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi ray with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/ray@2.59.0 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/ray@2.59.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi ray with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/ray@2.59.0 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/ray@2.59.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi ray with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/ray@2.59.0 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/ray@2.59.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi ray with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/ray@2.59.0 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/ray@2.59.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi scipy with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/scipy@1.18.1 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/scipy@1.18.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi scipy with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/scipy@1.18.1 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/scipy@1.18.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi scipy with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/scipy@1.18.1 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/scipy@1.18.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi scipy with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/scipy@1.18.1 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/scipy@1.18.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi scipy with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/scipy@1.18.1 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/scipy@1.18.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi scipy with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/scipy@1.18.1 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/scipy@1.18.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi scipy with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/scipy@1.18.1 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/scipy@1.18.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi scipy with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/scipy@1.18.1 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/scipy@1.18.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi scipy with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/scipy@1.18.1 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/scipy@1.18.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi scipy with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/scipy@1.18.1 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/scipy@1.18.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi scipy with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/scipy@1.18.1 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/scipy@1.18.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi scipy with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/scipy@1.18.1 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/scipy@1.18.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi scipy with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/scipy@1.18.1 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/scipy@1.18.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi scipy with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/scipy@1.18.1 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/scipy@1.18.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi scipy with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/scipy@1.18.1 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/scipy@1.18.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi scipy with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/scipy@1.18.1 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/scipy@1.18.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi scipy with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/scipy@1.18.1 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/scipy@1.18.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
	Potential vulnerability: pypi scipy with risk level "medium" Location: Package overview From: uv.lock → pypi/nemo-gym@0.7.0 → pypi/scipy@1.18.1 ℹ Read more on: This package | This alert | Navigating potential vulnerabilities Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore pypi/scipy@1.18.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.	Warn
See 25 more rows in the dashboard

View full report

[earakely-scale]

NeMo RL GRPO through this server, on GPU

NeMo RL main (dd2e251, its Gym pinned at 3f6f205) trained Qwen3-4B through this resources server on the GridSim v1 training bundle. The bundle was served by agent_env.openenv.build_app on the host, and the server ran in url mode, written into NeMo RL's Gym checkout by python -m agent_env.training.nemo_gym.

Setup. It mirrors the bundle's earlier GRPO run with this repo's TRL helper:

• the same SFT warm start, recreated with that run's script;
• LoRA r 32, α 64, all linear;
• 4 days × 8 rollouts a step, 40 steps;
• temperature 1.1;
• AdamW at 2e-5 with 3 warmup steps, then cosine;
• clip 0.2 / 0.28, β 0;
• advantages with a mean baseline and no std normalization;
• vLLM's hermes tool parser, thinking off;
• validation on the bundle's 200 eval days, one sampled rollout each.

The hardware was 4× L40S (g6e.12xlarge), with vLLM colocated.

Mean reward on the 200 eval days	step 0 (SFT)	step 20	step 40
NeMo RL through this server	0.546	0.679	0.672
TRL GRPOTrainer through agent_env.training.trl (1× L40S)	0.569	0.562	0.614

One sampled rollout per day leaves about ±0.065 of noise (95%) between two means. NeMo RL's +0.13 by step 20 is outside that; TRL's +0.045 by step 40 isn't. The two runs share their data, model, LoRA and sampling, but not their loss normalization. TRL's dr_grpo divides by max_completion_length (1,024), while NeMo RL's token-level loss averages over the tokens generated. That likely gives NeMo RL a larger step at the same LR; I haven't isolated it.

What the run shows about the integration:

• Token continuity: NeMo RL's checks passed across tool calls (generation KL error about 0.001).
• Masking: no rollout was masked, in training or in validation.
• Waiting for room: validation sends all 200 rows at once against a bundle server limited to 64 episodes, and every rollout was scored, waiting its turn rather than being masked (0285161).
• Speed: a step took about 61 s, and a 200-day validation 2.5–4 min. The TRL run took about 190 s a step on one L40S.

About the GPU build, which isn't part of this PR. NeMo RL's container builds its CUDA code for sm 9.0/10.0 only, and no H100 capacity was available. So NeMo RL was built on bare metal for Ada (sm 8.9), with three local changes:

• DeepEP and DeepGEMM dropped from its vllm/automodel extras;
• nemo-automodel[moe] changed to nemo-automodel;
• a relock.

A dense model doesn't use those packages. On H100s, NeMo RL's own container should need none of this.

🤖 Generated with Claude Code