Remove NRL docs and Perflab baseline reporting - #2432
Conversation
3df166a to
f6ffc92
Compare
Greptile SummaryThe PR removes the local NRL harness documentation and Perflab release-reference reporting while retaining benchmark execution, nightly orchestration, artifacts, and ordinary Slack reporting.
|
| Filename | Overview |
|---|---|
| nemo_retriever/README.md | Replaces deleted local harness guides with an internal GitLab link that is inaccessible to the public README audience. |
| nemo_retriever/docs/cli/benchmarking.md | Redirects end-to-end benchmarking readers to the same internal-only NRB repository. |
| nemo_retriever/src/nemo_retriever/harness/cli.py | Removes reference-file loading and the public CLI option; the previously reported rule-governed compatibility break remains outstanding. |
| nemo_retriever/src/nemo_retriever/harness/slack.py | Removes release-reference rendering and public function parameters while preserving ordinary Slack payload generation. |
| ops/retriever-nightly/run-nightly.sh | Removes release-reference configuration propagation while retaining nightly execution and ordinary Slack posting. |
Prompt To Fix All With AI
### Issue 1
nemo_retriever/README.md:834-835
**Replacement benchmark link is inaccessible**
If a reader lacks access to NVIDIA's internal `gitlab-master.nvidia.com` domain, the sole replacement link for the deleted harness guides cannot be opened, leaving that reader without a usable path to the canonical end-to-end benchmark documentation.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Reviews (5): Last reviewed commit: "Merge upstream/main into legacy harness ..." | Re-trigger Greptile
| End-to-end Retriever experiments and benchmark orchestration are maintained in | ||
| the [NeMo Retriever Benchmark (NRB) repository](https://gitlab-master.nvidia.com/charlesb/nemo-retriever-benchmark/). |
There was a problem hiding this comment.
Replacement benchmark link is inaccessible
If a reader lacks access to NVIDIA's internal gitlab-master.nvidia.com domain, the sole replacement link for the deleted harness guides cannot be opened, leaving that reader without a usable path to the canonical end-to-end benchmark documentation.
Rule Used: When this PR changes user-facing code, configurati... (source)
Knowledge Base Used: Benchmark Harness
Prompt To Fix With AI
This is a comment left during a code review.
Path: nemo_retriever/README.md
Line: 834-835
Comment:
**Replacement benchmark link is inaccessible**
If a reader lacks access to NVIDIA's internal `gitlab-master.nvidia.com` domain, the sole replacement link for the deleted harness guides cannot be opened, leaving that reader without a usable path to the canonical end-to-end benchmark documentation.
**Rule Used:** When this PR changes user-facing code, configurati... ([source](.greptile))
**Knowledge Base Used:** [Benchmark Harness](https://app.greptile.com/nvidia-public-github/-/custom-context/knowledge-base/nvidia/nemo-retriever/-/docs/benchmark-harness.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Summary
--reference-fileandrelease_referencescompatibility surfaces entirelyWhy
Perflab is the canonical home for performance results and comparisons. This removes the ad hoc baseline presentation from NRL while preserving the harness implementation for the separate code decision.
Validation
git diff --checkpasses[[ -v "$name" ]]construct is rejected by system Bash 3.2