added new Brev launchable instructions - #733
Merged
jayavenkatesh19 merged 2 commits intoAug 4, 2026
Merged
Conversation
ncclementi
reviewed
Aug 4, 2026
ncclementi
left a comment
ncclementi
left a comment
Contributor
There was a problem hiding this comment.
Looks pretty good. I think the docker compose should be self contained. like the equivalent to launching things via UI.
| # Remove this entry if you are not adding a repository. | ||
| - /home/ubuntu/<name_of_your_github_repo>:/home/rapids/notebooks/<name_of_your_github_repo> | ||
| user: root | ||
| command: jupyter-lab --notebook-dir=/home/rapids/notebooks --ip=0.0.0.0 --no-browser --allow-root --NotebookApp.token='' --NotebookApp.allow_origin='*' |
Contributor
There was a problem hiding this comment.
I understand the 0.0.0.0 but I'm not sure if that's secure enough. Should we ask the Brev team about this?
Contributor
Author
There was a problem hiding this comment.
We can ask them about this yes, but while debugging I couldn't find a way to make the 127.0.0.1 work. I even tried without the --NotebookApp.token=' ' which disables authentication. We can ask the Brev team about how internet access/firewall rules are setup
| count: all | ||
| capabilities: [gpu] | ||
| environment: | ||
| - EXTRA_CONDA_PACKAGES # Value comes from a launch parameter of the same name |
Contributor
There was a problem hiding this comment.
If we give put thsi as an example, don't we need : and some packages to show teh exmaple?
I'm worried this dockerfile won't work as is.
Contributor
Author
There was a problem hiding this comment.
We don't need it, as this variable is populated from the value from the Launch Parameters section. I added hdbscan and umap in launch parameters and I could import those packages in the Notebooks
Comment on lines
+94
to
+97
| volumes: | ||
| # Repo cloned by the Source section, mounted alongside the example notebooks. | ||
| # Remove this entry if you are not adding a repository. | ||
| - /home/ubuntu/<name_of_your_github_repo>:/home/rapids/notebooks/<name_of_your_github_repo> |
Contributor
There was a problem hiding this comment.
I assume we remove this to mount the whole repo and not map it to the notebooks directory right? Just wanted to make sure
Contributor
Author
There was a problem hiding this comment.
The reason for doing this was to keep the example notebooks visible to the user. If we map to the notebooks directory, users can see the cudf, cugraph, cuml examples folders along with the cloned repo added as the fourth folder in the root of the mounted path
|
|
||
| ```yaml | ||
| environment: | ||
| - EXTRA_CONDA_PACKAGES |
Contributor
There was a problem hiding this comment.
Have you checked once the instance is launched if the extra packages installed work?
To see if they were properly installed?
Contributor
Author
There was a problem hiding this comment.
Yep I could import them into the notebook without any errors.
ncclementi
approved these changes
Aug 4, 2026
ncclementi
left a comment
ncclementi
left a comment
Contributor
There was a problem hiding this comment.
pending understanding on the 0.0.0.0 change.
Contributor
Author
|
Confirmed with the Brev team that they switched from using Cloudflare for secure links to an internal tool called Skybridge which only supports external access through the |
HPicatto
pushed a commit
to HPicatto/deployment
that referenced
this pull request
Aug 10, 2026
jayavenkatesh19
added a commit
that referenced
this pull request
Sep 1, 2026
* Update docs to use NVIDIA Sphinx theme (#712) * Update docs to use NVIDIA Sphinx theme * fix: artifacting in rendered page * fix: run precommit * rebase fix conflcits * small edit html title * clean up comment in version switcher * clean up comment in conf.py --------- Co-authored-by: Jaya Venkatesh <jjayabaskar@nvidia.com> Co-authored-by: ncclementi <natyclementi@gmail.com> * feat: add dagster-slurm + RAPIDS topic modeling on HPC/Slurm workflow example Adds a workflow example showing RAPIDS cuML (UMAP + HDBSCAN) running on a Slurm HPC cluster orchestrated with dagster-slurm: a partitioned CPU fan-out of gensim LDA jobs followed by GPU reduction/clustering stages in a separate packed pixi environment, with a CPU fallback path for GPU-less development. Includes UI screenshots from a real cluster run and a refined variant with metaxy sample-level incremental tracking. Refs #715 Signed-off-by: Hernan Picatto <h.picatto@gmail.com> * docs: address tutorial review feedback on the dagster-slurm example - add a quickstart and an explicit 'by hand vs. here' contrast up front - call out the local iteration loop (identical code, laptop to cluster) - state which assets run in local mode plus expected size/runtime - mention run-scoped Slurm allocations for the queue-wait case - mention multi-cluster targeting via SLURM_EDGE_NODE_* configuration - add an 'HPC in the middle, Dagster end to end' section - soften the topic_map claim (plot not among the screenshots) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Hernan Picatto <h.picatto@gmail.com> * docs: address review feedback on the topic modeling example - Quickstart block right after the intro; 'what you would normally do' contrast paragraph for first-time HPC users - Explicit note that local mode materializes 3 of 6 assets - Download-size / wall-clock expectations for the docker chain - Queue-wait answer (experimental session/het-job modes) and multi-cluster deployment note for experienced Slurm users - Note that all-sbatch is a choice; ingest/publish can run off-Slurm in the same graph (and must, on sites without compute-node internet) - Iteration-loop paragraph in the conclusion - Add the topic_map artifact from the cluster run as the closing image - Stable cell ids (nbformat 5) Signed-off-by: Hernan Picatto <h.picatto@gmail.com> * docs: add complete-run screenshots to the topic modeling example From a full end-to-end backfill on the real cluster (19m28s): the topic_map run view with cluster-count/path metadata, the topic_map preview rendering the scatter inline in the Dagster UI (closing the review gap about the promised 'streams a labeled meta-topic map' claim), and the all-green backfill overview with observed wall-clock numbers in the surrounding text. Signed-off-by: Hernan Picatto <h.picatto@gmail.com> * Default the deployment docs to CUDA 13 (#719) * change conf to PEP440 standards Signed-off-by: Jaya Venkatesh <jjayabaskar@nvidia.com> * Centralize CUDA/Python config in conf.py; default docs to CUDA 13 * Use standard conda packages for SageMaker (issue #520 resolved) * Pin AzureML to CUDA 12 with inline conda spec and Docker Hub image * Use centralized CUDA tokens for Modal base image and wheel * Template Coiled CUDA/container references; shield YAML jinja from prettier * Refresh Vertex AI note to reflect CUDA 13 environment * Inline Modal base image and drop cuda_docker_version * Templatize custom-docker wheel references * Move Databricks to CUDA 13 and replace preinstalled cupy * added warning on AzureML --------- Signed-off-by: Jaya Venkatesh <jjayabaskar@nvidia.com> * Fixes after testing on Databricks (#721) * Fix nightly pip version specifier to allow prereleases * Use pip version reference for Modal wheel install * change nightly versioning * Drop explicit cupy handling from Databricks init script * Update Databricks runtime guidance to 18 LTS ML * Reflect new Databricks Machine Learning runtime toggle * update runtime wording * Pass RAPIDS pip index to Modal wheel install * Remove stale pip/cupy warning from Vertex AI page (#725) * X-ORG-722: Publish API docs to docs.nvidia.com (#724) * X-ORG-722: Publish API docs to docs.nvidia.com * Lint * Update compute engine - add workarounds (#727) * update links to guide better the user to console * add note with workaround due to broken vmi tile * Apply suggestion --------- Co-authored-by: Jaya Venkatesh <jjayabaskar@nvidia.com> * X-ORG-722: Ensure gha-tools are installed (#728) * Update readme, better layout description (#732) update readme, better layout description * added new Brev launchable instructions (#733) * Add kaggle page in platforms (#734) add kaggle page in platforms * add Databricks Serverless (#736) * add Databricks Serverless * added intro links * remove dask-cuda from pip installation * replace scikit Signed-off-by: Hernan Picatto <h.picatto@gmail.com> * rewording Signed-off-by: Hernan Picatto <h.picatto@gmail.com> * make plot nicer Signed-off-by: Hernan Picatto <h.picatto@gmail.com> * reference example on HPC base Signed-off-by: Hernan Picatto <h.picatto@gmail.com> * test if rendering works Signed-off-by: Hernan Picatto <h.picatto@gmail.com> * test Signed-off-by: Hernan Picatto <h.picatto@gmail.com> * check render on web Signed-off-by: Hernan Picatto <h.picatto@gmail.com> * add metaxy Signed-off-by: Hernan Picatto <h.picatto@gmail.com> * remove issue Signed-off-by: Hernan Picatto <h.picatto@gmail.com> * restructure topic modeling example around RAPIDS on HPC * fix spacing in hpc page * address review comments --------- Signed-off-by: Hernan Picatto <h.picatto@gmail.com> Signed-off-by: Jaya Venkatesh <jjayabaskar@nvidia.com> Co-authored-by: Bradley Dice <bdice@bradleydice.com> Co-authored-by: Jaya Venkatesh <jjayabaskar@nvidia.com> Co-authored-by: ncclementi <natyclementi@gmail.com> Co-authored-by: Georg Heiler <georg.kf.heiler@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Joséphine Wolf Oberholtzer <166141926+josephine-wolf-oberholtzer@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updates the Brev launchable instructions to reflect the new UI changes on Brev.
While testing this out, I also found a few bugs and made some changes in the old Docker Compose YAML file:
backend(now renamed tojupyter) was not nested under services and was failing YAML parsing--ip=127.0.0.1changed to--ip=0.0.0.0. I could not reach the earlier Jupyter endpoint over the internet and this configuration change worked--notebook-dir=/notebooks. This was showing up in Jupyterlab as an empty folder and not with the examples as intendedEXTRA_CONDA_PACKAGESis now a launch configuration variable with the value being passed to the container. This way users can customize their packages at deploy time rather than the Launchable creation timeI added screenshots of every single section, and tested everything out end to end. This new configuration ensures a healthy Jupyterlab, with access to any cloned repositories and all of the example notebooks.