Repository navigation
Conversation
- The generic dashboard-training guidance misses the hard-coded realtime QoS - The getting-started tunnel is incompatible with training when MongoDB uses a non-default port - A remaining BELLA-specific command is both ungeneralized and invalid
|
|
||
| 3. Set up the database settings (read-only) and the AmSC MLflow API key: | ||
| ```bash | ||
| export SF_DB_HOST='127.0.0.1' |
There was a problem hiding this comment.
Removed: went away in a recent PR.
We still need to remove references to it in a few other locations: https://github.com/search?q=repo%3ABLAST-AI-ML%2Fsynapse+SF_DB_HOST&type=code
Co-authored-by: Edoardo Zoni <59625522+EZoni@users.noreply.github.com>
Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
|
LGTM for a major improvement. Would merge and improve small nits further in separate PRs. |
| ``` | ||
|
|
||
| ```{note} | ||
| This path is hardcoded for the BELLA deployment in {repo}`dashboard/parameters_manager.py`. |
There was a problem hiding this comment.
| This path is hardcoded for the BELLA deployment in {repo}`dashboard/parameters_manager.py`. | |
| This path is hardcoded for the BELLA deployment in {repo}`dashboard/parameters_manager.py` and {repo}`dashboard/model_manager.py`. |
| podman-hpc run --gpu -v /etc/localtime:/etc/localtime -v $HOME/db.profile:/root/db.profile -v /path/to/config.yaml:/app/ml/config.yaml --rm -it registry.nersc.gov/m558/superfacility/synapse-ml:latest python -u /app/ml/train_model.py --test --config_file /app/ml/config.yaml --model NN | ||
| podman-hpc run --gpu -v /etc/localtime:/etc/localtime --env-file $HOME/db-podman.profile -v <your_config_file>:/app/ml/config.yaml --rm -it registry.nersc.gov/m558/superfacility/synapse-ml:latest python -u /app/ml/train_model.py --test --config_file /app/ml/config.yaml --model NN |
There was a problem hiding this comment.
Just double checking, these changes in the code snippet,
- from
db.profiletodb-podman.profile - from
-vto--env-file
happen to fix issues that we had missed so far?
| conda activate base | ||
| conda-lock install --name synapse-gui environment-lock.yml | ||
| conda activate synapse-gui | ||
| export SF_DB_READONLY_PASSWORD='...' |
There was a problem hiding this comment.
| export SF_DB_READONLY_PASSWORD='...' | |
| export SF_DB_READONLY_PASSWORD='<your_password_here>' # Use SINGLE quotes around the password! |
| conda activate base | ||
| conda-lock install --name synapse-ml environment-lock.yml | ||
| conda activate synapse-ml | ||
| export SF_DB_READONLY_PASSWORD='...' |
There was a problem hiding this comment.
| export SF_DB_READONLY_PASSWORD='...' | |
| export SF_DB_READONLY_PASSWORD='<your_password_here>' # Use SINGLE quotes around the password! |
| :sync: bella-nersc | ||
|
|
||
| ```text | ||
| SF_DB_READONLY_PASSWORD=your_password_here |
There was a problem hiding this comment.
| SF_DB_READONLY_PASSWORD=your_password_here | |
| SF_DB_READONLY_PASSWORD=<your_password_here> |
| 3. Set up the database settings (read-only) and the AmSC MLflow API key: | ||
| ```bash | ||
| export SF_DB_HOST='127.0.0.1' | ||
| export SF_DB_READONLY_PASSWORD='your_password_here' # Use SINGLE quotes around the password! |
There was a problem hiding this comment.
| export SF_DB_READONLY_PASSWORD='your_password_here' # Use SINGLE quotes around the password! | |
| export SF_DB_READONLY_PASSWORD='<your_password_here>' # Use SINGLE quotes around the password! |
| ```bash | ||
| export SF_DB_HOST='127.0.0.1' | ||
| export SF_DB_READONLY_PASSWORD='your_password_here' # Use SINGLE quotes around the password! | ||
| export AM_SC_API_KEY='your_amsc_api_key_here' # Required when MLflow tracking_uri is AmSC |
There was a problem hiding this comment.
| export AM_SC_API_KEY='your_amsc_api_key_here' # Required when MLflow tracking_uri is AmSC | |
| export AM_SC_API_KEY='<your_amsc_api_key_here>' # Required when MLflow tracking_uri is AmSC |
|
|
||
| 3. Set up the database settings (read-only) and the AmSC MLflow API key: | ||
| ```bash | ||
| export SF_DB_READONLY_PASSWORD='your_password_here' # Use SINGLE quotes around the password! |
There was a problem hiding this comment.
| export SF_DB_READONLY_PASSWORD='your_password_here' # Use SINGLE quotes around the password! | |
| export SF_DB_READONLY_PASSWORD='<your_password_here>' # Use SINGLE quotes around the password! |
| :sync: bella-nersc | ||
|
|
||
| ```bash | ||
| export SF_DB_READONLY_PASSWORD='your_password_here' # Use SINGLE quotes around the password! |
There was a problem hiding this comment.
| export SF_DB_READONLY_PASSWORD='your_password_here' # Use SINGLE quotes around the password! | |
| export SF_DB_READONLY_PASSWORD='<your_password_here>' # Use SINGLE quotes around the password! |
| 4. Run the Docker container: | ||
| ```bash | ||
| docker run --network=host -v /etc/localtime:/etc/localtime -v $PWD/ml:/app/ml -e SF_DB_HOST='127.0.0.1' -e SF_DB_READONLY_PASSWORD='your_password_here' -e AM_SC_API_KEY='your_amsc_api_key_here' synapse-gui | ||
| docker run --network=host -v /etc/localtime:/etc/localtime -v $PWD/ml:/app/ml -e SF_DB_READONLY_PASSWORD='your_password_here' -e AM_SC_API_KEY='your_amsc_api_key_here' synapse-gui |
There was a problem hiding this comment.
| docker run --network=host -v /etc/localtime:/etc/localtime -v $PWD/ml:/app/ml -e SF_DB_READONLY_PASSWORD='your_password_here' -e AM_SC_API_KEY='your_amsc_api_key_here' synapse-gui | |
| docker run --network=host -v /etc/localtime:/etc/localtime -v $PWD/ml:/app/ml -e SF_DB_READONLY_PASSWORD='<your_password_here>' -e AM_SC_API_KEY='<your_amsc_api_key_here>' synapse-gui |
| For debugging, you can enter the container without starting the app: | ||
| ```bash | ||
| docker run --network=host -v /etc/localtime:/etc/localtime -v $PWD/ml:/app/ml -e SF_DB_HOST='127.0.0.1' -e SF_DB_READONLY_PASSWORD='your_password_here' -e AM_SC_API_KEY='your_amsc_api_key_here' -it synapse-gui bash | ||
| docker run --network=host -v /etc/localtime:/etc/localtime -v $PWD/ml:/app/ml -e SF_DB_READONLY_PASSWORD='your_password_here' -e AM_SC_API_KEY='your_amsc_api_key_here' -it synapse-gui bash |
There was a problem hiding this comment.
| docker run --network=host -v /etc/localtime:/etc/localtime -v $PWD/ml:/app/ml -e SF_DB_READONLY_PASSWORD='your_password_here' -e AM_SC_API_KEY='your_amsc_api_key_here' -it synapse-gui bash | |
| docker run --network=host -v /etc/localtime:/etc/localtime -v $PWD/ml:/app/ml -e SF_DB_READONLY_PASSWORD='<your_password_here>' -e AM_SC_API_KEY='<your_amsc_api_key_here>' -it synapse-gui bash |
| export SF_DB_HOST='127.0.0.1' | ||
| export SF_DB_READONLY_PASSWORD='...' | ||
| export AM_SC_API_KEY='...' | ||
| export <database.password_ro_env>='...' |
There was a problem hiding this comment.
| export <database.password_ro_env>='...' | |
| export <database.password_ro_env>='<your_password_here>' # Use SINGLE quotes around the password! |
|
@EZoni do you like this finish this PR this week (do not over-do it) and we can merge a first generalized version? |
Generalize our docs to be "General Pattern" + "Example Project: ..." style.
That way we:
Follow-up to #341