Skip to content

Record job and DB ID in slurm job #40

Description

@janosh

sbatch has a handy --comment that allows adding metadata to jobs:

sbatch -h | grep comment
      --comment=name          arbitrary comment

i'd like to use this to connect my slurm jobs back to jobs in the database either via their jobflow or their database ID. i saw there's no explicit support for comment in SlurmIO:

class SlurmIO(BaseSchedulerIO):
header_template: str = """
#SBATCH --partition=$${partition}
#SBATCH --job-name=$${job_name}
#SBATCH --nodes=$${nodes}
#SBATCH --ntasks=$${ntasks}
#SBATCH --ntasks-per-node=$${ntasks_per_node}
#SBATCH --cpus-per-task=$${cpus_per_task}
#SBATCH --mem=$${mem}
#SBATCH --mem-per-cpu=$${mem_per_cpu}
#SBATCH --hint=$${hint}
#SBATCH --time=$${time}
#SBATCH --exclude=$${exclude_nodes}
#SBATCH --account=$${account}
#SBATCH --mail-user=$${mail_user}
#SBATCH --mail-type=$${mail_type}
#SBATCH --constraint=$${constraint}
#SBATCH --gres=$${gres}
#SBATCH --requeue=$${requeue}
#SBATCH --nodelist=$${nodelist}
#SBATCH --propagate=$${propagate}
#SBATCH --licenses=$${licenses}
#SBATCH --output=$${qout_path}
#SBATCH --error=$${qerr_path}
#SBATCH --qos=$${qos}
#SBATCH --priority=$${priority}
#SBATCH --array=$${array}
#SBATCH --exclusive=$${exclusive}
$${qverbatim}"""

maybe qverbatim is meant as an escape hatch for situations like this? didn't find any docs on it. also not sure how to set qverbatim. this raises

QResources(
    processes=16, job_name="name", qverbatim="test"
).as_dict()
>>> TypeError: QResources.__init__() got an unexpected keyword argument 'qverbatim'

maybe like this? didn't try yet but either way would be good to document how to pass qverbatim

QResources(
    processes=16, job_name="name", scheduler_kwargs={"qverbatim": "test"}
).as_dict()

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions