rxembed → g-xTB → ORCA workflow

This repository runs a database-backed computational chemistry workflow on a Slurm cluster. It takes SMILES from a CSV file, generates and optimizes conformers with rxembed and g-xTB, creates ORCA PBE0 single-point inputs, and submits those inputs as Slurm arrays.

The SQLite database is the workflow's source of truth. It records which molecules are waiting, running, complete, or failed, together with the paths to their generated files. This makes each stage easy to inspect and allows later stages to work only on jobs completed by the previous stage.

Repository and run-directory layout

workflow/
├── README.md
├── src/                         # Python workflow code
├── scripts/                     # Slurm launchers
├── desired/                     # Example run directory
│   └── config/
│       └── settings.yaml        # Example configuration
└── db/                          # Example database snapshot
    ├── jobs.db
    └── backup.db

The input dataset and generated Embed/, Inputs/, ORCA/, and logs/ directories are not included here because they can be large.

The launchers must be submitted from a run directory immediately below the repository root. For example, a populated desired/ run will look like:

workflow/
├── scripts/
├── src/
└── desired/                     # run commands from here
    ├── config/settings.yaml
    ├── desired_ligands.csv
    ├── db/
    │   ├── jobs.db
    │   └── backup.db
    ├── logs/
    ├── Embed/
    ├── Inputs/
    └── ORCA/

Paths in settings.yaml are relative to the directory containing config/. Consequently, the example setting input.db: db/jobs.db means desired/db/jobs.db, not the top-level example at workflow/db/jobs.db. Either seed a new database in the run directory, restore a consistent backup there, or intentionally change input.db and backup.mirror_path.

Do not copy a live jobs.db file while the workflow is writing to it. Use the periodically generated backup.db, or SQLite's online backup mechanism, so the restored database is consistent with its WAL state.

Cluster requirements

Before starting a run, check that the cluster provides:

• Slurm commands including sbatch, squeue, sacct, and scancel;
• Python with SQLite 3.35 or newer;
• the Python dependencies used by the workflow: PyYAML, pandas, RDKit, NetworkX, xyzgraph, and rxembed;
• an executable xTB installation;
• ORCA 6.1.0 and the matching OpenMPI installation; and
• a shared filesystem visible from the submission and compute nodes.

The supplied launchers contain cluster-specific defaults:

• db_maker.sh, backupper.sh, make_input.sh, and submit_orca.sh use the micromamba environment named Complex;
• submit_embedder.sbatch looks for rxembed at /groups/kemi/hteahan/rxembed and xTB at the path assigned to XTB_EXE;
• the Slurm partition defaults to kemi1; and
• the generated ORCA array script uses ORCA and OpenMPI paths under /groups/kemi/.

If running elsewhere, update these paths, environment names, partitions, and any site-specific Slurm directives before submitting jobs. The embedding launcher also accepts useful environment overrides:

RXEMBED_REPO=/absolute/path/to/rxembed \
XTB_EXE=/absolute/path/to/xtb \
RXEMBED_CONDA=my_environment \
sbatch ../scripts/submit_embedder.sbatch

Preparing a new run

Start in the repository root, copy the example run directory, and create the directories Slurm and the workflow will write to:

cp -a desired my_run
cd my_run
mkdir -p db logs Embed Inputs ORCA

You can also use desired/ itself as the run directory. Keeping separate run directories is usually clearer when testing different datasets or settings.

1. Add the input CSV

By default, place the input file at desired_ligands.csv. Its header must contain:

ligand_ID,smiles,ligand_smiles,csd_ID

• ligand_ID is required and identifies the ligand in outputs and the database.
• smiles is the structure passed to the embedding stage.
• ligand_smiles is also stored in the database.
• csd_ID is optional; omit the entire column if it is not available.

The names of the two SMILES columns and the CSV path can be changed under db_maker in config/settings.yaml.

2. Review config/settings.yaml

At minimum, check the input paths, Slurm partitions, executable paths, resources, chemistry settings, and ORCA query. The supplied configuration is a worked example rather than a universal production configuration.

The most operationally important sections are:

Section	What it controls
db_maker	Input CSV and column names used to seed the database
input	Shared SQLite database path and initial job type
backup	Consistent database mirror and backup interval
embedding	Geometry, conformers per stereoisomer, and total conformer cap
parallel	Concurrent SMILES and stereoisomer workers in the controller
xtb	xTB executable, resources, partition, timeout, and array concurrency
pruning	Energy and structural pruning before and after g-xTB
output	Embedding result directory; the launcher sets it to Embed/
make_input	Connectivity checks, energy window, and ORCA input template
orca	ORCA directories, resources, partition, and array sizes

All relative paths are resolved from the run directory. Environment variables inside values are expanded, so the example xtb.executable: "$XTB_EXE" uses the value exported by the launcher.

Resource settings that must agree with one another:

• The embedding controller needs parallel.smiles_workers × parallel.isomer_workers CPUs. The example uses 4 × 4 = 16, matching --cpus-per-task=16 in submit_embedder.sbatch.
• Each g-xTB array element separately requests xtb.cpus_per_task and xtb.memory. xtb.array_parallelism is the maximum number of those array elements allowed to run at once.
• make_input.workers should not exceed the CPUs requested by make_input.sh.
• make_input.cpus and make_input.memory_mb are written into each generated ORCA input and should match orca.cpus and orca.memory_mb.
• orca.array_size limits the total inputs in one array; orca.batch_size is the %N limit on simultaneously running array tasks.

3. Create or restore the database

For a new run, seed the database from the CSV:

bash ../scripts/db_maker.sh

The command creates db/jobs.db, enables WAL mode, creates the jobs table, and inserts one embedding/pending row per unique ligand_ID and smiles pair. Running it again is safe for already inserted pairs: existing rows are skipped.

To resume an existing run instead, place a consistent database backup at the path configured by input.db. The generated geometry and input files referred to by that database must also exist at their recorded paths.

Running the workflow

Run these commands from the populated run directory, such as my_run/ or desired/.

1. Start database backups

sbatch ../scripts/backupper.sh

This is a long-running job that copies input.db to backup.mirror_path using SQLite's online backup API at the configured interval. Record the returned Slurm job ID so it can be cancelled after the workflow finishes.

2. Embed and optimize conformers

sbatch ../scripts/submit_embedder.sbatch

The controller claims embedding/pending database rows, generates stereoisomers and conformers with rxembed, prunes them, and submits g-xTB child arrays. Successful jobs are written below Embed/ and marked embedding/complete; failures are marked embedding/failed with an error in the database.

The normal target is embedding.n_confs per stereoisomer. If that would exceed embedding.max_total_confs for one SMILES, every stereoisomer receives the same reduced count. For example, with a cap of 200, 16 stereoisomers receive 12 conformers each.

Wait for the embedding controller to finish before generating ORCA inputs.

3. Generate ORCA inputs

sbatch ../scripts/make_input.sh

This stage takes completed embedding rows, checks conformer connectivity, applies the per-isomer energy window, and writes ORCA inputs below Inputs/. Each job also receives an input_manifest.csv and graph_check.json so the selection can be audited. Successful rows become SP/inputs_created.

4. Submit and monitor ORCA arrays

sbatch ../scripts/submit_orca.sh

One controller claims complete database rows, flattens their conformer inputs into Slurm arrays, and monitors them through sacct. A database row is never split between arrays. Results are written below a unique ORCA/submission_<id>/ directory and successful rows become SP/orca_completed with their extracted PBE0 energies stored in the database.

Once that database update has been committed, the matching task_XXXXX/ directories are removed from the submission directory. Tasks belonging to other database rows in the same Slurm array are left in place.

If this controller is restarted, it first resumes monitoring rows already in orca_submitted before claiming new work.

5. Stop the backup job

After all required work is complete and a final backup has been written:

scancel BACKUP_JOB_ID

Monitoring a run

Slurm writes controller output and errors below logs/:

squeue -u "$USER"
tail -f logs/submit-embedder-JOB_ID.out
tail -f logs/input_JOB_ID.out
tail -f logs/orca_JOB_ID.out
tail -f logs/backup_JOB_ID.out

To see the database queue at any point:

sqlite3 db/jobs.db \
  "SELECT job_type, status, COUNT(*) AS jobs
   FROM jobs
   GROUP BY job_type, status
   ORDER BY job_type, status;"

To inspect failures without dumping every record:

sqlite3 -header -column db/jobs.db \
  "SELECT id, ligand_id, job_type, status, error
   FROM jobs
   WHERE error IS NOT NULL
   ORDER BY id
   LIMIT 50;"

The main successful state progression is:

embedding/pending
  → embedding/running
  → embedding/complete
  → embedding/inputting
  → SP/inputs_created
  → SP/orca_claimed
  → SP/orca_submitted
  → SP/orca_completed

Intermediate states such as running, inputting, and orca_claimed mean a controller had claimed the row. If a controller was killed at exactly that point, inspect its log and Slurm state before changing the database. Do not blindly reset statuses while an old controller or child array may still be running.

Output files

For a successful run, the persistent files are organized as follows:

Embed/
└── job_<database-id>_ligand_<ligand-id>/
    └── <isomer>/
        ├── optimized_ensemble.xyz
        ├── lowest.xyz
        ├── conformer_manifest.csv
        └── conformers/conformer_XXXX.xyz

Inputs/
└── job_<database-id>_ligand_<ligand-id>/
    ├── input_manifest.csv
    ├── graph_check.json
    └── <isomer>/conformers/conformer_XXXX.inp

ORCA/
└── submission_<unique-id>/
    ├── submit.slurm
    ├── batch_array.txt
    └── task_XXXXX/
        ├── conformer_XXXX.out
        ├── conformer_XXXX.err
        └── optional ORCA result files

Workflow Python code and the rxembed package are copied to node-local temporary storage for execution. Configuration, databases, curated geometries, ORCA inputs, and final results remain on the shared filesystem. g-xTB and ORCA scratch work is isolated per array task to avoid filename collisions.

Rerunning and recovery notes

• Database claiming is status-based. A stage ignores rows that are not in its configured input status.
• Seeding again adds missing CSV records but does not overwrite existing rows.
• Rerunning the input generator processes remaining embedding/complete rows.
• Rerunning the ORCA controller resumes submitted work, then processes remaining SP/inputs_created rows.
• An embedding row left as running, an input row left as inputting, or an ORCA row left as orca_claimed requires manual investigation; these states are not automatically reset.
• Keep the database and its referenced Embed/, Inputs/, and ORCA/ trees together when moving or restoring a run.

Developer checks

These structural checks can be run from the repository root. Some historical checkouts may not include the test directory.

python -m compileall -q src
bash -n scripts/*.sh scripts/*.sbatch