You can attack this in a few different stages. A lot of what you’re interested in will be found at various university or national lab sites (I Googled “sbatch example” for the one below)

If you’re good with doing a “make -j” to parallelize a make compilation over multiple CPUs in a single computer to build one executable, you can adapt the first example from the Multi-Threaded SMP Job section of [1], using the same number of cpus-per-task as you use for your -j flag.
Once that’s going, you can use a script similar to the one at [2] to submit an array of concurrent build jobs, and they can spread out across multiple computers as needed. The built-in job array support will only change an environment variable SLURM_ARRAY_TASK_ID for each job in the array, but you can use that value to select a folder to cd into, or to do other things.
For a higher-level abstraction that will submit a bunch of build jobs, wait for them all to finish, and then archive the resulting artifacts, a pipeline tool like Snakemake [3] can track the individual tasks, and work with batch jobs or with local programs on the submission host. Two Slurm-related profiles for Snakemake are at [4] (simpler) and [5] (more comprehensive).

From: slurm-users <slurm-users-bounces@lists.schedmd.com> on behalf of Duane Ellis <duane@duaneellis.com>
Date: Wednesday, January 3, 2024 at 2:41 PM
To: slurm-users@lists.schedmd.com <slurm-users@lists.schedmd.com>
Subject: [slurm-users] Slurp for sw builds

External Email Warning

This email originated from outside the university. Please use caution when opening attachments, clicking links, or responding to requests.

________________________________

In my case I would like to use a slurm cluster for a sw ci/cd like solution for building sw images

My current scripted full system build takes 3-5 hours and is done serially we could easily find places where we can choose to build things in parallel hence the idea is to spawn parallel builds on the Linux slurm cluster

example: we have a list of images we iterate over in a for loop to build each thing the steps are: cd somedir then type make or run a shell script in that directory

The last step after the for loop would be wait for all of the child builds to complete

Once all child jobs are done we have a single job that combines or packages all the intermediate images

Really want to use slurm because our FPGA team will have a giant slurm linux cluster for Xilinix FPGA builds and those nodes can do what we need for sw purposes (reusing the existing cluster is a huge win for us)

My question is this:

Can somebody point me to some sw build examples for or using slurm? All I can seem to find is how to install

I see the srun and sbatch command man pages but no good examples

Bonus would be something that integrates into a gitlab runner example or Jenkins in some way

All I can seem to find is how to install and administer slurm not a how to use slurm

Sent from my iPhone