Hi, Is there an option in slurm to launch a custom script at the time of job submission through sbatchor salloc? The script should run with submit user permission in submit area. The idea is that we need to enquire something which characterises our job’s requirement like CPUslots, memory etc from a central server and we do need read access to user area prior to that. In our use case the user doesn’t necessarily know beforehand what kind of resource his job needs.(Hence, the need for such a script which will contact the server with user area info.) Based on it we can modify the job a little later. A post submit script, if available, would inform us the slurm job id as well, it would get called just after the job has entered the system and prior to its scheduling.
Thanks,Bhaskar.
Sent from Yahoo Mail for iPad
Not sure if I understand your use case, but if I do I am not sure if Slurm provides that functionality. If it doesn't (and if my understanding is correct), you can still achieve your goal by:
1) removing sbatch and salloc from user's path 2) writing your own custom scripts named sbatch (and hard/symbolic linked to salloc) which does what you want and at the end (with $0 variable or similar thing) it actually invokes full-path to sbatch and salloc -- actually it could do that after, rather than before, if you so prefer
Hope this helps
On Mon, Oct 28, 2024 at 11:59 AM Bhaskar Chakraborty via slurm-users < slurm-users@lists.schedmd.com> wrote:
Hi,
Is there an option in slurm to launch a custom script at the time of job submission through sbatch or salloc? The script should run with submit user permission in submit area.
The idea is that we need to enquire something which characterises our job’s requirement like CPU slots, memory etc from a central server and we do need read access to user area prior to that.
In our use case the user doesn’t necessarily know beforehand what kind of resource his job needs. (Hence, the need for such a script which will contact the server with user area info.)
Based on it we can modify the job a little later. A post submit script, if available, would inform us the slurm job id as well, it would get called just after the job has entered the system and prior to its scheduling.
Thanks, Bhaskar.
Sent from Yahoo Mail for iPad https://mail.onelink.me/107872968?pid=nativeplacement&c=Global_Acquisition_YMktg_315_Internal_EmailSignature&af_sub1=Acquisition&af_sub2=Global_YMktg&af_sub3=&af_sub4=100000604&af_sub5=EmailSignature__Static_
-- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-leave@lists.schedmd.com
On 10/28/24 10:56 am, Bhaskar Chakraborty via slurm-users wrote:
Is there an option in slurm to launch a custom script at the time of job submission through sbatch or salloc? The script should run with submit user permission in submit area.
I think you are after the cli_filter functionality which can run plugins in that environment. There is a Lua plugin for that which will allow you to write your code in something a little less fraught than C.
https://slurm.schedmd.com/cli_filter_plugins.html
There is example Lua code for this here:
https://github.com/SchedMD/slurm/blob/master/etc/cli_filter.lua.example
All the best, Chris
Thanks for the help. Shall try to check the client plugin option 1st else have to revert to the other alternative which, though more of a workaround, also looks good & promising. -Bhaskar. On Tuesday 29 October, 2024 at 06:49:12 am IST, Christopher Samuel via slurm-users slurm-users@lists.schedmd.com wrote:
On 10/28/24 10:56 am, Bhaskar Chakraborty via slurm-users wrote:
Is there an option in slurm to launch a custom script at the time of job submission through sbatch or salloc? The script should run with submit user permission in submit area.
I think you are after the cli_filter functionality which can run plugins in that environment. There is a Lua plugin for that which will allow you to write your code in something a little less fraught than C.
https://slurm.schedmd.com/cli_filter_plugins.html
There is example Lua code for this here:
https://github.com/SchedMD/slurm/blob/master/etc/cli_filter.lua.example
All the best, Chris
Have you already seen if SrunProlog or TaskProlog could be useful for your case? The documentation is here: https://slurm.schedmd.com/prolog_epilog.html
-- Fabio
From: Bhaskar Chakraborty via slurm-users slurm-users@lists.schedmd.com Sent: Monday, October 28, 2024 6:56 PM To: slurm-users@schedmd.com Subject: [EXTERNAL] [slurm-users] Job pre / post submit scripts
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Hi,
Is there an option in slurm to launch a custom script at the time of job submission through sbatch or salloc? The script should run with submit user permission in submit area.
The idea is that we need to enquire something which characterises our job’s requirement like CPU slots, memory etc from a central server and we do need read access to user area prior to that.
In our use case the user doesn’t necessarily know beforehand what kind of resource his job needs. (Hence, the need for such a script which will contact the server with user area info.)
Based on it we can modify the job a little later. A post submit script, if available, would inform us the slurm job id as well, it would get called just after the job has entered the system and prior to its scheduling.
Thanks, Bhaskar. Sent from Yahoo Mail for iPadhttps://mail.onelink.me/107872968?pid=nativeplacement&c=Global_Acquisition_YMktg_315_Internal_EmailSignature&af_sub1=Acquisition&af_sub2=Global_YMktg&af_sub3=&af_sub4=100000604&af_sub5=EmailSignature__Static_
NICE SRL, viale Monte Grappa 3/5, 20124 Milano, Italia, Registro delle Imprese di Milano Monza Brianza Lodi REA n. 2096882, Capitale Sociale: 10.329,14 EUR i.v., Cod. Fisc. e P.IVA 01133050052, Societa con Socio Unico
-
Bhaskar Chakraborty -
Christopher Samuel -
Davide DelVento -
Di Bernardini, Fabio