[slurm-users] Slurm strigger configuration

[Christopher Benjamin Coffey]

Killian, thank you very much! Never noticed the perm flag!

Best,
Chris

—
Christopher Coffey
High-Performance Computing
Northern Arizona University
928-523-1167
 

On 9/19/18, 10:01 AM, "slurm-users on behalf of Kilian Cavalotti" <slurm-users-bounces at lists.schedmd.com on behalf of kilian.cavalotti.work at gmail.com> wrote:

    On Wed, Sep 19, 2018 at 9:21 AM Christopher Benjamin Coffey
    <Chris.Coffey at nau.edu> wrote:
    > The only thing that I've gotten working so far is this:
    > sudo -u slurm bash -c "strigger --set -D -n cn15 -p /common/adm/slurm/triggers/nodestatus"
    >
    > So, that will run the nodestatus script which emails when the node cn15 gets set into drain state. What I'd like to do, which I haven't put time into figuring out, is how to setup a persistent trigger that can run when ANY node goes into drain state. Let me know if you figure that out. As you can see above, the trigger has to be setup by the slurm user.
    
    strigger takes a "--flags=perm" option, which makes the trigger
    permanent and doesn't purge it after the event happened. So it doesn't
    need tb re-armed in the trigger script for the next event.
    
    Also, not specifying any job id nor node name in the strigger command
    will make the trigger apply to all nodes. That's how we set our
    default "down" trigger for all nodes on our cluster:
    # su -s /bin/bash -c "strigger --set --down
    --program=/share/admin/scripts/slurm/triggers/down.sh --flags=perm"
    slurm
    
    Also note that the script given to "--program" needs to be executable
    by user Slurm on the controller node(s).
    
    Cheers,
    -- 
    Kilian