I have integrated Podman with Slurm as per the docs (https://slurm.schedmd.com/containers.html#podman-scrun) and when I do a test run:
"podman run hello-world" (this runs fine)
----
$ podman run alpine hostname
executable file `/usr/bin/hostname` not found in $PATH: No such file or directory
srun: error: slurm1: task 0: Exited with exit code 1
-----
$ podman run alpine printenv SLURM_JOB_ID
executable file `/usr/bin/printenv` not found in $PATH: No such file or directory
srun: error: slurm1: task 0: Exited with exit code 1
scrun: error: run_command_waitpid_timeout: waitpid(67537): No child processes
---
podman run alpine uptime
11:31:28 up 5:32, 0 users, load average: 0.00, 0.00, 0.00
scrun: error: run_command_waitpid_timeout: waitpid(68160): No child processes
------
I built a small image from python:alpine3.19 which just prints "hello world" and numbers from 1 to 10. Here is a run:
$ podman run -it --rm hello-python
$ podman run -it --rm hello-python
Hello, world!
Numbers from 1 to 10: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
----
No error with my image. Also I tested podman on another machine without Slurm. Podman with its default runtime prints the hostname fine with "podman run alpine hostname". So something to do with its integration with Slurm.
What can I do to diagnose the problem?