<html>
  <head>

    <meta http-equiv="content-type" content="text/html; charset=UTF-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <p><span lang="EN-AU"></span>
    </p>
    <p class="MsoNormal"><span lang="EN-AU">Hi, </span></p>
    <p class="MsoNormal"><span lang="EN-AU">I am in the process of
        setting up Slurm using Amazon cloud infrastructure. All is going
        well, I can elastically start and stop nodes when jobs run.  I
        am running into a few small teething issues, that are probably
        due to me not understanding some of the terminology here. At a
        high level all the nodes given to end users in the cloud are
        hyper threaded, so I want to use my nodes as hyper threaded
        nodes.  All nodes are running centos7 latest. I would also like
        the jobs to be run in a cgroup and not migrate around after it
        starts. As I said before I think most of it is working except
        for the few issues below here.
      </span></p>
    <p class="MsoNormal"><span lang="EN-AU"> </span></p>
    <p class="MsoNormal"><span lang="EN-AU">My use case is, I have an in
        house built binary application that is single threaded and does
        no  message passing or anything like that. The application is
        not memory bound it is only compute bound.</span></p>
    <p class="MsoNormal"><span lang="EN-AU"> </span></p>
    <p class="MsoNormal"><span lang="EN-AU">So on a node I would like to
        be able to run 16 instances in parallel. As can be seen below if
        I launch the single app via srun it runs on each thread on a
        CPU.  However if I run the via sbatch command as can be seen it
        only runs on CPU 0-7 instead of CPU 0-15. </span></p>
    <p class="MsoNormal"><span lang="EN-AU"> </span></p>
    <p class="MsoNormal"><span lang="EN-AU">Another question would be
        how would be the best way to retry failed jobs, I can rerun the
        batch again, but I only want to rerun a single step in the
        batch?</span></p>
    <p class="MsoNormal"><span lang="EN-AU"> </span></p>
    <p class="MsoNormal"><span lang="EN-AU">Please see below for the
        output of various commands as well as my slurm.conf file as
        well.</span></p>
    <p class="MsoNormal"><span lang="EN-AU"> </span></p>
    <p class="MsoNormal"><span lang="EN-AU">Many thanks</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Nathan. </span></p>
    <p class="MsoNormal"><span lang="EN-AU"> </span></p>
    <p class="MsoNormal"><span lang="EN-AU">______________________________________________________________________</span></p>
    <p class="MsoNormal"><span lang="EN-AU">btuser@bt_slurm_login001[domain
        ]% slurmd -V</span></p>
    <p class="MsoNormal"><span lang="EN-AU">slurm 18.08.6-2</span></p>
    <p class="MsoNormal"><span lang="EN-AU">______________________________________________________________________</span></p>
    <p class="MsoNormal"><span lang="EN-AU">btuser@bt_slurm_login001[domain
        ]% cat nathan.batch.sh</span></p>
    <p class="MsoNormal"><span lang="EN-AU">#!/bin/bash</span></p>
    <p class="MsoNormal"><span lang="EN-AU">#SBATCH
        --job-name=nathan_test</span></p>
    <p class="MsoNormal"><span lang="EN-AU">#SBATCH --ntasks=1</span></p>
    <p class="MsoNormal"><span lang="EN-AU">#SBATCH --array=1-32</span></p>
    <p class="MsoNormal"><span lang="EN-AU">#SBATCH --ntasks-per-core=2</span></p>
    <p class="MsoNormal"><span lang="EN-AU">hostname</span></p>
    <p class="MsoNormal"><span lang="EN-AU">srun --hint=multithread -n1 
        --exclusive   --cpu_bind=threads cat /proc/self/status | grep -i
        cpus_allowed_list</span></p>
    <p class="MsoNormal"><span lang="EN-AU"> </span></p>
    <p class="MsoNormal"><span lang="EN-AU"> </span></p>
    <p class="MsoNormal"><span lang="EN-AU">btuser@bt_slurm_login001[domain
        ]%</span></p>
    <p class="MsoNormal"><span lang="EN-AU">btuser@bt_slurm_login001[domain
        ]% sbatch Nathan.batch.sh</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Submitted batch job 106491</span></p>
    <p class="MsoNormal"><span lang="EN-AU">btuser@bt_slurm_login001[domain
        ]% cat slurm-106491_*</span></p>
    <p class="MsoNormal"><span lang="EN-AU">btuser@bt_slurm_login001[domain
        ]% cat slurm-106491_*</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-89.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      1</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-89.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      2</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-89.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      3</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-89.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      4</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-89.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      5</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-89.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      6</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-89.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      7</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-90.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      0</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-90.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      1</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-90.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      2</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-88.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      0</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-90.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      3</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-90.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      4</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-90.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      5</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-90.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      6</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-90.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      7</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-91.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      0</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-91.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      1</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-91.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      2</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-91.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      3</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-91.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      4</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-88.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      1</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-91.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      5</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-91.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      6</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-91.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      7</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-88.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      2</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-88.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      3</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-88.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      4</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-88.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      5</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-88.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      6</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-88.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      7</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ip-10-0-8-89.ec2.internal</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      0</span></p>
    <p class="MsoNormal"><span lang="EN-AU">______________________________________________________________________</span></p>
    <p class="MsoNormal"><span lang="EN-AU">btuser@bt_slurm_login001[domain
        ]% srun -n32  --exclusive   --cpu_bind=threads cat
        /proc/self/status | grep -i
cpus_allowed_list                                                                                                         </span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      12</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      13</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      15</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      0</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      8</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      1</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      9</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      2</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      10</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      11</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      4</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      5</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      6</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      14</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      7</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      3</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      0</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      8</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      1</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      9</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      2</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      10</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      3</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      11</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      4</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      12</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      5</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      6</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      14</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      7</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      15</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cpus_allowed_list:      13</span></p>
    <p class="MsoNormal"><span lang="EN-AU">______________________________________________________________________</span></p>
    <p class="MsoNormal"><span lang="EN-AU">[sysadmin@ip-10-0-8-88 ~]$
        slurmd -C</span></p>
    <p class="MsoNormal"><span lang="EN-AU">NodeName=ip-10-0-8-88
        CPUs=16 Boards=1 SocketsPerBoard=1 CoresPerSocket=8
        ThreadsPerCore=2 RealMemory=30986</span></p>
    <p class="MsoNormal"><span lang="EN-AU">UpTime=0-00:06:10</span></p>
    <p class="MsoNormal"><span lang="EN-AU">______________________________________________________________________</span></p>
    <p class="MsoNormal"><span lang="EN-AU">Cloud server stats:</span></p>
    <p class="MsoNormal"><span lang="EN-AU">[sysadmin@ip-10-0-8-88 ~]$
        lscpu  -e</span></p>
    <p class="MsoNormal"><span lang="EN-AU">CPU NODE SOCKET CORE
        L1d:L1i:L2:L3 ONLINE</span></p>
    <p class="MsoNormal"><span lang="EN-AU">0   0    0      0   
        0:0:0:0       yes</span></p>
    <p class="MsoNormal"><span lang="EN-AU">1   0    0      1   
        1:1:1:0       yes</span></p>
    <p class="MsoNormal"><span lang="EN-AU">2   0    0      2   
        2:2:2:0       yes</span></p>
    <p class="MsoNormal"><span lang="EN-AU">3   0    0      3   
        3:3:3:0       yes</span></p>
    <p class="MsoNormal"><span lang="EN-AU">4   0    0      4   
        4:4:4:0       yes</span></p>
    <p class="MsoNormal"><span lang="EN-AU">5   0    0      5   
        5:5:5:0       yes</span></p>
    <p class="MsoNormal"><span lang="EN-AU">6   0    0      6   
        6:6:6:0       yes</span></p>
    <p class="MsoNormal"><span lang="EN-AU">7   0    0      7   
        7:7:7:0       yes</span></p>
    <p class="MsoNormal"><span lang="EN-AU">8   0    0      0   
        0:0:0:0       yes</span></p>
    <p class="MsoNormal"><span lang="EN-AU">9   0    0      1   
        1:1:1:0       yes</span></p>
    <p class="MsoNormal"><span lang="EN-AU">10  0    0      2   
        2:2:2:0       yes</span></p>
    <p class="MsoNormal"><span lang="EN-AU">11  0    0      3   
        3:3:3:0       yes</span></p>
    <p class="MsoNormal"><span lang="EN-AU">12  0    0      4   
        4:4:4:0       yes</span></p>
    <p class="MsoNormal"><span lang="EN-AU">13  0    0      5   
        5:5:5:0       yes</span></p>
    <p class="MsoNormal"><span lang="EN-AU">14  0    0      6   
        6:6:6:0       yes</span></p>
    <p class="MsoNormal"><span lang="EN-AU">15  0    0      7   
        7:7:7:0       yes</span></p>
    <p class="MsoNormal"><span lang="EN-AU">______________________________________________________________________</span></p>
    <p class="MsoNormal"><span lang="EN-AU"># slurm.conf file generated
        by configurator easy.html.</span></p>
    <p class="MsoNormal"><span lang="EN-AU">SlurmctldHost=bt_slurm_master</span></p>
    <p class="MsoNormal"><span lang="EN-AU">MailProg=/dev/null</span></p>
    <p class="MsoNormal"><span lang="EN-AU">MpiDefault=none</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ProctrackType=proctrack/cgroup</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ReturnToService=1</span></p>
    <p class="MsoNormal"><span lang="EN-AU">SlurmctldPidFile=/var/run/slurmd/slurmctld.pid</span></p>
    <p class="MsoNormal"><span lang="EN-AU">SlurmdPidFile=/var/run/slurmd/slurmd.pid</span></p>
    <p class="MsoNormal"><span lang="EN-AU">SlurmdSpoolDir=/var/spool/slurmctld/slurmd</span></p>
    <p class="MsoNormal"><span lang="EN-AU">SlurmUser=slurm</span></p>
    <p class="MsoNormal"><span lang="EN-AU">StateSaveLocation=/var/spool/slurmctld</span></p>
    <p class="MsoNormal"><span lang="EN-AU">SwitchType=switch/none</span></p>
    <p class="MsoNormal"><span lang="EN-AU">TaskPlugin=task/cgroup</span></p>
    <p class="MsoNormal"><span lang="EN-AU">TaskPluginParam=Threads</span></p>
    <p class="MsoNormal"><span lang="EN-AU">SlurmdTimeout=500</span></p>
    <p class="MsoNormal"><span lang="EN-AU">FastSchedule=1</span></p>
    <p class="MsoNormal"><span lang="EN-AU">SchedulerType=sched/backfill</span></p>
    <p class="MsoNormal"><span lang="EN-AU">SelectType=select/cons_res</span></p>
    <p class="MsoNormal"><span lang="EN-AU">SelectTypeParameters=CR_CPU</span></p>
    <p class="MsoNormal"><span lang="EN-AU">AccountingStorageType=accounting_storage/none</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ClusterName=simplecluster</span></p>
    <p class="MsoNormal"><span lang="EN-AU">JobAcctGatherType=jobacct_gather/none</span></p>
    <p class="MsoNormal"><span lang="EN-AU">PropagatePrioProcess=2</span></p>
    <p class="MsoNormal"><span lang="EN-AU">MaxTasksPerNode=16</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ResumeProgram=/bt/admin/slurm/etc/slurm_ec2_startup.sh</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ResumeTimeout=900  </span></p>
    <p class="MsoNormal"><span lang="EN-AU">ResumeRate=0             </span></p>
    <p class="MsoNormal"><span lang="EN-AU">SuspendProgram=/bt/admin/slurm/etc/slurm_ec2_shutdown.sh</span></p>
    <p class="MsoNormal"><span lang="EN-AU">SuspendTime=600       </span></p>
    <p class="MsoNormal"><span lang="EN-AU">SuspendTimeout=120 </span></p>
    <p class="MsoNormal"><span lang="EN-AU">TreeWidth=1024  </span></p>
    <p class="MsoNormal"><span lang="EN-AU">SuspendRate=0     </span></p>
    <p class="MsoNormal"><span lang="EN-AU">NodeName=ip-10-0-8-[2-100]
        CPUs=16 RealMemory=27648 Sockets=1 CoresPerSocket=8
        ThreadsPerCore=2  State=CLOUD</span></p>
    <p class="MsoNormal"><span lang="EN-AU">NodeName=bt_slurm_login00[1-10]
        RealMemory=512 State=DOWN</span></p>
    <p class="MsoNormal"><span lang="EN-AU">PartitionName=backtest
        Nodes=ip-10-0-8-[2-100] Default=YES MaxTime=300 Oversubscribe=NO
        State=UP</span></p>
    <p class="MsoNormal"><span lang="EN-AU">                                      
      </span></p>
    <p class="MsoNormal"><span lang="EN-AU">______________________________________________________________________</span></p>
    <p class="MsoNormal"><span lang="EN-AU"> </span></p>
    <p class="MsoNormal"><span lang="EN-AU">[sysadmin@bt_slurm_master
        ~]$ cat /etc/slurm/cgroup.conf</span></p>
    <p class="MsoNormal"><span lang="EN-AU">###</span></p>
    <p class="MsoNormal"><span lang="EN-AU">#</span></p>
    <p class="MsoNormal"><span lang="EN-AU"># Slurm cgroup support
        configuration file</span></p>
    <p class="MsoNormal"><span lang="EN-AU">#</span></p>
    <p class="MsoNormal"><span lang="EN-AU"># See man slurm.conf and man
        cgroup.conf for further</span></p>
    <p class="MsoNormal"><span lang="EN-AU"># information on cgroup
        configuration parameters</span></p>
    <p class="MsoNormal"><span lang="EN-AU">#--</span></p>
    <p class="MsoNormal"><span lang="EN-AU">CgroupAutomount=yes</span></p>
    <p class="MsoNormal"><span lang="EN-AU">CgroupMountpoint="/sys/fs/cgroup"</span></p>
    <p class="MsoNormal"><span lang="EN-AU">TaskAffinity=yes</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ConstrainCores=yes</span></p>
    <p class="MsoNormal"><span lang="EN-AU">ConstrainRAMSpace=no</span></p>
    <p class="MsoNormal"><span lang="EN-AU"> </span></p>
    <p class="MsoNormal"><span lang="EN-AU">______________________________________________________________________</span></p>
    <p class="MsoNormal"><span lang="EN-AU"> </span></p>
    <p class="MsoNormal"><span lang="EN-AU"> </span></p>
    <p class="MsoNormal"><span lang="EN-AU">JobId=107424
        ArrayJobId=107423 ArrayTaskId=1 JobName=aijing_test</span></p>
    <p class="MsoNormal"><span lang="EN-AU">   UserId=btuser(1001)
        GroupId=users(100) MCS_label=N/A</span></p>
    <p class="MsoNormal"><span lang="EN-AU">   Priority=4294901612
        Nice=0 Account=(null) QOS=(null)</span></p>
    <p class="MsoNormal"><span lang="EN-AU">   JobState=COMPLETED
        Reason=None Dependency=(null)</span></p>
    <p class="MsoNormal"><span lang="EN-AU">   Requeue=1 Restarts=0
        BatchFlag=1 Reboot=0 ExitCode=0:0</span></p>
    <p class="MsoNormal"><span lang="EN-AU">   DerivedExitCode=0:0</span></p>
    <p class="MsoNormal"><span lang="EN-AU">   RunTime=00:01:06
        TimeLimit=05:00:00 TimeMin=N/A</span></p>
    <p class="MsoNormal"><span lang="EN-AU">  
        SubmitTime=2019-05-17T01:55:56 EligibleTime=2019-05-17T01:55:56</span></p>
    <p class="MsoNormal"><span lang="EN-AU">   AccrueTime=Unknown</span></p>
    <p class="MsoNormal"><span lang="EN-AU">  
        StartTime=2019-05-17T01:55:56 EndTime=2019-05-17T01:57:02
        Deadline=N/A</span></p>
    <p class="MsoNormal"><span lang="EN-AU">   PreemptTime=None
        SuspendTime=None SecsPreSuspend=0</span></p>
    <p class="MsoNormal"><span lang="EN-AU">  
        LastSchedEval=2019-05-17T01:55:56</span></p>
    <p class="MsoNormal"><span lang="EN-AU">   Partition=backtest
        AllocNode:Sid=bt_slurm_login001:14002</span></p>
    <p class="MsoNormal"><span lang="EN-AU">  ReqNodeList=(null)
        ExcNodeList=(null)</span></p>
    <p class="MsoNormal"><span lang="EN-AU">   NodeList=ip-10-0-8-88</span></p>
    <p class="MsoNormal"><span lang="EN-AU">   BatchHost=ip-10-0-8-88</span></p>
    <p class="MsoNormal"><span lang="EN-AU">   NumNodes=1 NumCPUs=1
        NumTasks=1 CPUs/Task=1 ReqB:S:C:T=0:0:*:*</span></p>
    <p class="MsoNormal"><span lang="EN-AU">  
        TRES=cpu=1,node=1,billing=1</span></p>
    <p class="MsoNormal"><span lang="EN-AU">   Socks/Node=*
        NtasksPerN:B:S:C=0:0:*:* CoreSpec=*</span></p>
    <p class="MsoNormal"><span lang="EN-AU">     Nodes=ip-10-0-8-88
        CPU_IDs=0-1 Mem=0 GRES_IDX=</span></p>
    <p class="MsoNormal"><span lang="EN-AU">   MinCPUsNode=1
        MinMemoryNode=0 MinTmpDiskNode=0</span></p>
    <p class="MsoNormal"><span lang="EN-AU">   Features=(null)
        DelayBoot=00:00:00</span></p>
    <p class="MsoNormal"><span lang="EN-AU">   OverSubscribe=OK
        Contiguous=0 Licenses=(null) Network=(null)</span></p>
    <p class="MsoNormal"><span lang="EN-AU">  
        Command=/bt/data/backtester/destination_tables_94633/batch_run.sh</span></p>
    <p class="MsoNormal"><span lang="EN-AU">  
        WorkDir=/bt/data/backtester/destination_tables_94633</span></p>
    <p class="MsoNormal"><span lang="EN-AU">  
        StdErr=/bt/data/backtester/destination_tables_94633/slurm-107423_1.out</span></p>
    <p class="MsoNormal"><span lang="EN-AU">   StdIn=/dev/null</span></p>
    <p class="MsoNormal"><span lang="EN-AU">  
        StdOut=/bt/data/backtester/destination_tables_94633/slurm-107423_1.out</span></p>
    <p class="MsoNormal"><span lang="EN-AU">   Power=</span></p>
    <p class="MsoNormal"><span lang="EN-AU"> </span></p>
  </body>
</html>