Sponsored Content
Top Forums Shell Programming and Scripting Creating script with multiple job arrays Post 303026243 by idbemad on Thursday 22nd of November 2018 08:39:22 AM
Old 11-22-2018
Creating script with multiple job arrays

Hello everyone,

First of all this is my first post and im fairly new to working with Unix and creating scripts etc. so there will probably be wrong phrases used.

Lets get to my questions.

I have multiple scripts that submit Slurms/Jobs to the cluster starting like this and doing certain tasks:

Code:
#!/bin/bash
#SBATCH --partition=
#SBATCH --ntasks=
#SBATCH --time=0-10:00
#SBATCH --mem-per-cpu=48G
#SBATCH --job-name=basic_test
#SBATCH --mail-type=END,FAIL
#SBATCH --mail-user=           
#SBATCH --array=1-14

...


I have to run multiple of those scripts AFTER all the Slurms from the preceeding job are finished. Now I want to write a script that does exactly that. I already tried something like this:

Code:
#!/bin/bash

sbatch ~/testscript1.sh

wait

sbatch ~/testscript2.sh


the problem seems to be that the skript thinks that testscript1.sh is done as soon as it submitted all the slurms in the array to the server and then it starts testscript2.sh. This leads to problems because testscript2.sh works with some files that are only created when the submitted slurms from testscript1.sh are done.

Thank you for your help

Last edited by Scrutinizer; 11-22-2018 at 09:53 AM.. Reason: code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

shell: creating different arrays based on function argument

hi, I was wondering if there was a good way to create an array within a function, where the name is based on a passed argument? I tried this: _____________________________ func(){ #take in 1st arg as the arrayname arrayName=$1 let i=0 while read line do arrayName=${line} let i+=1... (5 Replies)
Discussion started by: nix21
5 Replies

2. Shell Programming and Scripting

Creating an unknown number of arrays

I need to create arrays like this: cnt=0 { while read myline; do if ];then firstpass="${myline##<meas>}" meas="${firstpass%%</meas>}" tempmeas="${meas%%;*}" MEAS$cnt=$tempmeas print $cnt print ${MEAS'$cnt'} ... (2 Replies)
Discussion started by: ajgwin
2 Replies

3. Shell Programming and Scripting

Joining two arrays and then creating a variable

Hello all... I'd like to create a variable from an array element from two arrays. In my search for answers I found this code for bash that joins two arrays and then started to work with it. I had got to work once and then foolishly without saving the code, I started to edit it for ksh and... (4 Replies)
Discussion started by: carlos25
4 Replies

4. Shell Programming and Scripting

Multiple instances of the job in shell script.

Hi, Please let us know how to create a multiple instances of a job in the shell script. Thanks. Gangegowda K.G (1 Reply)
Discussion started by: Gangegowda
1 Replies

5. Shell Programming and Scripting

Need help in creating arrays using shell

Hi, I need help in creating a array in shell scirpt. I have a file which has following details. hostname devices device1 device 2 de abcdmhs10 1234 2343 2353 3343 3435 2343 bcdfmhs11 2343 2443 3434 8874 0343 3434 (5 Replies)
Discussion started by: jpkumar10
5 Replies

6. Shell Programming and Scripting

creating multiple sub-/directories using a shell script

0 Hi, I am looking for a way of creating multiple directories using the mkdir -p command in a shell script. I'm working with an Ubuntu machine and try to do something like that: #!/bin/sh ... (3 Replies)
Discussion started by: frymor
3 Replies

7. Shell Programming and Scripting

How to write bash script for creating user on multiple Linux hosts?

I wonder whether someone can help me with what I'm trying to achieve Basically, the objective is one script to create new user on more than 70 linux hosts if required. Everything works apart from the highlighted part. It gave me an output passwd: Unknown user name ''. when try to set... (35 Replies)
Discussion started by: fugeulu
35 Replies

8. Shell Programming and Scripting

How to put the multiple job on-hold using shell script?

Hi.. I need to put multiple jobs on ON HOLD in Autosys.. please help me on this. For Example: 1)ABCD_EFGH_IJKL 2)abcd_efgh_ijkl (2 Replies)
Discussion started by: Maanjesh
2 Replies

9. Shell Programming and Scripting

Shell script for creating multiple users with password

for UserName in `cat users` ; do useradd -d /u02 -s /usr/libexec/openssh/sftp-server -G ftp-users $UserName ; PassWord=$( echo $( tr '' '' <<< ${UserName:0:1} )${UserName:1} ) ; echo "$PassWord@123" | passwd $UserName --stdin ; done can some one explain what the bold text do Please use... (5 Replies)
Discussion started by: James0806
5 Replies

10. Shell Programming and Scripting

Creating a script to download images from multiple cameras using ghoto2

I should start by saying that I am totally new to linux... I am trying to create a script that downloads images from multiple cameras into a specific folder on my machine. Ideally renaming the images as they are downloaded. I have installed gphoto2 which as a command line interface that... (0 Replies)
Discussion started by: benpems
0 Replies
cron(1M)						  System Administration Commands						  cron(1M)

NAME
cron - clock daemon SYNOPSIS
/usr/sbin/cron DESCRIPTION
cron starts a process that executes commands at specified dates and times. You can specify regularly scheduled commands to cron according to instructions found in crontab files in the directory /var/spool/cron/crontabs. Users can submit their own crontab file using the crontab(1) command. Commands which are to be executed only once can be submitted using the at(1) command. cron only examines crontab or at command files during its own process initialization phase and when the crontab or at command is run. This reduces the overhead of checking for new or changed files at regularly scheduled intervals. As cron never exits, it should be executed only once. This is done routinely by way of the svc:/system/cron:default service. The file /etc/cron.d/FIFO file is used as a lock file to prevent the execution of more than one instance of cron. cron captures the output of the job's stdout and stderr streams, and, if it is not empty, mails the output to the user. If the job does not produce output, no mail is sent to the user. An exception is if the job is an at(1) job and the -m option was specified when the job was submitted. cron and at jobs are not executed if your account is locked. Jobs and processses execute. The shadow(4) file defines which accounts are not locked and will have their jobs and processes executed. Setting cron Jobs Across Timezones The timezone of the cron daemon sets the system-wide timezone for cron entries. This, in turn, is by set by default system-wide using /etc/default/init. If some form of daylight savings or summer/winter time is in effect, then jobs scheduled during the switchover period could be executed once, twice, or not at all. Setting cron Defaults To keep a log of all actions taken by cron, you must specify CRONLOG=YES in the /etc/default/cron file. If you specify CRONLOG=NO, no log- ging is done. Keeping the log is a user configurable option since cron usually creates huge log files. You can specify the PATH for user cron jobs by using PATH= in /etc/default/cron. You can set the PATH for root cron jobs using SUPATH= in /etc/default/cron. Carefully consider the security implications of setting PATH and SUPATH. Example /etc/default/cron file: CRONLOG=YES PATH=/usr/bin:/usr/ucb: This example enables logging and sets the default PATH used by non-root jobs to /usr/bin:/usr/ucb:. Root jobs continue to use /usr/sbin:/usr/bin. The cron log file is periodically rotated by logadm(1M). FILES
/etc/cron.d Main cron directory /etc/cron.d/FIFO Lock file /etc/default/cron cron default settings file /var/cron/log cron history information /var/spool/cron Spool area /etc/cron.d/queuedefs Queue description file for at, batch, and cron /etc/logadm.conf Configuration file for logadm ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ SEE ALSO
svcs(1), at(1), crontab(1), sh(1), logadm(1M), svcadm(1M), queuedefs(4), shadow(4), attributes(5), smf(5) NOTES
The cron service is managed by the service management facility, smf(5), under the service identifier: svc:/system/cron:default Administrative actions on this service, such as enabling, disabling, or requesting restart, can be performed using svcadm(1M). The ser- vice's status can be queried using the svcs(1) command. DIAGNOSTICS
A history of all actions taken by cron is stored in /var/cron/log and possibly in /var/cron/olog. SunOS 5.10 5 Aug 2004 cron(1M)
All times are GMT -4. The time now is 04:06 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy