Sponsored Content
Top Forums Shell Programming and Scripting Creating script with multiple job arrays Post 303026243 by idbemad on Thursday 22nd of November 2018 08:39:22 AM
Old 11-22-2018
Creating script with multiple job arrays

Hello everyone,

First of all this is my first post and im fairly new to working with Unix and creating scripts etc. so there will probably be wrong phrases used.

Lets get to my questions.

I have multiple scripts that submit Slurms/Jobs to the cluster starting like this and doing certain tasks:

Code:
#!/bin/bash
#SBATCH --partition=
#SBATCH --ntasks=
#SBATCH --time=0-10:00
#SBATCH --mem-per-cpu=48G
#SBATCH --job-name=basic_test
#SBATCH --mail-type=END,FAIL
#SBATCH --mail-user=           
#SBATCH --array=1-14

...


I have to run multiple of those scripts AFTER all the Slurms from the preceeding job are finished. Now I want to write a script that does exactly that. I already tried something like this:

Code:
#!/bin/bash

sbatch ~/testscript1.sh

wait

sbatch ~/testscript2.sh


the problem seems to be that the skript thinks that testscript1.sh is done as soon as it submitted all the slurms in the array to the server and then it starts testscript2.sh. This leads to problems because testscript2.sh works with some files that are only created when the submitted slurms from testscript1.sh are done.

Thank you for your help

Last edited by Scrutinizer; 11-22-2018 at 09:53 AM.. Reason: code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

shell: creating different arrays based on function argument

hi, I was wondering if there was a good way to create an array within a function, where the name is based on a passed argument? I tried this: _____________________________ func(){ #take in 1st arg as the arrayname arrayName=$1 let i=0 while read line do arrayName=${line} let i+=1... (5 Replies)
Discussion started by: nix21
5 Replies

2. Shell Programming and Scripting

Creating an unknown number of arrays

I need to create arrays like this: cnt=0 { while read myline; do if ];then firstpass="${myline##<meas>}" meas="${firstpass%%</meas>}" tempmeas="${meas%%;*}" MEAS$cnt=$tempmeas print $cnt print ${MEAS'$cnt'} ... (2 Replies)
Discussion started by: ajgwin
2 Replies

3. Shell Programming and Scripting

Joining two arrays and then creating a variable

Hello all... I'd like to create a variable from an array element from two arrays. In my search for answers I found this code for bash that joins two arrays and then started to work with it. I had got to work once and then foolishly without saving the code, I started to edit it for ksh and... (4 Replies)
Discussion started by: carlos25
4 Replies

4. Shell Programming and Scripting

Multiple instances of the job in shell script.

Hi, Please let us know how to create a multiple instances of a job in the shell script. Thanks. Gangegowda K.G (1 Reply)
Discussion started by: Gangegowda
1 Replies

5. Shell Programming and Scripting

Need help in creating arrays using shell

Hi, I need help in creating a array in shell scirpt. I have a file which has following details. hostname devices device1 device 2 de abcdmhs10 1234 2343 2353 3343 3435 2343 bcdfmhs11 2343 2443 3434 8874 0343 3434 (5 Replies)
Discussion started by: jpkumar10
5 Replies

6. Shell Programming and Scripting

creating multiple sub-/directories using a shell script

0 Hi, I am looking for a way of creating multiple directories using the mkdir -p command in a shell script. I'm working with an Ubuntu machine and try to do something like that: #!/bin/sh ... (3 Replies)
Discussion started by: frymor
3 Replies

7. Shell Programming and Scripting

How to write bash script for creating user on multiple Linux hosts?

I wonder whether someone can help me with what I'm trying to achieve Basically, the objective is one script to create new user on more than 70 linux hosts if required. Everything works apart from the highlighted part. It gave me an output passwd: Unknown user name ''. when try to set... (35 Replies)
Discussion started by: fugeulu
35 Replies

8. Shell Programming and Scripting

How to put the multiple job on-hold using shell script?

Hi.. I need to put multiple jobs on ON HOLD in Autosys.. please help me on this. For Example: 1)ABCD_EFGH_IJKL 2)abcd_efgh_ijkl (2 Replies)
Discussion started by: Maanjesh
2 Replies

9. Shell Programming and Scripting

Shell script for creating multiple users with password

for UserName in `cat users` ; do useradd -d /u02 -s /usr/libexec/openssh/sftp-server -G ftp-users $UserName ; PassWord=$( echo $( tr '' '' <<< ${UserName:0:1} )${UserName:1} ) ; echo "$PassWord@123" | passwd $UserName --stdin ; done can some one explain what the bold text do Please use... (5 Replies)
Discussion started by: James0806
5 Replies

10. Shell Programming and Scripting

Creating a script to download images from multiple cameras using ghoto2

I should start by saying that I am totally new to linux... I am trying to create a script that downloads images from multiple cameras into a specific folder on my machine. Ideally renaming the images as they are downloaded. I have installed gphoto2 which as a command line interface that... (0 Replies)
Discussion started by: benpems
0 Replies
init.d(4)                                                                                                                                init.d(4)

NAME
init.d - initialization and termination scripts for changing init states SYNOPSIS
/etc/init.d /etc/init.d is a directory containing initialization and termination scripts for changing init states. These scripts are linked when appro- priate to files in the rc?.d directories, where `?' is a single character corresponding to the init state. See init(1M) for definitions of the states. The service management facility (see smf(5)) is the preferred mechanism for service initiation and termination. The init.d and rc?.d direc- tories are obsolete, and are provided for compatibility purposes only. Applications launched from these directories by svc.startd(1M) are incomplete services, and will not be restarted on failure. File names in rc?.d directories are of the form [SK]nn<init.d filename>, where S means start this job, K means kill this job, and nn is the relative sequence number for killing or starting the job. When entering a state (init S,0,2,3,etc.) the rc[S0-6] script executes those scripts in /etc/rc[S0-6].d that are prefixed with K followed by those scripts prefixed with S. When executing each script in one of the /etc/rc[S0-6] directories, the /sbin/rc[S0-6] script passes a single argument. It passes the argument 'stop' for scripts prefixed with K and the argument 'start' for scripts prefixed with S. There is no harm in applying the same sequence number to multiple scripts. In this case the order of execution is deterministic but unspecified. Guidelines for selecting sequence numbers are provided in README files located in the directory associated with that target state. For example, /etc/rc[S0-6].d/README. Absence of a README file indicates that there are currently no established guidelines. Do not put /etc/init.d in your $PATH. Having this directory in your $PATH can cause unexpected behavior. The programs in /etc/init.d are associated with init state changes and, under normal circumstances, are not intended to be invoked from a command line. Example 1: Example of /sbin/rc2. When changing to init state 2 (multi-user mode, network resources not exported), /sbin/rc2 is initiated by the svc.startd(1M) process. The following steps are performed by /sbin/rc2. 1. In the directory /etc/rc2.d are files used to stop processes that should not be running in state 2. The filenames are prefixed with K. Each K file in the directory is executed (by /sbin/rc2) in alphanumeric order when the system enters init state 2. See example below. 2. Also in the rc2.d directory are files used to start processes that should be running in state 2. As in Step 1, each S file is executed. Assume the file /etc/init.d/netdaemon is a script that will initiate networking daemons when given the argument 'start', and will terminate the daemons if given the argument 'stop'. It is linked to /etc/rc2.d/S68netdaemon, and to /etc/rc0.d/K67netdaemon. The file is executed by /etc/rc2.d/S68netdaemon start when init state 2 is entered and by /etc/rc0.d/K67netdaemon stop when shutting the system down. svcs(1), init(1M), svc.startd(1M), svccfg(1M), smf(5) Solaris now provides an expanded mechanism, which includes automated restart, for applications historically started via the init script mechanism. The Service Management Facility (introduced in smf(5)) is the preferred delivery mechanism for persistently running applica- tions. Existing init.d scripts will, however, continue to be executed according to the rules in this manual page. The details of execution in relation to managed services are available in svc.startd(1M). On earlier Solaris releases, a script named with a suffix of '.sh' would be sourced, allowing scripts to modify the environment of other scripts executed later. This behavior is no longer supported; for altering the environment in which services are run, see the setenv sub- command in svccfg(1M). /sbin/rc2 has references to the obsolescent rc.d directory. These references are for compatibility with old INSTALL scripts. New INSTALL scripts should use the init.d directory for related executables. The same is true for the shutdown.d directory. 17 Aug 2005 init.d(4)
All times are GMT -4. The time now is 02:01 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy