Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Writing single script with mutiple sbatch parts
Top Forums Shell Programming and Scripting Writing single script with mutiple sbatch parts Post 303026357 by idbemad on Monday 26th of November 2018 07:08:12 AM
Old 11-26-2018
Writing single script with mutiple sbatch parts
Hello,

I wrote several scripts that work with job arrays (sbatch) that look like this:


Code:
#!/bin/bash
#SBATCH --partition=carl.p
#SBATCH --ntasks=4
#SBATCH --time=0-10:00
#SBATCH --mem-per-cpu=48G
#SBATCH --job-name=basic_test
#SBATCH --mail-type=END,FAIL
#SBATCH --mail-user=
#SBATCH --array=1-48



These scripts all do different things and some need to be finished before other scripts can start.

I already figured out how to write a jobdependency script that times which scripts need to be started and which need to wait until specific scripts are finished. Now I want to know if it is possible to combine all those different scripts into one big script. I want to do this because I dont want to copy all the different scripts everytime i want to use them and I also want to use the same variables in the different scripts so that I only have to define them once. This is just to make everything more user friendly.

I wanted to do something like this



Code:
#!/bin/bash
#SBATCH --partition=carl.p
#SBATCH --ntasks=4
#SBATCH --time=0-10:00
#SBATCH --mem-per-cpu=48G
#SBATCH --job-name=basic_test
#SBATCH --mail-type=END,FAIL
#SBATCH --mail-user=
#SBATCH --array=1-12      



Here would be the first script that submits 12 jobs only containing the code up to the next SBATCH input...

CODE
CODE 
CODE
...




           #!/bin/bash
           #SBATCH --partition=carl.p
           #SBATCH --ntasks=4
           #SBATCH --time=0-10:00
           #SBATCH --mem-per-cpu=48G
           #SBATCH --job-name=basic_test
           #SBATCH --mail-type=END,FAIL
           #SBATCH --mail-user=
           #SBATCH --array=1-48           
           #SBATCH --dependency:afterok Job_id_of_first_script
  
            Here would be the second script which needs to run the 48 jobs from the second SBATCH input only once

            CODE
            CODE
            CODE
            ...

and so on.

These scripts should all work seperately so I dont want the first SBATCH to start a new SBATCH for the second job everytime. I have not tried anything to do this and I just wanted to know if this is at all possible.

Thanks in advance
 

10 More Discussions You Might Find Interesting

1. IP Networking

Script to assign mutiple IP's to 1 nic

I need a script to assign multiple IP addresses to a single nic. The machine is running CentOS. Any help would be appreciated. (2 Replies)
Discussion started by: ron310
2 Replies

2. Shell Programming and Scripting

Display mutiple line in single line

Hi All, I had a file called Input.txt, i need to group up in a single line as 1=ttt and the no of lines may vary bewteen the 1=ttt cat Input.txt 1=ttt,2=xxxxxx, 3=4545 44545, 4=66667 7777, 5=77723 1=ttt, 2=xxxxxx, 3=34436 66 3545, 4=66666, 5=ffffff, 6=uuuuuuu 1=ttt, 2=xxxxxx,... (4 Replies)
Discussion started by: manosubsulo
4 Replies

3. Shell Programming and Scripting

writing the content of two files to a single file horizontally

Hi all, assume that i am having two files file.1 and file.2 these are the contents of file.1 these are the contents of file.2 i want these two contents to be written in another single file like this. can you please post your suggestions. Thanks in advance, ... (3 Replies)
Discussion started by: anishkumarv
3 Replies

4. Shell Programming and Scripting

Format Parts of Multiline Section to Single Line

Hello, I have an input file that I need formatted. I was hoping I could use bash to get this done. Title: Kitchen Blender Washer Dishes Title: Bathroom Toilet Sink Title: Bedroom Bed Desired output would be similar to Results("Blender","Washer","Dishes") (1 Reply)
Discussion started by: jl487
1 Replies

5. Shell Programming and Scripting

Need a script to add mutiple printers in solaris box

Hi , i need to configure around 80 printers in a server. can someone please help me with the script. i have a file that has printer name and its ip. like. printer1 1.1.1.1 printer2 0.0.0.0 and so on.. can some one please help me to do it via script. i am using solaris 10 ... (0 Replies)
Discussion started by: chidori
0 Replies

6. UNIX for Dummies Questions & Answers

awk script combining mutiple commands

Hi, I am pretty new to the unix community and have encountered a problem that I am trying to solve. I have 2 files one of which is called passwd file that looks like the following Sample Output daemon:x:1:1:daemon:/usr/sbin:/bin/sh bin:x:2:2:bin:/bin:/bin/sh sys:x:3:3:sys:/dev:/bin/sh ... (1 Reply)
Discussion started by: raven905
1 Replies

7. Shell Programming and Scripting

Incrementing parts of ten digits number by parts

I have number in file which contains date and serial number: 2013101000. The last two digits are serial number (00). So maximum of serial number is 100. After reaching 100 it becomes 00 with incrementing 10 which is day with max 31. after reaching 31 it becomes 00 and increments 10... (31 Replies)
Discussion started by: Natalie
31 Replies

8. Shell Programming and Scripting

Convert mutiple spaces file to single tab

I have the following file I wanted to convert mutiple spaces to tab: I tried cat filename | tr ' ' '\t' or sed 's/ */ /' FILE but it looses the format 5557263102 5557263102 5552074858 5726310211 5557263102 5557263102 5557263103 5557263103 2142406768 ... (2 Replies)
Discussion started by: amir07
2 Replies

9. Shell Programming and Scripting

awk to remove mutiple values from specific pattern, leaving a single value

In the awk below I am trying to remove all instances after a ; (semi-colon) or , (comma) in the ANN= pattern. I am using gsub to substitute an empty string in these, so that ANN= is a single value (with only one value in it the one right after the ANN=). Thank you :). I have comented my awk and... (11 Replies)
Discussion started by: cmccabe
11 Replies

10. UNIX for Beginners Questions & Answers

How to ignore mutiple strings when using shell script?

Hi All, I am trying to use below syntax to find ignore multiple locations while searching for a file. find / -name "$serviceitem" ! -size 0 2>&1 |egrep -v "tmp|docker|WinSxS|Permission|HISTORY|alternatives|bearer11ssl|manifest" I tried to assign all the ignore strings to one variable... (2 Replies)
Discussion started by: sravani25
2 Replies

LEARN ABOUT MOJAVE

locale::codes::script

Locale::Codes::Script(3pm)				 Perl Programmers Reference Guide				Locale::Codes::Script(3pm)

NAME

       Locale::Codes::Script - standard codes for script identification

SYNOPSIS

	  use Locale::Codes::Script;

	  $script  = code2script('phnx');		      # 'Phoenician'
	  $code    = script2code('Phoenician'); 	      # 'Phnx'
	  $code    = script2code('Phoenician',
				 LOCALE_CODE_NUMERIC);	      # 115

	  @codes   = all_script_codes();
	  @scripts = all_script_names();

DESCRIPTION

       The "Locale::Codes::Script" module provides access to standards codes used for identifying scripts, such as those defined in ISO 15924.

       Most of the routines take an optional additional argument which specifies the code set to use. If not specified, the default ISO 15924
       four-letter codes will be used.

SUPPORTED CODE SETS

       There are several different code sets you can use for identifying scripts. A code set may be specified using either a name, or a constant
       that is automatically exported by this module.

       For example, the two are equivalent:

	  $script = code2script('phnx','alpha');
	  $script = code2script('phnx',LOCALE_SCRIPT_ALPHA);

       The codesets currently supported are:

       alpha, LOCALE_SCRIPT_ALPHA
	   This is a set of four-letter (capitalized) codes from ISO 15924 such as 'Phnx' for Phoenician.  It also includes additions to this set
	   included in the IANA language registry.

	   The Zxxx, Zyyy, and Zzzz codes are not used.

	   This is the default code set.

       num, LOCALE_SCRIPT_NUMERIC
	   This is a set of three-digit numeric codes from ISO 15924 such as 115 for Phoenician.

ROUTINES

       code2script ( CODE [,CODESET] )
       script2code ( NAME [,CODESET] )
       script_code2code ( CODE ,CODESET ,CODESET2 )
       all_script_codes ( [CODESET] )
       all_script_names ( [CODESET] )
       Locale::Codes::Script::rename_script  ( CODE ,NEW_NAME [,CODESET] )
       Locale::Codes::Script::add_script  ( CODE ,NAME [,CODESET] )
       Locale::Codes::Script::delete_script  ( CODE [,CODESET] )
       Locale::Codes::Script::add_script_alias	( NAME ,NEW_NAME )
       Locale::Codes::Script::delete_script_alias  ( NAME )
       Locale::Codes::Script::rename_script_code  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Codes::Script::add_script_code_alias  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Codes::Script::delete_script_code_alias	( CODE [,CODESET] )
	   These routines are all documented in the Locale::Codes::API man page.

SEE ALSO

       Locale::Codes
	   The Locale-Codes distribution.

       Locale::Codes::API
	   The list of functions supported by this module.

       http://www.unicode.org/iso15924/
	   Home page for ISO 15924.

       http://www.iana.org/assignments/language-subtag-registry
	   The IANA language subtag registry.

AUTHOR

       See Locale::Codes for full author history.

       Currently maintained by Sullivan Beck (sbeck@cpan.org).

COPYRIGHT

	  Copyright (c) 1997-2001 Canon Research Centre Europe (CRE).
	  Copyright (c) 2001-2010 Neil Bowers
	  Copyright (c) 2010-2013 Sullivan Beck

       This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

perl v5.18.2							    2013-11-04						Locale::Codes::Script(3pm)
All times are GMT -4. The time now is 06:00 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy