Submitting multiple qsub jobs


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Submitting multiple qsub jobs
# 1  
Old 10-30-2015
Submitting multiple qsub jobs

Hi all,
Today, I want to ask how to submit multiple qsub jobs.

I want to submit 100 .sh files for the simulations.
The name of files is
run_001.sh,
run_002.sh,
run_003.sh,
.....
.....
run_100.sh

Submitting each file manually are time-consuming, hence, I want to make another .sh file (runAllFile.sh) for just submitting those files at once.
The code of runAllFile.sh is as follows:

--------------------------------------------------------
Code:
#!/usr/bin/tcsh

set currDir = `pwd`

foreach i0 (run_*.sh)
	qsub -d ${currDir} ${i0}
end

--------------------------------------------------------

I think if I submit the runAllFile.sh file, it will submit all run_*.sh files to the machine.
However, I have received an error when I try to submit runAllFile.sh by "qsub runAllFile.sh".

This is an error message:
Code:
-bash: /usr/bin/tcsh: bad interpreter: No such file or directory

Could you please help me solve this problem?


Thanks for your time and consideration.

Last edited by Don Cragun; 10-30-2015 at 06:58 PM.. Reason: Add CODE and ICODE tags.
# 2  
Old 10-30-2015
Please use code tags as required by forum rules!

You seem to be running the bash shell. The script's shebang requests the tcsh shell. Does the file /usr/bin/tcsh exit?

Why don't you use bash to run all the .sh files?
# 3  
Old 10-30-2015
You could also get an error message like that if you created runAllFile.sh with an editor that uses DOS (<carriage-return><newline>) line terminators instead of UNIX/Linux (<newline>) line terminators.

So, if tcsh on your system is in /usr/bin/tcsh (instead of /bin/tcsh where it is located on many systems) try:
Code:
od -c runAllFile.sh

and look for characters displayed as \r.
# 4  
Old 10-31-2015
Thanks for your kind reply.
I can solve the issue by changing "
Code:
/usr/bin/tcsh

" to "
Code:
/bin/tcsh

"

However, I received another error message.

Code:
line 5: syntax error near unexpected token `('
line 5: `foreach i0 (run_*.sh)'

I think my code looks correct, so that it is hard to find the reason.



Thanks.
# 5  
Old 10-31-2015
You dont need the () there, unless you encapsule ls like $(ls run_*.sh), which would be unnecesary.
Not sure if it would require a leading ./run_*.sh.

hth
# 6  
Old 10-31-2015
No no, if this is tcsh, foreach needs the brackets!
And the syntax looks correct to me.
Code:
tcsh --version

?
# 7  
Old 11-01-2015
Thanks for your valuable comments.

I found that, if this is "bash", "foreach" does not work.
I use "for" instead.
Hence, I have modified the code as follows:
Code:
#!/bin/bash

set currDir = `pwd`

for i0 in 'run_*.sh'
do
	qsub -d ${currDir} ${i0}
done

Nonetheless, I have received an error:
Code:
line 7: qsub: command not found


Could you please help me solve this problem?




Thank you for your time and consideration.

Last edited by syg3434; 11-01-2015 at 12:52 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to run multiple cron jobs?

I have two scripts which I'm tying to run one after the other- this is what I've tried: 00 14 * * * /path/one.sh && /path/two.sh I've also tried putting each script on a different line: 00 14 * * * /path/one.sh 00 14 * * * /path/two.sh Can this be done? (1 Reply)
Discussion started by: $shell_Learner
1 Replies

2. Shell Programming and Scripting

Shell script to run multiple jobs and it's dependent jobs

I have multiple jobs and each job dependent on other job. Each Job generates a log and If job completed successfully log file end's with JOB ENDED SUCCESSFULLY message and if it failed then it will end with JOB ENDED with FAILURE. I need an help how to start. Attaching the JOB dependency... (3 Replies)
Discussion started by: santoshkumarkal
3 Replies

3. Shell Programming and Scripting

Help submitting jobs to cluster

Hi I am new to submitting jobs. I am trying to submit my perl file to the cluster. This is what my shell file looks like (shell1.sh): #!/bin/sh #$ -S /bin/sh cd data/projects/mydir/abbc perl autocorro.pl followed by qsub shell1.sh It takes the qsub, but does nothing. I check... (1 Reply)
Discussion started by: theawknewbie
1 Replies

4. Shell Programming and Scripting

How to View multiple Cron jobs

Hi, I ran two crontab commands using: crontab program1 crontab program2 However when I type crontab -l only the second cron job shows up, how do I see all cron jobs running and how do I edit all at the same time Thanks in Advance S:D (10 Replies)
Discussion started by: walforum
10 Replies

5. Shell Programming and Scripting

Multiple jobs reading from same file

I have a sequence of tasks that I routinely run and I'm trying to parallelize certain portions of the sequence. Specifically, there are 3 tasks which all read from the same file, each performing different operations and writing to their own seperate file. I was wondering if I could execute these... (3 Replies)
Discussion started by: erichpowell
3 Replies

6. UNIX for Advanced & Expert Users

Problem Running qsub multiple jobs

Hello, I have a perl script that takes in one file of input and outputs to a file. I would like to run this script on several input files, and I would like to run it with qsub, something like this: Input files: FileListDssp.txt.numaa FileListDssp.txt.numab FileListDssp.txt.numac etc.. ... (1 Reply)
Discussion started by: InfoSeeker
1 Replies

7. Shell Programming and Scripting

Multiple jobs in perl

Hello, I would like to write a perl script which executes several jobs. The key thing is I only want 4 jobs to be executed at one time (that's because my machine as 4 cpu, and I want one job per cpu). Is there any way that I can get perl to co-operate with me in this? Thanks! (1 Reply)
Discussion started by: amcrisan
1 Replies

8. Shell Programming and Scripting

monitoring multiple jobs Unix

I have a list of jobs. their expected start time and their expected duration. my needs are monitor multiple jobs (starting at different time and their execution time also differs) Need to mail if any of the job running longer than its expected duration. Whats the efficient way to scripting... (1 Reply)
Discussion started by: vikram3.r
1 Replies

9. UNIX for Advanced & Expert Users

Submitting jobs remotely...Experts help reqd

Is there any way I can submit a job to a remote machine and return immediately without withing for the job to finish? What I mean is this...using rsh I can submit a job to a remote machine like this: rsh remotemac1 job.sh But this doesn't return untill the job has finished and as a... (3 Replies)
Discussion started by: arijit
3 Replies

10. UNIX for Dummies Questions & Answers

.netrc multiple ftp jobs to same machine

I have an ftp user, which has been setup to run ftp jobs to a specific machine (different jobs). for the first job i created .netrc in the ftp users home directory and added the appropriate commands machine FTPBOX01 login user1 password xxx macdef init etc etc get file bye I use the... (3 Replies)
Discussion started by: hcclnoodles
3 Replies
Login or Register to Ask a Question