12-01-2009
Problem Running qsub multiple jobs
Hello,
I have a perl script that takes in one file of input and outputs to a file. I would like to run this script on several input files, and I would like to run it with qsub, something like this:
Input files:
FileListDssp.txt.numaa
FileListDssp.txt.numab
FileListDssp.txt.numac
etc..
and code
[tcsh shell]
foreach file ( FileListDssp.txt.num* )
qsub -cwd PerlScript.pl $file
## this outputs to a file Output.txt (command is within PerlScript)
echo DONE $file >>Status.txt ##be aware whether job submitted or not
end
If I do this, I get multiple jobs submitted, which is fine, but the output is jumbled because everything is sent to one file in no particular order.
Accordingly, I want to run a loop that executes each qsub job one at a time (to get the ordered output).
Can anyone kindly advise as to how to use qhold or qrls to prevent a new job from executing until the previous one is done? And how do I incorporate them into the loop?
OR, do I need to run all the jobs and then develop a different loop with qhold and qrls? How would I do that?
Your advice is greatly appreciated!
Thanks,
DG
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
I'm new to cron and I would like to know if this cron entry looks right.
Every monday to friday, at 9:15 AM execute the script called done_att.sh
Every monday to friday, at 3:15 PM execute the script called start_att.sh
15 09 * * /u54/jobs/done_att.sh
45 15 * *... (1 Reply)
Discussion started by: ted
1 Replies
2. UNIX for Dummies Questions & Answers
I would like to add a job using cron. I have following questions
- cron jobs are launched using what login-id
e.g. is the script launced using my login-id
- In which directory would the cron jobs be launched ?
- where do the environment variables required in the script being lauched come... (3 Replies)
Discussion started by: sharanbr
3 Replies
3. UNIX for Dummies Questions & Answers
I have some cron jobs that are taking a might long time. That is because they are screen scrapers and I inserted an extra long pause (10 minutes between screens) to be extra polite and minimize network traffic. I was going to have them run once a day but now I'm thinking once a week.
How do I... (2 Replies)
Discussion started by: siegfried
2 Replies
4. UNIX for Advanced & Expert Users
Before I get a million and one suggestions on all the things that could be wrong with my scripts - let me assure everyone that this is not the issue.
i have a RH EL ES4 server that stopped running all cron jobs except hourly, daily, weekly.
This has affected all users including root. No... (7 Replies)
Discussion started by: bill_k_Lopez
7 Replies
5. OS X (Apple)
I have a cron job set up to run a script on my Leopard Server every night. The job is set up as root, and when I run the script manually, it runs fine.
The problem is that it is not doing anything when it runs at night. In the cron log, it says it ran, but did not output the results to the file it... (3 Replies)
Discussion started by: yodomino6
3 Replies
6. UNIX for Dummies Questions & Answers
im doing at now <return>
touch ~/Desktop/file.txt
<ctrl d>
and file is never created though it says job 7 at Wed Mar 17 15:15:55 2010
why? (4 Replies)
Discussion started by: glev2005
4 Replies
7. Shell Programming and Scripting
I need to process 50 sqlplus scripts which are listed in a text file. I need to develop a shell script that'll read this file and run these sqlplus scripts. At any point of time, the number of sqlplus scripts running shouldn't exceed 6. If any of the sqlplus scripts completes successfully then... (17 Replies)
Discussion started by: gctex
17 Replies
8. UNIX for Dummies Questions & Answers
hi all,
I have 3 individual scripts to perform the task . 2nd script should run only after the 1st script and 3rd script must run only after first 2 scripts are executed successfully.
i want to have a single script that calls all this 3 scripts .this single script should execute the 2nd script... (1 Reply)
Discussion started by: Rahul619
1 Replies
9. UNIX for Advanced & Expert Users
Hi all,
Today, I want to ask how to submit multiple qsub jobs.
I want to submit 100 .sh files for the simulations.
The name of files is
run_001.sh,
run_002.sh,
run_003.sh,
.....
.....
run_100.sh
Submitting each file manually are time-consuming, hence, I want to make another .sh file... (11 Replies)
Discussion started by: syg3434
11 Replies
10. Shell Programming and Scripting
I have multiple jobs and each job dependent on other job.
Each Job generates a log and If job completed successfully log file end's with JOB ENDED SUCCESSFULLY message and if it failed then it will end with JOB ENDED with FAILURE.
I need an help how to start.
Attaching the JOB dependency... (3 Replies)
Discussion started by: santoshkumarkal
3 Replies
LEARN ABOUT CENTOS
crontabs
CRONTABS(4) Crontabs users' Manual CRONTABS(4)
NAME
crontabs - configuration and scripts for running periodical jobs
SYNOPSIS
run-parts [--list|--test]<directory>
DESCRIPTION
Crontabs is a historical name for the run-parts script and the system crontab. The run-parts script runs all executables in the specified
directory. Run-parts runs all executables in the specified directory. The execution of files can be allowed or denied by creating file
jobs.allow or jobs.deny which worked similar as other allow/deny config files. The file must be created in the specified directory.
--list print names of all files (not limited to executables), but don't run them. This option can't be used with test option.
--test print names of files, which would be run.
Randomization of jobs can be configured in the /etc/sysconfig/run-parts file. To enable randomization of jobs, set the RANDOMIZE
parameter to 1 and set the RANDOM parameter to an integer which determines a random seed. Additionally, you may configure the RAN-
DOMTIME parameter (again, by specifying an integer) to provide an additional level of randomization. Jobs are not randomized when
the RANDOM and RANDOMTIME parameters are set to 0. Values in these two parameters must be set to 1 or larger to provide a good
enough randomization.
Randomization of cron jobs can be useful for shared networks, where multiple cron jobs executed at once can cause spikes in traffic,
especially during daily jobs. With randomized jobs, the workload is evenly distributed throughout the day.
EXAMPLE OF CONFIGURATION FILE
RANDOMIZE=1
RANDOM=4
RANDOMTIME=8
Historically the crontab file contained configuration which called run-parts on files in cron.{daily,weekly,monthly} directories. These
jobs are now run indirectly through anacron to prevent conflicts between cron and anacron. That means the anacron package has to be
installed if the jobs in these directories should be running. Refer to the anacron(8) how to limit the time of day of the job execution.
EXAMPLE
/etc/cron.daily/jobs.deny could contain for example 0logwatch which forbid execution of this script.
SEE ALSO
anacron(8), crontab(5)
Marcela Malaova 2012-08-29 CRONTABS(4)