Sponsored Content
Top Forums UNIX for Advanced & Expert Users Problem Running qsub multiple jobs Post 302376470 by InfoSeeker on Tuesday 1st of December 2009 12:28:59 PM
Old 12-01-2009
Question Problem Running qsub multiple jobs

Hello,
I have a perl script that takes in one file of input and outputs to a file. I would like to run this script on several input files, and I would like to run it with qsub, something like this:

Input files:

FileListDssp.txt.numaa
FileListDssp.txt.numab
FileListDssp.txt.numac
etc..

and code

[tcsh shell]
foreach file ( FileListDssp.txt.num* )
qsub -cwd PerlScript.pl $file
## this outputs to a file Output.txt (command is within PerlScript)
echo DONE $file >>Status.txt ##be aware whether job submitted or not
end

If I do this, I get multiple jobs submitted, which is fine, but the output is jumbled because everything is sent to one file in no particular order.
Accordingly, I want to run a loop that executes each qsub job one at a time (to get the ordered output).

Can anyone kindly advise as to how to use qhold or qrls to prevent a new job from executing until the previous one is done? And how do I incorporate them into the loop?

OR, do I need to run all the jobs and then develop a different loop with qhold and qrls? How would I do that?

Your advice is greatly appreciated!
Thanks,
DG
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

running cron jobs

Hi, I'm new to cron and I would like to know if this cron entry looks right. Every monday to friday, at 9:15 AM execute the script called done_att.sh Every monday to friday, at 3:15 PM execute the script called start_att.sh 15 09 * * /u54/jobs/done_att.sh 45 15 * *... (1 Reply)
Discussion started by: ted
1 Replies

2. UNIX for Dummies Questions & Answers

running cron jobs

I would like to add a job using cron. I have following questions - cron jobs are launched using what login-id e.g. is the script launced using my login-id - In which directory would the cron jobs be launched ? - where do the environment variables required in the script being lauched come... (3 Replies)
Discussion started by: sharanbr
3 Replies

3. UNIX for Dummies Questions & Answers

How to tell which cron jobs are still running?

I have some cron jobs that are taking a might long time. That is because they are screen scrapers and I inserted an extra long pause (10 minutes between screens) to be extra polite and minimize network traffic. I was going to have them run once a day but now I'm thinking once a week. How do I... (2 Replies)
Discussion started by: siegfried
2 Replies

4. UNIX for Advanced & Expert Users

Seriously, cron jobs not running.

Before I get a million and one suggestions on all the things that could be wrong with my scripts - let me assure everyone that this is not the issue. i have a RH EL ES4 server that stopped running all cron jobs except hourly, daily, weekly. This has affected all users including root. No... (7 Replies)
Discussion started by: bill_k_Lopez
7 Replies

5. OS X (Apple)

cron jobs running, but not...

I have a cron job set up to run a script on my Leopard Server every night. The job is set up as root, and when I run the script manually, it runs fine. The problem is that it is not doing anything when it runs at night. In the cron log, it says it ran, but did not output the results to the file it... (3 Replies)
Discussion started by: yodomino6
3 Replies

6. UNIX for Dummies Questions & Answers

at jobs not running

im doing at now <return> touch ~/Desktop/file.txt <ctrl d> and file is never created though it says job 7 at Wed Mar 17 15:15:55 2010 why? (4 Replies)
Discussion started by: glev2005
4 Replies

7. Shell Programming and Scripting

Running jobs in parallel

I need to process 50 sqlplus scripts which are listed in a text file. I need to develop a shell script that'll read this file and run these sqlplus scripts. At any point of time, the number of sqlplus scripts running shouldn't exceed 6. If any of the sqlplus scripts completes successfully then... (17 Replies)
Discussion started by: gctex
17 Replies

8. UNIX for Dummies Questions & Answers

Execution problem in running multiple scripts

hi all, I have 3 individual scripts to perform the task . 2nd script should run only after the 1st script and 3rd script must run only after first 2 scripts are executed successfully. i want to have a single script that calls all this 3 scripts .this single script should execute the 2nd script... (1 Reply)
Discussion started by: Rahul619
1 Replies

9. UNIX for Advanced & Expert Users

Submitting multiple qsub jobs

Hi all, Today, I want to ask how to submit multiple qsub jobs. I want to submit 100 .sh files for the simulations. The name of files is run_001.sh, run_002.sh, run_003.sh, ..... ..... run_100.sh Submitting each file manually are time-consuming, hence, I want to make another .sh file... (11 Replies)
Discussion started by: syg3434
11 Replies

10. Shell Programming and Scripting

Shell script to run multiple jobs and it's dependent jobs

I have multiple jobs and each job dependent on other job. Each Job generates a log and If job completed successfully log file end's with JOB ENDED SUCCESSFULLY message and if it failed then it will end with JOB ENDED with FAILURE. I need an help how to start. Attaching the JOB dependency... (3 Replies)
Discussion started by: santoshkumarkal
3 Replies
CONDOR_QSUB(1)							   User Commands						    CONDOR_QSUB(1)

NAME
condor_qsub - minimalistic qsub emulation for Condor SYNOPSIS
condor_qsub [OPTIONS] [<command> [<command_args>]] DESCRIPTION
Minimalistic emulation of SGE's qsub for Condor. The primary purpose of this emulation is to allow SGE-style submission of dependent jobs without the need to specify the full dependency graph at submission time. This implementation is neither as efficient as Condor's DAGMan, nor as functional as SGE's qsub/qalter. It merely serves as a minimal adaptor to be able to use software original written to interact with SGE in a Condor pool. In general condor_qsub behaves just like qsub. However, only a fraction of the original functionality is available. The following list of options only describes the differences in the behavior of SGE's qsub and this emulation. Qsub options not listed here are not supported. OPTIONS
-b <y|n> If 'y', command and arguments given on the command line are wrapped into a shell script which is then submitted to Condor. --condor-keep-files This is a non-SGE option. If given, it will prevent condor_qsub from deleting temporary files (generated submit files, sentinel jobs). This is mostly useful for debugging. -cwd If given, this option will cause the 'initialdir' value in the Condor submit file to be set to the current directory. -e <filename|path> Name of the file to contain the STDERR output of the job. By default this will be job_name.ejob_id[.task_id]. If an existing direc- tory is specified, the file will be placed inside this directory using the default schema for the filename. -h,--help Print usage summary and option list. -hold_jid <jid> If given, the job will be submitted in 'hold' state. Along with the actual job a 'sentinel' job will be submitted to Condor's local universe. This sentinel watches the specified job and releases the submitted job whenever the job has completed. The sentinel observes SGE's behavior to detect job exiting with code 100 and not start depedent job in this case. If a cluster id of an array job is given the dependent job will only be released after all individual jobs of a cluster have completed. -l <ressource spec> This option is currently ignored. -m <a|e|n><...> SGE's notification labels will be translated (approximately) into Condor's notifications states (Never, Error, Complete). -M <email> Added as 'notify_user' to the submit file. -N <jobname> Determines the default name of logfile (stdout, stderr). -o <filename|path> See -e option, but for a job's stdout. -p <int> Added a 'priority' to the submit file. -r <y|n> This option is currently ignored. -S <shell> Path to a shell binary for script execution. -shell <y|n> This option is currently ignored. -t <start>[-<stop>[:<step>]] Task ID specification for array job submissions. -q <queue name> This option is permanently ignored, as Condor doesn't have multiple queues. -V If given, 'getenv = True' is added to the submit file. --version Print version information and exit. This script is released under the Apache V2.0 License. It was written by Michael Hanke <michael.hanke@gmail.com>. condor_qsub 0.1 September 2012 CONDOR_QSUB(1)
All times are GMT -4. The time now is 06:07 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy