11-20-2008
Writing a script to work sequentially with bsub
Hi,
I am REALLY a newbie of Unix: hope this question is not too odd.
Here is my problem:
I have to submit jobs to a remote platform (an IBM BCX/5120).
To submit I have to
bsub < filename.jcf
and the .jcf file has usually the following content:
#!/bin/bash
#BSUB <commands of the bsub>
/usr/local/stata/stata -b //stata is StataMP for statistical analysis
Unfortunately, each job cannot last more than 6 hours (usually each last around 4 hours), and I would love to make them start sequentially:
any idea about how could I do it?
If you can help me, you'll save my sleep: I won't wake up every 6 hours in order to make the jobs work.
Moreover, since the weekend is approaching and my girlfriend would get pretty angry if I don't dedicate her enough time during the following 3 days, you'll probably save even my relationship!
Thank you very much,
f
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Howdie everyone...
I have a shell script RemoveFiles.sh
Inside this file, it only has two commands as below:
rm -f ../../reportToday/temp/*
rm -f ../../report/*
My problem is that when i execute this script, nothing happened. Files remained unremoved. I don't see any error message as it... (2 Replies)
Discussion started by: cheongww
2 Replies
2. Shell Programming and Scripting
Hi,
I want to sort my files under a directory and execute them sequentially.
For ex:
my diir contains files: a_5.sql, ab_2.sql,abc_3.sql, acbd_1 ,ab_4.sql, etc. I want to execute the files based on the last number after underscore sequentially(i.e.. _1,_2,etc) .
Can anybody help me? (5 Replies)
Discussion started by: MuraliKrisna
5 Replies
3. Shell Programming and Scripting
Can a script that calls another script exit and end while the called script is still running? If so, how can this be made to happen? (1 Reply)
Discussion started by: Harleyrci
1 Replies
4. UNIX for Dummies Questions & Answers
Yesterday I was testing out some cron commands and when i was check the /var/log/cron with vi editor I accidentally did :wq instead. Since then the log has not updated and I'm assuming no cronjob is running because looking at the log there's activities everyday
there's some default stuff that... (5 Replies)
Discussion started by: orionoreo
5 Replies
5. Shell Programming and Scripting
Hi!
I am currently using the bsub command to submit jobs to a cluster and I need to specify the name of the node to which i want the job sent. What do I add to my .pbs script to specify this?
Cheers (0 Replies)
Discussion started by: mancino
0 Replies
6. Shell Programming and Scripting
Hi,
I'm trying to write a script where i'm trying to grep the PID and the associated file and list them. Then execute the KILL command sequentially on the listed PID's for ".tra" files
====================================================
ps -aux | grep mine
adm 27739 0.2 0.8 1131588... (12 Replies)
Discussion started by: murali1687
12 Replies
7. Shell Programming and Scripting
My script work on Linux but not work in sun os.
my script.
logFiles="sentLog1.log sentLog2.log"
intial_time="0 0"
logLocation="/usr/local/tomcat/logs/"
sleepTime=600
failMessage=":: $(tput bold)Log not update$(tput rmso) = "
successMessage="OK"
arr=($logFiles)... (7 Replies)
Discussion started by: ooilinlove
7 Replies
8. Shell Programming and Scripting
Hi,
The problem detail is follows
I have three individual scripts .
SCRIPT A
sh -x sqoop_channels_nc_daily_01.sh & sh -x sqoop_channels_nc_daily_02.sh & sh -x sqoop_channels_nc_daily_03.sh
SCRIPT B
sh -x sqoop_contacts_nc_daily_01.sh & sh -x sqoop_contacts_nc_daily_02.sh & sh -x... (1 Reply)
Discussion started by: H_bansal
1 Replies
9. UNIX for Dummies Questions & Answers
Hi everyone,
I must write a Linux script for a biological analysis. I have an example for that which involves the "bsub" and "bsub-o" command, respectively. I cannot find a description for this command. Can you please help me? (2 Replies)
Discussion started by: Max_W
2 Replies
10. Shell Programming and Scripting
Hello,
When I'm trying to send bsub job using script that executes fine in command line, I get nothing.
I do this for testing purposes, and the script scr_test is just one line:
pwd > outfile
So when I'm executing it in command line:
$ ./scr_test
it works fine producing the outfile with... (2 Replies)
Discussion started by: Sergey Aliev
2 Replies
LEARN ABOUT DEBIAN
cd-hit-2d-para
CD-HIT-2D-PARA.PL(1) User Commands CD-HIT-2D-PARA.PL(1)
NAME
cd-hit-2d-para.pl - divide a big clustering job into pieces to run cd-hit-2d or cd-hit-est-2d jobs
SYNOPSIS
cd-hit-2d-para.pl options
DESCRIPTION
This script divide a big clustering job into pieces and submit jobs to remote computers over a network to make it parallel. After
all the jobs finished, the script merge the clustering results as if you just run a single cd-hit-2d or cd-hit-est-2d.
You can also use it to divide big jobs on a single computer if your computer does not have enough RAM (with -L option).
Requirements:
1 When run this script over a network, the directory where you
run the scripts and the input files must be available on all the remote hosts with identical path.
2 If you choose "ssh" to submit jobs, you have to have
passwordless ssh to any remote host, see ssh manual to know how to set up passwordless ssh.
3 I suggest to use queuing system instead of ssh,
I currently support PBS and SGE
4 cd-hit-2d cd-hit-est-2d cd-hit-div cd-hit-div.pl must be
in same directory where this script is in.
Options
-i input filename for 1st db in fasta format, required
-i2 input filename for 2nd db in fasta format, required
-o output filename, required
--P program, "cd-hit-2d" or "cd-hit-est-2d", default "cd-hit-2d"
--B filename of list of hosts, requred unless -Q or -L option is supplied
--L number of cpus on local computer, default 0 when you are not running it over a cluster, you can use this option to divide a big
clustering jobs into small pieces, I suggest you just use "--L 1" unless you have enough RAM for each cpu
--S Number of segments to split 1st db into, default 2
--S2 Number of segments to split 2nd db into, default 8
--Q number of jobs to submit to queue queuing system, default 0 by default, the program use ssh mode to submit remote jobs
--T type of queuing system, "PBS", "SGE" are supported, default PBS
--R restart file, used after a crash of run
-h print this help
More cd-hit-2d/cd-hit-est-2d options can be speicified in command line
Questions, bugs, contact Weizhong Li at liwz@sdsc.edu
cd-hit-2d-para.pl 4.6-2012-04-25 April 2012 CD-HIT-2D-PARA.PL(1)