Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Request for help with SGE submission script manipulation
Top Forums Shell Programming and Scripting Request for help with SGE submission script manipulation Post 302565685 by crunchgargoyle on Tuesday 18th of October 2011 01:16:23 PM
Old 10-18-2011
[SOLVED] Request for help with Bash script manipulation
Hi,

I have the following SGE submission (HPC calculation) script, which is just a Bash script:

Code:
#!/bin/bash -l

#$ -S /bin/bash
#$ -l h_rt=1:00:0
#$ -l mem=4G
#$ -N XXX
#$ -pe qlc 24
#$ -P XXX
#$ -wd /home/uccaxxx/Scratch/222PdT/3vac/c0001/

mpirun -m $TMPDIR/machines -np $NSLOTS $HOME/bin/new_vasp

What I'd like to do is take the following line within that script:

Code:
#$ -wd /home/uccaxxx/Scratch/222PdT/3vac/c0001/

... looking, specifically, at the 'c0001' element.

I'd then like to, for example, say that I want 101 new scripts like this with only the 'c0001' element in that line changing accordingly. So, I would have 101 scripts where that element changes from c0001 to c0002 ... c0033 ... all the way to c0101.

The script is currently called 'script_0001', and I'd like the new scripts' filenames to change to script_0002 etc. all the way through to script_0101 according to how the c0001 element in the line in question is changed.

Ideally I'd like to produce something that can do what I need in Python or Bash as these are languages I am familiar with.

I started writing something in Bash which fell down and this is what I have in Python so far, which is a long way off and which I have rather hit the wall with:

Code:
#!/usr/bin/python

from __future__ import print_function

with open('scriptnew', mode='w') as outfile:
    with open('script') as infile:
        for line in infile.readlines():
                    if line[4:6]=='-wd':
                            a = int(line[42])
                line = line[42] + int(a)
                        print(line, end='', file=outfile)

Any help (or statements informing me that this not at all simple) would be greatly appreciated.
Last edited by crunchgargoyle; 10-19-2011 at 06:36 AM.. Reason: Solved
 

7 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Help in job submission!

I am attempting to submit alot of jobs to this CONDOR program, in which it utilizes alot of computers and puts a single job onto each. Anyways, im trying to use the same program (autodock) to analyze different input files (dpf files), where each input file has a different name but same file... (0 Replies)
Discussion started by: olifu02
0 Replies

2. UNIX for Dummies Questions & Answers

g03 problems on rocks 5.3 with sge queue.

Dear all, I'm a new guy doing some quntum calculation with gaussian on my rocks clusters. Recently, my cluster upgraded to rocks 5.3, but something strange happened. Everything goes correct except a serious problem. when I summit my gaussian job with script to sge queue, the job does run,... (0 Replies)
Discussion started by: liuzhencc
0 Replies

3. UNIX and Linux Applications

Looking to reduce the number a cpus available for SGE

Hey all Im looking to reduce the number of cpus available on a certain node in our cluster available for jobs using SGE. i.e. we have one node that has 24 cpus available for jobs on SGE, i would like to reduce that to 16. Thanks (1 Reply)
Discussion started by: olifu02
1 Replies

4. UNIX for Dummies Questions & Answers

Text manipulation help request

Hello Unix.com I want to know how can I extract from a large email list only the users. 1@a.com 2@b.com 3@c.com I want to extract only 1, 2, 3. Thanks in advance. Galford D. Weller (2 Replies)
Discussion started by: galford
2 Replies

5. UNIX for Beginners Questions & Answers

Sun Grid Engine (SGE) scripts - processors?

Hi, I was trying to run a program that calls 8 processors (with max. RAM of 2 GB per processor). I want to run this program on my cluster that runs SGE. The cluster has 2 nodes, and each node has 62 cores, and 248GB/node. Currently, I use the scripts below, but the program (softx below) crashes... (0 Replies)
Discussion started by: pc2001
0 Replies

6. Shell Programming and Scripting

SGE submit script

Hi, I'm trying to edit my working bash script to an SGE script in order to submit it as a job to the cluster. Currently I have: #!/bin/bash # Perform fastqc on files in a specified directory. for ((j=1; j <=17; j++)) do directory=/data4/una/batch"$j"/ ... (0 Replies)
Discussion started by: una1992
0 Replies

7. Shell Programming and Scripting

Submit a qsub script SGE

Hello, I'm working on a SGE cluster and trying to echo the variables passed to the qsub command. My script is as follows #!/bin/bash #$ -V #$ -cwd #$ -pe make 10 if ; then echo "Variable is not set" else echo "Variable says: $StatedUserName" fi and I run the... (1 Reply)
Discussion started by: nans
1 Replies

LEARN ABOUT DEBIAN

cd-hit-para

CD-HIT-PARA.PL(1)						   User Commands						 CD-HIT-PARA.PL(1)

NAME

       cd-hit-para.pl - divide a big clustering job into pieces to run cd-hit or cd-hit-est jobs

SYNOPSIS

       cd-hit-para.pl options

DESCRIPTION

	      This  script  divide a big clustering job into pieces and submit jobs to remote computers over a network to make it parallel.  After
	      all the jobs finished, the script merge the clustering results as if you just run a single cd-hit or cd-hit-est.

	      You can also use it to divide big jobs on a single computer if your computer does not have enough RAM (with -L option).

   Requirements:
	      1 When run this script over a network, the directory where you

	      run the scripts and the input files must be available on all the remote hosts with identical path.

	      2 If you choose "ssh" to submit jobs, you have to have

	      passwordless ssh to any remote host, see ssh manual to know how to set up passwordless ssh.

	      3 I suggest to use queuing system instead of ssh,

	      I currently support PBS and SGE

	      4 cd-hit cd-hit-2d cd-hit-est cd-hit-est-2d

	      cd-hit-div cd-hit-div.pl must be in same directory where this script is in.

       Options

       -i input filename in fasta format, required

       -o output filename, required

       --P program, "cd-hit" or "cd-hit-est", default "cd-hit"

       --B filename of list of hosts,

	      requred unless -Q or -L option is supplied

       --L number of cpus on local computer, default 0

	      when you are not running it over a cluster, you can use this option to divide a big clustering jobs into small pieces, I suggest you
	      just use "--L 1" unless you have enough RAM for each cpu

       --S Number of segments to split input DB into, default 64

       --Q number of jobs to submit to queue queuing system, default 0

	      by default, the program use ssh mode to submit remote jobs

       --T type of queuing system, "PBS", "SGE" are supported, default PBS

       --R restart file, used after a crash of run

       -h print this help

       More cd-hit/cd-hit-est options can be speicified in command line

	      Questions, bugs, contact Weizhong Li at liwz@sdsc.edu

cd-hit-para.pl 4.6-2012-04-25					    April 2012							 CD-HIT-PARA.PL(1)
All times are GMT -4. The time now is 10:07 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy