How to Control Number of Processes Running Post: 302421203

Sponsored Content

Top Forums Shell Programming and Scripting How to Control Number of Processes Running Post 302421203 by drbones on Thursday 13th of May 2010 07:26:59 PM

05-13-2010

Registered User

How to Control Number of Processes Running

Hi

Is there a way to count how many processes a script has started, count how many of these have finished, and make the script wait if their difference goes over a given threshold?

I am using a script to repeatedly execute a code (~100x) which converts 2 data files into one .plt which is in the format for Tecplot. These files are large, 300-1000MB.
The code reads an input file which is altered before each instance of the code is executed. I've done this by hand (never again), and could get about 5-6 instances going at a time (ie the 1st process would finish before I could start the 7th). (The input is read at the very beginning so I can alter it while code is running.)
My problem is once I scripted this process, MANY instances can get going at once, and the computer gets bogged down. I inserted a 'sleep' command, but this isn't robust enough, and depending on other loads on the computer the same problem can happen.

see script below

Thanks!
drbones

Code:

 1 #!/usr/bin/env tcsh
  2 foreach name (file_a_01*0)
  3 echo $name
  4
  5 set number=`echo $name | sed s/file_a_//`
  6 echo $number
  7
  8 set scrfile=`echo scr | sed s/r/r$number/`
  9 sed '2 s/[0-9]*/'$number'/' <code.in >new
 10 mv new code.in
 11 ./code.exe>$scrfile&
 12 sleep 20
 13 end

drbones

View Public Profile for drbones

Find all posts by drbones

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Determine Number of Processes Running

I am pretty new to unix, and I have a project to do. Part of the project asks me to determine the number of processes running and assign it to a variable. I know how to every part of the project but determine the number of processes running. How can I get just the number of processes...

2. Shell Programming and Scripting

Keep a certain number of background processes running

I've got a bit of code I'm trying to work on... What i want to happen is ... at all times have four parallel mysql dump and imports running. I found the follow code snippet on the forum and modified it to work by starting four concurrent processes but it waits until all four are done before...

3. Shell Programming and Scripting

Need help with running processes script

I'm doing a script with the Shell. I need that it only show the number of running processes. Ex: echo "There are `command` running processes" Thnx! Pd: Sorry the idiom. I'm spanish.

4. Shell Programming and Scripting

Help in running two processes in parellel

I have a script something like this: #!/usr/bin/ksh CLASSPATH=/apps/opt/db2udb/admin/db2bdt/sqllib/java/db2java.zip:/apps/opt/db2udb/admin/db2bdt/sqllib/java/db2jcc.jar:/apps/opt/db2udb/admin/db2bdt/sqllib/function:$CLASSPATH export CLASSPATH ...

5. UNIX for Dummies Questions & Answers

How do you print the number of processes that each user is currently running in Unix?

Ok, so I know there's a way to do this, but I've been trying to find out all afternoon with no luck. I think it should print out something like this: 1 bin 2 daemon 6 duo Where the numbers on the left are the number of processes being run by the user whose name is listed on the right. Is...

6. Shell Programming and Scripting

how to know the running processes.

Hi can anybody help me regarding this.. i want know the output of ps -ef with explanation. how can we know the running processess. this is the output of ps -elf F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD 19 T root 0 0 0 0 SY ...

7. Solaris

Running processes on GZ/LZ

Hi guys just a question is it normal to see running process on a non-global zone in the global zone... processes such as cron.

8. Linux

Running processes

Hi guys is it normal to have 5-10 cron/syslog processes running... in my case i got 10 cron process running.

9. UNIX for Dummies Questions & Answers

How do I find the number of processes running on root?

Is there a certain man command I'm missing here? I searched in ps but I couldn't find something that would give me the number of processes running on root. I only want to see the number of processes, not the processes itself.

10. UNIX for Beginners Questions & Answers

Many processes running at the same time

Hello everybody , I launched cron to execute a task every hour but the job takes more than hour that's why I'm getting more than 1000 cron processes running at the same time !!! My question is how to tell cron not to execute unless the job terminated in order to have only one process running ....

LEARN ABOUT X11R4

startpar

STARTPAR(8)						      System Manager's Manual						       STARTPAR(8)

NAME

       startpar - start runlevel scripts in parallel

SYNOPSIS

       startpar [-p par] [-i iorate] [-t timeout] [-T global_timeout] [-a arg] prg1 prg2 ...
       startpar [-p par] [-i iorate] [-t timeout] [-T global_timeout] -M [ boot|start|stop]

DESCRIPTION

       startpar  is  used  to run multiple run-level scripts in parallel.  The degree of parallelism on one CPU can be set with the -p option, the
       default is full parallelism. An argument to all of the scripts can be provided with the -a option.  Processes blocked by pending  I/O  will
       cause  new  process  creation  to be weighted by the iorate factor 800.	To change this factor the option -i can be used to specify another
       value.  The amount weight=(nblockedxiorate)/1000 will be subtracted from the total number  of  processes  which	could  be  started,  where
       nblocked is the number of processes currently blocked by pending I/O.

       The  output  of	each  script is buffered and written when the script exits, so output lines of different scripts won't mix. You can modify
       this behaviour by setting a timeout.

       The timeout set with the -t option is used as buffer timeout. If the output buffer of a script is not empty and the last output was timeout
       seconds ago, startpar will flush the buffer.

       The  -T option timeout works more globally. If no output is printed for more than global_timeout seconds, startpar will flush the buffer of
       the script with the oldest output. Afterwards it will only print output of this script until it is finished.

       The -M option switches startpar into a make(1) like behaviour.  This option takes three different arguments:  boot,  start,  and  stop  for
       reading .depend.boot or .depend.start or .depend.stop respectively in the directory /etc/init.d/.  By scanning the boot and runlevel direc-
       tories in /etc/init.d/ it then executes the appropriate scripts in parallel.

FILES

       /etc/init.d/.depend.boot
       /etc/init.d/.depend.start
       /etc/init.d/.depend.stop

SEE ALSO

       init(8) insserv(8).

COPYRIGHT

       2003,2004 SuSE Linux AG, Nuernberg, Germany.
       2007 SuSE LINUX Products GmbH, Nuernberg, Germany.

AUTHOR

       Michael Schroeder <mls@suse.de>
       Takashi Iwai <tiwai@suse.de>
       Werner Fink <werner@suse.de>

								     Jun 2003							       STARTPAR(8)