Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Start process on X number of files and then wait for the next batch
Top Forums Shell Programming and Scripting Start process on X number of files and then wait for the next batch Post 303034599 by busyboy on Tuesday 30th of April 2019 09:22:12 AM
Old 04-30-2019
the exact narration is

execute 10 jobs in either background or foreground -- wait for all them to finish and then start the next 10 or whatever count is coming from the next batch of files.

I think I'll have to go with the Plan-B ( execute 10 jobs at a given time , and start a new if there are less than 10 jobs running ) --

but what if 2 or more background processes are finished before the sleep is timed out, You see your code is running only 1 job in the else case.


Code:
for single_job in ${job_list}
do
   active_jobs=$(ps -f|grep -c slee[p])            #  This is the important bit!
   if [ $active_count -ge 10 ]
   then
      echo "All running at full capacity"
      sleep 1    # Pick a suitable check frequency
   else
      "${single_job}" &                 <----- Should I do the calculation on this point as well , how many jobs are running and how many to start ?
   fi
done

echo "Finished them with up-to-ten running"

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Process Wait on DG UX

Does anyone know what the equivalent command to pwait on Solaris is on DG/UX. I need my script to kick off a process and wait till it is complete before continuing with the script. (4 Replies)
Discussion started by: fabbas
4 Replies

2. UNIX for Dummies Questions & Answers

Append value(batch number) to start of records

Hi all, I am new to unix shell scripting and I am trying to append batch number that comes in Trailer record to the detailed record. TR|20080312|22881 |000000005|20080319|2202 LN|20080312|077777722220 |0000100000017|ABS LN|20080312|000799439326 |0000709943937|AA TR|20080313|22897 ... (6 Replies)
Discussion started by: kiran_418
6 Replies

3. Shell Programming and Scripting

wait command - cat it wait for not-chile process?

Did not use 'wait' yet. How I understand by now the wait works only for child processes, started background. Is there any other way to watch completion of any, not related process (at least, a process, owned by the same user?) I need to start a background process, witch will be waiting... (2 Replies)
Discussion started by: alex_5161
2 Replies

4. UNIX for Advanced & Expert Users

wait process

can any one please give me clear idea of wait process in UNIX system. I am using AIX 5.3 and see loots of wait process. I have very basic concept of wait process. If CPU has nothing to do then a wait process is generated per CPU. But i want know the detail how is it forked. Is wait a jombe... (2 Replies)
Discussion started by: pchangba1
2 Replies

5. Shell Programming and Scripting

Ksh Script to get the start minute of n number of process

How can i write a script.? which lists all X process and gets the start minute of each of them. thanks (1 Reply)
Discussion started by: Anteus
1 Replies

6. Solaris

Number of files - in start of year

Is there any way to find "Number of files" that exists on my solaris parition in the starting of 2009 year ? I know ctime or mtime will not help and unix wouldnt store creation time. Only hope i can see ( and i am not sure if that will help ) is that my system is up from last 2 years without... (5 Replies)
Discussion started by: rajwinder
5 Replies

7. Shell Programming and Scripting

How to make the parent process to wait for the child process

Hi All, I have two ksh script. 1st script calls the 2nd script and the second script calls an 'C' program. I want 1st script to wait until the 'C' program completes. I cant able to get the process id for the 'C' program (child process) to make the 1st script to wait for the second... (7 Replies)
Discussion started by: sennidurai
7 Replies

8. Shell Programming and Scripting

Using Make to batch process files

Hello all, I have a make question, and I was hoping somebody here might be able to point me in the right direction. Here is my issue; I have a command-line tool that I use to run a conversion on an input XML file that results in an output binary file. However, this particular tool needs to... (1 Reply)
Discussion started by: jujumbura
1 Replies

9. Shell Programming and Scripting

Executing a batch of files within a shell script with option to refire the individual files in batch

Hello everyone. I am new to shell scripting and i am required to create a shell script, the purpose of which i will explain below. I am on a solaris server btw. Before delving into the requirements, i will give youse an overview of what is currently in place and its purpose. ... (2 Replies)
Discussion started by: goddevil
2 Replies

10. Windows & DOS: Issues & Discussions

How to start a vbs from a windows batch file?

Morning, I'm trying to execute a vbs from a .bat file. Can someone tell me what the difference is between these statements: start c:\lib\runit.vbc c:\lib\runit.vbs When I run the batch with the 'start' parameter it doesn't seem to do anything. (1 Reply)
Discussion started by: Grueben
1 Replies

LEARN ABOUT OSF1

queuedefs

queuedefs(4)						     Kernel Interfaces Manual						      queuedefs(4)

NAME

       queuedefs - Queue description file for at, batch, and cron commands

DESCRIPTION

       The  queuedefs  file describes the characteristics of the queues managed by cron or specifies other characteristics for cron.  Each noncom-
       ment line in this file describes either one queue or a cron characteristic.  Each uncommented line should be in one of the  following  for-
       mats.

	    q.[njobj][nicen][nwaitw] max_jobs=mjobs log=lcode

       The  fields  in these lines are as follows: The name of the queue.  Defined queues are as follows: The default queue for jobs started by at
       The default queue for jobs started by batch The default queue for jobs run from a crontab file

	      Queues d to z are also available for local use.  The maximum number of jobs that can be run simultaneously in  the  queue;  if  more
	      than  njob jobs are ready to run, only the first njob jobs will be run.  The others will be initiated as currently running jobs ter-
	      minate.  The nice(1) value to give to all jobs in the queue that are not run with a user ID of superuser.  The number of seconds	to
	      wait  before rescheduling a job that was deferred because more than njob jobs were running in that queue, or because the system-wide
	      limit of jobs executing (max_jobs) has been reached.  The maximum number of active jobs from all queues that  may  run  at  any  one
	      time.  The default is 25 jobs.  Logging level of messages sent to a log file.  The default is 4.	Defined levels are as follows:

	      level-code   level

		  0	   None

		  1	   Low

		  2	   Medium

		  3	   High

		  4	   Full

       Lines beginning with # are comments, and are ignored.

EXAMPLES

       The  following  file specifies that the b queue, for batch jobs, can have up to 50 jobs running simultaneously; that those jobs will be run
       with a nice value of 20.  If a job cannot be run because too many other jobs are running, cron will wait 60 seconds before trying again	to
       run it.	All other queues can have up to 100 jobs running simultaneously;  they will be run with a nice value of 2.  If a job cannot be run
       because too many other jobs are running cron will wait 60 seconds before trying again to run it.  b.50j20n60w

       The following file specifies that a total of 25 active jobs will be allowed by cron over all the queues at any one time, and cron will  log
       all messages to the log file.  The last two lines are comments that are ignored.  max_jobs=25 log=4 # This is a comment # And so is this

FILES

       Main cron directory The default location for the queue description file.

RELATED INFORMATION

       Commands: at(1), cron(8), crontab(1), nice(1)
       delim off

																      queuedefs(4)
All times are GMT -4. The time now is 07:46 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy