Keep up constant number of parallel processes


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Keep up constant number of parallel processes
# 8  
Old 02-14-2013
It works absolutely fine for me, copy-pasted letter for letter. What's your system? What's your shell?
# 9  
Old 02-14-2013
Being a power user, I wrote birth control in c: bctl runs N command lines from stdin at a time:
Code:
$ cat mysrc/bctl.c
/*************************************************************************
* $RCSfile: $
* $Source:  $
* $Revision:  $
* $Date:  $
* $Author: $
* $State: $
* $Locker:  $
*************************************************************************/
#include <stdio.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <errno.h>
#include <unistd.h>
#include <strings.h>
static  char    usage[] =
"\n"
"Usage: bctl <num_sim> [ -v ] [ -l <lines_per> ]\n"
"\n"
"Reads <lines_per> (default one) line commands from standard input and\n"
"executes <num_sim> of them at a time, under the ksh.  The order of\n"
"termination has no control over the execution.  Keeps <num_sim> lines\n"
"running, and when standard input is exhausted, waits for all chilren to\n"
"terminate.  With -v, reports any abnormal child exit.\n"
"\n" ;
static  int     l = 1 ;
static  int     lct = 0 ;
static  int     v = 0 ;
static  int     i ;
static  int     x = 0 ;
static  int     c = 0 ;
static  int     cpid ;
static  int     cstat ;
static  char    buf[65536];
static  char    *bufp = buf ;
static  char    *argv2[4] = { "/usr/bin/ksh", "-c", buf, NULL };
static  siginfo_t  wsinfo ;
static  int     p_wait()
{
        cstat = 0 ;
        if ( 0 > ( cpid = wait( &cstat ) ) )
        {
                if ( errno == ECHILD )
                        return 0 ;
                perror( "wait()" );
                exit( 1 );
        }
        if ( !v )
                return 1 ;
        switch ( cstat & 0xff )
        {
        case 0:
                if ( cstat )
                        fprintf( stderr, "\nProcess %d exit %d\n",
                                                cpid, cstat>>8 );
                break ;
        case WSTOPFLG :
                fprintf( stderr, "\nProcess %d stopped on signal %d\n",
                                        cpid, cstat>>8 );
                break ;
        default:
                if ( !( cstat & 0xff00 ) )
                        fprintf( stderr, "\nProcess %d term by sig %d\n",
                                                cpid, cstat & 0xff );
                else
                        fprintf( stderr, "\nProcess %d wait(%d-%d)\n",
                                                cpid, cstat>>8, cstat & 0xff );
                break ;
        }
        return 1 ;
}
main( int argc, char **argv )
{
        for ( i = 1 ; i < argc ; i++ )
        {
                if ( !strcmp( argv[i], "-v" ) )
                {
                        v = 1 ;
                        continue ;
                }
                if ( !strcmp( argv[i], "-l" )
                  && ++i < argc
                  && ( l = atoi( argv[i] ) ) > 0 )
                {
                        continue ;
                }
                if ( 2 > ( x = atoi( argv[i] ) ) )
                {
                        fputs( usage, stderr );
                        exit( 1 );
                }
        }
        while ( fgets( bufp, sizeof buf + buf - bufp, stdin ) )
        {
                if ( !( bufp = strchr( bufp, '\n' ) ) )
                {
                        fputs( "\nFatal: Command too large!\n", stderr );
                        exit( 1 );
                }
                else
                        bufp++ ;
                if ( ++lct < l )
                {
                        continue ;
                }
                else
                {
                        bufp = buf ;
                        lct = 0 ;
                }
                if ( c == x )
                {
                        p_wait();
                }
                else
                        c++ ;
                switch( vfork() )
                {
                case -1:
                        perror( "vfork()" );
                        exit( 1 );
                case 0: /* child */
                        execvp( *argv2, argv2 );
                        perror( "execvp(/usr/bin/ksh)" );
                        exit( 1 );
                default: /* parent */
                        break ;
                }
        }
        if ( ferror( stdin ) )
        {
                perror( "stdin" );
                exit( 1 );
        }
        while ( p_wait() )
        {
                /* nothing else */
        }
        if ( lct )
        {
                fprintf( stderr,
"bctl: discarding incomplete command at EOF:\n"
"%s\n"
                        , buf );
        }
        exit( 0 );
}
/*************************************************************************
*
* Revision History:
*                        
* $Log:  $
*
**************************  End of File  ********************************/

Some complex children fool it into excess with grandchild signals. Minor adjustments to the wait() part and shell path are necessary as you go from system to system.
Login or Register to Ask a Question

Previous Thread | Next Thread

7 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

(bash) Script Processes in Parallel

Hello all, I tried to parralise my treatments but after a while 'ps -ef' display all child process <defunct> (zombie) Parent bash script to process all files (>100000) in directory: for filename in /Data/*.txt; do ./child_pprocess.sh $filename & done exit(0)I understand that the... (1 Reply)
Discussion started by: namnetes
1 Replies

2. Shell Programming and Scripting

Monitoring processes in parallel and process log file after process exits

I am writing a script to kick off a process to gather logs on multiple nodes in parallel using "&". These processes create individual log files. Which I would like to filter and convert in CSV format after they are complete. I am facing following issues: 1. Monitor all Processes parallelly.... (5 Replies)
Discussion started by: shunya
5 Replies

3. Shell Programming and Scripting

Deleting all the parallel processes launched when the main script receives a ctrl+c

Hi, I have a shell script that creates 2 parallel processes. When I press ctrl+c, i want the parallel process to get killed as well. #!/bin/bash cmd1="script1.py" cmd2="script2.py" ${cmd1} & pid1=$! echo ${pid1} ${cmd2} & pid2=$! (7 Replies)
Discussion started by: sana.usha
7 Replies

4. Shell Programming and Scripting

Parallel processes to INC- and DEC-rement shared counter

QUESTION: How do I run processes in parallel, so that the counter (in counter.txt) would vary in value (instead of just "0" and "1")? That is, how to not sequentially run inc.sh and dec.sh? The shared counter (a single number starting as 0) is in a file counter.txt. counter.sh is (supposed to... (2 Replies)
Discussion started by: courteous
2 Replies

5. Shell Programming and Scripting

Retention of Variable Value when a script is called by different processes in parallel- Linux 2.6.9

Hi, I have a generic FTP script which will be called by 28 different processes in parallel (through a GUI tool) may or may not be at the exact moment (there could be a delay of about a minute or so). ./FTP.ksh 1 (1 through 28) This script after importing file from remote m/c... (1 Reply)
Discussion started by: dips_ag
1 Replies

6. Shell Programming and Scripting

Trim not constant number of symbols ?

Hello, I need to trim zeros from left side: #echo $var1 00023456 But number of zeros is not constant. How do I do that ? thanks Vilius (4 Replies)
Discussion started by: vilius
4 Replies

7. Shell Programming and Scripting

How to run processes in parallel?

In a korn shell script, how can I run several processes in parallel at the same time? For example, I have 3 processes say p1, p2, p3 if I call them as p1.ksh p2.ksh p3.ksh they will run after one process finishes. But I want to run them in parallel and want to display "Process p1... (3 Replies)
Discussion started by: sbasak
3 Replies
Login or Register to Ask a Question