Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: script for multi-threaded bash processes
Top Forums Shell Programming and Scripting script for multi-threaded bash processes Post 302550568 by hfreyer on Friday 26th of August 2011 06:54:16 AM
Old 08-26-2011
Code:
# get maximum number of thread slots
concurrent=`echo $1 | sed -e 's/--threads=//'`
# get number of tasks to be done
maxprocs=`cat $serverset | wc -l`
# wait time between two task checks
tick=2

# initialize running info to a value which cannot be a valid PID
for ((i=0; i<$concurrent; i+=1 )); do running[$i]=123456789; done
# Zero tasks are started at the beginning
ran=0
# LOOP until no more running background jobs are present (see condition below)
until
    # while not all tasks are started
    while [ $ran -lt $maxprocs ]; do
        # loop over all thread slots
        for ((p=0; p<$concurrent; p+=1 )); do
            # get running PID info of the thread slot
            proc=${running[$p]}
            # check if a process with that PID is (still) running
            # in the init case this will result in "not running"
            ps -p $proc | fgrep $proc >/dev/null
            # if the thread slot is in state "not running"
            if [ $? -ne '0' ] ; then
                # occupy the thread slot with a background job ...
                runproc $ran &
                # ... and remember its PID
                running[$p]=$!
                # increase started tasks count
                ((ran+=1))
                # if all tasks to be done are started, leave thread slot loop
                if [ $ran -ge $maxprocs ]; then break 1; fi
            fi
        done
        sleep $tick
    done
    sleep $tick
# LOOP until no more running background jobs are present
do [ `jobs -r|wc -l` -eq 0 ]
done
# to be sure: final wait for complete finish of all child processes
wait

 

10 More Discussions You Might Find Interesting

1. Programming

multi-threaded server, pthreads, sleep

I am trying to writa a multi-client & multi-threaded TCP server. There is a thread pool. Each thread in the pool will handle requests of multiple clients. But here I have a problem. I find a solution but it is not how it must be... i think. When threads working without sleep(1) I can't... (0 Replies)
Discussion started by: Parahat Melayev
0 Replies

2. Programming

HOWTO: Calculate the balance of work in multi-threaded app.

I was wondering if anyone could give me a good idea how to calculate how balanced the threading is on a multi-threaded application. I want a percentage, such as "threads are 80% balanced." This is the way I am currently going about it, maybe it is good, maybe not. First, whenever a thread... (2 Replies)
Discussion started by: DreamWarrior
2 Replies

3. AIX

multi threaded program is hanging

I have a Multithreaded program which is hanging on AIX. OS Version: AIX 5.2 and thread library version : 5.2.0.75 We Initiate the process with 50 threads..when we are disconnecting from the process it hangs.There is lots of other stuff involved here.I am just sending the piece of the problem with... (0 Replies)
Discussion started by: hikrishn
0 Replies

4. Shell Programming and Scripting

In need of multi threaded perl assistance

I need to write a perl script to execute external programs and grab the output and return code. Each program should be killed if it has not completed within X seconds. Imagine that the script goes something like this : @commands = &get_commands(); foreach $cmd (@commands) { $pid =... (4 Replies)
Discussion started by: SandmanCL
4 Replies

5. UNIX for Advanced & Expert Users

Multi-threaded encryption @ Fedora 11

Hello, are any of the encryption programs capable of true multi-threading ? Friend of mine tells me that he's been running some testing on Fedora 11 and that the kernel doesn't support multi-threading at that level. I've been looking into TrueCrypt, encfs and both calm to support... (0 Replies)
Discussion started by: TehOne
0 Replies

6. Linux

Multi-threaded encryption @ Fedora 11

Hello, are any of the encryption programs capable of true multi-threading ? Friend of mine tells me that he's been running some testing on Fedora 11 and that the kernel doesn't support multi-threading at that level. I've been looking into TrueCrypt, encfs and both calm to support... (1 Reply)
Discussion started by: TehOne
1 Replies

7. Programming

multi-threaded memory leak

Hello All : I write a .c program to test the exactually resource the memory leak as follows： 1 #include <stdio.h> 2 #define NUM 100000 3 void *Thread_Run(void * arg){ 4 //TODO 5 //pthread_datch(pthread_self()); 6 int socket= (int)arg; 7 ... (1 Reply)
Discussion started by: aobai
1 Replies

8. UNIX for Dummies Questions & Answers

Bash: Regulating the number of processes a script can spawn

I've been working on some scripts in which I spawn some background processes. I'd like to be able to limit the number of processes, but have my script spawn additional processes as previous tasks finish. So, let's say I have 20 tasks to complete. Any given task could take from 1 to 10 minutes. ... (7 Replies)
Discussion started by: treesloth
7 Replies

9. Programming

Deallocating memory in multi-threaded environment.

I'm having a hard time figuring out how to manage deallocation of memory in multithreaded environments. Specifically what I'm having a hard time with is using a lock to protect a structure, but when it's time to free the structure, you have to unlock the lock to destroy the lock itself. Which will... (5 Replies)
Discussion started by: gngrwzrd
5 Replies

10. Shell Programming and Scripting

(bash) Script Processes in Parallel

Hello all, I tried to parralise my treatments but after a while 'ps -ef' display all child process <defunct> (zombie) Parent bash script to process all files (>100000) in directory: for filename in /Data/*.txt; do ./child_pprocess.sh $filename & done exit(0)I understand that the... (1 Reply)
Discussion started by: namnetes
1 Replies

LEARN ABOUT OSF1

kernel_thread_w_arg

kernel_thread_w_arg(9r) 												   kernel_thread_w_arg(9r)

NAME

       kernel_thread_w_arg - General: Starts a kernel thread with a calling argument passed in

SYNOPSIS

       thread_t kernel_thread_w_arg(
	       task_t task,
	       void (*start) (void),
	       void* argument );

ARGUMENTS

       Specifies  a  pointer to a task structure.  This pointer identifies the task in which the kernel_thread_w_arg routine starts the newly cre-
       ated kernel thread.  Specifies a pointer to a routine that is the entry point for the newly created kernel thread.  Specifies the  argument
       that kernel_thread_w_arg passes to the entry point specified in start.

DESCRIPTION

       The  kernel_thread_w_arg  routine  creates  and	starts a kernel thread in the specified task at the specified entry point with a specified
       argument. The kernel_thread_w_arg routine passes the specified argument to the newly created kernel thread.  The  kernel_thread_w_arg  rou-
       tine  creates and starts a kernel thread with timeshare scheduling. A kernel thread created with timeshare scheduling means that its prior-
       ity degrades if it consumes an inordinate amount of CPU resources.  A kernel module should call kernel_thread_w_arg only  for  long-running
       tasks. A kernel module should always attach a kernel thread to the ``first task.''

NOTES

       This routine is actually a convenience wrapper for the thread_create routine (which creates the kernel thread) and the thread_start routine
       (which starts the newly created kernel thread).

       The kernel_thread_w_arg routine behaves identically to kernel_isrthread except that with kernel_thread_w_arg you can pass  an  argument	to
       the entry point for the newly created kernel thread.

RETURN VALUES

       Upon  successful completion, kernel_thread_w_arg returns a pointer to the thread structure associated with the kernel thread started at the
       specified entry point. Kernel modules can use this pointer as a handle to a specific kernel thread in calls to other kernel threads-related
       routines.

SEE ALSO

       Routines: kernel_isrthread(9r), thread_create(9r), thread_start(9r)

       Data Structures: task(9s), thread(9s)

															   kernel_thread_w_arg(9r)
All times are GMT -4. The time now is 01:15 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy