Sponsored Content
Top Forums Shell Programming and Scripting In need of multi threaded perl assistance Post 302281028 by SandmanCL on Wednesday 28th of January 2009 04:48:27 AM
Old 01-28-2009
In need of multi threaded perl assistance

I need to write a perl script to execute external programs and grab the output and return code. Each program should be killed if it has not completed within X seconds.

Imagine that the script goes something like this :

@commands = &get_commands();

foreach $cmd (@commands) {
$pid = execute($cmd, $timeout);
push (@pids, $pid);
}

# pid included above in case it's needed

&wait_for_completion_or_timeout;

foreach (@commands) {
if ($output{$cmd}) {
print "Output of $cmd was $output{$cmd}, it comleted in $duration{$cmd} seconds\n";
} else {
print "$cmd did not complete in the allowed time and was killed";
}

... what could the execute function look like ?

Currently I have something roughly like this

sub execute {
my $cmd = shift;
my $pid;
unless ($pid = fork) {
exec("$cmd > /tmp/out.$$");
}
return $pid; # returning this pid, just in case :-)
}


This works, but I don't feel it's very elegant. I have to keep track of all pids and kill processes that take more than X seconds using 'ps' to track processes and 'kill' to kill them. Ideally I'd like something platform independent. I've looked into multi threaded programming but I'm unfamiliar with this and I'm stomped.

Example of my attempted code to achieve the same as above:

sub execute {
my $cmd = shift;
system("$cmd >/tmp/out.$$");
}

foreach $cmd (@commands) {
my $thread[$i++] = threads->create(\&execute,$cmd);
}

If I run this code I get an error like
A thread exited while 5 threads were running.

... where do I go from here ? How do I wait until each thread has completed, and how would I kill slow threads ?

Should I be looking at a completely different approach ?


Any assistance appreciated !
 

10 More Discussions You Might Find Interesting

1. Programming

multi-threaded server, pthreads, sleep

I am trying to writa a multi-client & multi-threaded TCP server. There is a thread pool. Each thread in the pool will handle requests of multiple clients. But here I have a problem. I find a solution but it is not how it must be... i think. When threads working without sleep(1) I can't... (0 Replies)
Discussion started by: Parahat Melayev
0 Replies

2. Programming

HOWTO: Calculate the balance of work in multi-threaded app.

I was wondering if anyone could give me a good idea how to calculate how balanced the threading is on a multi-threaded application. I want a percentage, such as "threads are 80% balanced." This is the way I am currently going about it, maybe it is good, maybe not. First, whenever a thread... (2 Replies)
Discussion started by: DreamWarrior
2 Replies

3. Shell Programming and Scripting

Assistance with Perl and HTTP

I need to query a http site and then parse the xml results, this works well if I use the string in IE but I require an automated solution. I have tried using the following as well as HTTP::Request, nothing seems to work any suggestions would be appreciated, I have tried diffrnt things I found on... (7 Replies)
Discussion started by: bryanthomas
7 Replies

4. AIX

multi threaded program is hanging

I have a Multithreaded program which is hanging on AIX. OS Version: AIX 5.2 and thread library version : 5.2.0.75 We Initiate the process with 50 threads..when we are disconnecting from the process it hangs.There is lots of other stuff involved here.I am just sending the piece of the problem with... (0 Replies)
Discussion started by: hikrishn
0 Replies

5. UNIX for Advanced & Expert Users

Multi-threaded encryption @ Fedora 11

Hello, are any of the encryption programs capable of true multi-threading ? Friend of mine tells me that he's been running some testing on Fedora 11 and that the kernel doesn't support multi-threading at that level. I've been looking into TrueCrypt, encfs and both calm to support... (0 Replies)
Discussion started by: TehOne
0 Replies

6. Linux

Multi-threaded encryption @ Fedora 11

Hello, are any of the encryption programs capable of true multi-threading ? Friend of mine tells me that he's been running some testing on Fedora 11 and that the kernel doesn't support multi-threading at that level. I've been looking into TrueCrypt, encfs and both calm to support... (1 Reply)
Discussion started by: TehOne
1 Replies

7. Programming

multi-threaded memory leak

Hello All : I write a .c program to test the exactually resource the memory leak as follows: 1 #include <stdio.h> 2 #define NUM 100000 3 void *Thread_Run(void * arg){ 4 //TODO 5 //pthread_datch(pthread_self()); 6 int socket= (int)arg; 7 ... (1 Reply)
Discussion started by: aobai
1 Replies

8. UNIX for Dummies Questions & Answers

Assistance with combining, sorting and saving multi files into one new file

Good morning. I have a piece of code that is currently taking multiple files and using the CAT.exe command to combine into one file that is then sorted in reverse order based on the 3rd field of the file, then displayed on screen. I am trying to change this so that the files are being combined into... (4 Replies)
Discussion started by: jaacmmason
4 Replies

9. Programming

Deallocating memory in multi-threaded environment.

I'm having a hard time figuring out how to manage deallocation of memory in multithreaded environments. Specifically what I'm having a hard time with is using a lock to protect a structure, but when it's time to free the structure, you have to unlock the lock to destroy the lock itself. Which will... (5 Replies)
Discussion started by: gngrwzrd
5 Replies

10. Shell Programming and Scripting

script for multi-threaded bash processes

hey everyone, I'm having some trouble breaking down some code. It's simple a control script that takes machines meant to be backed up from a list. Then according to that will run multi-threaded processes up until the specified thread limit. for example if there are 4 machines to be backed up,... (2 Replies)
Discussion started by: terrell
2 Replies
IPC::Open2(3pm) 					 Perl Programmers Reference Guide					   IPC::Open2(3pm)

NAME
IPC::Open2 - open a process for both reading and writing using open2() SYNOPSIS
use IPC::Open2; $pid = open2(*CHLD_OUT, *CHLD_IN, 'some cmd and args'); # or without using the shell $pid = open2(*CHLD_OUT, *CHLD_IN, 'some', 'cmd', 'and', 'args'); # or with handle autovivification my($chld_out, $chld_in); $pid = open2($chld_out, $chld_in, 'some cmd and args'); # or without using the shell $pid = open2($chld_out, $chld_in, 'some', 'cmd', 'and', 'args'); waitpid( $pid, 0 ); my $child_exit_status = $? >> 8; DESCRIPTION
The open2() function runs the given $cmd and connects $chld_out for reading and $chld_in for writing. It's what you think should work when you try $pid = open(HANDLE, "|cmd args|"); The write filehandle will have autoflush turned on. If $chld_out is a string (that is, a bareword filehandle rather than a glob or a reference) and it begins with ">&", then the child will send output directly to that file handle. If $chld_in is a string that begins with "<&", then $chld_in will be closed in the parent, and the child will read from it directly. In both cases, there will be a dup(2) instead of a pipe(2) made. If either reader or writer is the null string, this will be replaced by an autogenerated filehandle. If so, you must pass a valid lvalue in the parameter slot so it can be overwritten in the caller, or an exception will be raised. open2() returns the process ID of the child process. It doesn't return on failure: it just raises an exception matching "/^open2:/". However, "exec" failures in the child are not detected. You'll have to trap SIGPIPE yourself. open2() does not wait for and reap the child process after it exits. Except for short programs where it's acceptable to let the operating system take care of this, you need to do this yourself. This is normally as simple as calling "waitpid $pid, 0" when you're done with the process. Failing to do this can result in an accumulation of defunct or "zombie" processes. See "waitpid" in perlfunc for more information. This whole affair is quite dangerous, as you may block forever. It assumes it's going to talk to something like bc, both writing to it and reading from it. This is presumably safe because you "know" that commands like bc will read a line at a time and output a line at a time. Programs like sort that read their entire input stream first, however, are quite apt to cause deadlock. The big problem with this approach is that if you don't have control over source code being run in the child process, you can't control what it does with pipe buffering. Thus you can't just open a pipe to "cat -v" and continually read and write a line from it. The IO::Pty and Expect modules from CPAN can help with this, as they provide a real tty (well, a pseudo-tty, actually), which gets you back to line buffering in the invoked command again. WARNING
The order of arguments differs from that of open3(). SEE ALSO
See IPC::Open3 for an alternative that handles STDERR as well. This function is really just a wrapper around open3(). perl v5.18.2 2013-11-04 IPC::Open2(3pm)
All times are GMT -4. The time now is 10:37 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy