SIGCHLD interrupts its own handler


 
Thread Tools Search this Thread
Top Forums Programming SIGCHLD interrupts its own handler
# 1  
Old 03-16-2010
SIGCHLD interrupts its own handler

Hi. I have a program whose job it is to manage 15 child processes. Sometimes these children die (sometimes deliberately other times with a SEGV). This causes a SIGCHLD to be sent to my program which uses waitpid() in the signal handler to gather information and, in most cases, restart the child.
The problem I am having I that under very high loads I am seeing SIGCHLDs sent while the first is still being processed. I can see from my log that the intention was to restart both but only the last one actually gets restarted leading to a gradual haemorrhage of children which eventually causes the whole system to stop responding.

I am using posix threads and have omitted any mutex in the signal handler because I thought it would be atomic. Obviously not. I am a bit scared to put a mutex there; what happens if the same thread is interrupted again while the mutex is held? Deadlock would be worse than the current situation.

I don't want to ignore signals while in the handler either; it is most important that all SIGCHLDs are honoured and the child restarted. I have seen a deferred solution where one thread is dedicated to catching signals and the main program looks at this from time to time. I don't think this will work too well though because I need to call waitpid straight after I get the signal; it needs to wait for the right child status after all.

Any pointers in the right direction would be most welcome.

Cheers;
Jeremy
# 2  
Old 03-16-2010
Make your SIGCHLD handler a loop that calls waitpid with WNOHANG in case any signals were missed while you were processing and only quits when it finds there's none left to process. Otherwise its hard to guarantee you catch them all.
# 3  
Old 03-17-2010
Hi Corona688. You are right. I already use WNOHANG with waitpid in a loop but only until that child has been dealt with. This is still liable to be interrupted though.

However you gave me an idea: I am using threads, I don't really need signals anyway and they mess with my head. What I did was start a thread and get it to sit in a loop with a blocking call to waitpid. As children die it gets unblocked, deals with it and loops back to blocking again. All in a nice serial manner.

I can simply ignore SIGCHLD now, that thread will pick up any children needing attention via waitpid.

Now, try as I might, I cannot break the system and I have it running in production today.

Cheers;

Jeremy
# 4  
Old 03-18-2010
Quote:
Originally Posted by jrichemont
Hi Corona688. You are right. I already use WNOHANG with waitpid in a loop but only until that child has been dealt with. This is still liable to be interrupted though.

However you gave me an idea: I am using threads, I don't really need signals anyway and they mess with my head. What I did was start a thread and get it to sit in a loop with a blocking call to waitpid. As children die it gets unblocked, deals with it and loops back to blocking again. All in a nice serial manner.

I can simply ignore SIGCHLD now, that thread will pick up any children needing attention via waitpid.

Now, try as I might, I cannot break the system and I have it running in production today.

Cheers;

Jeremy
Good job, Jeremy!

You just discovered the way we usually deal with asynchronous signal in multi-threaded application. We spend a thread that waits (and consumes) synchronously the signal.

The function to wait synchronously for a signal is called sigwait(). You need in addition to ensure that the signal shall be delivered to the right thread, that is the one blocked in sigwait(). Pthreads offers such possibility with pthread_sigmask().

In your particular application, you luckier. First you can use waitpid(). And second, you can ignore SIGCHLD "signal redirection", since the default signal action for SIGCHLD is IGN.

Cheers,
Loïc.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

using SIGCHLD

I'm testing out how to use SIGCHLD and I had a question about intercepting the signal and executing an action in the signal handler. signal(SIGCHLD,countdown); What I'm trying to achieve is be able to printf(Hello) every second that child is set to sleep. I'm setting sleep = 3; so... (1 Reply)
Discussion started by: l flipboi l
1 Replies

2. UNIX for Dummies Questions & Answers

about concept of Interrupts.

Hi all, I am new here ,i want to know about interrupts in detail.What r Interrupts .how they r handeled. Thanx in adavnce. (1 Reply)
Discussion started by: vishwasrao
1 Replies

3. Programming

SIGCHLD trace problem

Hello, I'd like to know whether it is possible to let the parent know who kills its child process. The case is likely as below: if there are four processes, we call them A, B, C and D. B is the child of A, and can be killed by both C and D. if B is killed, then A will receive SIGCHLD from B.... (7 Replies)
Discussion started by: aaronwong
7 Replies

4. UNIX for Dummies Questions & Answers

Who sent the process SIGCHLD ?

I want to know whicj process send the signal SIGCHLD to the parent's child. Thank you in advance, (1 Reply)
Discussion started by: Puntino
1 Replies

5. UNIX for Dummies Questions & Answers

About SIGCHLD

When the SIGCHLD is sent? SIGCHLD is sent either a child exits spontaneously (e.g. exit(0)) or it is killed ? thank you in advance (3 Replies)
Discussion started by: Puntino
3 Replies

6. Programming

adv reqd on SIGCHLD on accept call

Hi, I have small problem. In my (concurrent)server programm, I am handling accept problem from client like this. sample of server code. /*******************/ end = 0; while (! end ) { sockfd = accept(...) if(sockfd == -1) { if (errno == EINTR) /* this is bcoz... (5 Replies)
Discussion started by: stevenjagan
5 Replies

7. Programming

When is SIGCHLD is raised.

Hi, I have 2 processes X and Y. Y is exec() from X. In Y i have an exit handler, which is called when i return from main. With in exit handler i delete and object which in turn calls the destructor of the object, which terminates all the threads of Y. I believe that SIGCHLD is raised by Y as... (4 Replies)
Discussion started by: supersumanth
4 Replies

8. Programming

signal handler for SIGCHLD

Hi, I have an c++ application which uses the function fork and execvp(). The parent does not wait until the child ends. The parents just creates children and let them do their stuff. You can see the parent program as a batch-manager. I have added a SIGCHLD handler to the program: void... (3 Replies)
Discussion started by: jens
3 Replies

9. UNIX for Advanced & Expert Users

catch SIGCHLD signal in parent process

I want to catch SIGCHLD signal in parent process. I can't use wait() system call to catch SIGCHLD according to project requirment. Operating system linux 3.1 can any one have a solution for this. Thanking you, ranjan (2 Replies)
Discussion started by: ranjan
2 Replies

10. Programming

Need help with SIGCHLD

Hello everybody, this is my first post on this forum. I have a program that has a child process that sleeps for 5 second and exit. I'm suppose to modify this program so that when the child exits, the parent reports the exit status of the child, so I also have to deal with SIGINT and SIGQUIT. Can... (1 Reply)
Discussion started by: Unlimited Sky
1 Replies
Login or Register to Ask a Question