UNIX for Dummies Questions & Answers

Hello everyone!

Here's my problem (I'm a newbie):
I use a program/calculator which launches several parallel processes at each iteration. Then it expects to receive a response file from each of the processes before continuing to the next iteration:

Launch_program ---> starts 30 processes ---> Expects 30 results_files -|
^-------<---------<-------<------------v

The tricky part is that the processes do not returning directly any results file: instead they create a flag file which triggers the computing of all the 30 operations at once and returns 30 results files.

The only solution I've found to run the program is to make each process check for the existence of the results file before exiting with a while(1) loop:
(in C):

Code:

/* Wait until a results file is created*/
 while (1)
 {
  if( access( "result_fileXXX", F_OK ) != -1 )
   {      
  break;
   } 
 }

However, this obviously consumes a lot of resources... Is there any way to launch a process, make it "hibernate" and then wake it up with a signal??
In this case the signal would be the creation of the results file, but I do not want the process to be checking its existing continuously.

Any ideas? Is this even possible in unix systems?

Thanks in advance for your input!

There's lots of ways to do this in UNIX.

The simplest way to fix your loop is to add a usleep(100000); at the bottom of your while(1) loop: This will put the process to sleep for at least 100 milliseconds per loop, preventing it from checking for the file more than 10 times per second. The process literally goes to sleep, consuming no CPU time until the delay is up. This may also mean a lag of up to a tenth of a second between a new file being created and the new file being used.

Other, more direct methods, may depend on what your system actually is.

One concern: Is this actually a flag file, i.e. this file is created only after all the data is finished? Or is it one of the data files, which you might catch in the middle of being written to instead of after it's done?

I'm using Linux, if I type uname -a I get this (plus motherboard stuff):

Linux myhostname 2.6.20-1.2948.fc6

Given that the time needed to calculate the results file can vary from less than one second to many hours, isn't there any other solution to really make the process hibernate until a wake up call is received? If I use usleep() this will no doubt represent a great improvement, but does not really solve the problem, at least as I expected .

I suspect you may need to redesign your application for this but, the best and proper way to wait for child processes to finish is simply wait() -- or its related call waitpid(): waitpid(child_pid, &stat, 0); Call this for each child you've created. Your process will sleep until all your children return. You can use WEXITSTATUS(stat) after each call to see what return code they gave.

Thanks, that's what I was looking for... even if I don't know if I'll be able to use it in my specific case, since no child processes are directly launched (I use a flag file to initiate the calculations).
The main problem is that I cannot exit the process before the results files are created...

Indeed, I think my application will need somme major redesign!

You've made things much harder on yourself by using files instead of IPC, but all isn't lost. Linux can watch for filesystem changes with the inotify system.

This will make your code unportable though, since inotify is Linux-only, and only relatively modern Linux at that... wait() and waitpid() on the other hand should be available on any sane UNIX and UNIX-like...

You could also use sockets instead of files. Open a socket to your daemon program, write the data you want written, read the data you want read, and close the connection. read() will sleep whenever it's waiting for the other end to send. TCP sockets and UNIX domain sockets should both be doable and quite portable.

Thank you very much for your input! It was really helpful!