read() blocks process until the stream is closed


 
Thread Tools Search this Thread
Top Forums Programming read() blocks process until the stream is closed
# 1  
Old 01-25-2010
read() blocks process until the stream is closed

Hi @all,

i really stuck in programming a tool with bidirectional process communication (popen(cmd, "rw") ... something like that ;-)).

Here is the code:

Code:
  if(pipe(p_stdin) != 0 || pipe(p_stdout) != 0) {
    fprintf(stderr, "Aufruf von pipe() schlug fehl.\n");
    exit(1);
  }

  if((pid = fork()) < 0) {
    fprintf(stderr, "Aufruf von fork() schlug fehl.\n");
    exit(1);

  // kinderprozess
  } else if(pid == 0) {
    close(p_stdin[WRITE]);
    dup2(p_stdin[READ], READ);
    close(p_stdout[READ]);
    dup2(p_stdout[WRITE], WRITE);

    // factor aurufen
    execl("/bin/sh", "sh", "-c", "factor", NULL);
    perror("execl");
    exit(1);
  }

  close(p_stdin[READ]);
  close(p_stdout[WRITE]);

  // zahlen einlesen
  do {
    assert(fgets(buf, 128, stdin) != NULL);
    
    if((number = atoi(buf)) > 0) {
      // zahl an factor übergeben
      buf[length = strlen(buf)] = '\n';
      assert(write(p_stdin[WRITE], buf, length + 1) != -1);

      // ergebniss auslesen
      close(p_stdin[WRITE]); // <- this musst be removed
      assert(read(p_stdout[READ], buf, 128) != -1);
      printf("Die Zufallszahl %d hat die Primfaktoren %s", number, buf + length + 1);
    }
  } while(number > 0);

intersting are only the last 10 lines. I try to write to factor, then read, than write again (...) until 0 is given. This wont work, because read(p_stdout[READ], ....) always blocks the programm, waiting for some better waeather are someso. I tried to use fsync() between write and read, but it didnt help. Only fclose() solve the problem, but if p_stdin[WRITE] is closed, i can only iterate the loop one times.

Can someone help me? Thx :-)
# 2  
Old 01-25-2010
Hallo Jens,

Quote:
intersting are only the last 10 lines. I try to write to factor, then read, than write again (...) until 0 is given. This wont work, because read(p_stdout[READ], ....) always blocks the programm, waiting for some better waeather are someso. I tried to use fsync() between write and read, but it didnt help. Only fclose() solve the problem, but if p_stdin[WRITE] is closed, i can only iterate the loop one times.

Can someone help me? Thx :-)
Of course, we can help. It's just a matter how much you're willing to pay Smilie
More seriously, I think your problem is simply related to the fact that stdout is line buffered. If you switch to unbuffered mode in the child, this may perhaps solve your problem:
Code:
 // kinderprozess
  } else if(pid == 0) {
    close(p_stdin[WRITE]);
    dup2(p_stdin[READ], READ);
    close(p_stdout[READ]);
    dup2(p_stdout[WRITE], WRITE);
    setvbuf(stdout, NULL, _IONBF, 0);
    ...

Another alternative would be that the factor program fflush each message it writes to stdout.

Besides that, a few comment to your code:
- you may want to use the constant STDIN_FILENO and STDOUT_FILENO defined in <unistd.h> instead of your READ and WRITE constant.
- Once you dup2, you may close the descriptor you have duplicated (p_stdin[READ] and p_stdout[WRITE])

Viel Glück,
Loïc.
# 3  
Old 01-26-2010
Hi Loic,

thx for your tips - unfortunately "setvbuf(stdout, NULL, _IONBF, 0)" does not solve the problem (I think its the same like a explizit fsync() call). Any other ideas?

Greetz Jens
# 4  
Old 01-26-2010
Hi Jens,

yeah, my bad. I prototyped using a parent/child... But setvbuf() settings are not preserved across an exec()! Besides that, I am not sure that it has to do with your specific problem... I need to dig further. Perhaps you could tell us what this 'factor' is? a shell script?

Bis Dann,
Loïc.
# 5  
Old 01-26-2010
Hi! "factor" is part of the gnu coreutils and calculates the prime factors fo a given number :-)
# 6  
Old 01-27-2010
Hallo Jens,

so I have a good and a bad news. The good news: I found the reason for your problem. The stream stdout is normally "line-buffered", but become block-buffered (e.g. 8Kb) when stdout is not connected to a terminal (for instance, when stdout is redirected to a pipe or a file)... This can be simply seen if you invoke factor as follows
Code:
factor | more
123
456
(no output)
x
factor: `x' is not a valid positive integer
123: 3 41
456: 2 2 2 3 19

The 'x' causes a message on stderr stream, and stdout to be flushed.

There are solutions to this problem, but those are not necessarily straightforward. That's the bad news. Basically, you have to let the child process think that it is attached to a pseudo-terminal. On Linux, this can be achieved using forkpty(). More info on this technique can be found here: rmathew: Terminal Sickness

If you want to play with bidirectional pipes, I suggest you to have a binary that you control (e.g. a shell script, or a C-program from you) where you can flush the stdout stream.

Cheers,
Loïc
# 7  
Old 01-27-2010
Do you have gmp installed?
factor source is attached.... This would be the simple way to deal with your problem - as Loic suggested.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Row blocks to column blocks

Hello, Searched for a while and found some "line-to-column" script. My case is similar but with multiple fields each row: S02 Length Per S02 7043 3.864 S02 54477 29.89 S02 104841 57.52 S03 Length Per S03 1150 0.835 S03 1321 0.96 S03 ... (9 Replies)
Discussion started by: yifangt
9 Replies

2. UNIX for Dummies Questions & Answers

Read data from given filename or stream

I have a script that takes 2 parameters (say) as mandatory script1.sh a b The 3rd parameter can be filename which it should process or it can come through a pipeline stream The script should work both ways: script1.sh a b filec or cat filec | script1.sh a b How to put logic in the... (1 Reply)
Discussion started by: ysrini
1 Replies

3. Shell Programming and Scripting

how to split this file into blocks and then send these blocks as input to the tool called Yices?

Hello, I have a file like this: FILE.TXT: (define argc :: int) (assert ( > argc 1)) (assert ( = argc 1)) <check> # (define c :: float) (assert ( > c 0)) (assert ( = c 0)) <check> # now, i want to separate each block('#' is the delimeter), make them separate files, and then send them as... (5 Replies)
Discussion started by: paramad
5 Replies

4. UNIX for Dummies Questions & Answers

Convert 512-blocks to 4k blocks

I'm Unix. I'm looking at "df" on Unix now and below is an example. It's lists the filesystems out in 512-blocks, I need this in 4k blocks. Is there a way to do this in Unix or do I manually convert and how? So for container 1 there is 7,340,032 in size in 512-blocks. What would the 4k block be... (2 Replies)
Discussion started by: rockycj
2 Replies

5. Shell Programming and Scripting

[Video stream] network stream recording with mplayer

Hi I used this command: mplayer http://host/axis-cgi/mjpg/video.cgi -user root -passwd root \ -cache 1024 -fps 25.0 -nosound -vc ffh264 \ -demuxer 3 -dumpstream -dumpfile output.avi It's ok but... Video Playing is very fast! Why? Is it a synch problem? What parameter I have to use for... (1 Reply)
Discussion started by: takeo.kikuta
1 Replies

6. Shell Programming and Scripting

How to read text in blocks

Hi, I have file which contains information written in blocks (every block is different). Is it possible to read every block one by one to another file (one block per file). The input is something like this <block1> <empty line> <block2> <empty line> ... ... ... <block25> <empty... (0 Replies)
Discussion started by: art84_)LV
0 Replies

7. Programming

read input-process-output

Can you help me ? I want to write a program ,which can open a input file (input.txt) and run as child process ,then write to output file (output.txt)....... char inFile="input.txt"; char outFile="output.txt"; int main(int argc, char **argv) { pid_t pid=1; int no=0; // no. of... (5 Replies)
Discussion started by: cupid1575
5 Replies

8. UNIX for Dummies Questions & Answers

ssh_exchange_identification: Connection closed by remote host Connection closed

Hi Everyone, Good day. Scenario: 2 unix servers -- A (SunOS) and B (AIX) I have an ftp script to sftp 30 files from A to B which happen almost instantaneously i.e 30 sftp's happen at the same time. Some of these sftp's fail with the following error: ssh_exchange_identification: Connection... (1 Reply)
Discussion started by: jeevan_fimare
1 Replies

9. Programming

How to know a new file is in process of creating? It has not been closed.

I am programming some data loader of oracle with unix c, when I find new data file, then read it to database and delete it. but one issue, if the file is in process of creating, not been closed yet. I will read zero or part of data content, this will cause problem. I want to know whether some unix... (2 Replies)
Discussion started by: linkjack
2 Replies

10. UNIX for Advanced & Expert Users

Stream Read And Write Queues

Is there any possibility that a Stream Read and Write queues will interchange messages of any kind. If so what are the different possiblites and under what circumstances ? Thanks in advance. (4 Replies)
Discussion started by: S.P.Prasad
4 Replies
Login or Register to Ask a Question