You can fork off child processes for each blocking i/o, but they are hard to communicate with,
you can spin off concurrent threads for each blocking i/o, still pretty tricky to communicate between,
you can do non-blocking I/O using O_NONBLOCK (O_NDELAY is obsolete, cannot tell read() EOF) with fcntl() or
you can use poll() or select() to wait for the first ready device or stream.
I am a poll() fan, but select is handy for cases with many fd. Basically, you build a poll() array with a struct for each fd, with bits for the currend needs: input sensing, output sensing, no sensing or both. If you are doing out of band, priority messages, you are on your own, as they can complicate this and i/o generally. You are in the land of raw I/O now, open-read-write-close or socket-bind-listen-accept-send-receive-close or the like, forget FILE*:stdin-stdout-stderr-fflush-ferror-popen-fgets-printf. Poll will tell you who in the array it found willing, or that it timed out. You get called for EOF, too. If poll() returns the offset of that fd, you should be able to do an i/o on that fd in the sensed direction now, but no number of available bytes is guaranteed. As your desire to do I/O changes, you can adjust the entries in the array so they do not call, like when you have nothing to write yet. If you do not like the array, you can present one fd at a time as a trivial array with no timer to test if it is ready, somewhat like O_NONBLOCK, then move to another fd, and you can use poll( null, 0, # ) to sleep # milliseconds (usually, you do not get the cpu back "on time"). The timer is somewhat important, as you do not want to loop the CPU when out of things to do. That is why the array method is best: you check all your open fd at once, get called for the first that is ready, and sleep a little bit only if all are idle, and if there is extended idleness, you still get the CPU now and then to do housekeeping, like detecting extended idleness, logging a heartbeat or changing the log at midnight.
I wrote this a while back, and I think it works! It is hard coded to 5120 as the Solaris pipe buffer size, but in HP-UX, for instance, it should be 8192. I found it far lower in overhead than threads. The single thread can keep track of buffers more easily!
Code:
pipebuf1m.c
#include <stdio.h>
#include <poll.h>
#include <errno.h>
#include <unistd.h>
#include <stdlib.h>
#include <strings.h>
typedef struct lls {
struct lls *next ;
size_t sz ;
char buf[4];
} llt ;
int main( int argc, char **argv ){
char *buf ;
struct pollfd pfds[2];
llt *head = NULL ;
llt **tail = &head ;
llt *tmpllp ;
llt *newllp = NULL ;
size_t cwo = 0 ;
int ret ;
int eof_ct = 0 ;
int buf_ct = 0 ;
int buf_lim = 128 ;
if ( argc == 3
&& !strcmp( argv[1], "-l" )
&& 0 < atol( argv[2] ) ){
buf_lim = atol( argv[2] );
} else if ( argc > 1 ){
fputs(
"\n"
"Usage: pipebuf [ -l <buf_limit> ]\n"
"\n"
"Copies standard input to standard output, buffering data in RAM/VM when\n"
"incoming data can be read faster the outgoing data can be written, up to\n"
"<buf_limit> (default 128) buffers of up to 5120 characters.\n"
"\n",
stderr );
exit( 1 );
}
pfds[0].fd = 0 ;
pfds[0].events = POLLIN | POLLRDNORM | POLLRDBAND | POLLPRI ;
pfds[0].revents = 0 ;
pfds[1].fd = 1 ;
pfds[1].events = 0 ;
pfds[1].revents = 0 ;
while ( eof_ct < 128 || pfds[1].events ) {
if ( ( pfds[0].events
&& pfds[1].events
&& 1 > poll( pfds, 2, -1 ))
|| ( pfds[0].events
&& !pfds[1].events
&& 1 > poll( pfds, 1, -1 ))
|| ( !pfds[0].events
&& pfds[1].events
&& 1 > poll( pfds + 1, 1, -1 ))){
if ( errno == EINTR
|| errno == EAGAIN ){
continue ;
}
perror( "pipebuf:poll(stdin, stdout)" );
exit( 1 );
}
if ( pfds[1].revents ){
if ( ! pfds[1].events ){
fprintf( stderr,
"Unsolicited error or EOF poll event on stdout.\n" );
exit( 2 );
}
if ( 0 > ( ret = write( 1, head->buf + cwo, head->sz - cwo ))){
if ( errno == EINTR
|| errno == EAGAIN ){
continue ;
}
perror( "pipebuf:write(stdout)" );
exit( 1 );
}
pfds[1].revents = 0;
if ( ( cwo += ret ) == head->sz ){
cwo = 0 ;
tmpllp = head ;
if ( !( head = head->next )){ /* emptied linked list */
tail = &head ;
pfds[1].events = 0 ;
}
free( tmpllp );
buf_ct-- ;
}
if ( head ){
continue ; /* Keep writing until it might block */
}
} /* end if write revents */
if ( buf_ct >= buf_lim ){
poll( 0, 0, 1 );
continue ;
}
if ( pfds[0].revents ){
if ( !newllp
&& !( newllp = (llt*)malloc( sizeof ( llt ) - 4 + 5120 ))){
if ( errno == EAGAIN
|| errno == EINTR ){
continue ;
}
perror( "pipebuf:umem_alloc(5120+)" );
exit( 1 );
}
buf_ct++ ;
if ( 0 > ( ret = read( 0, newllp->buf, 5120 ))){
if ( errno == EINTR
|| errno == EAGAIN ){
continue ;
}
perror( "pipebuf:read(stdin)" );
exit( 1 );
}
pfds[0].revents = 0 ;
/* EOF stdin or 128 zero length messages in a row */
if ( !ret ){
if ( ++eof_ct > 128 ){
pfds[0].events = 0 ; /* Stop reading, may need to write */
}
continue ;
}
eof_ct = 0 ; /* Got something, reset eof counter */
newllp->sz = ret ;
newllp->next = NULL ;
*tail = newllp ;
tail = &newllp->next ;
newllp = NULL ;
pfds[1].events = POLLWRNORM ;
} /* end if read revents */
} /* end while */
exit( 0 );
}
Last edited by DGPickett; 03-01-2011 at 03:46 PM..
This not a joke but a quite serious question to maybe have your point of view about this very topic of content on the net. So I start this poll to ask the users if they can imagine that the so called content industry of former times sooner or later or anyway will regain lost ground or not? Do you... (1 Reply)
1) Do you use LDAP on AIX? (as a client)
2) If yes, what LDAP server technology do you use:
a) IDS (or ITDS) - IBM Tivoli Directory Server
b) AD
c) openLDAP
d) other - please list.
I ask, because I am looking at openLDAP as well as IDS and am wondering if there is a clear preference I... (4 Replies)
Hi all,
Im trying to do a simple program which ask the user for a unix command with the arguments. The program fork and the two process communicate with pipes. The child process call execvp with the command and the father process read the result of the execvp via the pipe.
This program works... (11 Replies)
Hi ,
i have a requirement in which i have to ftp files to unix from windows and vice versa.
I have to encrypt files in windows which will then be decrypted in unix and vice versa.
Now the process needs to be automated ..therefore when windows server or unix server recieves the files a shell... (5 Replies)
I have to write a script where I poll a txt file for data (30 min interval)
Dependent on the data read, the script should return a message.
It should look something like the "code" below:
--
do while <data recived>
sleep 30m
read data from file
Done
If <data> x return "A"
If... (1 Reply)
Hi,
Off late I had been looking at the differences b/w select() & poll() system calls. The requirement is to reduce the overhead, processor power in waiting for the data. In the kind of connections under consideration there would be very frequent data arriving on the sockets, so poll() fares... (12 Replies)
Hi,
As far as I known, kqueue/kevent model can be used to improve the efficiency of systems event dispatching. I m wondering whether kqueue/kevent is same as the real-time OS event model. I also want to know when writing multiplexing app in real-time OS, what APIs need to be used for... (0 Replies)
