Sponsored Content
Top Forums UNIX for Advanced & Expert Users named pipe with persistent buffer Post 302102383 by heck on Wednesday 10th of January 2007 05:31:28 AM
Old 01-10-2007
Error

Namara i did like your suggestions in the other thread. Really, i also wanted to come up with an answer but couldn't order my mind right already. I see this thread as a "subproblem".

And ahm... i really think i understand the problem folks. Data which had never the chance to be written to disc can never be recovered(after crash).
JA, THIS IS LIKE THAT! I know.
I didnt want to dispute about this, we dont need to.

The conceptual discussion is aiming at a design of the system wich behaves as well as possible in all thinkable situations. _Recuding_ the POSSIBLE data loss.
So i realized, as you can see in my thread post, that i have to save the data to disc as soon as possbile.

So far my aimings and my concept are NOT nonsense. It is NOT a "Yes you need your computer powered up to run your program" like problem.


Focus was on two questions.
1. i have to save to disc a soon as possible, but also i need to further process the data. What is more clever?
a.) If i duplicate data as being written to disc AND to Pipe.Disc will never be read if pipe reader (proc2) signals that he has written the pipe data to disc itself. It is just to be transaction save.
b.) I store it in a physical file which i can be read anyway from proc2. So i can forget the pipe.

2. Is there anything like a named pipe with a persistent buffer (disc). (My answer: yes MQseries from IBM - just slightly an overkill) no Really?


And i conclude, yes it is a little bit of an uncommom problem. But were simply a the point, where we cant say, "ahh if the Machine screws up sometime, everything is screwed up anyway."
We have to be able to say. If the Machine screws up, we are prepared the best we can. It is not a devil which will never gonna happen. It is a case which can and over the years WILL happen in PRODUCTION environment, and we have to be prepared to handle it the best we can.
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

IPC using named pipe

Hi All, I am facing a vague issue while trying to make two process talk to each other using named pipe. read process ========= The process which reads, basically creates FIFO using mkfifo - ret_val = mkfifo(HALF_DUPLEX, 0666);) func. It then opens the pipe using open func - fd =... (1 Reply)
Discussion started by: sharanbr
1 Replies

2. Programming

IPC using named pipe

Hi All, I am facing a vague issue while trying to make two process talk to each other using named pipe. read process ========= The process which reads, basically creates FIFO using mkfifo - ret_val = mkfifo(HALF_DUPLEX, 0666) func. It then opens the pipe using open func - fd = open... (2 Replies)
Discussion started by: sharanbr
2 Replies

3. UNIX for Dummies Questions & Answers

Named PIPE

Gurus, I've a File Transaction Server, which communicates with other servers and performs some processing.It uses many Named PIPE's. By mistake i copied a named PIPE into a text file. I heard that PIPE files shouldn't be copied.Isn't it? Since it's a production box, i'm afraid on... (2 Replies)
Discussion started by: Tamil
2 Replies

4. UNIX for Advanced & Expert Users

will a named pipe always be size 0 on filesystem?

I did cat < myFile >> myPipe I was hoping that if I did ls -l, myPipe would now be holding the contents of myFile, and would be the same size. But it was 0. Also strange was that when I did the command above, cat did not return control back to the shell. Why? thanks (4 Replies)
Discussion started by: JamesByars
4 Replies

5. UNIX for Dummies Questions & Answers

Filtering mail into a named pipe

Hello, On my machine, all mail is stored in my /var/spool/mail. IS there a way to direct all mail that goes there into a namep pipe? Thank you, Dado (4 Replies)
Discussion started by: dadoprso
4 Replies

6. Shell Programming and Scripting

pipe to file named with date

I would like to pipe (redirect ? - what is the right term?) the output of my script to a file named with the current date. If I run this at a command prompt: date +'%Y%m%d" ...it returns "20110429" OK, that's good... so I try: ./script.sh > "'date +%Y%m%d'.csv" I get a file... (1 Reply)
Discussion started by: landog
1 Replies

7. Shell Programming and Scripting

Using Named pipe in shell script

Hi, I want to use a Named pipe to get input from a growing file for further processing. When I prototype this scenario using a while loop, the data is not written to the named pipe. This the script I use to get data into the Named pipe: #!/bin/ksh mkfifo pipe while (( n <= 10 )) do echo... (2 Replies)
Discussion started by: sudvishw
2 Replies

8. Shell Programming and Scripting

Named pipe performance

Hi, I am getting data into a Named pipe. Does Named pipe have any size restriction; I know it does not have any storage and it just passes on the data to the next process. I want to know, if there will be a difference in the Named pipe performance if the data input is more. (I am using DB2... (1 Reply)
Discussion started by: sudvishw
1 Replies

9. Programming

Named pipe behavior in Linux

Hi All ! I try to collect importent events from syslog and in my syslog conf, there is something like this: *.* |/logs/ipes/SLpipe1 I have a program, which opens this pipe and reads the messages from it. But how this pipe works ? Where can I probably read something about the details,... (3 Replies)
Discussion started by: mabra
3 Replies

10. UNIX for Dummies Questions & Answers

Named pipe hanging?

Ok, I can't seem to figure this out or find anything on the web about this. I'm on Sun Solaris, UNIX. I have the following test script: #!/bin/ksh touch test.file LOG=./tmp.log rm -f ${LOG} PIPE=./tmp.pipe mkfifo ${PIPE} trap "rm -f ${PIPE}" EXIT tee -a ${LOG} < ${PIPE} & ... (17 Replies)
Discussion started by: Ditto
17 Replies
FIFO(7) 						     Linux Programmer's Manual							   FIFO(7)

NAME
fifo - first-in first-out special file, named pipe DESCRIPTION
A FIFO special file (a named pipe) is similar to a pipe, except that it is accessed as part of the file system. It can be opened by multi- ple processes for reading or writing. When processes are exchanging data via the FIFO, the kernel passes all data internally without writ- ing it to the file system. Thus, the FIFO special file has no contents on the file system; the file system entry merely serves as a refer- ence point so that processes can access the pipe using a name in the file system. The kernel maintains exactly one pipe object for each FIFO special file that is opened by at least one process. The FIFO must be opened on both ends (reading and writing) before data can be passed. Normally, opening the FIFO blocks until the other end is opened also. A process can open a FIFO in nonblocking mode. In this case, opening for read only will succeed even if no-one has opened on the write side yet, opening for write only will fail with ENXIO (no such device or address) unless the other end has already been opened. Under Linux, opening a FIFO for read and write will succeed both in blocking and nonblocking mode. POSIX leaves this behavior undefined. This can be used to open a FIFO for writing while there are no readers available. A process that uses both ends of the connection in order to communicate with itself should be very careful to avoid deadlocks. NOTES
When a process tries to write to a FIFO that is not opened for read on the other side, the process is sent a SIGPIPE signal. FIFO special files can be created by mkfifo(3), and are indicated by ls -l with the file type 'p'. SEE ALSO
mkfifo(1), open(2), pipe(2), sigaction(2), signal(2), socketpair(2), mkfifo(3), pipe(7) COLOPHON
This page is part of release 3.25 of the Linux man-pages project. A description of the project, and information about reporting bugs, can be found at http://www.kernel.org/doc/man-pages/. Linux 2008-12-03 FIFO(7)
All times are GMT -4. The time now is 02:07 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy