12-30-2011
After following this thread for a week my view is unchanged. You cannot capture transient data reliably from transient data files using standard unix tools.
Imho 1.The system design needs attention. In particular the application which is writing these data files. You get a completely different scenario if the application writes files with unique file names and closes each file after processing.
Imho 2. This processing should be in the primary system. Too many times we find complex data processing in Shell tools which should be done in a high level database language.
Throughout this thread we have learnt absolutely nothing about the process which writes these transient data files or the reason for intercepting the files and we have seen no sign of test code, formal testing, and the results of such testing.
Imho 3. This thread is a complete waste of technical resources.
10 More Discussions You Might Find Interesting
1. Programming
here's the problem, i have two processes that i need to run and both process should be run at a total of 1 minute each. how do i do that? and one more here's what the processes do: the 1st process show the '+" sign infinitely while the 2nd process displays the "-" infinitely. how could i count the... (1 Reply)
Discussion started by: kelogs1347
1 Replies
2. Shell Programming and Scripting
Is there a simple command to find or check if a file has been modified within the last minute? The mtime parameter from find cmd only allow days.
I am trying to avoid timestamp parsing and do some sort of comparison as I'm real beginner at scripts. If this is the only way, any help is greatly... (3 Replies)
Discussion started by: ystee
3 Replies
3. Shell Programming and Scripting
How can i write a script.?
which lists all X process and gets the start minute of each of them.
thanks (1 Reply)
Discussion started by: Anteus
1 Replies
4. UNIX for Dummies Questions & Answers
Hello All,
i am a newbie and need some help when reading a csv file in a bourne shell script. I want to read 10 lines, then wait for a minute and then do a reading of another 10 lines and so on in the same way. I want to do this till the end of file.
Any inputs are appreciated
... (3 Replies)
Discussion started by: victor.s
3 Replies
5. Shell Programming and Scripting
Anyone knows how to use AWK to achieve the following
Sun Feb 12 00:41:01-00:41:59 Success:2 Fail:2
Sun Feb 12 00:42:01-00:42:59 Success:1 Fail:2
Sun Feb 12 01:20:01-01:20:59 Success:1 Fail:2
Mon Feb 13 22:41:01-22:41:59 Success:1 Fail:1
log file:
Success
Success
Fail
Fail
... (9 Replies)
Discussion started by: timmywong
9 Replies
6. Shell Programming and Scripting
Hi, I've been trying to develop a script that performs the parsing of a log every 1 minute and then generating some statistics. I'm fairly new to programming and this is why I come to ask if I can lend a hand.
this is my log:
xxxx 16/04/2012 17:00:52 - xxxx714 - E234 - Time= 119 ms.... (8 Replies)
Discussion started by: jockx
8 Replies
7. Shell Programming and Scripting
How to check log size every 10min. by script (can use crontab)
if log size not change with alert "Log not update"
Base run on SunOS 5.8 Generic_Virtual sun4u sparc SUNW,SPARC-Enterprise
logFiles="log1.log log2.log"
logLocation="/usr/home/test/log/"
Out put.
Tue Jan 31... (3 Replies)
Discussion started by: ooilinlove
3 Replies
8. Shell Programming and Scripting
Hi All,
Need your help on how i can sum up the values. I have a file that contains the count and the time. I wanted to add up all the first column having the same datestamp. Please see below.
INPUT
1721 2015-12-26 00:01
1440 2015-12-26 00:02
1477 2015-12-26 00:02
411 ... (4 Replies)
Discussion started by: ernesto
4 Replies
9. Shell Programming and Scripting
Hello,
I have a list of files, an example below:
-rw-r--r-- 1 smf_oper esg 910773 Jul 6 12:52 am1slc02_ACS_201607061242571_20346.cdr
-rw-r--r-- 1 smf_oper esg 995838 Jul 6 12:52 am1slc01_ACS_201607061243125_19895.cdr
-rw-r--r-- 1 smf_oper esg 557235 Jul 6 12:52... (5 Replies)
Discussion started by: nms
5 Replies
10. Shell Programming and Scripting
I am writing a script to kick off a process to gather logs on multiple nodes in parallel using "&". These processes create individual log files. Which I would like to filter and convert in CSV format after they are complete. I am facing following issues:
1. Monitor all Processes parallelly.... (5 Replies)
Discussion started by: shunya
5 Replies
LEARN ABOUT DEBIAN
io_pipe
io_pipe(3) Library Functions Manual io_pipe(3)
NAME
io_pipe - create a Unix pipe
SYNTAX
#include <io.h>
int io_pipe(int64 pfd[2]);
DESCRIPTION
io_pipe creates a new UNIX ``pipe.'' The pipe can receive data and provide data; any bytes written to the pipe can then be read from the
pipe in the same order.
A pipe is typically stored in an 8192-byte memory buffer; the exact number depends on the UNIX kernel. Bytes are written to the end of the
buffer and read from the beginning of the buffer. Once a byte has been read, it is eliminated from the buffer, making space for another
byte to be written; readers cannot ``rewind'' a pipe to read old data. Once 8192 bytes have been written to the buffer, the pipe will not
be ready for further writing until some of the bytes have been read. Once all the bytes written have been read, the pipe will not be ready
for further reading until more bytes are written.
io_pipe sets d[0] to the number of a new descriptor reading from the pipe, and sets d[1] to the number of a new descriptor writing to the
pipe. It then returns 1 to indicate success. If something goes wrong, io_pipe returns 0, setting errno to indicate the error; in this case
it frees any memory that it allocated for the new pipe, and it leaves d alone.
SEE ALSO
io_readfile(3), io_createfile(3), io_socketpair(3)
io_pipe(3)