How to read/process a .gz file, one line at a time?
Hello
I'm stuck trying to solve this KSH issue and I'm hoping someone out there can offer some suggestions.
I want to read lots of large .gz files one line at a time in order to compare its Error entries with a list of known errors. I can't simply do "foreach ERROR do gzcat *.gz |grep ${ERROR}" because I would have to parse every single large log file for each of the known errors and the time to do that would be days. So I want to parse the log files only once, read each line, compare the error, and increment the error count for each error. I know how to do most of this except the part about processing one line at a time from a .gz file.
I came up with something like this but it's not working. It's outputing all the lines in the file, instead:
This is the output that's coming out. It should output only the lines beginning with ERROR but it's showing everything such as the AUDIT lines. I want just the ERROR lines. And there is no carriage return.
AUDIT ; WebContainer : 2008-08-04 00:11:51,554 ; com.at.commons:A_EndRequest - Done preparing response for transaction for uri '/docroot/common' in 337 ms.^JAUDIT ; WebContainer : 2008-08-04 00:11:58,885 ; com.at.commons:A_BeginRequest - Received request for transaction for uri '/docroot/common'.^JAUDIT ; WebContainer : 20-08-08-04 00:11:59,136 ; com.at.commons:A_EndRequest - Done preparing response for transaction for uri '/docroot/common' in 251ms.^JAUDIT ; WebContainer : 2008-08-04 00:12:08,686 ; com.at.commons:A_BeginRequest - Received request for transaction for uri '/docroot/common'.^JAUDIT ; WebContainer : 2008-08-04 00:12:09,078 ; com.at.commons:A_EndRequest - Done preparing response for transaction for uri '/docroot/common' in 392 ms.: cannot open
Hi!
All!
I am wirting a shell script in which i want to read one line
at a time from the file and write it simultaneouly to other file one line at a time.
Please let me know about some shell utility which can help me out.
Thanx.
If further clarifications are needed then please let me know... (2 Replies)
Guys,
I've written the following to proces one line at a time and then process individual column within the line.. But it works only when there is one line. I know awk will simply fetch multiple lines when run against a file with many lines. How do i achieve this ?
cat ${PARFILE} |... (3 Replies)
Hi,
I have gps receiver log..its giving readings .like below
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GPSD,R=1
$GPGSV,3,1,11,08,16,328,40,11,36,127,00,28,33,283,39,20,11,165,00*71... (3 Replies)
I am using the while-loop to read a file.
The file has lines with null-terminated strings (words, actually.)
What I have by that reading - just a first word up to '\0'!
I need to have whole string up to 'new line' - (LF, 10#10, 16#A)
What I am doing wrong?
#make file 'grb' with... (6 Replies)
Hi,
i have file which contains data as below(Only sample shown, it may contain more data similar to the one shown here)
i need to read this file line by line and generate an output file like the one below
i.e based on N value the number of MSISDNs will vary, if N=1 then the following... (14 Replies)
Hi ,
I am trying to write an shell, which reads a text file (from a location) having a list of numbers of strictly 5 digits only ex: 33144
Now my script will check :
1) that each entry is only 5 digits & numeric only, no alphabets, & its not empty.
2)then it executes a shell script called... (8 Replies)
Hello,
I need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line... An example of entries in the file would be:
SRVXPAPI001 ERRO JUN24 07:28:34 1775
REASON= 0000, PROCID= #E506 #1065: TPCIPPR, INDEX= 003F
... (8 Replies)
Hi All,
Am trying to write wrapper shell/bash script on a utility tool for which i need to pass 2 files as arugment to execute utility tool.
Wraper script am trying is to do with above metion 2 files.
utility tool accepts :
a. userinfo file : which contains username
b. item file : which... (2 Replies)
Hi all,
I have a log file say Test.log that gets updated continuously and it has data in pipe separated format. A sample log file would look like:
<date1>|<data1>|<url1>|<result1>
<date2>|<data2>|<url2>|<result2>
<date3>|<data3>|<url3>|<result3>
<date4>|<data4>|<url4>|<result4>
What I... (3 Replies)
HI All,
I am aware of reading one file line by line but now my requirement is to read 2 file line by line as below explained.
I have 2 files as below file 1 with spaces and file 2 same as file1 but without space and () brackets
now i have to read both file line by line and use first line of... (13 Replies)
Discussion started by: abhaydas
13 Replies
LEARN ABOUT PHP
line
line(1) User Commands line(1)NAME
line - read one line
SYNOPSIS
line
DESCRIPTION
The line utility copies one line (up to and including a new-line) from the standard input and writes it on the standard output. It returns
an exit status of 1 on EOF and always prints at least a new-line. It is often used within shell files to read from the user's terminal.
EXIT STATUS
Exit status is:
0 Successful completion
>0 End-of-file on input.
ATTRIBUTES
See attributes(5) for descriptions of the following attributes:
+-----------------------------+-----------------------------+
| ATTRIBUTE TYPE | ATTRIBUTE VALUE |
+-----------------------------+-----------------------------+
|Availability |SUNWcsu |
+-----------------------------+-----------------------------+
SEE ALSO sh(1), read(2), attributes(5)SunOS 5.10 1 Feb 1995 line(1)