11-07-2011
grep from multiple lines in several gz files
Hello all,
I have been struggling to get grep work to my requirements. Basically I have to filter out patterns spread across multiple lines over hundreds of .gz files in a folder. And the output needs to be piped to a file.
Here is the example:
folder name: logs
files in this folder: log1.gz, log2.gz, .........,log400.gz
Each of these gz files contain various log files with separate transaction ids.
Here is a dummy example of a log file:
......
<transactionId1> [ignore this random string] ignore this line
<transactionId1> [ignore this random string] carrier: ER
<transactionId1> [ignore this random string] ignore this line
<transactionId1> [ignore this random string] ignore this line
<transactionId1> [ignore this random string] ticket number:
1234567890
<transactionId1> [ignore this random string] ignore this line
......
So when I run the script from the 'logs' folder, I want the output in this format:
transactionId1 ER 1234567890
transactionId2 RF 4566822347
transactionId3 FG 5673456834
......
Please note that in the example log above, the ticket number is actually on the next line. I have tried making this work with sed but failed. I would be glad to be advised by all you experts out there.
Many thanks in advance.
Regards,
Mandhan
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I 'm trying to grep 2 fieldds on 2 differnt lines. Like this:
psit > file
egrep -e '(NS|ES)' $file. Not working. If this succeeds then run next cmd else exit. Pls Help
Gundu (13 Replies)
Discussion started by: gundu
13 Replies
2. Shell Programming and Scripting
Hey guys:
I've been meaning to post this question for awhile...it is regarding grep. Let's say for example that the following entry is in logxx:
Wed Feb 2 07:44:11 <vsm> 91030 Line 5 Severity 1 Vps 6
Call Answered - DN:8753101 CLID:5164665761 PI:83
If I do a grep 91030... (27 Replies)
Discussion started by: cdunavent
27 Replies
3. Shell Programming and Scripting
Hi.
I have this format on a textfile:
VG Name /dev/vg00
PV Name /dev/dsk/c16t0d0
PV Name /dev/dsk/c18t0d0
PV Name /dev/dsk/c16t4d0
VG Name /dev/vg01
PV Name ... (6 Replies)
Discussion started by: jOOc
6 Replies
4. UNIX for Dummies Questions & Answers
I want to search files (basically .cc files) in /xx folder and subfolders.
Those files (*.cc files) must contain #include "header.h" AND x() function.
I am writing it another way to make it clear,
I wanna list of *.cc files that have 'header.h' & 'x()'. They must have two strings, header.h... (2 Replies)
Discussion started by: ritikaSharma
2 Replies
5. UNIX for Dummies Questions & Answers
Hi,
I will use below command for grep single string ("osuser" is search string)
ex: find . -type f | xarg grep -il osuser
but i have one more string "v$session"
here i want to grep in which file these two strings are present.
any help is appreciated,
Thanks in advance.
Gagan (2 Replies)
Discussion started by: gagan4599
2 Replies
6. UNIX for Dummies Questions & Answers
hi
i have kind of below text in a file.
I want to get a complete paragraph starting with START and ending with before another START) which has a particular string say XYZ or ABC
START XYZ hshjghkjh 45 ljkfd
fldjlj d jldf
START 3493u ABC 454
4545454
4545454 45454
4545454
START ...... (3 Replies)
Discussion started by: reldb
3 Replies
7. UNIX for Advanced & Expert Users
How do you grep 'select * from table_name' string from a script if the select * and from table_name are on 2 different lines ? like
select *
from table_name
Any help would be greatly appreciated !!!
Thanks
RDR (4 Replies)
Discussion started by: RDR
4 Replies
8. UNIX for Dummies Questions & Answers
I want to grep multiple lines from a text file. I want to grep all lines containing X,Y and NA in a single command. How do I go about doing that?
This is what my text files look like:
rs1983866 0.0983 10 100016313
rs1983865 0.5994 X 100016339
rs1983864 0.3272 11 100017453
rs7077266... (2 Replies)
Discussion started by: evelibertine
2 Replies
9. Shell Programming and Scripting
Hi All,
I want to use egrep on multiple files and the results should be output to multiple files. I am using the below code in my shell script(working in Ksh shell). However with this code I am not attaining the desired results.
#!/bin/ksh
(
a="/path/file1"
b="path/file2"
for file in... (4 Replies)
Discussion started by: am24
4 Replies
10. Shell Programming and Scripting
Hello Gurus,
I have a multiple pipe separated files which have records going over multiple Lines. End of line separator is \n and records going over multiple lines have <CR> as separator. below is example from one file.
1|ABC DEF|100|10
2|PQ
RS
T|200|20
3| UVWXYZ|300|30
4| GHIJKL|400|40... (7 Replies)
Discussion started by: dJHa
7 Replies
LEARN ABOUT DEBIAN
plan9-grep
GREP(1) General Commands Manual GREP(1)
NAME
grep, g - search a file for a pattern
SYNOPSIS
grep [ option ... ] pattern [ file ... ]
g [ option ... ] pattern [ file ... ]
DESCRIPTION
Grep searches the input files (standard input default) for lines that match the pattern, a regular expression as defined in regexp(7) with
the addition of a newline character as an alternative (substitute for |) with lowest precedence. Normally, each line matching the pattern
is `selected', and each selected line is copied to the standard output. The options are
-c Print only a count of matching lines.
-h Do not print file name tags (headers) with output lines.
-e The following argument is taken as a pattern. This option makes it easy to specify patterns that might confuse argument parsing,
such as -n.
-i Ignore alphabetic case distinctions. The implementation folds into lower case all letters in the pattern and input before interpre-
tation. Matched lines are printed in their original form.
-l (ell) Print the names of files with selected lines; don't print the lines.
-L Print the names of files with no selected lines; the converse of -l.
-n Mark each printed line with its line number counted in its file.
-s Produce no output, but return status.
-v Reverse: print lines that do not match the pattern.
-f The pattern argument is the name of a file containing regular expressions one per line.
-b Don't buffer the output: write each output line as soon as it is discovered.
Output lines are tagged by file name when there is more than one input file. (To force this tagging, include /dev/null as a file name
argument.)
Care should be taken when using the shell metacharacters $*[^|()= and newline in pattern; it is safest to enclose the entire expression in
single quotes '...'. An expression starting with '*' will treat the rest of the expression as literal characters.
G invokes grep with -n and forces tagging of output lines by file name. If no files are listed, it searches all files matching
*.C *.b *.c *.h *.m *.cc *.java *.cgi *.pl *.py *.tex *.ms
SOURCE
/src/cmd/grep
/bin/g
SEE ALSO
ed(1), awk(1), sed(1), sam(1), regexp(7)
DIAGNOSTICS
Exit status is null if any lines are selected, or non-null when no lines are selected or an error occurs.
GREP(1)