11-07-2011
grep from multiple lines in several gz files
Hello all,
I have been struggling to get grep work to my requirements. Basically I have to filter out patterns spread across multiple lines over hundreds of .gz files in a folder. And the output needs to be piped to a file.
Here is the example:
folder name: logs
files in this folder: log1.gz, log2.gz, .........,log400.gz
Each of these gz files contain various log files with separate transaction ids.
Here is a dummy example of a log file:
......
<transactionId1> [ignore this random string] ignore this line
<transactionId1> [ignore this random string] carrier: ER
<transactionId1> [ignore this random string] ignore this line
<transactionId1> [ignore this random string] ignore this line
<transactionId1> [ignore this random string] ticket number:
1234567890
<transactionId1> [ignore this random string] ignore this line
......
So when I run the script from the 'logs' folder, I want the output in this format:
transactionId1 ER 1234567890
transactionId2 RF 4566822347
transactionId3 FG 5673456834
......
Please note that in the example log above, the ticket number is actually on the next line. I have tried making this work with sed but failed. I would be glad to be advised by all you experts out there.
Many thanks in advance.
Regards,
Mandhan
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I 'm trying to grep 2 fieldds on 2 differnt lines. Like this:
psit > file
egrep -e '(NS|ES)' $file. Not working. If this succeeds then run next cmd else exit. Pls Help
Gundu (13 Replies)
Discussion started by: gundu
13 Replies
2. Shell Programming and Scripting
Hey guys:
I've been meaning to post this question for awhile...it is regarding grep. Let's say for example that the following entry is in logxx:
Wed Feb 2 07:44:11 <vsm> 91030 Line 5 Severity 1 Vps 6
Call Answered - DN:8753101 CLID:5164665761 PI:83
If I do a grep 91030... (27 Replies)
Discussion started by: cdunavent
27 Replies
3. Shell Programming and Scripting
Hi.
I have this format on a textfile:
VG Name /dev/vg00
PV Name /dev/dsk/c16t0d0
PV Name /dev/dsk/c18t0d0
PV Name /dev/dsk/c16t4d0
VG Name /dev/vg01
PV Name ... (6 Replies)
Discussion started by: jOOc
6 Replies
4. UNIX for Dummies Questions & Answers
I want to search files (basically .cc files) in /xx folder and subfolders.
Those files (*.cc files) must contain #include "header.h" AND x() function.
I am writing it another way to make it clear,
I wanna list of *.cc files that have 'header.h' & 'x()'. They must have two strings, header.h... (2 Replies)
Discussion started by: ritikaSharma
2 Replies
5. UNIX for Dummies Questions & Answers
Hi,
I will use below command for grep single string ("osuser" is search string)
ex: find . -type f | xarg grep -il osuser
but i have one more string "v$session"
here i want to grep in which file these two strings are present.
any help is appreciated,
Thanks in advance.
Gagan (2 Replies)
Discussion started by: gagan4599
2 Replies
6. UNIX for Dummies Questions & Answers
hi
i have kind of below text in a file.
I want to get a complete paragraph starting with START and ending with before another START) which has a particular string say XYZ or ABC
START XYZ hshjghkjh 45 ljkfd
fldjlj d jldf
START 3493u ABC 454
4545454
4545454 45454
4545454
START ...... (3 Replies)
Discussion started by: reldb
3 Replies
7. UNIX for Advanced & Expert Users
How do you grep 'select * from table_name' string from a script if the select * and from table_name are on 2 different lines ? like
select *
from table_name
Any help would be greatly appreciated !!!
Thanks
RDR (4 Replies)
Discussion started by: RDR
4 Replies
8. UNIX for Dummies Questions & Answers
I want to grep multiple lines from a text file. I want to grep all lines containing X,Y and NA in a single command. How do I go about doing that?
This is what my text files look like:
rs1983866 0.0983 10 100016313
rs1983865 0.5994 X 100016339
rs1983864 0.3272 11 100017453
rs7077266... (2 Replies)
Discussion started by: evelibertine
2 Replies
9. Shell Programming and Scripting
Hi All,
I want to use egrep on multiple files and the results should be output to multiple files. I am using the below code in my shell script(working in Ksh shell). However with this code I am not attaining the desired results.
#!/bin/ksh
(
a="/path/file1"
b="path/file2"
for file in... (4 Replies)
Discussion started by: am24
4 Replies
10. Shell Programming and Scripting
Hello Gurus,
I have a multiple pipe separated files which have records going over multiple Lines. End of line separator is \n and records going over multiple lines have <CR> as separator. below is example from one file.
1|ABC DEF|100|10
2|PQ
RS
T|200|20
3| UVWXYZ|300|30
4| GHIJKL|400|40... (7 Replies)
Discussion started by: dJHa
7 Replies
LEARN ABOUT DEBIAN
root-tail
ROOTTAIL(1) General Commands Manual ROOTTAIL(1)
NAME
root-tail - print text directly to X11 root window
SYNOPSIS
root-tail [-g|--geometry GEOMETRY] [-fn|--font FONTSPEC] [--color color] [--reload SEC COMMAND] [--shade] [--outline] [--minspace]
[--noflicker] [-f|--fork] [--reverse] [--whole] [--partial] [--update] [--cont STRING] [--wordwrap] [--justify] [--noinitial] [--frame]
[-id ID] [-i|--interval SECONDS] [-V] file1[,color[,desc]] [file2[,color[,desc]]]
DESCRIPTION
Displays a given file anywhere on your X11 root window, i.e. it is kind of tail -f for multiple files using your desktop background as out-
put window.
All non-option arguments on the command line are files to be logged. A null desc (example: "/var/log/messages,red,") will prevent the
printing of a description and the []'s.
General Options
--color COLOR
Use COLOR as default.
--font | -fn FONTSPEC
Use font FONTSPEC. This can be either a fixed width font like -fn fixed or any font using -fn '-*-*-*-*-*-*-*-*-*-*-*-*-*-*' with
the appropriate fields filled out (see xfontsel). Specifying a different FONTSPEC before each filename will cause each file to be
displayed in a different font.
-f | --fork
Forks to the background.
--reverse
Display the files in reverse order, with the newest lines at the top.
--whole
Only display whole lines. If the last line of a file doesn't yet end with a newline character then wait until it does before dis-
playing it. This is the default if more than one file is being displayed.
--partial
This is the opposite of the --whole option (see above). It displays lines even if they don't yet have a newline at the end. This is
the default if only one file is being displayed.
--update
Update partial lines 'in place' if they are still on the screen when they are updated in their files. Using --update automatically
turns on --partial.
--cont STRING
When partial lines are broken into two lines in the display, prefix the 2nd line with STRING. Defaults to "|| ". Specify the
"--whole" argument to ensure partial lines are never displayed, or specify "--update" to attempt to "repair" broken lines in-place.
--cont-color COLOR
Use COLOR when displaying the continuation string (as optionally specified with the --cont option above).
--wordwrap
The default behaviour is to fit as much as possible onto each line of output, even if this means splitting a word between one line
and the next. The --wordwrap argument splits lines at spaces if possible.
--justify
After wrapping long lines, attempt to justify the text to produce a smooth right-hand margin. Implies --wordwrap.
--reload SEC COMMAND
Re-display the file(s) and run COMMAND every SEC seconds. The default is to never re-display the file(s).
--shade
Add black shading to the font.
--outline
Add a black outline to the font (making redraws quite a bit slower).
--minspace
Use minimum linespace even when using shading or outlining. This might result in leftover pixels (dependign on font and logfile con-
tent).
--noflicker
Use slower but flicker-free update.
--noinitial
Don't display the end of the file(s) initially.
-id ID Use the given window ID for output instead of the root window.
-i | --interval SECONDS
Use the specified sleeping interval between checks instead of the default 2.4 seconds. Fractional values are OK.
-V Print version information.
--frame
Draw a frame around the selected area. This is useful when trying to find the perfect geometry.
EXAMPLE
root-tail -g 800x250+100+50 -font 10x20 /var/log/messages,green -font 12x24 /var/log/secure,red,'ALERT'
BUGS
Some desktop environments open a virtual root window and make it difficult to share it. If you cannot see anything after starting root-
tail, try to find a setting "allow programs on desktop" or similar, or manually specify a window id.
Should you happen to find any bugs please fix them and send me a diff.
NOTE: This program was modified by Marc Lehmann <pcg@goof.com>, who couldn't reach the original author. Please direct bug-reports etc. to
pcg@goof.com.
http://root-tail.plan9.de/
4.2 Berkeley Distribution 2004-03-27 ROOTTAIL(1)