Sponsored Content
Top Forums Shell Programming and Scripting grep from multiple lines in several gz files Post 302571573 by mandhan on Monday 7th of November 2011 06:57:36 PM
Old 11-07-2011
grep from multiple lines in several gz files

Hello all,

I have been struggling to get grep work to my requirements. Basically I have to filter out patterns spread across multiple lines over hundreds of .gz files in a folder. And the output needs to be piped to a file.

Here is the example:
folder name: logs
files in this folder: log1.gz, log2.gz, .........,log400.gz
Each of these gz files contain various log files with separate transaction ids.
Here is a dummy example of a log file:

......
<transactionId1> [ignore this random string] ignore this line
<transactionId1> [ignore this random string] carrier: ER
<transactionId1> [ignore this random string] ignore this line
<transactionId1> [ignore this random string] ignore this line
<transactionId1> [ignore this random string] ticket number:
1234567890

<transactionId1> [ignore this random string] ignore this line
......

So when I run the script from the 'logs' folder, I want the output in this format:
transactionId1 ER 1234567890
transactionId2 RF 4566822347
transactionId3 FG 5673456834
......


Please note that in the example log above, the ticket number is actually on the next line. I have tried making this work with sed but failed. I would be glad to be advised by all you experts out there.

Many thanks in advance.

Regards,
Mandhan
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Grep on multiple lines

I 'm trying to grep 2 fieldds on 2 differnt lines. Like this: psit > file egrep -e '(NS|ES)' $file. Not working. If this succeeds then run next cmd else exit. Pls Help Gundu (13 Replies)
Discussion started by: gundu
13 Replies

2. Shell Programming and Scripting

grep multiple lines

Hey guys: I've been meaning to post this question for awhile...it is regarding grep. Let's say for example that the following entry is in logxx: Wed Feb 2 07:44:11 <vsm> 91030 Line 5 Severity 1 Vps 6 Call Answered - DN:8753101 CLID:5164665761 PI:83 If I do a grep 91030... (27 Replies)
Discussion started by: cdunavent
27 Replies

3. Shell Programming and Scripting

grep multiple lines

Hi. I have this format on a textfile: VG Name /dev/vg00 PV Name /dev/dsk/c16t0d0 PV Name /dev/dsk/c18t0d0 PV Name /dev/dsk/c16t4d0 VG Name /dev/vg01 PV Name ... (6 Replies)
Discussion started by: jOOc
6 Replies

4. UNIX for Dummies Questions & Answers

grep command to find multiple strings in multiple lines in a file.

I want to search files (basically .cc files) in /xx folder and subfolders. Those files (*.cc files) must contain #include "header.h" AND x() function. I am writing it another way to make it clear, I wanna list of *.cc files that have 'header.h' & 'x()'. They must have two strings, header.h... (2 Replies)
Discussion started by: ritikaSharma
2 Replies

5. UNIX for Dummies Questions & Answers

Grep multiple strings in multiple files using single command

Hi, I will use below command for grep single string ("osuser" is search string) ex: find . -type f | xarg grep -il osuser but i have one more string "v$session" here i want to grep in which file these two strings are present. any help is appreciated, Thanks in advance. Gagan (2 Replies)
Discussion started by: gagan4599
2 Replies

6. UNIX for Dummies Questions & Answers

grep in multiple lines

hi i have kind of below text in a file. I want to get a complete paragraph starting with START and ending with before another START) which has a particular string say XYZ or ABC START XYZ hshjghkjh 45 ljkfd fldjlj d jldf START 3493u ABC 454 4545454 4545454 45454 4545454 START ...... (3 Replies)
Discussion started by: reldb
3 Replies

7. UNIX for Advanced & Expert Users

grep across multiple lines

How do you grep 'select * from table_name' string from a script if the select * and from table_name are on 2 different lines ? like select * from table_name Any help would be greatly appreciated !!! Thanks RDR (4 Replies)
Discussion started by: RDR
4 Replies

8. UNIX for Dummies Questions & Answers

Grep multiple lines

I want to grep multiple lines from a text file. I want to grep all lines containing X,Y and NA in a single command. How do I go about doing that? This is what my text files look like: rs1983866 0.0983 10 100016313 rs1983865 0.5994 X 100016339 rs1983864 0.3272 11 100017453 rs7077266... (2 Replies)
Discussion started by: evelibertine
2 Replies

9. Shell Programming and Scripting

Grep strings on multiple files and output to multiple files

Hi All, I want to use egrep on multiple files and the results should be output to multiple files. I am using the below code in my shell script(working in Ksh shell). However with this code I am not attaining the desired results. #!/bin/ksh ( a="/path/file1" b="path/file2" for file in... (4 Replies)
Discussion started by: am24
4 Replies

10. Shell Programming and Scripting

Removing carriage returns from multiple lines in multiple files of different number of columns

Hello Gurus, I have a multiple pipe separated files which have records going over multiple Lines. End of line separator is \n and records going over multiple lines have <CR> as separator. below is example from one file. 1|ABC DEF|100|10 2|PQ RS T|200|20 3| UVWXYZ|300|30 4| GHIJKL|400|40... (7 Replies)
Discussion started by: dJHa
7 Replies
ROOTTAIL(1)						      General Commands Manual						       ROOTTAIL(1)

NAME
root-tail - print text directly to X11 root window SYNOPSIS
root-tail [-g|--geometry GEOMETRY] [-fn|--font FONTSPEC] [--color color] [--reload SEC COMMAND] [--shade] [--outline] [--minspace] [--noflicker] [-f|--fork] [--reverse] [--whole] [--partial] [--update] [--cont STRING] [--wordwrap] [--justify] [--noinitial] [--frame] [-id ID] [-i|--interval SECONDS] [-V] file1[,color[,desc]] [file2[,color[,desc]]] DESCRIPTION
Displays a given file anywhere on your X11 root window, i.e. it is kind of tail -f for multiple files using your desktop background as out- put window. All non-option arguments on the command line are files to be logged. A null desc (example: "/var/log/messages,red,") will prevent the printing of a description and the []'s. General Options --color COLOR Use COLOR as default. --font | -fn FONTSPEC Use font FONTSPEC. This can be either a fixed width font like -fn fixed or any font using -fn '-*-*-*-*-*-*-*-*-*-*-*-*-*-*' with the appropriate fields filled out (see xfontsel). Specifying a different FONTSPEC before each filename will cause each file to be displayed in a different font. -f | --fork Forks to the background. --reverse Display the files in reverse order, with the newest lines at the top. --whole Only display whole lines. If the last line of a file doesn't yet end with a newline character then wait until it does before dis- playing it. This is the default if more than one file is being displayed. --partial This is the opposite of the --whole option (see above). It displays lines even if they don't yet have a newline at the end. This is the default if only one file is being displayed. --update Update partial lines 'in place' if they are still on the screen when they are updated in their files. Using --update automatically turns on --partial. --cont STRING When partial lines are broken into two lines in the display, prefix the 2nd line with STRING. Defaults to "|| ". Specify the "--whole" argument to ensure partial lines are never displayed, or specify "--update" to attempt to "repair" broken lines in-place. --cont-color COLOR Use COLOR when displaying the continuation string (as optionally specified with the --cont option above). --wordwrap The default behaviour is to fit as much as possible onto each line of output, even if this means splitting a word between one line and the next. The --wordwrap argument splits lines at spaces if possible. --justify After wrapping long lines, attempt to justify the text to produce a smooth right-hand margin. Implies --wordwrap. --reload SEC COMMAND Re-display the file(s) and run COMMAND every SEC seconds. The default is to never re-display the file(s). --shade Add black shading to the font. --outline Add a black outline to the font (making redraws quite a bit slower). --minspace Use minimum linespace even when using shading or outlining. This might result in leftover pixels (dependign on font and logfile con- tent). --noflicker Use slower but flicker-free update. --noinitial Don't display the end of the file(s) initially. -id ID Use the given window ID for output instead of the root window. -i | --interval SECONDS Use the specified sleeping interval between checks instead of the default 2.4 seconds. Fractional values are OK. -V Print version information. --frame Draw a frame around the selected area. This is useful when trying to find the perfect geometry. EXAMPLE
root-tail -g 800x250+100+50 -font 10x20 /var/log/messages,green -font 12x24 /var/log/secure,red,'ALERT' BUGS
Some desktop environments open a virtual root window and make it difficult to share it. If you cannot see anything after starting root- tail, try to find a setting "allow programs on desktop" or similar, or manually specify a window id. Should you happen to find any bugs please fix them and send me a diff. NOTE: This program was modified by Marc Lehmann <pcg@goof.com>, who couldn't reach the original author. Please direct bug-reports etc. to pcg@goof.com. http://root-tail.plan9.de/ 4.2 Berkeley Distribution 2004-03-27 ROOTTAIL(1)
All times are GMT -4. The time now is 04:24 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy