Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Grep and exttract multiple lines

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grep and exttract multiple lines
# 1  
Old 04-21-2014
Grep and exttract multiple lines
Hello:
I am trying to use grep in cygwin to do the following, however I am unable to get the output in the desired format. Please see and let me know how to solve this
Code:
Input log file
20140403 07:29:26 IN:CTRL=[CIN=CIN1]:TYP=TYP1:DCN=DCN1:DATA=[xxxxx xxxx xxxxx
xxxx xxxx xxx xxx xxx xx xxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx   xxxxxxxxxxx  xxxxxx  xxxxxx  xxxx xxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx]
20140403 07:25:26 IN:CTRL=[CIN=CIN2]:TYP=TYP1:DCN=DCN2:DATA=[xxxxx xxxx xxxxx
xxxx xxxx xxx xxx xxx xx xxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  xxxx xxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx]
20140403 07:26:26 IN:CTRL=[CIN=CIN3]:TYP=TYP2:DCN=DCN3:DATA=[xxxxx xxxx xxxxx
xxxx xxxx xxx xxx xxx xx xxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  xxxx xxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx]
20140403 07:29:26 IN:CTRL=[CIN=CIN4]:TYP=TYP2:DCN=DCN4:DATA=[xxxxx xxxx xxxxx
xxxx xxxx xxx xxx xxx xx xxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  xxxx xxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx]
20140403 07:32:26 OUT:CTRL=[CIN=CIN1]:TYP=TYP1:DCN=DCN1:DATA=[xxxxx xxxx xxxxx
xxxx xxxx xxx xxx xxx xx xxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  xxxx xxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx]
20140403 07:42:26 OUT:CTRL=[CIN=CIN2]:TYP=TYP1:DCN=DCN2:DATA=[xxxxx xxxx xxxxx
xxxx xxxx xxx xxx xxx xx xxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  xxxx xxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx xxx   xxxxxxxxxxx  xxxxxx  xxxxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx]
20140403 07:45:26 OUT:CTRL=[CIN=CIN3]:TYP=TYP2:DCN=DCN3:DATA=[xxxxx xxxx xxxxx
xxxx xxxx xxx xxx xxx xx xxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  xxxx xxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx]

Desired output
Code:
----------------
20140403 07:29:26 IN:CTRL=[CIN=CIN1]:TYP=TYP1:DCN=DCN1:DATA=[xxxxx xxxx xxxxx
xxxx xxxx xxx xxx xxx xx xxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx   xxxxxxxxxxx  xxxxxx  xxxxxx  xxxx xxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx]
20140403 07:32:26 OUT:CTRL=[CIN=CIN1]:TYP=TYP1:DCN=DCN1:DATA=[xxxxx xxxx xxxxx
xxxx xxxx xxx xxx xxx xx xxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  xxxx xxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx]
-----------------
20140403 07:25:26 IN:CTRL=[CIN=CIN2]:TYP=TYP1:DCN=DCN2:DATA=[xxxxx xxxx xxxxx
xxxx xxxx xxx xxx xxx xx xxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  xxxx xxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx]

20140403 07:42:26 OUT:CTRL=[CIN=CIN2]:TYP=TYP1:DCN=DCN2:DATA=[xxxxx xxxx xxxxx
xxxx xxxx xxx xxx xxx xx xxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  xxxx xxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx xxx   xxxxxxxxxxx  xxxxxx  xxxxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx]
--------------------------------
20140403 07:26:26 IN:CTRL=[CIN=CIN3]:TYP=TYP2:DCN=DCN3:DATA=[xxxxx xxxx xxxxx
xxxx xxxx xxx xxx xxx xx xxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  xxxx xxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx]

20140403 07:45:26 OUT:CTRL=[CIN=CIN3]:TYP=TYP2:DCN=DCN3:DATA=[xxxxx xxxx xxxxx
xxxx xxxx xxx xxx xxx xx xxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  xxxx xxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx]
---------------
20140403 07:29:26 IN:CTRL=[CIN=CIN4]:TYP=TYP2:DCN=DCN4:DATA=[xxxxx xxxx xxxxx
xxxx xxxx xxx xxx xxx xx xxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  xxxx xxx
xxx   xxxxxxxxxxx  xxxxxx  xxxxx]

I have the list of all CINs, when I grep I would only get the first line, but my intention is to get the DATA between [ xxxx] as well. Please suggest.
# 2  
Old 04-22-2014
Change the bold parts per your requirement.
Code:
awk '{if(NR > 1 && $0 ~ /20[0-9]+ [0-9]+:[0-9]+:[0-9]+ [OI][UN][T:]/) print ""}1' file | \
awk '$1 ~ pat' RS= FS='\n' pat='CIN1'

# 3  
Old 04-22-2014
... a basic awk approach ...
Code:
awk '
	$0~/IN:CTRL=/ {f="I"; i++}
	$0~/OUT:CTRL=/ {f="O"; o++}
	f=="I" {IN[i]=IN[i] RS $0}
	f=="O" {OUT[o]=OUT[o] RS $0}
	END {for (n in IN)
			print "----------------" IN[n] (n in OUT? "\n"OUT[n]:"") }
' file

# 4  
Old 07-17-2014
Quote:
Originally Posted by jethrow
... a basic awk approach ...
Code:
awk '
    $0~/IN:CTRL=/ {f="I"; i++}
    $0~/OUT:CTRL=/ {f="O"; o++}
    f=="I" {IN[i]=IN[i] RS $0}
    f=="O" {OUT[o]=OUT[o] RS $0}
    END {for (n in IN)
            print "----------------" IN[n] (n in OUT? "\n"OUT[n]:"") }
' file

Could you please explain what the script at 1 thru 4 is doing?
# 5  
Old 07-17-2014
Above script works only if the INs and OUTs are in the same order. If that can't be assured, try this modification of jethrow's proposal:
Code:
awk     'match ($0, /CIN[0-9]+/)        {IX=0+substr($0, RSTART+3, RLENGTH-3)}          # find the CIN- no. and save in index var
         match ($0, /[^ ]*:CTRL=/)      {TP=substr($0, RSTART, 1)}                      # find the record type (IN / OUT) and save in type var
         TP=="I"                        {IN[IX]=IN[IX] RS $0}                           # assemble record in IN array for that index    
         TP=="O"                        {OUT[IX]=OUT[IX] RS $0}                         # assemble record in OUT array for that index    
         END                            {for (n in IN) print "----------------" IN[n],  # print all the IN records from the array          
                                                (n in OUT? "\n"OUT[n]:"") }             # and corresp. OUT record if available
        ' file

Last edited by RudiC; 07-17-2014 at 05:35 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Exclude multiple lines using grep

Hi, I'm working on a shell script that reports service status on a database server. There are some services that are in disabled status that the script should ignore and only check the services that are in Enabled status. I output the service configuration to a file and use that information to... (5 Replies)
Discussion started by: senthil3d
5 Replies

2. Shell Programming and Scripting

Grep and display multiple lines

Hi guys, I have a log file that generates multiple logs about a query. <query time='2016-04-13 13:01:50.825'> <PagingRequestHandler> <Before>brand:vmu</Before> <After>brand:vmu</After> </PagingRequestHandler> <GroupDeviceFilterHandler> <Before>brand:vmu</Before> ... (3 Replies)
Discussion started by: Junaid Subhani
3 Replies

3. Shell Programming and Scripting

grep from multiple lines in several gz files

Hello all, I have been struggling to get grep work to my requirements. Basically I have to filter out patterns spread across multiple lines over hundreds of .gz files in a folder. And the output needs to be piped to a file. Here is the example: folder name: logs files in this folder:... (4 Replies)
Discussion started by: mandhan
4 Replies

4. UNIX for Dummies Questions & Answers

Grep multiple lines

I want to grep multiple lines from a text file. I want to grep all lines containing X,Y and NA in a single command. How do I go about doing that? This is what my text files look like: rs1983866 0.0983 10 100016313 rs1983865 0.5994 X 100016339 rs1983864 0.3272 11 100017453 rs7077266... (2 Replies)
Discussion started by: evelibertine
2 Replies

5. UNIX for Advanced & Expert Users

grep across multiple lines

How do you grep 'select * from table_name' string from a script if the select * and from table_name are on 2 different lines ? like select * from table_name Any help would be greatly appreciated !!! Thanks RDR (4 Replies)
Discussion started by: RDR
4 Replies

6. UNIX for Dummies Questions & Answers

grep in multiple lines

hi i have kind of below text in a file. I want to get a complete paragraph starting with START and ending with before another START) which has a particular string say XYZ or ABC START XYZ hshjghkjh 45 ljkfd fldjlj d jldf START 3493u ABC 454 4545454 4545454 45454 4545454 START ...... (3 Replies)
Discussion started by: reldb
3 Replies

7. UNIX for Dummies Questions & Answers

grep command to find multiple strings in multiple lines in a file.

I want to search files (basically .cc files) in /xx folder and subfolders. Those files (*.cc files) must contain #include "header.h" AND x() function. I am writing it another way to make it clear, I wanna list of *.cc files that have 'header.h' & 'x()'. They must have two strings, header.h... (2 Replies)
Discussion started by: ritikaSharma
2 Replies

8. Shell Programming and Scripting

grep multiple lines

Hi. I have this format on a textfile: VG Name /dev/vg00 PV Name /dev/dsk/c16t0d0 PV Name /dev/dsk/c18t0d0 PV Name /dev/dsk/c16t4d0 VG Name /dev/vg01 PV Name ... (6 Replies)
Discussion started by: jOOc
6 Replies

9. Shell Programming and Scripting

grep multiple lines

Hey guys: I've been meaning to post this question for awhile...it is regarding grep. Let's say for example that the following entry is in logxx: Wed Feb 2 07:44:11 <vsm> 91030 Line 5 Severity 1 Vps 6 Call Answered - DN:8753101 CLID:5164665761 PI:83 If I do a grep 91030... (27 Replies)
Discussion started by: cdunavent
27 Replies

10. Shell Programming and Scripting

Grep on multiple lines

I 'm trying to grep 2 fieldds on 2 differnt lines. Like this: psit > file egrep -e '(NS|ES)' $file. Not working. If this succeeds then run next cmd else exit. Pls Help Gundu (13 Replies)
Discussion started by: gundu
13 Replies
Login or Register to Ask a Question