Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Extarcting data from a file that matches two different patterns

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extarcting data from a file that matches two different patterns
# 1  
Old 07-30-2008
Extarcting data from a file that matches two different patterns
Say I have data like shown below. I want all the data that contains XXXX and where it finds XXXX then I also want the Timestamp and the Error fields. I have tried egrep "XXXX|Timestamp" > test.txt. That pulls out the data correctly but gives me two records for each message and it also pulls the line containg timestamp for data that contains DDDD into test.txt file. Is there any way around my problem?

============
Message Discriptor
zSACZXCZCXb,nATimestampABpDSaErrordsbASD
dfvdfgvADSasdASXXXXDasdASDasdASDasdASDAds
============
Message Discriptor
casTEST1DCZXCZXCZTimestampXCacsErrorJ,ADasdb,nADSadsbASD
adfsadsasdZXZcdawsdqadaxax
ADSasdASDasxvxcXXXXvdASDasdASDasdASDAds
============
Message Discriptor
casDCacsTEST1XZCTimestampcJ,ADasdb,nADSadsbASDError
ADSasdASDaxcvxDDDDDvcxvcsdASDasdASDasdASDAds
============
Message Discriptor
casDCacsxTEST1cvTimestampzxcErrorvJ,ADasdb,nADSadsbASD
sazdfcafdafdsadsadsads
ADSasdASDaXXXXsdASDvxcvxvcasdASDasdASDAds
============
Message Discriptor
casDCacsxcvzxcvJ,ADaTimestampTEST1dfErrorgsdfgadfgsdb,nADSadsbASD
ADSasdASDasDDDDdASDvxcvxvdfgdfgdgfdgfcasdASDasdASDAds
============
Message Discriptor
casDCacsxTEST1cvzxTimestampcvJ,ADErrorasdb,nADSadsbASD
ADSasdASDaXXXXsdASDvxcvxvcasdASDasdASDAds
casDCacsxTEST1cvzxcvJ,ADasdb,nADSadsbASD
sazdfcafdafdsadsadsads
============
Message Discriptor
casDCacsxTzxczcEST1cvzxTimestampcvJ,ADErrorasdb,nADSadsbASD
ADSasdASDaDDDDsdASDvxcvxvcasdASDasdASDAds
# 2  
Old 07-30-2008
Hammer & Screwdriver Perhaps a start to a solution...
Since you have so much of the sample in a "test" format, it was difficult to understand what you are trying to retrieve for output. However, take a look at the following commands and resulting output:

Code:
> sed <sample_log "s/Message/~Message/g" | tr "\n" " " | tr "~" "\n" | grep "XXXX"
Message Discriptor zSACZXCZCXb,nATimestampABpDSaErrordsbASD dfvdfgvADSasdASXXXXDasdASDasdASDasdASDAds ============ 
Message Discriptor casTEST1DCZXCZXCZTimestampXCacsErrorJ,ADasdb,nADSadsbASD adfsadsasdZXZcdawsdqadaxax ADSasdASDasxvxcXXXXvdASDasdASDasdASDAds ============ 
Message Discriptor casDCacsxTEST1cvTimestampzxcErrorvJ,ADasdb,nADSadsbASD sazdfcafdafdsadsadsads ADSasdASDaXXXXsdASDvxcvxvcasdASDasdASDAds ============ 
Message Discriptor casDCacsxTEST1cvzxTimestampcvJ,ADErrorasdb,nADSadsbASD ADSasdASDaXXXXsdASDvxcvxvcasdASDasdASDAds casDCacsxTEST1cvzxcvJ,ADasdb,nADSadsbASD sazdfcafdafdsadsadsads ============

From this, I think you can append additional cut/grep commands to further refine the data.
# 3  
Old 07-30-2008
Code:
grep -B 1 "XXXX"

?
# 4  
Old 07-30-2008
Quote:
Originally Posted by gugs
Say I have data like shown below. I want all the data that contains XXXX and where it finds XXXX then I also want the Timestamp and the Error fields. I have tried egrep "XXXX|Timestamp" > test.txt. That pulls out the data correctly but gives me two records for each message and it also pulls the line containg timestamp for data that contains DDDD into test.txt file. Is there any way around my problem?
[...]
Could you post the desired output given your sample data?
# 5  
Old 07-30-2008
Another way using awk :
Code:
awk -v RS='=' 'NF && /XXXX/ {printf("============%s", $0)}' inputfile

Output:
Code:
============
Message Discriptor
zSACZXCZCXb,nATimestampABpDSaErrordsbASD
dfvdfgvADSasdASXXXXDasdASDasdASDasdASDAds
============
Message Discriptor
casTEST1DCZXCZXCZTimestampXCacsErrorJ,ADasdb,nADSadsbASD
adfsadsasdZXZcdawsdqadaxax
ADSasdASDasxvxcXXXXvdASDasdASDasdASDAds
============
Message Discriptor
casDCacsxTEST1cvTimestampzxcErrorvJ,ADasdb,nADSadsbASD
sazdfcafdafdsadsadsads
ADSasdASDaXXXXsdASDvxcvxvcasdASDasdASDAds
============
Message Discriptor
casDCacsxTEST1cvzxTimestampcvJ,ADErrorasdb,nADSadsbASD
ADSasdASDaXXXXsdASDvxcvxvcasdASDasdASDAds
casDCacsxTEST1cvzxcvJ,ADasdb,nADSadsbASD
sazdfcafdafdsadsadsads

Jean-Pierre.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash - Find files excluding file patterns and subfolder patterns

Hello. For a given folder, I want to select any files find $PATH1 -f \( -name "*" but omit any files like pattern name ! -iname "*.jpg" ! -iname "*.xsession*" ..... \) and also omit any subfolder like pattern name -type d \( -name "/etc/gconf/gconf.*" -o -name "*cache*" -o -name "*Cache*" -o... (2 Replies)
Discussion started by: jcdole
2 Replies

2. Shell Programming and Scripting

awk to print match or non-match and select fields/patterns for non-matches

In the awk below I am trying to output those lines that Match between file1 and file2, those Missing in file1, and those missing in file2. Using each $1,$2,$4,$5 value as a key to match on, that is if those 4 fields are found in both files the match, but if those 4 fields are not found then missing... (0 Replies)
Discussion started by: cmccabe
0 Replies

3. Shell Programming and Scripting

Egrep patterns in a file and limit number of matches to print for each pattern match

Hi I need to egrep patterns in a file and limit number of matches to print for each matched pattern. -m10 option is not working out in my sun solaris 5.10 Please guide me the options to achieve. if i do head -10 , i wont be getting all pattern match results as output since for a... (10 Replies)
Discussion started by: ananan
10 Replies

4. Shell Programming and Scripting

How to print line if two lines above it matches patterns.?

Hi, I could only find examples to print line before/after a match, but I'd need to print line after two separate lines matching. E.g.: From the below log entry, I would need to print out the 1234. This is from a huge log file, that has a lot of entries with "CLIENT" and "No" entries (+ other... (3 Replies)
Discussion started by: Juha
3 Replies

5. Shell Programming and Scripting

Find matches and write the data before it

Hi all I am here for help once again I have two files One file is like this with one columns F2 B2 CAD KGM HTC CSP Second file is like this in 5 columns where firs column contain sometime entries of first file with space and other entries (12 Replies)
Discussion started by: Priyanka Chopra
12 Replies

6. Shell Programming and Scripting

Replace column that matches specific pattern, with column data from another file

Can anyone please help with this? I have 2 files as given below. If 2nd column of file1 has pattern foo1@a, find the matching 1st column in file2 & replace 2nd column of file1 with file2's value. file1 abc_1 foo1@a .... abc_1 soo2@a ... def_2 soo2@a .... def_2 foo1@a ........ (7 Replies)
Discussion started by: prashali
7 Replies

7. UNIX for Dummies Questions & Answers

Pipe binary file matches grep results to file

I am using grep to match a pattern, but the output is strange. $ grep -r -o "pattern" * Gives me: Binary file foo1 matches Binary file foo2 matches Binary file foo3 matches To find the lines before/after, I then have to use the following on each file: $ strings foo1 | grep -A1 -B1... (0 Replies)
Discussion started by: chipperuga
0 Replies

8. Shell Programming and Scripting

Capturing data between patterns

Hii, Friends, I want your help in one of my problem. My problem is as follows. I have a flat file as follows (Just a sample) MHT011 01(DOT)8750707asdfas8609 03(DOT)ASD3453ASD 09(DOT)876JHT87 11(DOT)sfd324ert TTT077 01(MOB)876786klj897 06(MOB)876JHT87 07(MOB)sfd324ert... (4 Replies)
Discussion started by: anushree.a
4 Replies

9. Shell Programming and Scripting

Searching patterns in 1 file and deleting all lines with those patterns in 2nd file

Hi Gurus, I have a file say for ex. file1 which has 3500 lines in it which are different account numbers and another file (file2) which has 230000 lines in it. I want to read all the lines in file1 and delete all those lines from file2 which has that same pattern as in file1. I am not quite... (4 Replies)
Discussion started by: toms
4 Replies

10. Shell Programming and Scripting

Extarcting particular fields from a file

Hi, I have a file called status.txt Now that file has got fields and values like this: Owner Workitem progress targetdate Peter Development 50% 26/04/2006 Now I have one more file called status1.txt that has got only the field names: Owner ... (2 Replies)
Discussion started by: sendhilmani123
2 Replies
Login or Register to Ask a Question