Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

text filtering

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting text filtering
# 1  
Old 06-30-2012
text filtering
INPUT FILE:
Code:
Date: 10-JUN-12 12:00:00
B 0: 00 00 00 00 10 00 16 28
B 120: 00 00 00 39 53 32 86 29
 
Date: 10-JUN-12 12:00:10
B 0: 00 00 00 00 10 01 11 22
B 120: 00 00 00 29 23 32 16 29
Date: 10-JUN-12 12:00:20
B 0: 00 00 00 00 10 02 17 29
B 120: 00 00 35 51 42 66 14
Date: 10-JUN-12 12:00:30
B 0: 00 00 00 00 10 03 61 42
B 120: 00 00 00 44 33 52 21 52
Date: 10-JUN-12 12:00:40
B 0: 00 00 00 00 10 04 11 22
B 120: 00 00 12 87 10 01 13 42
Date: 10-JUN-12 12:00:50
B 0: 00 00 00 00 10 05 15 24
B 120: 00 00 12 87 10 01 13 42
Date: 10-JUN-12 12:01:00
B 0: 00 00 00 00 10 06 11 22
B 120: 00 00 12 87 10 01 13 42

Then repeats (the field after 10 on the the B 0: line increments from 00 to 06) with new times and new data (except 10 will always be in the same field on the B 0: line).


What I would like the output to be (sometimes data is missing so checks will have to be done)
I would like to find the B 0: line that contains 10 00 and then print the line above it if it contains Date: then if that checks out print the B 0: line that contains 10 00. Right after these 2 lines are printed I would like to find the B 0: line that contains 10 04 then print the line above it if contains Date:, then print the B 0: line that contains 10 04 that was just found, then print the line right below the B 0: line that contains 10 04 if it contains B 120:. Right after these 2 lines are printed I would like to find the B 0: line that contains 10 06 then print the line above it if contains Date:, then print the B 0: line that contains 10 06 that was just found, then print the line right below the B 0: line that contains 10 06 if it contains B 120:. I would like this done for the entire file (going to the next set of data). Sorry this is fairly confusing.

Below is the output I would like.
Code:
Date: 10-JUN-12 12:00:00
B 0: 00 00 00 00 10 00 16 28
Date: 10-JUN-12 12:00:40
B 0: 00 00 00 00 10 04 11 22
B 120: 00 00 12 87 10 01 13 42
Date: 10-JUN-12 12:01:00
B 0: 00 00 00 00 10 06 11 22
B 120: 00 00 12 87 10 01 13 42

If the input file was larger the next set of data might look like this:
Code:
Date: 10-JUN-12 12:01:10
B 0: 00 00 00 00 10 00 46 78
Date: 10-JUN-12 12:00:50
B 0: 00 00 00 00 10 04 61 82
B 120: 00 00 14 77 10 01 19 02
Date: 10-JUN-12 12:02:10
B 0: 00 00 00 00 10 06 77 55
B 120: 00 00 82 87 70 01 13 42

FYI: I am using solaris 10 and don't have any GNU products installed. I've tried using egrep without success.....I'm thinking awk would be better...but I'm not sure. If you know how to solve this problem...your help would be appreciated.
Last edited by Scrutinizer; 06-30-2012 at 07:45 PM.. Reason: code tags and cleaned spurious formatting
# 2  
Old 06-30-2012
Quote:
Originally Posted by thibodc
(sometimes data is missing so checks will have to be done)
Would you show a sample input file which has data missing? What I want to know is whether missing data means missing lines or some missing values in the lines. As far I can make out from your description of the problem, whole lines could be missing and the only lines always present are the B 0: lines. But, please confirm.


Check if this meets your requirement:
Code:
awk '/^B 0:/ && $7=="10" && $8=="04"{if(p ~ /^Date:/){print p;print $0;p=$0;getline;if($0 ~ /^B 120:/) print}
}    /^B 0:/ && $7=="10" && $8=="06"{if(p ~ /^Date:/){print p;print $0;p=$0;getline;if($0 ~ /^B 120:/) print}
}    /^B 0:/ && $7=="10" && $8=="00" {if(p ~ /^Date:/){print p;print $0}
}   {p=$0}' inputfile

Last edited by elixir_sinari; 06-30-2012 at 01:20 PM..
# 3  
Old 06-30-2012
The Date: line will be the only line that's there always including every field. B 0: and B 120: lines could be missing (the entire line not individual fields). However, I want the search to key on B 0: lines and find the other lines if they are there. So, if the B 0: line isn't there I don't want to print anything. Thanks for the help will the above meet this criteria? Why is line 3 of your code last shouldn't it be first...since that should be the first thing printed? Sorry I'm new to awk do I need the } in front of each line of code?
# 4  
Old 06-30-2012
In that case, the following should work (not tested fully though):

Code:
awk '/^Date:/{times++} /^Date:/ && times==8 {for(i=1;a[i];i++){
split(a[i],f)
if (a[i] ~ /^B 0:/ && f[7]=="10" && f[8]=="00") {if(a[i-1] ~ /^Date:/){print a[i-1];print a[i]}}
if (a[i] ~ /^B 0:/ && f[7]=="10" && f[8]=="04") {if(a[i-1] ~ /^Date:/){print a[i-1];print a[i];j=i+1;if(a[j] ~ /^B 120:/) print a[j]}}
if (a[i] ~ /^B 0:/ && f[7]=="10" && f[8]=="06") {if(a[i-1] ~ /^Date:/){print a[i-1];print a[i];j=i+1;if(a[j] ~ /^B 120:/) print a[j]}}
} times=1;i=0;for(j in a) delete a[j]} {a[++i]=$0} END{
if(times!=8) {
for(i=1;a[i];i++){
split(a[i],f)
if (a[i] ~ /^B 0:/ && f[7]=="10" && f[8]=="00") {if(a[i-1] ~ /^Date:/){print a[i-1];print a[i]}}
if (a[i] ~ /^B 0:/ && f[7]=="10" && f[8]=="04") {if(a[i-1] ~ /^Date:/){print a[i-1];print a[i];j=i+1;if(a[j] ~ /^B 120:/) print a[j]}}
if (a[i] ~ /^B 0:/ && f[7]=="10" && f[8]=="06") {if(a[i-1] ~ /^Date:/){print a[i-1];print a[i];j=i+1;if(a[j] ~ /^B 120:/) print a[j]}}}}}' inputfile

Last edited by elixir_sinari; 06-30-2012 at 03:14 PM..
This User Gave Thanks to elixir_sinari For This Post:
thibodc
# 5  
Old 06-30-2012
Could you explain the new code...thanks again for your help.
# 6  
Old 07-01-2012
Quote:
Originally Posted by thibodc
Could you explain the new code...thanks again for your help.
As you already mentioned, you need your logic to be applied on sets of data. So, first we need to identify a set. As the Date: lines will always be there, the only way (I could think) of identifying sets is a series of 7 Date: lines. So, the 8th occurrence of Date: line will signal the end of a set (and beginning of a new one) and so on. So, going on these lines, I am storing each of the lines in a set in an array and when a new set begins, I am applying your logic to the stored lines (of the previous set) and printing out the lines.

As I am using a new set to mark the end of the previous set, I need to repeat your logic (only for the last set) in the END section.
This User Gave Thanks to elixir_sinari For This Post:
thibodc
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Filtering text with awk

I need to filter a file that is composed like that: >Cluster 0 0 292nt, >last294258;size=1;... * >Cluster 1 0 292nt, >last111510;size=1;... * 1 290nt, >last136280;size=1;... at -/98.62% 2 292nt, >last217336;size=1;... at +/99.66% 3 292nt, >last280937;size=1;... at -/99.32% >Cluster 2... (6 Replies)
Discussion started by: pedro88
6 Replies

2. Shell Programming and Scripting

Filtering data from text to csv

Hello, Is there a way to filerter data from a text file as shown below to a Column e.g. hostname nfsmount as two separate column. Currently I could get hostname and the mount is appearing below.. using this script #! /bin/bash for i in `cat fqdn.txt` do echo "$i ............ " >>... (3 Replies)
Discussion started by: Cy Pqa
3 Replies

3. AIX

Need help with filtering

Hi!! I have a bit of a task here and filtering/scripting not my strongest. I have to collect info of approx 1100 hdiskpower.so i have appended all the hdisk into a text file and i need it to run the command lscfg -vl to confirm if the drive is symmetrix. here's what i have so far at... (3 Replies)
Discussion started by: vpundit
3 Replies

4. Shell Programming and Scripting

Parsing and filtering multiline text into comma separated line

I have a log file that contains several reports with following format. <Start of delimiter> Report1 header Report1 header continue Report1 header continue Record1 header Record1 header continue Record1 header continue field1 field2 field3 field4 ------... (1 Reply)
Discussion started by: yoda9691
1 Replies

5. Shell Programming and Scripting

Filtering out text with awk

(0 Replies)
Discussion started by: nilekyle
0 Replies

6. Shell Programming and Scripting

Please help me to do some filtering

I have to grep a pattern. scenario is like :- Suppose "/etc/sec/one" is a string, i need to check if this string contains "one" using any utility something like if /etc/sec/one | grep ; then Thanks in advance Renjesh Raju (3 Replies)
Discussion started by: Renjesh
3 Replies

7. Shell Programming and Scripting

text processing and filtering scripting

Still new to bash. Using debian lenny 5, bash version 3.2.39. I'm working on three scripts. I need help completing them. One script that inputs a plain text file, echo then chop it up into separate whitespace-delimited strings as an output. Not sure how to do this... for example, the... (4 Replies)
Discussion started by: l20N1N
4 Replies

8. Shell Programming and Scripting

filtering text

Hi how can I filter the text using this one. SAMPLE servervmpool -listall|tail -11 ================================================================================ pool number: 112 pool name: Net-Ora-1wk description: Net-Ora-1wk max partially full: 0... (12 Replies)
Discussion started by: kenshinhimura
12 Replies

9. Shell Programming and Scripting

Another text filtering question

I want to remove everything from a file but the word following the search word. Example: crap crap crap crap SearchWord WordToKeep crap crap crap How would I do this with say awk or grep? Thank you! (4 Replies)
Discussion started by: DethLark
4 Replies

10. UNIX for Dummies Questions & Answers

Filtering text from a string

I'm trying to write a script which prints out the users who are loged in. Printing the output of the "users" command isn't the problem. What I want is to filter out my own username. users | grep -v (username) does not work because the whole line in which username exists is suppressed. If... (5 Replies)
Discussion started by: Cozmic
5 Replies
Login or Register to Ask a Question