awk data filtering


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk data filtering
# 1  
Old 07-11-2012
awk data filtering

I am trying to filter out some data with awk. If someone could help me that would be great. Below is my input file.

Date: 10-JUN-12 12:00:00
B 0: 00 00 00 00 10 00 16 28
B 120: 00 00 00 39 53 32 86 29
Date: 10-JUN-12 12:00:10
B 0: 00 00 00 00 10 01 11 22
B 120: 00 00 00 29 23 32 16 29
Date: 10-JUN-12 12:00:20
B 0: 00 00 00 00 10 02 17 29
B 120: 00 00 35 51 42 66 14
Date: 10-JUN-12 12:00:30
B 0: 00 00 00 00 10 03 61 42
B 120: 00 00 00 44 33 52 21 52
Date: 10-JUN-12 12:00:40
B 0: 00 00 00 00 10 04 11 22
B 120: 00 00 12 87 10 01 13 42
Date: 10-JUN-12 12:00:50
B 0: 00 00 00 00 10 05 15 24
B 120: 00 00 12 87 10 01 13 42
Date: 10-JUN-12 12:01:00
B 0: 00 00 00 00 10 06 11 22
B 120: 00 00 12 87 10 01 13 42

The above input file is one block of date (i.e. field 8 of the B 0: line increments from 00 to 06 then repeats; with Date:, B 0: (except field 8 will be 10 (its 10 in the above input file) or 11) and B 120: lines changing).


So, I would like to find the B 0: line that contains 10 or 11 in field 7 and 00 in field 8 and print the line directly before it (the Date: line) and then print that B 0: line it found. Next, I would like awk to find the B 0: line that contains 10 or 11 in field 7 and 04 in field 8 and print the line directly before it (the Date: line), then print the found B 0: line, then print the line directly after the B 0: line (the B 120: line). Finally, I would like awk to find the found B 0: line that contains 10 or 11 in field 7 and 06 in field 8 and print the line directly before it (the Date: line), then print the found B 0: line, then print the line directly after the found B 0: line (the B 120: line). I would like this repeated for every block of data in the order described (again my input file in the example above only covers one blocks worth).

Ideally, I would also like the following to also occur...but if someone doesn't know how to do it..then I would be more than greatful to have the awk line needed to get the previous paragraph done. From time to time parts of data are missing. Missing data will consist always of 3 lines (Date:, B 0: and B 120: lines). So, its possible that one or more of the search lines may be missing (line B 0: with 10 or 11 in field 7 and 00 in field 8, and/or line B 0: with 10 or 11 in field 7 and 04 in field 8, and or line B 0: with 10 or 11 in field 7 and 06 in field 8). If this is the case I would like awk not to print anything and move on to the next block of data (find the next line b 0: with 10 or 11 in field 7 and 00 in field 8) and begin search and print again.


What I would like for an output file (this pattern would repeat if the input file had more than one block of data):

Date: 10-JUN-12 12:00:00
B 0: 00 00 00 00 10 00 16 28
Date: 10-JUN-12 12:00:40
B 0: 00 00 00 00 10 04 11 22
B 120: 00 00 12 87 10 01 13 42
Date: 10-JUN-12 12:01:00
B 0: 00 00 00 00 10 06 11 22
B 120: 00 00 12 87 10 01 13 42

Hopefully this isn't too confusing....I've try quite a few things and haven't had any luck.
# 2  
Old 07-11-2012
Smilie

use some code tags... Be clear in whatever you try to explain us Smilie
# 3  
Old 07-12-2012
How about this:

Code:
awk '$1" "$2=="B 0:"&&$7~"^(10|11)$"&&$8==w {
print p;print; if(w>2) {getline;print};w=w<8?w+2:0; if(w==2)w=4} {p=$0}' infile

This User Gave Thanks to Chubler_XL For This Post:
# 4  
Old 07-12-2012
Thanks for the info...I'll give it a try tomorrow...could you explain the code?
# 5  
Old 07-12-2012
p stores previous line
w cycles thru 0, 4 and 6
getline;print - fetches next line from infile and prints it.
$1" "$2=="B 0:" field 1 is "B" and field 2 is "0:"

print p;print print previous (stored line) and current line.
This User Gave Thanks to Chubler_XL For This Post:
# 6  
Old 07-12-2012
Great...thanks for the help. Sorry for the confusion...but I just realized my data actually goes from b 0: field 8 being 00 to 1F. So, I need w to cycle thru 0 (no change to code), 0F (instead of 4) and 1F (instead of 6). Just to be clear 00 to 1F is in hex so (00, 01, 02, 03, 04, 05, 06, 07, 08, 09, 0A, 0B, 0C, 0D, 0E, 0F, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 1A, 1B, 1C, 1D, 1E, 1F). How would you change the code for that? Thanks again.

Last edited by thibodc; 07-12-2012 at 01:37 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need help Filtering Data from an API

Hi Everyone, I need help on figuring out a way to filter some data that I get back from an API. Im able to get all the data that Im looking for but I would like to know a way for me to filter it better. The data that Im getting back is basically 2 rows of data as seen here. Row 1 ... (25 Replies)
Discussion started by: TheStruggle
25 Replies

2. Shell Programming and Scripting

Filtering out the data with dates

Hi, I have some data like seen below. format : apple(hhmm mm/dd).fruit apple(2345 03/25).fruit apple(2345 05/06).fruit orange(0443 05/02).fruit orange(0345 05/05).fruit orange(2134 05/04).fruit grape(0930 04/24).fruit grape(2330 03/30).fruit I need to get the data which are... (1 Reply)
Discussion started by: jayadanabalan
1 Replies

3. Homework & Coursework Questions

awk - filtering data by if --> into an array

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: my data in csv-format ... ... 13/08/2012,16:30,303.30,5.10,3,2,2,1,9360.0,322... (13 Replies)
Discussion started by: IMPe
13 Replies

4. Shell Programming and Scripting

awk - filtering data by if --> into an array

Hi my data is in csv-format ... ... 13/08/2012,16:30,303.30,5.10,3,2,2,1,9360.0,322 13/08/2012,16:40,305.50,5.00,3,2,2,1,12360.0,322 13/08/2012,16:50,319.90,3.80,3,2,1,0,2280.0 13/08/2012,17:00,326.10,3.50,3,2,1,1,4380.0,321 13/08/2012,17:10,333.00,3.80,3,3,1,0,2280.0... (1 Reply)
Discussion started by: IMPe
1 Replies

5. Shell Programming and Scripting

Filtering data using AWK

Hi , i have file with delimiter as "|" and data in Double codes for all fields. how to filter data in a column like awk -F"|" '$1="asdf" {print $0}' test. ex : "asdf"|"zxcv" Thanks, Soma (1 Reply)
Discussion started by: challamsomu
1 Replies

6. Shell Programming and Scripting

Help with pulling / filtering data from a .csv

Good day Gurus, I have a csv file that contains an inventory of active servers. This csv file contains a well over a hundred systems (IBM, SUN, HP). It also contains those systems details. See below for an example hostA,invver,1.02,20100430 hostA,date,08/30/2010,06:18 hostA,use,"Unknown... (4 Replies)
Discussion started by: LuffyDMonkey
4 Replies

7. Shell Programming and Scripting

help need in filtering data

Hello Gurus, Please help me out of the problem. I ve a input file as below input clock; input a; //reset all input b; //input comment output c; output d; output e; input f; //output comment I need the output as follows: \\Inputs (1 Reply)
Discussion started by: user_prady
1 Replies

8. UNIX for Dummies Questions & Answers

Filtering Data

file1 contain: (this just a small sample of data it may have thousand of lines) 1 aaa 1/01/1975 delhi 2 bbb 2/03/1977 mumbai 3 ccc 1/01/1975 mumbai 4 ddd 2/03/1977 chennai 5 aaa 1/01/1975 kolkatta 6 bbb 2/03/1977 bangalore program: nawk '{ idx= $2 SUBSEP $3 arr = (idx in arr) ?... (2 Replies)
Discussion started by: bobo
2 Replies

9. Shell Programming and Scripting

Filtering Data

Hi All, I have the below input and expected ouput. I need a code which can scan through this input file and if the number in column1 is more than 1 , it will print out the whole line, else it will output "No Re-occurrence". Can anybody help ? Input: 1 vvvvv 20 7 7 23 0 64 6 zzzzzz 11 5... (7 Replies)
Discussion started by: Raynon
7 Replies

10. UNIX for Dummies Questions & Answers

Filtering out data ...

I have following command which tells me File size in GBs which are greater than 0.01GBs recursively in a dir structure. ls -l -R | awk '{ if ($5/1073741824 >= 0.01) print $9, $5/1073741824 }' But there are some files whom I dont have enough permissions, after executing this script gives me... (1 Reply)
Discussion started by: videsh77
1 Replies
Login or Register to Ask a Question