How to filter data


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers How to filter data
# 1  
Old 10-23-2009
How to filter data

Hi all,

I require assistance in this.

I'd like to filter the following data to extract WADRAIN.

RAIN, 2003-01-01 00:00, 1, WADRAIN, 900, 1, 24109, 1011, 1.6, 0, 6,
RAIN, 2003-01-01 00:00, 1, DLY3208, 900, 1, 30747, 1011, 1.8, 0, 6,
RAIN, 2003-01-01 00:00, 1, WADRAIN, 900, 1, 24775, 1011, 2.8, 0, 6,
RAIN, 2003-01-01 00:00, 1, WADRAIN, 900, 1, 1622, 1011, 2.7, 0, 6,
RAIN, 2003-01-01 00:00, 1, WADRAIN, 900, 1, 1624, 1011, 1.8, 0, 6,
RAIN, 2003-01-01 00:00, 1, WAMRAIN, 900, 31, 24953, 1011, 84.5, 0, 6,
RAIN, 2003-01-01 00:00, 0, DLY3208, 900, 1, 24953, 1012, 1, 0, 1,
RAIN, 2003-01-01 00:00, 1, WADRAIN, 900, 31, 24953, 1022, 84, 0, 22576,

The eventual output I want is;

RAIN, 2003-01-01 00:00, 1, WADRAIN, 900, 1, 24109, 1011, 1.6, 0, 6,
RAIN, 2003-01-01 00:00, 1, WADRAIN, 900, 1, 24775, 1011, 2.8, 0, 6,
RAIN, 2003-01-01 00:00, 1, WADRAIN, 900, 1, 1622, 1011, 2.7, 0, 6,
RAIN, 2003-01-01 00:00, 1, WADRAIN, 900, 1, 1624, 1011, 1.8, 0, 6,
RAIN, 2003-01-01 00:00, 1, WAMRAIN, 900, 31, 24953, 1011, 84.5, 0, 6,
RAIN, 2003-01-01 00:00, 1, WADRAIN, 900, 31, 24953, 1022, 84, 0, 22576,

Thanks!
# 2  
Old 10-23-2009
Hi.

Code:
man grep

# 3  
Old 10-24-2009
Thanks scottn,

I did

> grep -H "WADRAIN" infile.txt > outfile.txt

and got what I wanted.

This is relatively straightforward but I also want to filter data from other columns (containing 1s and 0s) after extracting "WADRAIN". For example, how do I filter the rows containing 1s from, say, column 4?

Example:

RAIN, 2003-01-01 00:00, 1, WADRAIN, 900, 1, 24109, 1011, 1.6, 0, 6,
RAIN, 2003-01-01 00:00, 0, WADRAIN, 900, 1, 24775, 1011, 2.8, 0, 6,
RAIN, 2003-01-01 00:00, 1, WADRAIN, 900, 1, 1622, 1011, 2.7, 0, 6,
RAIN, 2003-01-01 00:00, 0, WADRAIN, 900, 1, 1624, 1011, 1.8, 0, 6,
RAIN, 2003-01-01 00:00, 1, WAMRAIN, 900, 31, 24953, 1011, 84.5, 0, 6,
RAIN, 2003-01-01 00:00, 0, WADRAIN, 900, 31, 24953, 1022, 84, 0, 22576,

Desired result:
RAIN, 2003-01-01 00:00, 1, WADRAIN, 900, 1, 24109, 1011, 1.6, 0, 6,
RAIN, 2003-01-01 00:00, 1, WADRAIN, 900, 1, 1622, 1011, 2.7, 0, 6,
RAIN, 2003-01-01 00:00, 1, WAMRAIN, 900, 31, 24953, 1011, 84.5, 0, 6,

Thanks!
# 4  
Old 10-24-2009
then use awk...
Code:
awk -F"," '/WADRAIN/&&$3==1{print}' filename

# 5  
Old 10-24-2009
Thanks vidyadhar85,

Could you tell me what the syntax means?
# 6  
Old 10-24-2009
Hi.

It means if the line contains the string WADRAIN and field three contains the number 1, then print the line

Fields are separated by a comma (-F",")

If you want WAMRAIN too, then:

Code:
awk -F, '/WA[DM]RAIN/ && $3' filename


Last edited by Scott; 10-24-2009 at 02:38 PM.. Reason: added WAMRAIN!
# 7  
Old 10-24-2009
Thanks scottn!
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Filter Data based on codes

Hello, I have a question on how to filter the data on multiple columns. The problem is I have one table with 25 columns, 4500 rows. I need to filter out the data based on some codes like 'XXXX', I have 25 codes to check on that table. My requirement is that which ever row has this code I... (1 Reply)
Discussion started by: sandeep309
1 Replies

2. Shell Programming and Scripting

Need to filter data based on sysdate

I have a file from which I need to filter out certain lines when field 17 is less than sysdate. The file has date in YYYYMMDD HH:MI:SS format. Sample file is as below: PRUM,67016800 ,CC ,C1,67016800 ,00,Y,Y,2 ,US,BX,BOX ... (5 Replies)
Discussion started by: mady135
5 Replies

3. Shell Programming and Scripting

Filter more data in single column

Hi All, One of my source file column name RelationshipNumber. I need to filter below mentioned records in RelationshipNumber column. RelationshipNumber: S45678 D89763 Y09246579 A91234 If it is available in above mentioned column, then I need to print the entire line from my source... (2 Replies)
Discussion started by: suresh_target
2 Replies

4. Shell Programming and Scripting

How to filter out data file...?

Hi... I would like to filter out my data file....in two different way 1st way is like this, I will take one example..here... The script should ask like this. Enter min value in first column Enter max value in first column Enter min value in second column Enter max value in... (5 Replies)
Discussion started by: nex_asp
5 Replies

5. UNIX for Dummies Questions & Answers

Need to filter data from a file

Hi, I have a file with hundreds of records. There are four fields on each line, separated by semicolons. Name Height (meters) Country Continent (Africa,Asia,Europe,North America,Oceania,South America,The Poles) I need to Write the command to find display how many mountains appear... (1 Reply)
Discussion started by: erora
1 Replies

6. Linux

filter data

hi all, here is my question. i have the following data Block 1 a b c d e f g h i Block 2 j k l m n o p q r i need to grep all information from block 1 i.e 'a to i' (6 Replies)
Discussion started by: yprudent
6 Replies

7. Shell Programming and Scripting

Filter data and send email

Need script....We get sar file sa15 with 7 days data which gets appended.. we want filter and parse the outptu such that we should filter only last 48 hours data and then email it to out team. we read -- sar -f sa15 Please help me with the script either by use sar itself or even sar and awk... (10 Replies)
Discussion started by: noorm
10 Replies

8. Shell Programming and Scripting

extract data from a data matrix with filter criteria

Here is what old matrix look like, IDs X1 X2 Y1 Y2 10914061 -0.364613333 -0.362922333 0.001691 -0.450094667 10855062 0.845956333 0.860396667 0.014440333 1.483899333... (7 Replies)
Discussion started by: ssshen
7 Replies

9. Windows & DOS: Issues & Discussions

Filter data from text file

Hi All We have got a text file, which has data dumped from 60 tables. From these 60 tables of data we need data from 4 tables only. I tried assigning line numbers to filter out data, but it is not working as intended. below is the sample file ----Table1----- 3,dfs,43,df 4,sd,5,edd... (18 Replies)
Discussion started by: b_sri
18 Replies

10. UNIX for Dummies Questions & Answers

filter data

Hi I have two files and their names are donotcall.txt, filter_it.txt. I want output.txt file with all data that is in filter_it.txt file but not in donotcall.txt. Can anybody help me to write 1-2 lines unix code for this. donotcall.txt contains following data. Each row represent phone... (3 Replies)
Discussion started by: deep.singh
3 Replies
Login or Register to Ask a Question