Filtering records from 1 file based on some manipulation doen on second file


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Filtering records from 1 file based on some manipulation doen on second file
# 1  
Old 12-21-2011
Filtering records from 1 file based on some manipulation doen on second file

Hi,

I am looking for an awk script which should help me to meet the following requirement:

File1 has records in following format

Code:
INF: FAILEd RECORD AB1234
INF: FAILEd RECORD PQ1145
INF: FAILEd RECORD AB3215
INF: FAILEd RECORD AB6114
.........................
.........................

Each record(basically the ID 'AB1234', 'PQ1145' and so on) of File1, should be validated against the following File2, which has format like:
Code:
AB0001-AB1111
AB1112-AB2222
.............
.............
PQ1111-PQ2222
.............
XY1234-XY5678
.............

Note that File2 has different ranges for code AB, PQ, XY and so on, so validation won't be straight forward and hence code should be compared first and then the ID should check if it lies in any of the ranges above, associated to that particular code. If it does then the record should be filtered out from File1.

I'll really appreciate if anyone please help me out. Thanks

Last edited by zxmaus; 12-21-2011 at 04:38 AM..
# 2  
Old 12-21-2011
Wrong language, but Perl would handle this well...

So match the text part and see if the numeric part falls within any of the resulting ranges
I'm not sure how you want to perform your validation, so take a look at the following and feel free to ask if you hit a snag
Code:
~/tmp]$ perl -e '
open (RANGES,"File2"); 
while(<RANGES>){
   $ranges{$1}=[$2,$3] if (/^([A-Z]{2})(\d{4})-\1(\d{4})/);
}
while (<>){
   chomp;
   print "$1 $2 - ", $ranges{$1}[0],"=>",$ranges{$1}[1],"\n" if /\s([A-Z]{2})(\d{4})$/;
}' File1
AB 1234 - 1112=>2222
PQ 1145 - 1111=>2222
AB 3215 - 1112=>2222
AB 6114 - 1112=>2222

This User Gave Thanks to Skrynesaver For This Post:
# 3  
Old 12-21-2011
Thanks @Skrynesaver, but this requirement needs to fit in an existing shell script, thats why I need to write in a shell script only. Though I am able to do it through a simple shell scripting, but I want to optimise it by using awk which 'does more in less code'.
Appreciate your thoughts on this.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Filtering records of a csv file based on a value of a column

Hi, I tried filtering the records in a csv file using "awk" command listed below. awk -F"~" '$4 ~ /Active/{print }' inputfile > outputfile The output always has all the entries. The same command worked for different users from one of the forum links. content of file I was... (3 Replies)
Discussion started by: sunilmudikonda
3 Replies

2. Shell Programming and Scripting

Need help with filtering records in a file

Hi, I have following records in a file more file1.txt setting applicaction ABC for user setting applicaction CDE for user setting applicaction XXX for user logging applicaction XXX for user I need to filter out records which have strings " setting... (5 Replies)
Discussion started by: manid
5 Replies

3. Shell Programming and Scripting

Delete the records in file based on lookup file.

Hi I have two files one.txt and two.txt one.txt 123 324 456 235 456 two txt abc one 000 123 abc abc one 000 456 abc abc one 000 122 abc abc one 000 111 abc My question here is, the records which are present in one.txt has to deleted in second file two.txt my output result... (2 Replies)
Discussion started by: Ganesh L
2 Replies

4. Shell Programming and Scripting

Filtering first file columns based on second file column

Hi friends, I have one file like below. (.csv type) SNo,data1,data2 1,1,2 2,2,3 3,3,2 and another file like below. Exclude data1 where Exclude should be treated as column name in file2. I want the output shown below. SNo,data2 1,2 2,3 3,2 Where my data1 column got removed from... (2 Replies)
Discussion started by: ks_reddy
2 Replies

5. Shell Programming and Scripting

File manipulation based on values in file

Hi, using Shell to do some file manipulation here. Input - input.txt "2006/2007", "string1a","string2v","stringf3" "2006/2007", "string12b","string30c","string10d" "2006/2007", "string22","string22","string11" "2007/2008", "string1a","string2v","stringf3" "2007/2008",... (5 Replies)
Discussion started by: pgop
5 Replies

6. Shell Programming and Scripting

select records from one file based on a second file

Hi all: I have two files: file1: 74 DS 9871 199009871 1 1990 4 1 165200 Sc pr de te sa ox 1.0 1.0 13.0000 35.7560 5.950 3.0 3.0 13.0100 35.7550 5.970 ** 74 DS 99004 74DS99004 6738 1990 4 1 165200 Eb pr de te sa ox 1.0 1.0 13.0000 ... (7 Replies)
Discussion started by: rleal
7 Replies

7. Shell Programming and Scripting

filtering records based on numeric field value in 8th position

I have a ";" delimited file.Whcih conatins a number fileds of length 4 charcters in 8th position But there is a alphanumeric charcters like : space, ";" , "," , "/" , "23-1" , "23 1" , "aqjhdj" , "jun-23" , "APR-04" , "4:00AM" , "-234" , "56784 ", "." , "+" "_" , "&" , "*" , "^" , "%" , "!"... (2 Replies)
Discussion started by: indusri
2 Replies

8. Shell Programming and Scripting

filtering one file based on results from other- AGAIN

I have asked this question here before and got the answer too. Unfortunately I used only one record as an example and the script works fine for one record but not for more than one record. Can anybody help me with writing a script for the data that I want to use from one file based on the... (13 Replies)
Discussion started by: digipak
13 Replies

9. Shell Programming and Scripting

filtering one file based on results from other

Can anybody help me with writing a script for the data that I want to use from one file based on the data from another file. I have file1 in this form; (the first field represents a well name and the second field represents the depth of interest) FILE1 -------- DATA_35_0 ... (2 Replies)
Discussion started by: digipak
2 Replies

10. UNIX for Dummies Questions & Answers

Filtering records of a file based on a value of a column

Hi all, I would like to extract records of a file based on a condition. The file contains 47 fields, and I would like to extract only those records that match a certain value in one of the columns, e.g. COL1 COL2 COL3 ............... COL47 1 XX 45 ... (4 Replies)
Discussion started by: risk_sly
4 Replies
Login or Register to Ask a Question