Matching Numbers in Bash/AWK


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Matching Numbers in Bash/AWK
# 1  
Old 04-19-2011
Matching Numbers in Bash/AWK

Hi,

I need to match up some numbers in one file to the closest numbers in other file and produce an output file.

File one (f1.txt) is laid out like this
 
PCode Lon Lat
AB10 1AA 57.148235 -2.096648
BB2 3JD 53.728563 -2.47852
LU4 9ET 51.912308 -0.4785
etc,,,,

File two (F2.txt) is liad out like this
 
Node Lon Lat
1 48.39 -13.0
2 48.39 -12.89
..........
7079 51.91 -2.0
10779 53.78 -2.44
17322 57.09 -2.11
etc,,,

And I would like the output file to read
 
Pcode Lon Lat Lon Lat Node
AB10 1AA 57.148235 -2.096648 57.09 -2.11 17322
BB2 3JD 53.728563 -2.47852 53.78 -2.44 10779
LU4 9ET 51.912308 -0.4785 51.91 -2.0 7079
As you can see I need to find the closest match from collumn 2 on boths files and collumn 3 on both files and output the above.


Many thanks,
Ian
# 2  
Old 04-19-2011
What you are asking requires substantial logic. How do you determine 'close enough'
We need an exact statement of how you intend to determine 'closeness' and what to do when no close match is found.

Is it 1%? You cannot just 'eyeball' data with programming languages easily.
# 3  
Old 04-19-2011
Hi Jim,

Perhaps nearest should be the key. How I'm thinking is that as the files are looping the nearest match can be found and that is the one to use.

So $2 on file one has a number of 53.728563 and the nearest match on $2 on file two is 53.78.
Once that number has been found then loop though $3 on both files, but only look at the 53.78 numbers (the number we found on $2) there will be quite a few on file 2 and return then nearest match which would be -2.44 and print out the rest..........

I dont think I'm making this sound easy am I? But please stay with me on this :-)

Cheers,
Ian

---------- Post updated at 01:17 PM ---------- Previous update was at 01:07 PM ----------

Hang on.....I will try and explain this again..

If I have one file called A.txt with
1
2
3
4
5
And I have another with
1.1
2.1
1.3
5.1
0.9
Is it possible to return a output file like this
1 0.9
1 1.1
1 1.3
2 2.1
5 5.1

As you can see I have matched up the nearest number from one file to another... I'm not sure whether this is possible to do, but this is what I desire :-)

Many thanks for looking at this

Cheers,
Ian
# 4  
Old 04-21-2011
how about you try below perl?

Code:
open $fh,"<f1.txt";
while(<$fh>){
	chomp;
	my @arr = split;
	$hash{$arr[2]}->{'SEQ'}=$.;
	$hash{$arr[2]}->{'VAL'}=$_;
	$hash{$arr[2]}->{'GAP'}=10000;
}
open $fh2,"<f2.txt";
while(<$fh2>){
	my @tmp = split;
	foreach my $key(keys %hash){
		my $new_gap = abs($key-$tmp[1]);
		if($new_gap < $hash{$key}->{'GAP'}){
			$hash{$key}->{'GAP'} = $new_gap;
			$hash{$key}->{'APPEND'}=$tmp[1]." ".$tmp[2]." ".$tmp[0];
		}
	}
}
foreach my $key(sort {$hash{$a}->{'SEQ'} <=> $hash{$b}->{'SEQ'}} keys %hash){
	print $hash{$key}->{'VAL'}," ", $hash{$key}->{'APPEND'},"\n";
}

This User Gave Thanks to summer_cherry For This Post:
# 5  
Old 04-21-2011
Hi Summer,

thanks for the script. I have not really tried perl, only bits and bobs, nothing major. I will have a go at it though, if you don't mind coaching me along every now and again. :-)
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to combine all matching dates and remove non-matching

Using the awk below I am able to combine all the matching dates in $1, but I can not seem to remove the non-matching from the file. Thank you :). file 20161109104500.0+0000,x,5631 20161109104500.0+0000,y,2 20161109104500.0+0000,z,2 20161109104500.0+0000,a,4117... (3 Replies)
Discussion started by: cmccabe
3 Replies

2. Shell Programming and Scripting

awk decimal point numbers matching

Hi All, Can some one help me in identifying the significance of character "$" ,Which is playing critical role in matching decimal point numbers as below. $ echo "01#.01"|awk '{if ($0 ~ /^+(\.*)?$/) print}' $ echo "01#.01"|awk '{if ($0 ~ /^+(\.*)?/) print}' 01#.01 $ Regards, Rmkganesh. (3 Replies)
Discussion started by: rmkganesh
3 Replies

3. Shell Programming and Scripting

Matching and retreiving numbers

Hi, I have one file 1.txt with one field consist of following Ids (shortlisted 10 but showing 3 here): 00052 00184 00607 and then second file 2.txt with three fields (very big file): 00052 00184 12.73062 00052 00598 13.51205 00052 00599 13.92554 00052 00600 13.73358... (2 Replies)
Discussion started by: bioinfo
2 Replies

4. Shell Programming and Scripting

Adding numbers matching with words

Hi All, I have a file which looks like this: abc 1 abc 2 abc 3 abc 4 abc 5 bcd 1 bcd 3 bcd 3 bcd 5 cde 7 This file is just a miniature version of what I really have. Original file is some 1 million lines long. I have tried to come up with the code for what I wish to accomplish... (1 Reply)
Discussion started by: shoaibjameel123
1 Replies

5. Shell Programming and Scripting

Regex/egrep matching numbers in brackets

Experts: I don't know that regular expressions will ever be easy for me, so if one of you guru's could help out, I'd appreciate it. I'm trying to match a line in our syslog, but I can't figure out how to match a number inside a bracket. This is what I'm trying to match. "Jul 16 00:01:34... (2 Replies)
Discussion started by: jdveencamp
2 Replies

6. UNIX for Dummies Questions & Answers

Matching numbers of characters in two lines

Dear all, I'm stuck on a certain problem regarding counting the number of characters in one line and then adjusting the number of characters of another line to this number. This was my original input data: @HWI-ST471_57:1:1:1231:2079/2... (4 Replies)
Discussion started by: DerSeb
4 Replies

7. Shell Programming and Scripting

How to find the matching numbers between 2 files Using R language??

Hi friends,, I need ur help very urgently.I have 2 files which have more than 5000 numbers,I want to find the matching numbers between 2 files using R language.I dont know how to use for loop in R.I tried to get a result,,but didnt get, file1 is like 1061909 1162391... (1 Reply)
Discussion started by: sureshraj
1 Replies

8. Shell Programming and Scripting

count numbers of matching rows and replace its value in another file

Hello all, can you help me in this problem, assume We have two txt file (file_1 and file_3) one is file_1 contains the data: a 0 b 1 c 3 a 7 b 4 c 5 b 8 d 6 . . . . and I need to count the lines with the matching data (a,b,..) and print in new file called file_2 such as the... (4 Replies)
Discussion started by: GoldenFalcon10
4 Replies

9. Shell Programming and Scripting

Matching in bash

I know how to do this in perl, but I want to do it in bash script. Hope some one could help me, many thanks! I have a file looks like this: AB_21423 ********* BC_123 ********* CD_343 ********* AB_289 ********* *************** *************** I... (5 Replies)
Discussion started by: zx1106
5 Replies

10. Shell Programming and Scripting

matching numbers

hi all I have a 2 files. both the files have some numbers and i want to find out each number in file1 is existing or not in file2. if not then put it into new file. if yes then also in a seperate file i can not use diff command as the files are different and no order has been defined. ... (2 Replies)
Discussion started by: infyanurag
2 Replies
Login or Register to Ask a Question