Search values between ranges in File1 within File2

cgkmal · #1 (**permalink**) 01-31-2009

Hi people,

I have 2 files, one with a list of non consecutive ranges (File1.txt), where each range begins with the value in column 1
and finishes with the value in column 2 in the same line, as can be seen above.

Code:

215312581156279  	215312581166279
215312582342558  	215312582357758
215312584725516  	215312584735016
215312589480032  	215312589490032
215312589491032  	215312589501032
215312589592032  	215312589602032
215312589614332  	215312589624332
215312589636632  	215312589646632
215312589658932  	215312589668932
215312589681232  	215312589691232
215312589703532  	215312589713532

In the 2nd file (File2.txt), I have less than 4 millions of lines and I need to search only those lines that contain the values
within the specified ranges in File1.txt and send the matching lines to a new file (File3.txt).

I know how to do with awk or even grep for a few values in the way showed below

Code:

awk '/215312581156279/||/215312581156280/||/215312581156281/' File2.txt > File3.txt

The above command would be for the first 3 values of the first range, but every range in File1.txt contain
easy 10,000 values and sometimes more.

How can I say to awk that I need to search from first value up to last value for each range?

Thanks in advance for any help on this.

Best regards.

Corona688 · #2 (**permalink**) 01-31-2009

Things like awk and grep use string matching, which are great for matching strings. Matching a range of numbers is not a string matching problem, since it means needing to actually process and compare the value of the numbers, not the strings representing them.

I'm sure better perl coders than me could shrink this program down to three lines of garbage, but this seems to work at least.

Code:

#!/usr/bin/perl

my $arr=[];
open FILE1, "file1";
# Get a list of number ranges, store it in an array of arrays
while($line=<FILE1>)
{
        if($line =~ /([0-9]+)[ \t]*([0-9]+)/)
        {
                $arr[++$#arr]=[$1, $2];
        }
}

close FILE1;

# Read in lines from file2
open FILE2, "file2";
while($line=<FILE2>)
{
        $match=0;

# Extract all numbers from $line
        while(($match == 0) && ($line =~ /([0-9]+)/g))
        {
# check each number against the ranges given
                for($n=0; ($n<$#arr)&&($match==0); $n++)
                {
# If it matches, stop hunting and print the line.
                        if(($arr[$n][0] <= $1) &&
                                ($arr[$n][1] >= $1))
                        {
                                print $line;
                                $match=1;
                        }
                }
        }
}

exit 0;

Franklin52 · #3 (**permalink**) 01-31-2009

Another one with awk, assuming you have only numbers in the second file, otherwise, replace $0 in the if statement with the desired field:

Code:

awk 'NR==FNR{a[++i]=$1;b[i]=$2;next}
{ for(j=1;j<=i;j++) {
    if($0>=a[j] && $0<=b[j]) {print;break}
  }
}' file1 file2

Regards

Corona688 · #4 (**permalink**) 01-31-2009

How large a value can awk use? These ranges are far higher values than the 32-bit max, I used perl specifically because it can handle arbitrarily large numbers.

cgkmal · #5 (**permalink**) 01-31-2009

Hey Corona688,

Many thanks for take your time to help me, I think it works but I receive an error when I run it, syntax error '}', I´ll try
to find out why this happens when I run some perl scripts in the emulator I used, it could be some libraries are missing.
I´ll try to test in other system, but really thanks.

Hi Franklin again,

Thanks a lot for the help, it works nice, nice when I replaced $0 with the desired field because you anticipated very good the
kind of file, the 2nd file it has numbers and letters. Using $0 didn´t show correct info, but when I replace with $1 or $2
worked perfectly man!!!.

Again thank both for your valuable help.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
Search, replace string in file1 with string from (lookup table) file2?	gstuart	Shell Programming and Scripting	9	06-08-2009 06:11 AM
awk/sed search lines in file1 matching columns in file2	floripoint	Shell Programming and Scripting	1	12-17-2008 11:36 PM
Awk Compare File1 File2 on f2	RacerX	Shell Programming and Scripting	4	10-27-2008 09:50 AM
match value from file1 in file2	myguess21	Shell Programming and Scripting	2	02-21-2008 11:39 AM
Awk Compare f1,f2,f3 of File1 with f1 of File2	RacerX	Shell Programming and Scripting	6	11-09-2007 01:34 AM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Switch to Linear Mode Hybrid Mode Switch to Threaded Mode	Rate This Thread:


	Powered by