Compare and Count...


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare and Count...
# 1  
Old 04-21-2011
Compare and Count...

Hello all.. i am new to shell scripting.. learning the things..i am struck at a place here..

I have divided one table in two two tables for comparision..

X1Left1000010025
X2Left10501075
X2Left30503075
X3Left90009025
Y1Left30003025
Y1Left45004525
Y2Left70007025
Y2Left82008225
Y2Left20002025

Q2T2start2stop2
X1Right10501075
X1Right30003025
X1Right2000020025
X2Right20502075
X2Right70507075
X3Right90509075
Y1Right40004025
Y1Right55005525
Y2Right77007725
Y2Right92009225
Y2Right1400014025

Now
i want to compare all X1's in first table with X1's in second table,
X2's with X2's like that and
subtract first table stop column from second table and
Out of all combinations display results of pairs which have difference less than 3000
QueryTypeStartStopQ2T2start2stop2Difference
X1Left10001025X1Right1050107550
X1Left10001025X1Right300030252000
X1Left20002025X1Right300030251000
X2Left10501075X2Right205020751000
X2Left30503075X2Right205020752000
X3Left90009025X3Right9050907550
         
Y1Left30003025Y1Right400040251000
Y1Left30003025Y1Right550055252500
Y1Left45004525Y1Right40004025500
Y1Left45004525Y1Right550055251000
Y2Left70007025Y2Right77007725700
Y2Left82008225Y2Right920092251000
Y2Left70007025Y2Right920092252200
Y2Left82008225Y2Right77007725500

Last edited by Scott; 04-21-2011 at 01:44 PM.. Reason: You can use a | in [TABLE] tags to make it look nicer
# 2  
Old 04-21-2011
It's not clear from your original posting exactly what the format is of the files that you are trying to manipulate. Can I check, are you dealing with flat-text files that are space-separated, or are you dealing with HTML files?
# 3  
Old 04-21-2011
I am dealing with tab delimited files here..
# 4  
Old 04-21-2011
Try this:
Code:
awk -v left="tab1.txt" -v right="tab2.txt" '
    BEGIN {
        # Read right-side into cache now
        while (getline < right)
        {
            if (!ridx) { rhead=$0; ridx=1 }
            else
            {
                rquery[ridx]=$1; rtype[ridx]=$2
                rstart[ridx]=$3; rstop[ridx]=$4
                ridx++
            }
        }

        # Process left-side one line at a time
        while (getline < left)
        {
            if (!lidx)
            {
                print $0 "\t" rhead "\tDifference"
                lidx=1
            }
            else
            {
                lquery=$1; lstop=$4

                # Find corresponding right-side
                for (i=1; i<ridx; i++)
                {
                    diff=rstop[i]-lstop
                    if (rquery[i]==lquery && diff < 3000)
                    {
                        print $0 "\t" rquery[i] "\t" rtype[i] "\t" \
                                rstart[i] "\t" rstop[i] "\t" diff
                    }
                }
            }
        }
    }'

This shows negative differences too, not sure what you wanted to do with those. Amend the code above to suit your exact need.

Regards,
Mark.
# 5  
Old 04-21-2011
Thanks mate.. works good for me.. i tried other way.. in three steps.. even that worked.. but this is just one code.. Thanks
  1. sort table 1 on first column using unix 'sort'
  2. sort table 2 on first column using unix 'sort'
  3. join both tables on first column using unix 'join'
  4. pipe the result into awk to calculate the difference and print the result
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to compare files and count

I am trying to compare 2 files and output the results in two files. The lines that are the same output to concordant.txt while the lines that do not match output to discordant.txt. Is there a way to count the lines after specific text (#CHROM POS ID REF ALT QUAL FILTER INFO FORMAT E1) in each... (2 Replies)
Discussion started by: cmccabe
2 Replies

2. Shell Programming and Scripting

Compare two files and count number of matching lines

Dear All, I would like to compare two files and return the number of matches found. Example File A Lx2 L1_Mus1 L1Md_T Lx5 L1M2 L1_Mus3 Lx3_Mus Lx9 Lx2A L1Md_A L1Md_F2 File B L1_Mus3 L1_Mus3 (3 Replies)
Discussion started by: paolo.kunder
3 Replies

3. Shell Programming and Scripting

Compare file1 header count with file2 line count

What I'm trying to accomplish. I receive a Header and Detail file for daily processing. The detail file comes first which holds data, the header is a receipt of the detail file and has the detail files record count. Before processing the detail file I would like to put a wrapper around another... (4 Replies)
Discussion started by: pone2332
4 Replies

4. Shell Programming and Scripting

FASTEN count line of dat file and compare with the CTRL file

Hi All, I thinking on how to accelerate the speed on calculate the dat file against the number of records CTRL file. There are about 300 to 400 folder directories that contains both DAT and CTL files. DAT contain all the flat files records CTL is the reference check file for the... (3 Replies)
Discussion started by: ckwan
3 Replies

5. Shell Programming and Scripting

Script to compare count of two csv files

Hi Guys, I need to write a script to compare the count of two csv files each having 5 columns. Everyday a csv file is recived. Now we need to compare the count of todays csv file with yesterday's csv file and if the total count of records is same in todays csv file and yesterday csv file out... (3 Replies)
Discussion started by: Vivekit82
3 Replies

6. Shell Programming and Scripting

Count, compare, execute ...

I'm working on a script that needs to execute only after directory B is equal in file count to directory A. It needs to check dir B every 5 min or so and when the file count is equal to A, it executes a script. I'm thinking of using wc -w but I'm not sure how to put it together. Any help... (7 Replies)
Discussion started by: scribling
7 Replies

7. Shell Programming and Scripting

compare 2 files and show count same content.

$ cat File1 Non HTTP response code:java.net.ConnectException225073X 000000005143329Load time: 402335410224 Non HTTP response code: ava.net.ConnectException206423X 000000005143330Load time: 402305687161 Non HTTP response code: ava.net.ConnectException290212X 000000005143331Load time:... (1 Reply)
Discussion started by: ooilinlove
1 Replies

8. Shell Programming and Scripting

Shell script to count number of ~ from each line and compare with next line

Hi, I have created one shell script in which it will count number of "~" tilda charactors from each line of the file.But the problem is that i need to count each line count individually, that means. if line one contains 14 "~"s and line two contains 15 "~"s then it should give an error msg.each... (3 Replies)
Discussion started by: Ganesh Khandare
3 Replies

9. Shell Programming and Scripting

Shell Script to compare files, check current date and count

Hello - I have written the following basic shell script to count files, compare files and look for a particular strings in a file. Problem 1: How do I define more than 1 file location? #!/bin/bash #this is a test script FILES=$(ls /home/student/bin/dir1, home/student/bin/dir2)... (0 Replies)
Discussion started by: DallasT
0 Replies

10. Shell Programming and Scripting

count and compare no of records in bash shell script.

consider this as a csv file. H,0002,0002,20100218,17.25,P,barani D,1,2,3,4,5,6,7,8,9,10,11 D,1,2,3,4,5,6,7,8,9,10,11 D,1,2,3,4,5,6,7,8,9,10,11 D,1,2,3,4,5,6,7,8,9,10,11 D,1,2,3,4,5,6,7,8,9,10,11 T,5 N i want to read the csv file and count the number of rows that start with D and... (11 Replies)
Discussion started by: barani75
11 Replies
Login or Register to Ask a Question