Compare data with reference from other file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare data with reference from other file
# 1  
Old 03-16-2016
Compare data with reference from other file

Gents,

Please can you help with this.

I have a big file (file2) which contends many records increment every 25 rows ( column 1 ).

Then I have other file as reference (file1).. column 1 to 11.

I want to compare that all values in file2 (column 2 to 12.) match with values in file1..(column 1 to 11)

If some errors then give a list of the records where there is the mismatch.

File1 ( reference )
Code:
            2000     -26.518     -26.099     -25.703     -25.328     -24.972     -24.634     -24.313     -24.006     -23.714     -23.435
            2020     -23.169     -22.915     -22.671     -22.438     -22.216     -22.002     -21.798     -21.602     -21.415     -21.236
            2040     -21.064     -20.900     -20.742     -20.592     -20.448     -20.311     -20.179     -20.054     -19.935     -19.821
            2060     -19.713     -19.611     -19.514     -19.422     -19.335     -19.253     -19.177     -19.105     -19.037     -18.975
            2080     -18.917     -18.864     -18.816     -18.772     -18.732     -18.697     -18.666     -18.640     -18.618     -18.600
            2100     -18.587     -18.578     -18.574     -18.573     -18.577     -18.586     -18.599     -18.616     -18.638     -18.664
            2120     -18.694     -18.729     -18.769     -18.813     -18.862     -18.915     -18.974     -19.037     -19.105     -19.178
            2140     -19.257     -19.340     -19.429     -19.523     -19.623     -19.728     -19.840     -19.957     -20.081     -20.210
            2160     -20.347     -20.490     -20.640     -20.798     -20.963     -21.136     -21.317     -21.507     -21.705     -21.913
            2180     -22.131     -22.359     -22.598     -22.849     -23.112     -23.388     -23.678     -23.983     -24.304     -24.642
            2200     -24.998     -25.375     -25.775     -26.198     -26.648     -27.128     -27.642     -28.192     -28.785     -29.428
            2220     -30.127     -30.893     -31.740     -32.684     -33.749     -34.970     -36.399     -38.117     -40.269     -43.147
            2240     -47.493     -56.627     -57.890     -47.900     -43.376     -40.416     -38.214     -36.461     -35.005     -33.761
            2260     -32.676     -31.715     -30.853     -30.072     -29.359     -28.703     -28.097     -27.534     -27.009     -26.518
            2280     -26.057     -25.623     -25.214     -24.826     -24.460     -24.112     -23.781     -23.467     -23.168     -22.883
            2300     -22.611     -22.351     -22.103     -21.866     -21.640     -21.423     -21.216     -21.018     -20.829     -20.648
            2320     -20.475     -20.310     -20.152     -20.001     -19.857     -19.720     -19.590     -19.465     -19.347     -19.235
            2340     -19.128     -19.028     -18.933     -18.843     -18.759     -18.680     -18.606     -18.538     -18.474     -18.415
            2360     -18.362     -18.313     -18.269     -18.229     -18.195     -18.165     -18.140     -18.119     -18.104     -18.092
            2380     -18.086     -18.084     -18.087     -18.094     -18.106     -18.123     -18.144     -18.170     -18.201     -18.237
            2400     -18.277     -18.323     -18.374     -18.429     -18.490     -18.556     -18.627     -18.703     -18.785     -18.873
            2420     -18.966     -19.066     -19.171     -19.282     -19.400     -19.524     -19.655     -19.793     -19.938     -20.090
            2440     -20.250     -20.418     -20.595     -20.780     -20.974     -21.177     -21.391     -21.614     -21.849     -22.095
            2460     -22.354     -22.626     -22.912     -23.213     -23.530     -23.864     -24.218     -24.591     -24.987     -25.407
            2480     -25.855     -26.332     -26.843     -27.391     -27.982     -28.623     -29.320     -30.085     -30.930     -31.873


file2
Code:
    1       2000     -26.518     -26.099     -25.703     -25.328     -24.972     -24.634     -24.313     -24.006     -23.714     -23.435
    1       2020     -23.169     -22.915     -22.671     -22.438     -22.216     -22.002     -21.798     -21.602     -21.415     -21.236
    1       2040     -21.064     -20.900     -20.742     -20.592     -20.448     -20.311     -20.179     -20.054     -19.935     -19.821
    1       2060     -19.713     -19.611     -19.514     -19.422     -19.335     -19.253     -19.177     -19.105     -19.037     -18.975
    1       2080     -18.917     -18.864     -18.816     -18.772     -18.732     -18.697     -18.666     -18.640     -18.618     -18.600
    1       2100     -18.587     -18.578     -18.574     -18.573     -18.577     -18.586     -18.599     -18.616     -18.638     -18.664
    1       2120     -18.694     -18.729     -18.769     -18.813     -18.862     -18.915     -18.974     -19.037     -19.105     -19.178
    1       2140     -19.257     -19.340     -19.429     -19.523     -19.623     -19.728     -19.840     -19.957     -20.081     -20.210
    1       2160     -20.347     -20.490     -20.640     -20.798     -20.963     -21.136     -21.317     -21.507     -21.705     -21.913
    1       2180     -22.131     -22.359     -22.598     -22.849     -23.112     -23.388     -23.678     -23.983     -24.304     -24.642
    1       2200     -24.998     -25.375     -25.775     -26.198     -26.648     -27.128     -27.642     -28.192     -28.785     -29.428
    1       2220     -30.127     -30.893     -31.740     -32.684     -33.749     -34.970     -36.399     -38.117     -40.269     -43.147
    1       2240     -47.493     -56.627     -57.890     -47.900     -43.376     -40.416     -38.214     -36.461     -35.005     -33.761
    1       2260     -32.676     -31.715     -30.853     -30.072     -29.359     -28.703     -28.097     -27.534     -27.009     -26.518
    1       2280     -26.057     -25.623     -25.214     -24.826     -24.460     -24.112     -23.781     -23.467     -23.168     -22.883
    1       2300     -22.611     -22.351     -22.103     -21.866     -21.640     -21.423     -21.216     -21.018     -20.829     -20.648
    1       2320     -20.475     -20.310     -20.152     -20.001     -19.857     -19.720     -19.590     -19.465     -19.347     -19.235
    1       2340     -19.128     -19.028     -18.933     -18.843     -18.759     -18.680     -18.606     -18.538     -18.474     -18.415
    1       2360     -18.362     -18.313     -18.269     -18.229     -18.195     -18.165     -18.140     -18.119     -18.104     -18.092
    1       2380     -18.086     -18.084     -18.087     -18.094     -18.106     -18.123     -18.144     -18.170     -18.201     -18.237
    1       2400     -18.277     -18.323     -18.374     -18.429     -18.490     -18.556     -18.627     -18.703     -18.785     -18.873
    1       2420     -18.966     -19.066     -19.171     -19.282     -19.400     -19.524     -19.655     -19.793     -19.938     -20.090
    1       2440     -20.250     -20.418     -20.595     -20.780     -20.974     -21.177     -21.391     -21.614     -21.849     -22.095
    1       2460     -22.354     -22.626     -22.912     -23.213     -23.530     -23.864     -24.218     -24.591     -24.987     -25.407
    1       2480     -25.855     -26.332     -26.843     -27.391     -27.982     -28.623     -29.320     -30.085     -30.930     -31.873
    2       2000     -90.000     -26.099     -25.703     -25.328     -24.972     -24.634     -24.313     -24.006     -23.714     -23.435
    2       2020     -23.169     -22.915     -22.671     -22.438     -22.216     -22.002     -21.798     -21.602     -21.415     -21.236
    2       2040     -21.064     -20.900     -20.742     -20.592     -20.448     -20.311     -20.179     -20.054     -19.935     -19.821
    2       2060     -19.713     -19.611     -19.514     -19.422     -19.335     -19.253     -19.177     -19.105     -19.037     -18.975
    2       2080     -18.917     -18.864     -18.816     -18.772     -18.732     -18.697     -18.666     -18.640     -18.618     -90.000
    2       2100     -18.587     -18.578     -18.574     -18.573     -18.577     -18.586     -18.599     -18.616     -18.638     -18.664
    2       2120     -18.694     -18.729     -18.769     -18.813     -18.862     -18.915     -18.974     -19.037     -19.105     -19.178
    2       2140     -19.257     -19.340     -19.429     -19.523     -19.623     -19.728     -19.840     -19.957     -20.081     -20.210
    2       2160     -20.347     -20.490     -20.640     -20.798     -20.963     -21.136     -21.317     -21.507     -21.705     -21.913
    2       2180     -22.131     -22.359     -22.598     -22.849     -23.112     -23.388     -23.678     -23.983     -24.304     -24.642
    2       2200     -24.998     -25.375     -25.775     -26.198     -26.648     -27.128     -27.642     -28.192     -28.785     -29.428
    2       2220     -30.127     -30.893     -31.740     -32.684     -33.749     -34.970     -36.399     -38.117     -40.269     -43.147
    2       2240     -47.493     -56.627     -57.890     -47.900     -43.376     -40.416     -38.214     -36.461     -35.005     -33.761
    2       2260     -32.676     -31.715     -30.853     -30.072     -90.000     -28.703     -28.097     -27.534     -27.009     -26.518
    2       2280     -26.057     -25.623     -25.214     -24.826     -24.460     -24.112     -23.781     -23.467     -23.168     -22.883
    2       2300     -22.611     -22.351     -22.103     -21.866     -21.640     -21.423     -21.216     -21.018     -20.829     -20.648
    2       2320     -20.475     -20.310     -20.152     -20.001     -19.857     -19.720     -19.590     -19.465     -19.347     -19.235
    2       2340     -19.128     -19.028     -18.933     -18.843     -18.759     -18.680     -18.606     -18.538     -18.474     -18.415
    2       2360     -18.362     -18.313     -18.269     -18.229     -18.195     -18.165     -18.140     -18.119     -18.104     -18.092
    2       2380     -18.086     -18.084     -18.087     -18.094     -18.106     -18.123     -18.144     -18.170     -18.201     -18.237
    2       2400     -18.277     -18.323     -18.374     -18.429     -18.490     -18.556     -18.627     -18.703     -18.785     -18.873
    2       2420     -18.966     -19.066     -19.171     -19.282     -19.400     -19.524     -19.655     -19.793     -19.938     -20.090
    2       2440     -20.250     -20.418     -20.595     -20.780     -20.974     -21.177     -21.391     -21.614     -21.849     -22.095
    2       2460     -22.354     -22.626     -22.912     -23.213     -23.530     -23.864     -24.218     -24.591     -24.987     -25.407
    2       2480     -25.855     -26.332     -26.843     -27.391     -27.982     -28.623     -29.320     -30.085     -30.930     -31.873
    3       2000     -26.518     -26.099     -25.703     -25.328     -24.972     -24.634     -24.313     -24.006     -23.714     -23.435
    3       2020     -23.169     -22.915     -22.671     -22.438     -22.216     -22.002     -21.798     -21.602     -21.415     -21.236
    3       2040     -21.064     -20.900     -20.742     -20.592     -20.448     -20.311     -20.179     -20.054     -19.935     -19.821
    3       2060     -19.713     -19.611     -19.514     -19.422     -19.335     -19.253     -19.177     -19.105     -19.037     -18.975
    3       2080     -18.917     -18.864     -18.816     -18.772     -18.732     -18.697     -18.666     -18.640     -18.618     -18.600
    3       2100     -18.587     -18.578     -18.574     -18.573     -18.577     -18.586     -18.599     -18.616     -18.638     -18.664
    3       2120     -18.694     -18.729     -18.769     -18.813     -18.862     -18.915     -18.974     -19.037     -19.105     -19.178
    3       2140     -19.257     -19.340     -19.429     -19.523     -19.623     -19.728     -19.840     -19.957     -20.081     -20.210
    3       2160     -20.347     -20.490     -20.640     -20.798     -20.963     -21.136     -21.317     -21.507     -21.705     -21.913
    3       2180     -22.131     -22.359     -22.598     -22.849     -23.112     -23.388     -23.678     -23.983     -24.304     -24.642
    3       2200     -24.998     -25.375     -25.775     -26.198     -26.648     -27.128     -27.642     -28.192     -28.785     -29.428
    3       2220     -30.127     -30.893     -31.740     -32.684     -33.749     -34.970     -36.399     -38.117     -40.269     -43.147
    3       2240     -47.493     -56.627     -57.890     -47.900     -43.376     -40.416     -38.214     -36.461     -35.005     -33.761
    3       2260     -32.676     -31.715     -30.853     -30.072     -29.359     -28.703     -28.097     -27.534     -27.009     -26.518
    3       2280     -26.057     -25.623     -25.214     -24.826     -24.460     -24.112     -23.781     -23.467     -23.168     -22.883
    3       2300     -22.611     -22.351     -22.103     -21.866     -21.640     -21.423     -21.216     -21.018     -20.829     -20.648
    3       2320     -20.475     -20.310     -20.152     -20.001     -19.857     -19.720     -19.590     -19.465     -19.347     -19.235
    3       2340     -19.128     -19.028     -18.933     -18.843     -18.759     -18.680     -18.606     -18.538     -18.474     -18.415
    3       2360     -18.362     -18.313     -18.269     -18.229     -18.195     -18.165     -18.140     -18.119     -18.104     -18.092
    3       2380     -18.086     -18.084     -18.087     -18.094     -18.106     -18.123     -18.144     -18.170     -18.201     -18.237
    3       2400     -18.277     -18.323     -18.374     -18.429     -18.490     -18.556     -18.627     -18.703     -18.785     -18.873
    3       2420     -18.966     -19.066     -19.171     -19.282     -19.400     -19.524     -19.655     -19.793     -19.938     -20.090
    3       2440     -20.250     -20.418     -20.595     -20.780     -20.974     -21.177     -21.391     -21.614     -21.849     -22.095
    3       2460     -22.354     -22.626     -22.912     -23.213     -23.530     -23.864     -24.218     -24.591     -24.987     -25.407
    3       2480     -25.855     -26.332     -26.843     -27.391     -27.982     -28.623     -29.320     -30.085     -30.930     -31.873
    4       2000     -90.000     -26.099     -25.703     -25.328     -24.972     -24.634     -24.313     -24.006     -23.714     -23.435
    4       2020     -23.169     -22.915     -22.671     -22.438     -22.216     -22.002     -21.798     -21.602     -21.415     -21.236
    4       2040     -21.064     -20.900     -20.742     -20.592     -20.448     -20.311     -20.179     -20.054     -19.935     -19.821
    4       2060     -19.713     -19.611     -19.514     -19.422     -19.335     -19.253     -19.177     -19.105     -19.037     -18.975
    4       2080     -18.917     -18.864     -18.816     -18.772     -18.732     -18.697     -18.666     -18.640     -18.618     -90.000
    4       2100     -18.587     -18.578     -18.574     -18.573     -18.577     -18.586     -18.599     -18.616     -18.638     -18.664
    4       2120     -18.694     -18.729     -18.769     -18.813     -18.862     -18.915     -18.974     -19.037     -19.105     -19.178
    4       2140     -19.257     -19.340     -19.429     -19.523     -19.623     -19.728     -19.840     -19.957     -20.081     -20.210
    4       2160     -20.347     -20.490     -20.640     -20.798     -20.963     -21.136     -21.317     -21.507     -21.705     -21.913
    4       2180     -22.131     -22.359     -22.598     -22.849     -23.112     -23.388     -23.678     -23.983     -24.304     -24.642
    4       2200     -24.998     -25.375     -25.775     -26.198     -26.648     -27.128     -27.642     -28.192     -28.785     -29.428
    4       2220     -30.127     -30.893     -31.740     -32.684     -33.749     -34.970     -36.399     -38.117     -40.269     -43.147
    4       2240     -47.493     -56.627     -57.890     -47.900     -43.376     -40.416     -38.214     -36.461     -35.005     -33.761
    4       2260     -32.676     -31.715     -30.853     -30.072     -90.000     -28.703     -28.097     -27.534     -27.009     -26.518
    4       2280     -26.057     -25.623     -25.214     -24.826     -24.460     -24.112     -23.781     -23.467     -23.168     -22.883
    4       2300     -22.611     -22.351     -22.103     -21.866     -21.640     -21.423     -21.216     -21.018     -20.829     -20.648
    4       2320     -20.475     -20.310     -20.152     -20.001     -19.857     -19.720     -19.590     -19.465     -19.347     -19.235
    4       2340     -19.128     -19.028     -18.933     -18.843     -18.759     -18.680     -18.606     -18.538     -18.474     -18.415
    4       2360     -18.362     -18.313     -18.269     -18.229     -18.195     -18.165     -18.140     -18.119     -18.104     -18.092
    4       2380     -18.086     -18.084     -18.087     -18.094     -18.106     -18.123     -18.144     -18.170     -18.201     -18.237
    4       2400     -18.277     -18.323     -18.374     -18.429     -18.490     -18.556     -18.627     -18.703     -18.785     -18.873
    4       2420     -18.966     -19.066     -19.171     -19.282     -19.400     -19.524     -19.655     -19.793     -19.938     -20.090
    4       2440     -20.250     -20.418     -20.595     -20.780     -20.974     -21.177     -21.391     -21.614     -21.849     -22.095
    4       2460     -22.354     -22.626     -22.912     -23.213     -23.530     -23.864     -24.218     -24.591     -24.987     -25.407
    4       2480     -25.855     -26.332     -26.843     -27.391     -27.982     -28.623     -29.320     -30.085     -30.930     -31.873

output

Code:
error in code 2
error in code 4

Appreciate your help
# 2  
Old 03-16-2016
No attempts from your side?

Try
Code:
awk '
NR == FNR       {for (i=1; i<=NF; i++) T[NR%25,i] = $i
                 next
                }
                {for (i=2; i<=NF; i++) if (T[FNR%25,i-1] != $i) ERR[$1] = 1
                }
END             {for (e in ERR) print "error in code", e 
                }
' file1 file2
error in code 2
error in code 4

This User Gave Thanks to RudiC For This Post:
# 3  
Old 03-16-2016
Hi RudiC.

Thanks for your help.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract data using a reference

Gents, If there the possibility can to extract data using a reference from other file. input.txt ( big file which contends all data output.txt ( data extracted ) selection.txt ( information to extract the data Example In file input.txt there is big data each record have 56 lines like... (3 Replies)
Discussion started by: jiam912
3 Replies

2. Shell Programming and Scripting

Compare the source definition file with data file

Hi How to compare the source definition file in unix with the data file . Please can you share me example if some one has done it before (3 Replies)
Discussion started by: Raj4fusion
3 Replies

3. Shell Programming and Scripting

Replace a value using a reference data from other file

Gents, Can you please help me to solve this case In my input file I have a values in column 49 which always need to be one, but sometimes the system create a value 2, in this case I need to go to search in the original file and replace the values in the row where the value 2 is and in the... (6 Replies)
Discussion started by: jiam912
6 Replies

4. Shell Programming and Scripting

Compare two files and write data to second file using awk

Hi Guys, I wanted to compare a delimited file and positional file, for a particular key files and if it matches then append the positional file with some data. Example: Delimited File -------------- Byer;Amy;NONE1;A5218257;E5218257 Byer;Amy;NONE1;A5218260;E5218260 Positional File... (3 Replies)
Discussion started by: Ajay Venkatesan
3 Replies

5. Shell Programming and Scripting

Compare 2 text file with 1 column in each file and write mismatch data to 3rd file

Hi, I need to compare 2 text files with around 60000 rows and 1 column. I need to compare these and write the mismatch data to 3rd file. File1 - file2 = file3 wc -l file1.txt 58112 wc -l file2.txt 55260 head -5 file1.txt 101214200123 101214700300 101250030067 101214100500... (10 Replies)
Discussion started by: Divya Nochiyil
10 Replies

6. Shell Programming and Scripting

Compare a file with all others then print off data

my script so far nawk -F, 'NR==FNR{a++;next} a{b++} END{for(i in b){if(b-1){print i";\t\t"b}else{print "NEW:"i";\t\t1"} } }' OFS=, 20111228.csv *.csv | sort NE:221478,SHELF:13,SLOT:4; 2 NE:221726,SHELF:8,SLOT:1; 2 NE:222318,SHELF:14,SLOT:1; 9... (20 Replies)
Discussion started by: llcooljatt
20 Replies

7. Shell Programming and Scripting

help to compare data from file using shell script to c++ program

hello ....i am new in shell script..I have one shell script which read inputs from file...and second is c++ pgm..i call my script from c++ programm..to compare file values to c++ pgm value.. our shell script is shubhi.sh #! /bin/sh while read line do echo -e"$line" done<file1 this script... (1 Reply)
Discussion started by: shubhig15
1 Replies

8. Shell Programming and Scripting

Compare and print out data only appear in file 1 problem

Below is the data content of file_1 and file_2: file_1 >sample_1 FKGJGPOPOPOQA ASDADWEEWERE ASDAWEWQWRW ASDASDASDASDD file_2 >sample_1 DRTOWPFPOPOQA ASDADWEEASDF ASDADRTYWRW ASDASDASDASDD I got try the following perl script. Unfortunately, it can't give my desired output result... (7 Replies)
Discussion started by: patrick87
7 Replies

9. UNIX for Dummies Questions & Answers

Compare Data in the same file

Dear Unix-Gurus, I'm trying to write a script to compare the data in a log file. Here's how my logfile will look like: 'List All A0 Data in Destination Server' A0567 A0678 A0789 List A0 Files in Source Server A0567 A0678 A0789 So if the file match in Source Server match Destination... (1 Reply)
Discussion started by: lweegp
1 Replies

10. Shell Programming and Scripting

Compare data in 2 files and delete if file exist

Hi there, I have written a script called "compare" (see below) to make comparison between 2 files namely test_put.log and Output_A0.log #!/bin/ksh while read file do found="no" while read line do echo $line | grep $file > /dev/null if then echo $file found found="yes" break fi... (3 Replies)
Discussion started by: lweegp
3 Replies
Login or Register to Ask a Question