Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Compare data in two files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare data in two files
# 1  
Old 02-15-2015
Compare data in two files
Gents,

Can you help please

I have a data base with lot information (file2) and I have some data in (file1) to compare.

Then the comparison should be done using the following keys:

Example ( values from file1 )
key1 = columns from 20-34 substr($0,20,15)
Code:
66705.00  19793

key2 = columns from 40-80 substr($0,40,41)
Code:
   1  2071  66500.00  19604.00  20016.001

Then using this keys need to compare that all information for key1 and range in key2. Matches in file2

The key1 is the same during 18 rows and then it change to other value, but the key2 changes every row.

So, the thing is to compare that all information in file1 match in file2..and when there is some difference..

Example in
file1
Code:
X  4960     22110  66705.00  19793.001 3441 40001  66704.00  19610.00  20016.001

file2
Code:
X              1   66705.00  19793.00  3441 36441  66704.00  19610.00  20016.001

values doesn't match in 40001 and 36441

The desired output will be:

66705.00 19793 has error.

If everything is ok not output will appears.

Please help me with this.
file1.txt (4.3 KB) 
file2.txt (8.5 KB) 
# 2  
Old 02-16-2015
You may want to try this one:
Code:
awk     'NR==FNR        {T[substr ($0, 20, 15) ".*" substr ($0, 40, 41)]=NR; next}
                        {for (t in T)   {if ($0 ~ t) {FOUND=1; break}
                                        }
                         if (!FOUND) print $3, $4 " has error."
                         FOUND=0
                        }
        ' /tmp/file1.txt /tmp/file2.txt

Last edited by RudiC; 02-16-2015 at 07:13 AM..
This User Gave Thanks to RudiC For This Post:
jiam912
# 3  
Old 02-16-2015
Hi RudiC

I did something like this and I got the desired output.
Code:
awk '{if(/^X/)print $0}' file1.txt > tmpX1
awk '{if(/^X/)print $0}' file2.txt > tmpX2
awk 'FNR == NR {c[substr($0,20,15)substr($0,52,5) ]++;next}c[(substr($0,20,15)substr($0,52,5))]-- == 1' tmpX1 tmpX2 >  tmpX3

cat tmpX1 | awk '{

				ln=substr($0,20,5);
				pt=substr($0,30,5);
				spr=substr($0,39,42);
				rpl=substr($0,52,5);
				rpl=substr($0,52,5);
				tape=$2;
				ffid=substr($0,8,8);
				printf( "%5d%5d%5d %s %4d %4d\n",ln,pt,rpl,spr,tape,ffid)}' | sort -k1n > tmp1

cat tmpX3 | awk '{

				ln=substr($0,20,5);
				pt=substr($0,30,5);
				spr=substr($0,39,42);
				rpl=substr($0,52,5);
				printf( "%5d%5d%5d %s\n",ln,pt,rpl,spr)}' | sort -k1n > tmp2


awk 'NR==FNR {a[substr($0,1,58)];next}!(substr($0,1,58) in a)' tmp2 tmp1 > tmp3

awk '{printf (" %5d/%5d has errors\n", substr($1,1,5),substr($1,6,5))}' tmp3		

rm -f *tmp*

Output

Code:
66705/19793 has errors

I will try your code now..
# 4  
Old 02-16-2015
Not all versions of cut offer the --output-delimiter=STRING option - mine does:
Code:
cut -c20-34,40-81 --output-delimiter=".*" /tmp/file2.txt | grep -vf- /tmp/file1.txt 
X  4960     22110  66705.00  19793.001 3441 40001  66704.00  19610.00  20016.001

---------- Post updated at 13:38 ---------- Previous update was at 13:34 ----------

and pipe it through cut again to only receive the two values you need:
Code:
 | cut -c20-34
66705.00  19793

This User Gave Thanks to RudiC For This Post:
jiam912
# 5  
Old 02-16-2015
Rudi C,
That cut version does not works for me Smilie

The previuos code works but take a lot of time and looks like as the files are not sorted as I did in my script .. your code got more data in the output

Many thanks for your help
# 6  
Old 02-16-2015
The awk script prints all lines in file2 that don't have a counterpart in file1. If it needs to be the other way round, exchange the file names, but check the printed fields, too.
This User Gave Thanks to RudiC For This Post:
jiam912
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare two files and write data to second file using awk

Hi Guys, I wanted to compare a delimited file and positional file, for a particular key files and if it matches then append the positional file with some data. Example: Delimited File -------------- Byer;Amy;NONE1;A5218257;E5218257 Byer;Amy;NONE1;A5218260;E5218260 Positional File... (3 Replies)
Discussion started by: Ajay Venkatesan
3 Replies

2. Shell Programming and Scripting

Compare columns of two files and retrieve data

Hi guys, I need your help. I have two files: file1 1 3 5 file2 1,XX 2,AA 3,BB 4,CC 5,DD I would like to compare the first column and where they are equal to write that output in a new file: 1,XX 3,BB (7 Replies)
Discussion started by: apenkov
7 Replies

3. Shell Programming and Scripting

Compare 2 files and match column data and align data from 3 column

Hello experts, Please help me in achieving this in an easier way possible. I have 2 csv files with following data: File1 08/23/2012 12:35:47,JOB_5330 08/23/2012 12:35:47,JOB_5330 08/23/2012 12:36:09,JOB_5340 08/23/2012 12:36:14,JOB_5340 08/23/2012 12:36:22,JOB_5350 08/23/2012... (5 Replies)
Discussion started by: asnandhakumar
5 Replies

4. Shell Programming and Scripting

compare date and time inside data of two files

i have two files with identical no of columns. 6th columns is date (MM/DD/YY format) and 7th columns is time (HH:MM:SS) format. I need to compare these two vaules and if the date & time is higher than fileA, save it on fileC; if the value is lower, then save it on fileD CONDITIONS... (7 Replies)
Discussion started by: ajiwww
7 Replies

5. Shell Programming and Scripting

Compare two files and get matching data

Hi, Can anyone help me to compare two files and get the matching data... say i have file1 and file2 ... file1 has 300 unique data with that i need to match with file2 to see how may are matching.. file2 have 1000 records. (4 Replies)
Discussion started by: zooby
4 Replies

6. Shell Programming and Scripting

Compare 3 files, delete data equals.

Hi, i have a problem, I have three files, file_1, File_2 file_3 and I need to compare the data with file_3 file_1, data that are equal to file_3 file_1 should be deleted, file_1 receive data and file_2 file_3. Ex: file_1 374905,2001, Selmar Santos, Técnico de Sistemas, U$3.000,00 789502,... (3 Replies)
Discussion started by: selmar
3 Replies

7. Shell Programming and Scripting

How to compare two flat files and get changed data

Hi, I need to compare two flat files (yesterday & today's data) and get only the changed data from flat files. In flat file i dont have data column or anything its just a string data in flat file.Can any one please let me know the script With Regds Shashi (3 Replies)
Discussion started by: jtshashidhar
3 Replies

8. Shell Programming and Scripting

How to compare data in two flat files and update them?

Hi All, I am giving an example similar to the problem I have. I have two data files of 10 columns each in which fields are delimited by comma(,). I need to compare compare the two files using the uniq col(col3). If there are any records in file1 and are not in file2 then I have check the value... (3 Replies)
Discussion started by: rajus19
3 Replies

9. Shell Programming and Scripting

Compare data in 2 files and delete if file exist

Hi there, I have written a script called "compare" (see below) to make comparison between 2 files namely test_put.log and Output_A0.log #!/bin/ksh while read file do found="no" while read line do echo $line | grep $file > /dev/null if then echo $file found found="yes" break fi... (3 Replies)
Discussion started by: lweegp
3 Replies

10. Shell Programming and Scripting

Compare data files

I would like to compare a data file (before and after a process has run) to identify if there are any differences. Can anyone help !! (1 Reply)
Discussion started by: ithomp
1 Replies
Login or Register to Ask a Question