Need help regarding comparison between two files through UNIX script

07-01-2015

Registered User

42, 0

Join Date: Jan 2015

Last Activity: 20 April 2020, 8:16 AM EDT

Posts: 42

Thanks Given: 11

Thanked 0 Times in 0 Posts

Hi RudiC/Mannu2525 ,
I was also using the sample data which I posted in post#3 .But not sure why in the mismatch details section ,below entries are coming though these entries are not present in sample source/target file.Also I dont need field count in the report.If the below entries can be removed ,then Field lebel Src & Tgt Mismatches should return correct count .

Code:

I ran the script like below.
./CompareScrpt1.sh FILE3_S1.txt FILE3_T1.txt

I just the modified file1 and file2 as $1 and $2 in the below mentioned script.

Code:

#!/bin/bash

echo "<<Analysis Report >>"
echo ""

echo "Src & tgt Count Validation"
echo "Source Count : `wc -l < $1`"
echo "Target Count : `wc -l < $2`"

awk -F\| 'BEGIN{ print "Mismatch in line", "Column Name", "Source Data", "Target Data"}
                {for (i=1; i<=NF; i++) arr[i]=$i
                 getline <FN
                 for (i=1; i<=NF; i++)
                                 {if (arr[i] != $i)
                                 print $1,i,arr[i], $i} }
' FN="$1" $2 > output.temp

echo ""
echo "Field lebel Src & Tgt Mismatches"
echo "No of Mismatches : `expr $(wc -l < output.temp) - 1`"
echo ""
echo "Mismatch Details:"
cat output.temp
rm output.temp
exit

After executing the above script ,analysis report is coming like below :

Code:

<<Analysis Report >>

Src & tgt Count Validation
Source Count : 4
Target Count : 4

Field lebel Src & Tgt Mismatches
No of Mismatches : 8

Mismatch Details:
Mismatch in line Column Name Source Data Target Data
000101 6 123455  123456
000101 20
000102 8 24 23
000102 20
000103 2 000004      3
000103 20
000104 6 3401    3400
000104 20

The above analysis report result is not displaying correct result,the analysis report should come like below :

Code:

<<Analysis Report >>

Src & tgt Count Validation
Source Count : 4
Target Count : 4

Field lebel Src & Tgt Mismatches
No of Mismatches : 4

Mismatch Details:
Mismatch in line Column Name Source Data Target Data
000101                6               123455       123456
000102                8               24              23
000103                2               000004       3
000104                6                3401          3400

If possible ,kindly look into the above mentioned issues.Also as I am new to unix script level ,if you kindly explain the block of code(specially the awk part) ,that will be really beneficial for my understanding.Thanks !

---------- Post updated 07-01-15 at 05:18 AM ---------- Previous update was 06-30-15 at 09:08 AM ----------

Hi RudiC/Mannu2525/All ,

I have tested the below mentinoed script for another sample file ,but analysis report does not provide us correct result.Please find below sample source & target file .
source file :

Code:

000001|     1|AQWWW|234,456.00  |     | 123456|     |41|abC| 0|xyZ|     |99900|0.00999|      |1|c|6|S|
000002|     2|11  4|1,234,456.99|     |      0|     |23|   |99|!  |     |00000|0.00000|      |2| |4|#|
000003|     3|!!@#$|0,000,001.10|     |      9|     | 0|XSW|12|  7|     |00100|0.00001|      |0|S|0| |
000004|     4|     |            |     |   3400|     | 2|r7!|72|xY1|     |01200|0.00045|      |9|W|3|2|

target file :

Code:

000001|100001|AQWWW|234,456.00  |     |123456|     |41|abC|00|xyZ|     |99900|0.00999|      |1|c|6|S
000002|000002|11  4|1,234,456.99|     |0|     |23|   |99|!  |     |0|0|      |2| |4|#
000003|000003|!!@#$|0,000,001.10|     |9|     |00|XSW|12|  7|     |100|0.00001|      |0|S|0| 
000004|000004|     |            |     |3400|     |02|r7!|72|xY1|     |1200|0.00045|      |9|W|3|2

script :

Code:

#!/bin/bash

echo "<<Analysis Report >>"
echo ""

echo "Src & tgt Count Validation"
echo "Source Count : `wc -l < $1`"
echo "Target Count : `wc -l < $2`"

awk -F\| 'BEGIN{ print "Mismatch in line", "Column Name", "Source Data", "Target Data"}
                {for (i=1; i<=NF; i++) arr[i]=$i
                 getline <FN
                 for (i=1; i<=NF; i++)
                                 {if (arr[i] != $i)
                                 print $1,i,arr[i], $i} }
' FN="$1" $2 > output.temp

echo ""
echo "Field lebel Src & Tgt Mismatches"
echo "No of Mismatches : `expr $(wc -l < output.temp) - 1`"
echo ""
echo "Mismatch Details:"
cat output.temp
rm output.temp
exit

After executing the above script ,analysis report is coming like below .In the mismatch details section below entries should not come.

000001 20
000002 20
000003 20
000004 20

Code:

<<Analysis Report >>

Src & tgt Count Validation
Source Count : 4
Target Count : 4

Field lebel Src & Tgt Mismatches
No of Mismatches : 5

Mismatch Details:
Mismatch in line Column Name Source Data Target Data
000001 2 100001      1
000001 20
000002 20
000003 20
000004 20

The above analysis report is not displaying correct result .The Report should display like below :

Code:

<Analysis Report >>

Src & tgt Count Validation
Source Count : 4
Target Count : 4

Field lebel Src & Tgt Mismatches
No of Mismatches : 1

Mismatch Details:
Mismatch in line Column Name Source Data Target Data
000001              2                  100001        1

It wud be really helpful if anyone can look into the above mentioned issues.Thanks !

Last edited by STCET22; 07-01-2015 at 07:35 AM.. Reason: added under code tags

STCET22

View Public Profile for STCET22

Find all posts by STCET22

07-01-2015

Registered User

15,129, 5,008

Join Date: Jul 2012

Last Activity: 4 May 2020, 4:31 PM EDT

Location: Aachen, Germany

Posts: 15,129

Thanks Given: 735

Thanked 5,008 Times in 4,483 Posts

Are you aware that file1 has 20 fields whereas file2 has 19? Your awk version might dislike that.
This is what I get running the awk script on your sample files

Code:

<<Analysis Report >>

Src & tgt Count Validation
Source Count : 4
Target Count : 4

Field lebel Src & Tgt Mismatches
No of Mismatches : 1

Mismatch Details :
Key Column Value    Column Name    Source Data    Target Data
000001    2         1    100001

, and, I'm afraid I can't be of any further help.

RudiC

View Public Profile for RudiC

Find all posts by RudiC

07-01-2015

Registered User

42, 0

Join Date: Jan 2015

Last Activity: 20 April 2020, 8:16 AM EDT

Posts: 42

Thanks Given: 11

Thanked 0 Times in 0 Posts

Hi RudyC ,
Thanks a lot for your reply.Issue got resolved now.

Last edited by STCET22; 07-01-2015 at 10:17 AM.. Reason: Issue got solved now

STCET22

View Public Profile for STCET22

Find all posts by STCET22

Shell Programming and Scripting

Need help regarding comparison between two files through UNIX script

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Comparison of files

Discussion started by: Priya Amaresh

2. Shell Programming and Scripting

UNIX file comparison

Discussion started by: rakeshtomar82

3. Shell Programming and Scripting

Comparison between two files through UNIX script

Discussion started by: STCET22

4. Homework & Coursework Questions

Unix script Unix script which counts no. of files/sub-files

Discussion started by: sam09

5. Solaris

Unix script Unix script which counts no. of files/sub-files

Discussion started by: sam09

6. Shell Programming and Scripting

comparison of 2 files using unix or awk

Discussion started by: Diya123

7. Shell Programming and Scripting

Comparison of two files (sh)

Discussion started by: shizik

8. Shell Programming and Scripting

comparison of 2 files

Discussion started by: kamal_418

9. UNIX for Dummies Questions & Answers

Comparison of 2 files in UNIX

Discussion started by: Dana Evans

10. UNIX for Dummies Questions & Answers

Unix comparison

Discussion started by: NewGuy100