To get the exact mismatches from two csv files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting To get the exact mismatches from two csv files
# 1  
Old 03-12-2017
To get the exact mismatches from two csv files

Hello Guys,
I am pretty new to unix shell scripting where in i need to compare two files which are comma separated files.

So here i go with the file contents

Code:
cty_id,grade_val,g_val_2,g_val_3
001,10,20,30
002,,,40
003,100,,10


grade_val,g_val_2,cty_id
10,20,001
41,,002
100,1,003

In both of my files cty_id is the key column where i would be searching by putting the cty_id number. For eg if i search for cty_id 001 then i should get below results

Code:
city_id 001 doest have any mismatch

if 002
Mismatch at col grade_val : Null - 41
if 003
Mismatch at col g_val : Null - 1

Like wise i need to compare based on the columns from two files and there can be mismatches in other columns for the same city_id's but need to compare only the columns which are present in both files.

I tried with below code but not able to get through

Code:
echo Please enter your cty_id:
read id
while read id; do grep "$id" file.csv; done < file_1.csv

# 2  
Old 03-12-2017
Not sure I understand your intentions nor requirements.

As a starting point. this post might help to compare fields that have a common header but different position in a line.
# 3  
Old 03-12-2017
Thanks RudiC for your suggestion,

What i would require is to compare the files based on the columns and extract the mismatches in below format if i input city_id.

For eg if i give 001 as input it will search in both files because the files will have only one record with 001 and compare column by column the values if they have any mismatch in column

Code:
column	         1st file	  2nd file
city_id	    1	            1
grade_val	   10	           10
g_val_2	   20	           20

There is no mismatch


Code:
city_id 001 doesn't have any mismatch
Mismatch at col grade_val : Null - 41
Mismatch at col g_val : Null - 1


Last edited by Don Cragun; 03-13-2017 at 02:46 AM.. Reason: Add missing CODE tags.
# 4  
Old 03-13-2017
Still some questions and guesses; but, based on that link given, try
Code:
awk '
NR == FNR       {if (NR == 1)   for (MX=n=NF; n>0; n--) REF[$n] = n
                 else           TMP[NR] = $0
                 next
                }

FNR == 1        {for (n=NF; n>0; n--)   {if ($n in REF) CMP[n]=REF[$n]
                                         if ($n == SRCH) NSR = n
                                         HD[n]  = $n
                                         NL     = "Null"
                                        }
                 next
                }

                {n = split (TMP[FNR], IT)
                 EQU = 1
                 for (i=1; i<=MX; i++)  {T = IT[CMP[i]]
                                         if ($i != T)   {print SRCH, $NSR ": mismatch at", HD[i] ":", $i?$i:NL, "-", T?T:NL
                                                         EQU = 0
                                                        }
                                        }
                 if (EQU) print SRCH, $NSR, "doesn´t have any mismatch."
                }


' FS="," SRCH="cty_id" file2 file1
cty_id 001 doesn´t have any mismatch.
cty_id 002: mismatch at grade_val: 40 - 41
cty_id 003: mismatch at g_val_2: Null - 1

This User Gave Thanks to RudiC For This Post:
# 5  
Old 03-14-2017
Thanks for your help RudiC, How i can filter with city_id i mean i can pass city id as input parameter to perform the mismatch

I got it i can use grep at the end

Also RudiC, can you let me know what questions you have so that i can help to sort out for you

Last edited by Master_Mind; 03-14-2017 at 06:46 AM.. Reason: Seeking clarification
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Using awk to output matches and mismatches between two files to one file

In the tab-delimited files, I am trying to match $1,$2,$3,$4,$5 in fiel1 with $1,$2,$3,$4,$5 in fiel2 and create and output file that lists what matches and what was not found (or doesn't match). However the awk below seems to skip the first line and does not produce the desired output. I think... (2 Replies)
Discussion started by: cmccabe
2 Replies

2. Shell Programming and Scripting

Using awk to output matches between two files to one file and mismatches to two others

I am trying to output the matches between $1 of file1 to $3 of file2 into a new file match. I am also wanting to output the mismatches between those same 2 files and fields to two separate new files called missing from file1 and missing from file2. The input files are tab-delimited, but the... (9 Replies)
Discussion started by: cmccabe
9 Replies

3. Shell Programming and Scripting

Match columns from two csv files and update field in one of the csv file

Hi, I have a file of csv data, which looks like this: file1: 1AA,LGV_PONCEY_LES_ATHEE,1,\N,1,00020460E1,0,\N,\N,\N,\N,2,00.22335321,0.00466628 2BB,LES_POUGES_ASF,\N,200,200,00006298G1,0,\N,\N,\N,\N,1,00.30887539,0.00050312... (10 Replies)
Discussion started by: djoseph
10 Replies

4. Shell Programming and Scripting

Compare two txt files,mismatches will be in new txt files

Hi, Below are the sample data for txt files. txt file 1 Partnumber|catgroup_id 10001082|46016 10001093|4680 10001093|386003 10001093|463004 10003251|683 10003251|63005 10003252|463005 10003252|4683 10003260|463005 10003260|4683 10003264|4683 10003264|463005 13420000|67... (5 Replies)
Discussion started by: Ankita Talukdar
5 Replies

5. Shell Programming and Scripting

Compare 2 csv files in ksh and o/p the difference in a new csv file

(say) I have 2 csv files - file1.csv & file2.csv as mentioned below: file1.csv ID,version,cost 1000,1,30 2000,2,40 3000,3,50 4000,4,60 file2.csv ID,version,cost 1000,1,30 2000,2,45 3000,4,55 6000,5,70 The... (7 Replies)
Discussion started by: Naresh101
7 Replies

6. Shell Programming and Scripting

Comparing 2 CSV files and sending the difference to a new csv file

(say) I have 2 csv files - file1.csv & file2.csv as mentioned below: file1.csv ID,version,cost 1000,1,30 2000,2,40 3000,3,50 4000,4,60 file2.csv ID,version,cost 1000,1,30 2000,2,45 3000,4,55 6000,5,70 ... (1 Reply)
Discussion started by: Naresh101
1 Replies

7. Shell Programming and Scripting

QUESTION1: grep only exact string. QUESTION2: find and replace only exact value with sed

QUESTION1: How do you grep only an exact string. I am using Solaris10 and do not have any GNU products installed. Contents of car.txt CAR1_KEY0 CAR1_KEY1 CAR2_KEY0 CAR2_KEY1 CAR1_KEY10 CURRENT COMMAND LINE: WHERE VARIABLE CAR_NUMBER=1 AND KEY_NUMBER=1 grep... (1 Reply)
Discussion started by: thibodc
1 Replies

8. UNIX for Advanced & Expert Users

format mismatches

Hi I am spooling the table values in file in unix ,but the problem is date format in table 'dd/mm/yyyy' ,when strong into file 'dd-mon-yyyy'.I want the same format to loaded into files also.Any idea. $ORACLE_HOME/bin/sqlplus -S "$db"<<! | tee -a >$INFA_HOME/server/infa_shared/CTRL.log ... (1 Reply)
Discussion started by: akil
1 Replies

9. Shell Programming and Scripting

Need to compare two csv files values and write into another csv file

Hi all, Am new to scripting. So i just need your ideas to help me out. Here goes my requirement. I have two csv files 1.csv 2.csv abc,1.24 abc,1 def,2.13 def,1 I need to compare the first column of 1.csv with 2.csv and if matches then need to compare... (2 Replies)
Discussion started by: chinnahyd
2 Replies

10. Shell Programming and Scripting

Matches and mismatches in perl

When we give an input sequence , the program should match with the pattern and give the matches and mismatches in the output. i will give you 2 small examples. if you cant get it pls let me know. i will try to give a clear idea. example 1: $a=APPLE; # let it be a pattern... (0 Replies)
Discussion started by: srisha
0 Replies
Login or Register to Ask a Question