Finding difference in two comma separated files in UINX


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Finding difference in two comma separated files in UINX
# 1  
Old 07-07-2011
Bug Finding difference in two comma separated files in UINX

Dear All,

I have requirement like this:
I have 2 camma seperated files
file1:
Code:
1,aaa,bbb,ccc,
2,bbb,ccc,ddd,
3,ccc,ddd,eee,

file2:
Code:
1,aaa,bbb,ccc,
2,bbb,ddd,ddd,
3,ccc,ddd,eee,

my requirement is I should get message in the out put like:

There is a difference in 3 rd filed in second record expected is ccc but actual is ddd .

How we can do it in unix.
I have written below code:
Code:
 export file1=$1
 export file2=$2
 export i=1
 rec_count1=`wc -l <  $file1`
 rec_count2=`wc -l <  $file2`
 
 while [  $i -le $rec_count1 ]
 do
  head -$i $file1 |tail -1 >temp_1.txt
  head -$i $file2 |tail -1 >temp_2.txt
  export j=1
  column_no1=`awk -F"," '{print NF}' temp_1.txt`
  column_no2=`awk -F"," '{print NF}' temp_2.txt`
  if [ $column_no1 != $column_no2 ]; then 
   echo "There is difference in no of fileds in record $i"
  else
  while [  $j -le $column_no1 ]
  do
   field1=`cut -f$j -d"," temp_1.txt`
   field2=`cut -f$j -d"," temp_2.txt`
   
   if [ "$field1" != "$field2" ];  then
    echo " There is a difference in column no=$j in record  no=$i  expected value is $field1 but actual value is $field2"
    
    
   fi
  j=`expr $j + 1`
  
  done
  fi
 i=`expr $i + 1`
 done


but it is taking lot of time for big files, How can we achive in better way, Please advise.

Thanks in Advance,
Moto Smilie

Last edited by Franklin52; 07-07-2011 at 04:58 AM.. Reason: Please use code tags for code and data samples, thank you
# 2  
Old 07-07-2011
why dont you use diff command ?

Code:
 
bash-3.00$ diff 1.txt 2.txt
2c2
< 2,bbb,ccc,ddd,
---
> 2,bbb,ddd,ddd,

# 3  
Old 07-07-2011
if I use diff I will be getting whole record and again it is difficult for me trace where is the difference if there are 200+ fileds and thousads of reords.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to extract fields from a CSV i.e comma separated where some of the fields having comma as value?

can anyone help me!!!! How to I parse the CSV file file name : abc.csv (csv file) The above file containing data like abv,sfs,,hju,',',jkk wff,fst,,rgr,',',rgr ere,edf,erg,',',rgr,rgr I have a requirement like i have to extract different field and assign them into different... (4 Replies)
Discussion started by: J.Jena
4 Replies

2. Shell Programming and Scripting

awk to parse comma separated field and removing comma in between number and double quotes

Hi Experts, Please support I have below data in file in comma seperated, but 4th column is containing comma in between numbers, bcz of which when i tried to parse the file the column 6th value(5049641141) is being removed from the file and value(222.82) in column 5 becoming value of column6. ... (3 Replies)
Discussion started by: as7951
3 Replies

3. Shell Programming and Scripting

Calculate difference in comma separated coordinates

Dear All, I would like some help to process the following file ID n_exons start end Zfp85-rs1 4 67747799,67751609,67752387,67755062, 67749722,67751705,67752514,67755134,... (2 Replies)
Discussion started by: paolo.kunder
2 Replies

4. UNIX for Dummies Questions & Answers

[solved] Comma separated values to space separated

Hi, I have a large number of files which are written as csv (comma-separated values). Does anyone know of simple sed/awk command do achieve this? Thanks! ---------- Post updated at 10:59 AM ---------- Previous update was at 10:54 AM ---------- Guess I asked this too soon. Found the... (0 Replies)
Discussion started by: lost.identity
0 Replies

5. Shell Programming and Scripting

Finding first difference between two files

Hi! I'd like to know if it is possible for a command to find the first difference between two large files, output that line from both file and stop, so no need to continue after that to save some computation time. I don't think looping through it will be efficient enough but that's the only... (6 Replies)
Discussion started by: Mojing
6 Replies

6. Shell Programming and Scripting

Need Help - comma inside double quote in comma separated csv,

Hello there, I have a comma separated csv , and all the text field is wrapped by double quote. Issue is some text field contain comma as well inside double quote. so it is difficult to process. Input in the csv file is , 1,234,"abc,12,gh","GH234TY",34 I need output like below,... (8 Replies)
Discussion started by: Uttam Maji
8 Replies

7. Shell Programming and Scripting

Comma separated file

Hi all, I have the following files types: FileA: 100, 23, 33, FileB: 22, 45, 78, and i want to make File C: 100,22 23,45 33,78 any nice suggestions for making it easy. (3 Replies)
Discussion started by: hen1610
3 Replies

8. Shell Programming and Scripting

finding difference between two files

Hi, I have two files one with 12486 lines second one with 13116 As per the comparsion between two files the count have 630 difference I used diff command to find the difference between two files but it's not understandable could any one suggest any command to get 630 records in a new... (4 Replies)
Discussion started by: thelakbe
4 Replies

9. Homework & Coursework Questions

Find the files and make them comma separated files

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: Hi All, I am new to unix, my requirement is like need to find the files like DATA_FUNCTION* and put those... (1 Reply)
Discussion started by: madsongtel
1 Replies

10. Shell Programming and Scripting

Deleting a column in multiple files that are comma separated

Hi, I have a directory that contains say 100 files named sequencially like input_1.25_50_C1.txt input_1.25_50_C2.txt input_1.25_50_C3.txt input_1.25_50_C4.txt .. .. .. input_1.25_50_C100.txt an example of the content in each of the file is: "NAME" "MEM.SHIP" "cgd1_10" "cgd1_10"... (9 Replies)
Discussion started by: Lucky Ali
9 Replies
Login or Register to Ask a Question