Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Compare two files (Many to one comparison)

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare two files (Many to one comparison)
# 1  
Old 01-23-2013
Compare two files (Many to one comparison)
Hello,

I am having output shown below names will change and can show different result and i want to compare with the checklist this file will not change it is standard settings.

If there is some discrepancy than the complete line of output should be shown with name.

I tried to use below code but it it didnt worked.

Code:
#!/bin/sh
a=0
while [ $a -lt 30 ]
do
sdiff ref $bsc > res
join -a 2 -a 1 -o 2.1,2.2,1.1,2.3 cell res > resjoin
          ./diff                                           
    a=`expr $a + 1`
done

Output

Code:
john Function=1 spare i[10] = 0 0 0 0 0 0 0 0 0 0
john Function=1 spareA i[10] = 0 0 0 0 0 0 0 0 0 0
john Function=1 spareB i[10] = 0 0 0 0 0 0 0 0 0 0
john Function=1 spareC i[10] = 0 0 0 0 0 0 0 0 0 0
raj Function=1 spare i[10] = 0 0 0 0 0 0 0 0 0 0
raj Function=1 spareA i[10] = 0 0 0 0 0 0 0 0 0 0
raj Function=1 spareB i[10] = 0 0 0 0 0 0 0 0 0 0
raj Function=1 spareC i[10] = 0 0 0 0 0 0 0 0 0 0
john Details=1 Time 65
raj Details=1 Time 65
john Risk=1 Offset 80
raj Risk=1 Offset 80
john 53 Activ A_w_S Data No 524288 16 0 ( 0, 22) (Lm)
john 46 Activ A_w_S SC_0_1 No 40000 10 0 ( 0, 22) (Scc)
john 55 Activ A_w_S - - - - - ( 0, 7) (LmSabp)
john 54 Activ A_w_S sap No 524288 12 0 ( 0, 7) (LmRn)
raj 53 Activ A_w_S Data No 524288 16 0 ( 0, 22) (Lm) 
raj 46 Activ A_w_S SC_0_1 No 40000 10 0 ( 0, 22) (Scc) 
raj 55 Activ A_w_S - - - - - ( 0, 7) (LmSabp) 
raj 54 Activ A_w_S sap No 524288 12 0 ( 0, 7) (LmRn)

checklist

Code:
Function=1 spare i[10] = 0 0 0 0 0 0 0 0 0 0
Function=1 spareA i[10] = 0 0 0 0 0 0 0 0 0 0
Function=1 spareB i[10] = 0 0 0 0 0 0 0 0 0 0
Function=1 spareC i[10] = 0 0 0 0 0 0 0 0 0 0
Details=1 Time 65
Risk=1 Offset 80
Activ A_w_S Data No 524288 16 0 ( 0, 22) (Lm)
Activ A_w_S SC_0_1 No 40000 10 0 ( 0, 22) (Scc)
Activ A_w_S - - - - - ( 0, 7) (LmSabp)
Activ A_w_S sap No 524288 12 0 ( 0, 7) (LmRn)

Thanks
Last edited by Scrutinizer; 01-28-2013 at 04:16 PM.. Reason: code tags; formatting
# 2  
Old 01-23-2013
Not sure if I understand what your input files to 'join' are, But if they are like the below try the supplied code:
Files:
Code:
Input file1: cell
Function=1 spare i[10] = 0 0 0 0 0 0 0 0 0 0
Function=1 spareA i[10] = 0 0 0 0 0 0 0 0 0 0
Function=1 spareB i[10] = 0 0 0 0 0 0 0 0 0 0
Function=1 spareC i[10] = 0 0 0 0 0 0 0 0 0 0
Details=1 Time 65
Risk=1 Offset 80
Activ A_w_S Data No 524288 16 0 ( 0, 22) (Lm)
Activ A_w_S SC_0_1 No 40000 10 0 ( 0, 22) (Scc)
Activ A_w_S - - - - - ( 0, 7) (LmSabp)
Activ A_w_S sap No 524288 12 0 ( 0, 7) (LmRn)

Input file2: res
john Function=1 spare i[10] = 0 0 0 0 0 0 0 0 0 0
john Function=1 spareA i[10] = 0 0 0 0 0 0 0 0 0 0
john Function=1 spareB i[10] = 0 0 0 0 0 0 0 0 0 0
john Function=1 spareC i[10] = 0 0 0 0 0 0 0 0 0 0
raj Function=1 spare i[10] = 0 0 0 0 0 0 0 0 0 0
raj Function=1 spareA i[10] = 0 0 0 0 0 0 0 0 0 0
raj Function=1 spareB i[10] = 0 0 0 0 0 0 0 0 0 0
raj Function=1 spareC i[10] = 0 0 0 0 0 0 0 0 0 0
john Details=1 Time 65
raj Details=1 Time 65
john Risk=1 Offset 80
raj Risk=1 Offset 80
john 53 Activ A_w_S Data No 524288 16 0 ( 0, 22) (Lm)
john 46 Activ A_w_S SC_0_1 No 40000 10 0 ( 0, 22) (Scc)
john 55 Activ A_w_S - - - - - ( 0, 7) (LmSabp)
john 54 Activ A_w_S sap No 524288 12 0 ( 0, 7) (LmRn)
raj 53 Activ A_w_S Data No 524288 16 0 ( 0, 22) (Lm)
raj 46 Activ A_w_S SC_0_1 No 40000 10 0 ( 0, 22) (Scc)
raj 55 Activ A_w_S - - - - - ( 0, 7) (LmSabp)
raj 54 Activ A_w_S sap No 524288 12 0 ( 0, 7) (LmRn)



Code:
a=0
while [ $a -lt 30 ]
do
  sdiff ref $bsc > res
  join -a 2 -a 1 -1 1 -2 2 -o 2.1,2.2,1.1,2.3 cell res > resjoin
  ./diff                                           
  a=`expr $a + 1`
done


Just remember 'join' will only join on one field so if you want to join on multiple fields you will need to concatenate the fields for input into join.
# 3  
Old 01-24-2013
getting the below mentioned error
Code:
Usage: sdiff [-l] [-s] [-o output] [-w #] file1 file2
test.sh: ./diff: not found
Usage: sdiff [-l] [-s] [-o output] [-w #] file1 file2
test.sh: ./diff: not found
Usage: sdiff [-l] [-s] [-o output] [-w #] file1 file2
test.sh: ./diff: not found
Usage: sdiff [-l] [-s] [-o output] [-w #] file1 file2
test.sh: ./diff: not found
Usage: sdiff [-l] [-s] [-o output] [-w #] file1 file2

Input File1:- cell is a standard file and will remain the same and Input File2 :- res will change.
We need to check discrpancy between file2 and file1 and if there is dicrepancy than we should get the dicrepancy written in a separate file as shown below
Code:
raj Details=1 Time 64

as in file1 (standard file) Time value is 65.
# 4  
Old 01-25-2013
Ok, I think understand what you want to do:
checklist file:
PHP Code:
Function=1 spare i[10] = 0 0 0 0 0 0 0 0 0 0
Function=1 spareA i[10] = 0 0 0 0 0 0 0 0 0 0
Function=1 spareB i[10] = 0 0 0 0 0 0 0 0 0 0
Function=1 spareC i[10] = 0 0 0 0 0 0 0 0 0 0
Details=1 Time 65
Risk=1 Offset 80
Activ A_w_S Data No 524288 16 0 ( 0, 22) (Lm)
Activ A_w_S SC_0_1 No 40000 10 0 ( 0, 22) (Scc)
Activ A_w_S - - - - - ( 0, 7) (LmSabp)
Activ A_w_S sap No 524288 12 0 ( 0, 7) (LmRn) 
File to check against checklist file, lets call: IFile
PHP Code:
john Function=1 spare i[10] = 0 0 0 0 0 0 0 0 0 0
john Function=1 spareA i[10] = 0 0 0 0 0 0 0 0 0 0
john Function=1 spareB i[10] = 0 0 0 0 0 0 0 0 0 0
john Function=1 spareC i[10] = 0 0 0 0 0 0 0 0 0 0
raj Function=1 spare i[10] = 0 0 0 0 0 0 0 0 0 0
raj Function=1 spareA i[10] = 0 0 0 0 0 0 0 0 0 0
raj Function=1 spareB i[10] = 0 0 0 0 0 0 0 0 0 0
raj Function=1 spareC i[10] = 0 0 0 0 0 0 0 0 0 0
john Details=1 Time 65
raj Details=1 Time 65
john Risk=1 Offset 80
raj Risk=1 Offset 80
john 53 Activ A_w_S Data No 524288 16 0 ( 0, 22) (Lm)
john 46 Activ A_w_S SC_0_1 No 40000 10 0 ( 0, 22) (Scc)
john 55 Activ A_w_S - - - - - ( 0, 7) (LmSabp)
john 54 Activ A_w_S sap No 524288 12 0 ( 0, 7) (LmRn)
raj 53 Activ A_w_S Data No 524288 16 0 ( 0, 22) (Lm)
raj 46 Activ A_w_S SC_0_1 No 40000 10 0 ( 0, 22) (Scc)
raj 55 Activ A_w_S - - - - - ( 0, 7) (LmSabp)
raj 54 Activ A_w_S sap No 524288 12 0 ( 0, 7) (LmRn) 
Am I correct in stating that you want to validate that each record in IFile should match a record in the checklist file?

See Examples below.
1st example, IFile Record:
Code:
raj Function=1 spare i[10] = 0 0 0 0 0 0 0 0 0 0
with 1st column stripped off should have a matching record in checklist file:
Function=1 spare i[10] = 0 0 0 0 0 0 0 0 0 0


2nd example, IFile Record:
Code:
john 55 Activ A_w_S - - - - - ( 0, 7) (LmSabp)
with 1st and 2nd column stripped off should have a matching record in checklist file:
Activ A_w_S - - - - - ( 0, 7) (LmSabp)

# 5  
Old 01-25-2013
Yes you are right, also please ensure that there are different names in IFile and if there is dicrepancy than we should get the dicrepancy written in a separate file as shown below

Code:
raj Details=1 Time 64

as in the checklist the value of this is 65 for Raj. John value is correct so discrepancy is not shown.
# 6  
Old 01-25-2013
I believe this will do what you want:
Code:
# Script:  test.sh
while read rec
do
  # Strips name and number column from beginning of record so that it can be
  # looked up CheckList file.
  lr=`echo $rec | sed -e 's#^[a-z]*\s\(.*\)$#\1#' -e 's#^[0-9][0-9]\s\(.*\)$#\1#'`
  grep -q -F "$lr" CheckList.txt
  if [[ $? -gt 0 ]]; then
    echo "$rec" >> NotFound.txt
  fi
done <t
exit

$ cat CheckList.txt
Function=1 spare i[10] = 0 0 0 0 0 0 0 0 0 0
Function=1 spareA i[10] = 0 0 0 0 0 0 0 0 0 0
Function=1 spareB i[10] = 0 0 0 0 0 0 0 0 0 0
Function=1 spareC i[10] = 0 0 0 0 0 0 0 0 0 0
Details=1 Time 65
Risk=1 Offset 80
Activ A_w_S Data No 524288 16 0 ( 0, 22) (Lm)
Activ A_w_S SC_0_1 No 40000 10 0 ( 0, 22) (Scc)
Activ A_w_S - - - - - ( 0, 7) (LmSabp)
Activ A_w_S sap No 524288 12 0 ( 0, 7) (LmRn)

$ cat t
john Function=1 spare i[10] = 0 0 0 0 0 0 0 0 0 0
john Function=1 spareA i[10] = 0 0 0 0 0 0 0 0 0 0
john Function=1 spareB i[10] = 0 0 0 0 0 0 0 0 0 0
john Function=1 spareC i[10] = 0 0 0 0 0 0 0 0 0 0
john Function=9 spareC i[10] = 0 0 0 0 0 0 0 0 0 0
raj Function=1 spare i[10] = 0 0 0 0 0 0 0 0 0 0
raj Function=1 spareA i[10] = 0 0 0 0 0 0 0 0 0 0
raj Function=1 spareB i[10] = 0 0 0 0 0 0 0 0 0 0
raj Function=1 spareC i[10] = 0 0 0 0 0 0 0 0 0 0
john Details=1 Time 65
raj Details=1 Time 65
john Risk=1 Offset 80
raj Risk=1 Offset 80
john 53 Activ A_w_S Data No 524288 16 0 ( 0, 22) (Lm)
john 46 Activ A_w_S SC_0_1 No 40000 10 0 ( 0, 22) (Scc)
john 55 Activ A_w_S - - - - - ( 0, 7) (LmSabp)
john 54 Activ A_w_S sap No 524288 12 0 ( 0, 7) (LmRn)
raj 53 Activ A_w_S Data No 524288 16 0 ( 0, 22) (Lm)
raj 46 Activ A_w_S SC_0_1 No 40000 10 0 ( 0, 22) (Scc)
raj 55 Activ A_w_S - - - - - ( 0, 7) (LmSabp)
raj 55 Activ A_w_S - - - - - ( 0, 7) (LmSabp)xxx

# Run script
Code:
$ test.sh

Code:
$ cat NotFound.txt
john Function=9 spareC i[10] = 0 0 0 0 0 0 0 0 0 0
raj 55 Activ A_w_S - - - - - ( 0, 7) (LmSabp)xxx

# 7  
Old 01-28-2013
Thanks for the reply i tried the code and used different workarounds but getting the below mentioned error.

Code:
grep: illegal option -- q
grep: illegal option -- F
Usage: grep -hblcnsviw pattern file . . .
test.sh: [[: not found
grep: illegal option -- q
grep: illegal option -- F
Usage: grep -hblcnsviw pattern file . . .
test.sh: [[: not found
grep: illegal option -- q
grep: illegal option -- F
Usage: grep -hblcnsviw pattern file . . .
test.sh: [[: not found
grep: illegal option -- q
grep: illegal option -- F
Usage: grep -hblcnsviw pattern file . . .
test.sh: [[: not found
grep: illegal option -- q
grep: illegal option -- F
Usage: grep -hblcnsviw pattern file . . .
test.sh: [[: not found
grep: illegal option -- q
grep: illegal option -- F
Usage: grep -hblcnsviw pattern file . . .
test.sh: [[: not found
grep: illegal option -- q
grep: illegal option -- F
Usage: grep -hblcnsviw pattern file . . .
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Comparison of files

I have the requirement I have two files cat fileA something anythg nothing everythg cat fileB everythg anythg Now i shld use fileB and compare every line at fileA and get the output as something nothing (3 Replies)
Discussion started by: Priya Amaresh
3 Replies

2. Shell Programming and Scripting

Comparison of two files

Hi all I have two files which I have to compare that whetehr there is soemthing common or not body, div, table, thead, tbody, tfoot, tr, th, td, p { font-family: "Liberation Sans"; font-size: x-small; } body, div, table, thead, tbody, tfoot,... (2 Replies)
Discussion started by: manigrover
2 Replies

3. Shell Programming and Scripting

Comparison of multiple files

Need a write script in bournce shell. Compare all the file contents and need to generate a report. Example : Having 10 trace.log files like below trace1.log, trace2.log .... trace10.log Need to compare all the 10 files contents and provide the report as below, Assume trace... (1 Reply)
Discussion started by: sureshmani
1 Replies

4. Shell Programming and Scripting

Comparison of fields in Files

Hello, I have two files with tab delimited data. The file will contain details something like below: FILENAME.A.B.C. 3 5 VALID PROCESSED I would have a bench mark file. I would be getting new files of the same format. My requirement is to compare a particular field for a... (3 Replies)
Discussion started by: Praveenkulkarni
3 Replies

5. Shell Programming and Scripting

Require compare command to compare 4 files

I have four files, I need to compare these files together. As such i know "sdiff and comm" commands but these commands compare 2 files together. If I use sdiff command then i have to compare each file with other which will increase the codes. Please suggest if you know some commands whcih can... (6 Replies)
Discussion started by: nehashine
6 Replies

6. Shell Programming and Scripting

Comparison of two files (sh)

Hi, I have a problem with comparison of two files file1 20100101 20090101 20080101 20071001 20121229 file2 19990112 12 456 7 20011131 19 20100101 2 567 1 987 17890709 123 555 and, sh script needs to compare of these two files and give out to me result: 20100101 2 567 1 987 it... (5 Replies)
Discussion started by: shizik
5 Replies

7. UNIX for Dummies Questions & Answers

Comparison between binary files

Hello, I would like to know if we can compare binary(non text) files in unix eg. exes Thanks Saurabh (3 Replies)
Discussion started by: smehra_20
3 Replies

8. Shell Programming and Scripting

comparison of 2 files

Kindly help on follows. I have 2 files. One file contains only one column of mobile numbers. And total records in a file 12 million. Second file contains 2 columns mobile numbers and balance. and total records 30 million. I want to find out balance of each data in file 1 corresponding to file 2.... (2 Replies)
Discussion started by: kamal_418
2 Replies

9. UNIX for Dummies Questions & Answers

Comparison of 2 files in UNIX

Hi, There are two files in UNIX system with some lines are exactly the same, some lines are not. I want to compare these two files.The 2 files (both the files have data in Column format )should be compared row wise and any difference in data for a particular row should lead to storage of data of... (32 Replies)
Discussion started by: Dana Evans
32 Replies

10. Shell Programming and Scripting

Comparison of two files in awk

Hi, I have two files file1 and file2 delimited by semicolon, And I want to compare column 2 and column3 of file1 to column3 and column 4 in file2. file1 -------- abc;cef;155.67;143_34; def;fgh;146.55;123.3; frg;hff;134.67;; yyy;fgh;134.78;35_45; file 2 --------- abc;cef;155.09;;... (12 Replies)
Discussion started by: jerome Sukumar
12 Replies
Login or Register to Ask a Question