Shell script needed for comparing two files


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Shell script needed for comparing two files
# 1  
Old 09-05-2015
Shell script needed for comparing two files

Hi,

I need shell script to compare the two files based on certain fields and output should contains the required fields based on result.pls find sample input files and required output

file 1
HTML Code:
COUNT, BNG_IP,PORT,OVLAN
    22 ,  10.238.60.129,1/1,2009
    144 ,  10.238.60.129,1/1,2251
      3 ,  10.238.60.129,1/1,2255
     48 ,  10.238.60.129,1/1,2294
     32 ,  10.238.60.129,1/1,2677
    137 ,  10.238.60.129,1/1,2682
     49 ,  10.238.60.129,1/1,2683
     17 ,  10.238.60.129,1/1,2684
    123 ,  10.238.60.129,1/1,2686
     74 ,  10.238.60.129,1/1,2731
      8 ,  10.238.60.129,1/1,2896
     39 ,  10.238.60.129,2/1,2002
      8 ,  10.238.60.129,2/1,2310
      9 ,  10.238.60.129,2/1,2312
      4 ,  10.238.60.129,2/1,2313
     13 ,  10.238.60.129,2/1,2314
      1 ,  10.238.60.129,2/1,2316
File 2
HTML Code:
  BNG_IP DSLAM_IP PORT 
10.238.60.129 , 10.232.0.9 , 1/1 , 2009:124
10.238.60.129 , 10.232.2.2 , 2/1 , 2302:122
10.238.60.129 , 10.232.2.3 , 2/1 , 2303:122
10.238.60.129 , 10.232.2.4 , 2/1 , 2304:122
10.238.60.129 , 10.232.2.6 , 2/1 , 2306:122
10.238.60.129 , 10.232.2.7 , 2/1 , 2307:122
10.238.60.129 , 10.232.2.8 , 2/1 , 2308:122
10.238.60.129 , 10.232.2.9 , 2/1 , 2309:122
10.238.60.129 , 10.232.2.10 , 2/1 , 2310:122
10.238.60.129 , 10.232.2.11 , 2/1 , 2311:122
10.238.60.129 , 10.232.2.12 , 2/1 , 2312:122
10.238.60.129 , 10.232.2.13 , 2/1 , 2313:122
10.238.60.129 , 10.232.2.16 , 2/1 , 2316:122
10.238.60.129 , 10.232.2.17 , 2/1 , 2317:122
10.238.60.129 , 10.232.2.18 , 2/1 , 2318:122
Script should compare the first file with second file for BNG_IP,PORT,OVLAN fields if its available in file 2 .it should give the corresponding DSLAM_IP from file2
Required out put will be as below

Output
HTML Code:
COUNT, BNG_IP,PORT,OVLAN,DSLAM_IP
 22 ,  10.238.60.129,1/1,2009 ,10.232.0.9
 8 ,  10.238.60.129,2/1,2310,10.232.2.10
  9 ,  10.238.60.129,2/1,2312,10.232.2.12
  4 ,  10.238.60.129,2/1,2313,10.232.2.13
 1 ,  10.238.60.129,2/1,2316,10.232.2.16

Last edited by surender reddy; 09-05-2015 at 06:24 PM..
# 2  
Old 09-05-2015
Any attempts from your side?

---------- Post updated at 23:46 ---------- Previous update was at 23:43 ----------

Am I correctly assuming OVLAN in file2 is the first part of the last field?
# 3  
Old 09-05-2015
yes. OVLAN in file2 is the first part of the last field
# 4  
Old 09-05-2015
Howsoever, try
Code:
awk -F, '
        {gsub (/ /,_)}
NR==FNR {split ($NF, T, ":")
         DSLAM[$1,$3,T[1]]=$2
         next
        }
FNR==1  {print $0 FS "DSLAM_IP"
         next
        }
($2,$3,$4) in DSLAM     {print $0 FS DSLAM[$2,$3,$4]}
' file2 file1
COUNT,BNG_IP,PORT,OVLAN,DSLAM_IP
22,10.238.60.129,1/1,2009,10.232.0.9
8,10.238.60.129,2/1,2310,10.232.2.10
9,10.238.60.129,2/1,2312,10.232.2.12
4,10.238.60.129,2/1,2313,10.232.2.13
1,10.238.60.129,2/1,2316,10.232.2.16

# 5  
Old 09-05-2015
Thanks .Its working .Can you pls explain the command in detail.
# 6  
Old 09-05-2015
You could try,
Code:
awk -F'[,:]' '                                       # separate by a coma or a colon
BEGIN{
    print "COUNT,BNG_IP,PORT,OVLAN,DSLAM_IP"         # start by displaying the header
}
{
    gsub(" ", "")                                    # remove any spaces to sanitize the input
}
FNR==NR {                                            # apply the following block rule to the first file
    I[$2,$3,$4]=$0                                   # make the fields 2,3,4 as an id for the array I and save the whole line in it
    next                                             # stop processing any more blocks and go to read the next record
}
I[$1,$3,$4]{                                         # if it gets here, it is processing the second file. Check there's something in for the key of fields
    print I[$1,$3,$4], $2                            # if an ID matches, display the whole record from file 1 + the added DSLAM_IP from file 2
}' OFS=',' surender_reddy.file1 surender_reddy.file2 # OFS (Output Field Separator) is set to be a coma

Code:
COUNT,BNG_IP,PORT,OVLAN,DSLAM_IP
22,10.238.60.129,1/1,2009,10.232.0.9
8,10.238.60.129,2/1,2310,10.232.2.10
9,10.238.60.129,2/1,2312,10.232.2.12
4,10.238.60.129,2/1,2313,10.232.2.13
1,10.238.60.129,2/1,2316,10.232.2.16

---------- Post updated at 05:30 PM ---------- Previous update was at 05:03 PM ----------

Quote:
Originally Posted by surender reddy
Thanks .Its working .Can you pls explain the command in detail.
Code:
awk -F, '                                             # use a coma as input separator
        {gsub (/ /,_)}				      # substitute any white space for an underscore to balance the input										
NR==FNR {split ($NF, T, ":")                          # for the first file, split the last field by colon and save it into an array T
         DSLAM[$1,$3,T[1]]=$2                         # create an array named DSLAM with the id made up from $1,$3,T[1] to save $2
         next                                         # move to the next record and skip the rest if this is file2
        }
FNR==1  {print $0 FS "DSLAM_IP"                       # only for the first line of the second file,since the first file has been processed already. Print header.
         next			                      # skip the rest if it is the record is the first line of the second file at stdin
        }
($2,$3,$4) in DSLAM     {print $0 FS DSLAM[$2,$3,$4]} # if there a key named $2,$3,$5 in the previous made array DSLAM, display current line + coma + content stored in DSLAM[key].
' file2 file1

This User Gave Thanks to Aia For This Post:
 
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need Help in comparing 2 text files in shell script

Hi All, I have 2 files like below vi f1 frog elephant rabit zebra dog vi f2 rabit dog ============== Now i want to comapre two files and the result will be frog (8 Replies)
Discussion started by: kumar85shiv
8 Replies

2. Shell Programming and Scripting

shell script for comparing two files

Hi, I have 2 files as below FILE1.dat co1|co2|co3 abnd|45.56|AZ dkny|30.2|PA sam|.23|VA FILE.CTL FILENAME|FILE1.dat NO OF RECORDS|3 CHECKSUM|75.99 Could you please help me to write a shell script to compare 1. TOTAL RECORDS from FILE1.dat with NO of RECORDS in FILE.CTL 2.... (8 Replies)
Discussion started by: dreamsportsteam
8 Replies

3. Shell Programming and Scripting

Comparing 2 files using shell script

Hi Experts, I have 2 files 1 file consists of 800 records and 2 file consists of 100 records with matching column as Membership_Num.So i need a script which will compare the 2 files and displays the output.As these are the files the script should take any delimter like (tab,comma) as input... (6 Replies)
Discussion started by: naveen.dasu
6 Replies

4. Shell Programming and Scripting

comparing 2 files in shell script

I have 2 files config1h.txt ----------------- BFMU=ENABLE,ID=PM THR=OFF,REP=ALL,CON=IACM,TIM=OFF;GPON collection strategy 1.3.6.1.2.1.2.2:8 1.3.6.1.2.1.2.2:7 1.3.6.1.4.1.637.61.1.35.11.4:4 1.3.6.1.4.1.637.61.1.35.11.4:3 1.3.6.1.4.1.637.61.1.35.10.1:2 1.3.6.1.4.1.637.61.1.35.10.1:43... (7 Replies)
Discussion started by: LavanyaP
7 Replies

5. Shell Programming and Scripting

Shell Script Needed to Read a text from a list files

Hi, Below is my issue which I desperately need and I want a shell script which can do this job. I need this script as I m planning to put this for a system health check. Please assist me. 1. There are 10 log files in a particular location. 2. open each log file. Goto to the end of the... (4 Replies)
Discussion started by: kashriram
4 Replies

6. Shell Programming and Scripting

help needed in a shell script for printing files

Hi All, I am writing a shell script to print all the files in a particular folder. This script is on solaris server . I am using the lp command to send the requests . lp -d printername filename . The output I get is the request id. Is there any way I can use the request Id to determine if... (1 Reply)
Discussion started by: shahshilpa
1 Replies

7. Shell Programming and Scripting

comparing two files using shell script

hi experts please help me to compare two files which are in different directory file1<file will be master file> (/home/rev/mas.txt} ex x1 x2 file2 <will be in different folder> (/home/rev/per/.....) ex x3 x4 the filesinside per folder i need to compare with master file and the files... (2 Replies)
Discussion started by: revenna
2 Replies

8. UNIX for Dummies Questions & Answers

shell script for comparing 2 files

Hi, how to read the 2 files and compare each other in shell script? i have 2 files test1 and test2, both files contains 20 character records.we have to compare file 1 records with file2, if exists then reject the record else we have to append it to test2 file. in file test1 around 100 single... (2 Replies)
Discussion started by: prashanth.spl
2 Replies

9. Shell Programming and Scripting

shell script comparing files in a file

There is a text file that contains the data in the following format: COLUMN1 COLUMN2 ABC 1 ABC 2 ABC 3 DEF 4 DEF 5 XYZ 7 We have to create a second text file... (4 Replies)
Discussion started by: raina_nalin
4 Replies
Login or Register to Ask a Question