Sponsored Content
Top Forums Shell Programming and Scripting Find duplicate value comparing 2 files and create an output Post 302170721 by ricky007 on Tuesday 26th of February 2008 02:25:56 PM
Old 02-26-2008
Find duplicate value comparing 2 files and create an output

I need a perl script which will create an output file after comparing two diff file in a directory path:
/export/home/abc/file1
/export/home/abc/file2

File Format: <IP>TAB<DeviceName><TAB>DESCRIPTIONS

file1:
10.1.2.1.3<tab>abc123def<tab>xyz.mm1.ppp.
#10.1.2.1.4<tab>abc345def<tab>xyz.mm2.ppp2.
10.1.2.1.5<tab>abc567def<tab>xyz.mm4.ppp3.
10.1.2.1.6<tab>abc789def<tab>xyz.mm5.ppp6.
#10.1.2.1.7<tab>abc009def<tab>xyz.mm9.ppp7

file2:

10.1.2.1.3<tab>abc123def<tab>xyz.mm1.ppp.
#10.1.2.1.4<tab>abc345def<tab>xyz.mm2.ppp2.
10.1.2.1.5<tab>abc009def<tab>xyz.mm7.ppp9
10.1.2.1.6<tab>abc567def<tab>xyz.mm4.ppp3.
#10.1.2.1.9<tab>abc009def<tab>xyz.mm9.ppp7

If IP and Device name both are match on both file1 and file2 then Out put file would say “duplicate IP and device found”. If just duplicate IP and or device found the output file will indicate “duplicate IP found” or “duplicate device found”

I don't care 3rd column; the script will look for match in 1st and/or 2nd column and must not ignore IP which has "#" in front of it.


OUTPUT:

Duplicate IP and device found:

10.1.2.1.3<tab>abc123def
#10.1.2.1.4<tab>abc345def

Duplicate IP Found:

10.1.2.1.6
10.1.2.1.5

Duplicate Device found
abc009def

Thank you very much. btw, this is not a home work.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Dynamically redirect output to duplicate files ???

Hi There are many posts in this forum regarding reditecting output, but mine is a different problem, please have a look. My shell script is redirecting output to a log file dynamically. That is it is using - exec > log1.txt 2>&1 Hence all the traces are appearing in the log1.txt. I want... (3 Replies)
Discussion started by: nsinha
3 Replies

2. Shell Programming and Scripting

Find duplicate value and create an

I need a perl script, which will run every midnight via cronjob and e-mail few users once it finds any duplicated value in a file which is located /etc/hosts, the file name is called hosts and the format of the file has 3 colums and some time 2 columns. The script will look for duplicate IP or... (3 Replies)
Discussion started by: ricky007
3 Replies

3. Shell Programming and Scripting

Find Duplicate files, not by name

I have a directory with images: -rw-r--r-- 1 root root 26216 Mar 19 21:00 020109.210001.jpg -rw-r--r-- 1 root root 21760 Mar 19 21:15 020109.211502.jpg -rw-r--r-- 1 root root 23144 Mar 19 21:30 020109.213002.jpg -rw-r--r-- 1 root root 31350 Mar 20 00:45 020109.004501.jpg -rw-r--r-- 1 root... (2 Replies)
Discussion started by: Ikon
2 Replies

4. Shell Programming and Scripting

Find duplicate files

What utility do you recommend for simply finding all duplicate files among all files? (4 Replies)
Discussion started by: kiasas
4 Replies

5. Shell Programming and Scripting

comparing in files and output

i have a file a with contents tom lasole jon gille sam and a file b with contents tom|1234|abcf|newyork,ohio,oregon sam|2345|drft|texas,london hyle|4444|befr|wisconsin neyo|2333|tdtt|ohio,jersey i want to compare records in file a to file b such that when the name tom is... (5 Replies)
Discussion started by: dealerso
5 Replies

6. Shell Programming and Scripting

removing duplicate records comparing 2 csv files

Hi All, I want to remove the rows from File1.csv by comparing a column/field in the File2.csv. If both columns matches then I want that row to be deleted from File1 using shell script(awk). Here is an example on what I need. File1.csv: RAJAK,ACTIVE,1 VIJAY,ACTIVE,2 TAHA,ACTIVE,3... (6 Replies)
Discussion started by: rajak.net
6 Replies

7. UNIX for Advanced & Expert Users

How to find duplicates contents in a files by comparing other files?

Hi Guys , we have one directory ...in that directory all files will be set on each day.. files must have header ,contents ,footer.. i wants to compare the header,contents,footer ..if its same means display an error message as 'files contents same' (7 Replies)
Discussion started by: Venkatesh1
7 Replies

8. Shell Programming and Scripting

Duplicate files and output list

Gents, I have a file like this. 1 1 1 2 2 3 2 4 2 5 3 6 3 7 4 8 5 9 I would like to get something like it 1 1 2 2 3 4 5 3 6 7 Thanks in advance for your support :b: (8 Replies)
Discussion started by: jiam912
8 Replies

9. Shell Programming and Scripting

Comparing Select Columns from two CSV files in UNIX and create a third file based on comparision

Hi , I want to compare first 3 columns of File A and File B and create a new file File C which will have all rows from File B and will include rows that are present in File A and not in File B based on First 3 column comparison. Thanks in advance for your help. File A A,B,C,45,46... (2 Replies)
Discussion started by: ady_koolz
2 Replies

10. Shell Programming and Scripting

Comparing two files in UNIX and create a new file similar to equi join

I have 2 files namely branch.txt file & RXD.txt file as below Ex:Branch.txt ========================= B1,Branchname1,city,country B2,Branchname2,city,country B3,Branchname3,city,country B4,Branchname4,city,country B5,Branchname5,city,country RXD file : will... (11 Replies)
Discussion started by: satece
11 Replies
DIFF(1) 						      General Commands Manual							   DIFF(1)

NAME
diff - differential file comparator SYNOPSIS
diff [ -efbh ] file1 file2 DESCRIPTION
Diff tells what lines must be changed in two files to bring them into agreement. If file1 (file2) is `-', the standard input is used. If file1 (file2) is a directory, then a file in that directory whose file-name is the same as the file-name of file2 (file1) is used. The normal output contains lines of these forms: n1 a n3,n4 n1,n2 d n3 n1,n2 c n3,n4 These lines resemble ed commands to convert file1 into file2. The numbers after the letters pertain to file2. In fact, by exchanging `a' for `d' and reading backward one may ascertain equally how to convert file2 into file1. As in ed, identical pairs where n1 = n2 or n3 = n4 are abbreviated as a single number. Following each of these lines come all the lines that are affected in the first file flagged by `<', then all the lines that are affected in the second file flagged by `>'. The -b option causes trailing blanks (spaces and tabs) to be ignored and other strings of blanks to compare equal. The -e option produces a script of a, c and d commands for the editor ed, which will recreate file2 from file1. The -f option produces a similar script, not useful with ed, in the opposite order. In connection with -e, the following shell program may help maintain multiple versions of a file. Only an ancestral file ($1) and a chain of version-to-version ed scripts ($2,$3,...) made by diff need be on hand. A `latest version' appears on the standard output. (shift; cat $*; echo '1,$p') | ed - $1 Except in rare circumstances, diff finds a smallest sufficient set of file differences. Option -h does a fast, half-hearted job. It works only when changed stretches are short and well separated, but does work on files of unlimited length. Options -e and -f are unavailable with -h. FILES
/tmp/d????? /usr/lib/diffh for -h SEE ALSO
cmp(1), comm(1), ed(1) DIAGNOSTICS
Exit status is 0 for no differences, 1 for some, 2 for trouble. BUGS
Editing scripts produced under the -e or -f option are naive about creating lines consisting of a single `.'. DIFF(1)
All times are GMT -4. The time now is 08:07 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy