Sponsored Content
Top Forums Shell Programming and Scripting Find duplicate value comparing 2 files and create an output Post 302170780 by ricky007 on Tuesday 26th of February 2008 03:57:11 PM
Old 02-26-2008
Find duplicate value comparing 2 files and create an output

Thanks, but I am facing complicated situation.
There are multiple files:

1. filenane.xxx
2. finename.abcnnxyz


So the scripts needs to compare filenane.xxx files with finename.abcnnxyz files.

In other works, compare all files filenane.xxx with finename.abcnnxyz look for duplciates. There are 15/20 diff filenane.xxx type of files and may be 10/15 finename.abcnnxyz of files.


Thanks.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Dynamically redirect output to duplicate files ???

Hi There are many posts in this forum regarding reditecting output, but mine is a different problem, please have a look. My shell script is redirecting output to a log file dynamically. That is it is using - exec > log1.txt 2>&1 Hence all the traces are appearing in the log1.txt. I want... (3 Replies)
Discussion started by: nsinha
3 Replies

2. Shell Programming and Scripting

Find duplicate value and create an

I need a perl script, which will run every midnight via cronjob and e-mail few users once it finds any duplicated value in a file which is located /etc/hosts, the file name is called hosts and the format of the file has 3 colums and some time 2 columns. The script will look for duplicate IP or... (3 Replies)
Discussion started by: ricky007
3 Replies

3. Shell Programming and Scripting

Find Duplicate files, not by name

I have a directory with images: -rw-r--r-- 1 root root 26216 Mar 19 21:00 020109.210001.jpg -rw-r--r-- 1 root root 21760 Mar 19 21:15 020109.211502.jpg -rw-r--r-- 1 root root 23144 Mar 19 21:30 020109.213002.jpg -rw-r--r-- 1 root root 31350 Mar 20 00:45 020109.004501.jpg -rw-r--r-- 1 root... (2 Replies)
Discussion started by: Ikon
2 Replies

4. Shell Programming and Scripting

Find duplicate files

What utility do you recommend for simply finding all duplicate files among all files? (4 Replies)
Discussion started by: kiasas
4 Replies

5. Shell Programming and Scripting

comparing in files and output

i have a file a with contents tom lasole jon gille sam and a file b with contents tom|1234|abcf|newyork,ohio,oregon sam|2345|drft|texas,london hyle|4444|befr|wisconsin neyo|2333|tdtt|ohio,jersey i want to compare records in file a to file b such that when the name tom is... (5 Replies)
Discussion started by: dealerso
5 Replies

6. Shell Programming and Scripting

removing duplicate records comparing 2 csv files

Hi All, I want to remove the rows from File1.csv by comparing a column/field in the File2.csv. If both columns matches then I want that row to be deleted from File1 using shell script(awk). Here is an example on what I need. File1.csv: RAJAK,ACTIVE,1 VIJAY,ACTIVE,2 TAHA,ACTIVE,3... (6 Replies)
Discussion started by: rajak.net
6 Replies

7. UNIX for Advanced & Expert Users

How to find duplicates contents in a files by comparing other files?

Hi Guys , we have one directory ...in that directory all files will be set on each day.. files must have header ,contents ,footer.. i wants to compare the header,contents,footer ..if its same means display an error message as 'files contents same' (7 Replies)
Discussion started by: Venkatesh1
7 Replies

8. Shell Programming and Scripting

Duplicate files and output list

Gents, I have a file like this. 1 1 1 2 2 3 2 4 2 5 3 6 3 7 4 8 5 9 I would like to get something like it 1 1 2 2 3 4 5 3 6 7 Thanks in advance for your support :b: (8 Replies)
Discussion started by: jiam912
8 Replies

9. Shell Programming and Scripting

Comparing Select Columns from two CSV files in UNIX and create a third file based on comparision

Hi , I want to compare first 3 columns of File A and File B and create a new file File C which will have all rows from File B and will include rows that are present in File A and not in File B based on First 3 column comparison. Thanks in advance for your help. File A A,B,C,45,46... (2 Replies)
Discussion started by: ady_koolz
2 Replies

10. Shell Programming and Scripting

Comparing two files in UNIX and create a new file similar to equi join

I have 2 files namely branch.txt file & RXD.txt file as below Ex:Branch.txt ========================= B1,Branchname1,city,country B2,Branchname2,city,country B3,Branchname3,city,country B4,Branchname4,city,country B5,Branchname5,city,country RXD file : will... (11 Replies)
Discussion started by: satece
11 Replies
DIFF(1) 						      General Commands Manual							   DIFF(1)

NAME
diff - differential file and directory comparator SYNOPSIS
diff [ -l ] [ -r ] [ -s ] [ -cefhn ] [ -biwt ] dir1 dir2 diff [ -cefhn ] [ -biwt ] file1 file2 diff [ -Dstring ] [ -biw ] file1 file2 DESCRIPTION
If both arguments are directories, diff sorts the contents of the directories by name, and then runs the regular file diff algorithm (described below) on text files which are different. Binary files which differ, common subdirectories, and files which appear in only one directory are listed. Options when comparing directories are: -l long output format; each text file diff is piped through pr(1) to paginate it, other differences are remembered and summarized after all text file differences are reported. -r causes application of diff recursively to common subdirectories encountered. -s causes diff to report files which are the same, which are otherwise not mentioned. -Sname starts a directory diff in the middle beginning with file name. When run on regular files, and when comparing text files which differ during directory comparison, diff tells what lines must be changed in the files to bring them into agreement. Except in rare circumstances, diff finds a smallest sufficient set of file differences. If nei- ther file1 nor file2 is a directory, then either may be given as `-', in which case the standard input is used. If file1 is a directory, then a file in that directory whose file-name is the same as the file-name of file2 is used (and vice versa). There are several options for output format; the default output format contains lines of these forms: n1 a n3,n4 n1,n2 d n3 n1,n2 c n3,n4 These lines resemble ed commands to convert file1 into file2. The numbers after the letters pertain to file2. In fact, by exchanging `a' for `d' and reading backward one may ascertain equally how to convert file2 into file1. As in ed, identical pairs where n1 = n2 or n3 = n4 are abbreviated as a single number. Following each of these lines come all the lines that are affected in the first file flagged by `<', then all the lines that are affected in the second file flagged by `>'. Except for -b, -w, -i or -t which may be given with any of the others, the following options are mutually exclusive: -e produces a script of a, c and d commands for the editor ed, which will recreate file2 from file1. In connection with -e, the fol- lowing shell program may help maintain multiple versions of a file. Only an ancestral file ($1) and a chain of version-to-version ed scripts ($2,$3,...) made by diff need be on hand. A `latest version' appears on the standard output. (shift; cat $*; echo '1,$p') | ed - $1 Extra commands are added to the output when comparing directories with -e, so that the result is a sh(1) script for converting text files which are common to the two directories from their state in dir1 to their state in dir2. -f produces a script similar to that of -e, not useful with ed, and in the opposite order. -n produces a script similar to that of -e, but in the opposite order and with a count of changed lines on each insert or delete com- mand. This is the form used by rcsdiff(1). -c produces a diff with lines of context. The default is to present 3 lines of context and may be changed, e.g to 10, by -c10. With -c the output format is modified slightly: the output beginning with identification of the files involved and their creation dates and then each change is separated by a line with a dozen *'s. The lines removed from file1 are marked with `- '; those added to file2 are marked `+ '. Lines which are changed from one file to the other are marked in both files with with `! '. Changes which lie within <context> lines of each other are grouped together on output. (This is a change from the previous ``diff -c'' but the resulting output is usually much easier to interpret.) -h does a fast, half-hearted job. It works only when changed stretches are short and well separated, but does work on files of unlimited length. -Dstring causes diff to create a merged version of file1 and file2 on the standard output, with C preprocessor controls included so that a compilation of the result without defining string is equivalent to compiling file1, while defining string will yield file2. -b causes trailing blanks (spaces and tabs) to be ignored, and other strings of blanks to compare equal. -w is similar to -b but causes whitespace (blanks and tabs) to be totally ignored. E.g., ``if ( a == b )'' will compare equal to ``if(a==b)''. -i ignores the case of letters. E.g., ``A'' will compare equal to ``a''. -t will expand tabs in output lines. Normal or -c output adds character(s) to the front of each line which may screw up the indenta- tion of the original source lines and make the output listing difficult to interpret. This option will preserve the original source's indentation. FILES
/tmp/d????? /usr/libexec/diffh for -h /bin/diff for directory diffs /bin/pr SEE ALSO
cmp(1), cc(1), comm(1), ed(1), diff3(1) DIAGNOSTICS
Exit status is 0 for no differences, 1 for some, 2 for trouble. BUGS
Editing scripts produced under the -e or -f option are naive about creating lines consisting of a single `.'. When comparing directories with the -b, -w or -i options specified, diff first compares the files ala cmp, and then decides to run the diff algorithm if they are not equal. This may cause a small amount of spurious output if the files then turn out to be identical because the only differences are insignificant blank string or case differences. 4th Berkeley Distribution October 21, 1996 DIFF(1)
All times are GMT -4. The time now is 07:39 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy