Sponsored Content
Top Forums Shell Programming and Scripting Script to Compare Multiple Files. Post 302387637 by thegeek on Sunday 17th of January 2010 08:53:47 PM
Old 01-17-2010
You can solve this in two ways.

find md5sum for all the files, and whose md5sum are same, the contents could also be the same.
Code:
md5sum *

Or try this tool in sourceforge, finddup,
finddup | Get finddup at SourceForge.net

This might have some bugs, but anyway this could find out and tell you the duplications...
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare multiple columns between 2 files

hello I need to compare 2 text files. File 1 has 2 columns and file 2 has 1 to many. Sample: File 1: 111 555 222 666 333 777 444 755 File 2: 000 110 113 114 844 111 555 999 202 777 865 098 023 222 313 499 065 655 333 011 890 777 433 (15 Replies)
Discussion started by: stevesmith
15 Replies

2. Shell Programming and Scripting

String compare in multiple files

Hi I have a requirement to process number of files matching a criteria. The resulted file would be processed indivdually looking for a particular string until another one found lines afterwards. Then look for the occurrence of another string in the result count and display/return the result. ... (13 Replies)
Discussion started by: arungn
13 Replies

3. Shell Programming and Scripting

Compare multiple files and print unique lines

Hi friends, I have multiple files. For now, let's say I have two of the following style cat 1.txt cat 2.txt output.txt Please note that my files are not sorted and in the output file I need another extra column that says the file from which it is coming. I have more than 100... (19 Replies)
Discussion started by: jacobs.smith
19 Replies

4. Shell Programming and Scripting

compare multiple files and get the difference

Hi all, i have 50 files .data should be same in these 50 files , so my task is to find the difference. i need a logic , which finds difference between all files and print in output file with file name where it found that difference . i tried below logic , but its not giving me what i want. let... (2 Replies)
Discussion started by: deepakiniimt
2 Replies

5. Shell Programming and Scripting

Compare multiple files with multiple number of columns

Hi, input file1 abcd 123 198 xyz1:0909090-0909091 ghij 234 999 xyz2:987654:987655 kilo 7890 7990 xyz3:12345-12357 prem 9 112 xyz5:97-1134 input file2 abcd 123 198 xyz1:0909090-0909091 -9.122 0 abed 88 98 xyz1:98989-090808 -1.234 1.345 ghij 234 999 xyz2:987654:987655 -10.87090909 5... (5 Replies)
Discussion started by: jacobs.smith
5 Replies

6. Shell Programming and Scripting

Compare multiple columns from 2 files

Hi, I need to compare multiple columns from 2 files. I can, for example, have these 2 files: file1: col1, col2, col3,col4 a,1,4,7 b,2,5,8 c,3,6,9file2: col1, col2, col3,col4 a,2,3,2 b,5,7,5 c,1,9,8As a result, I need for example the difference between the columns 2 and 4: col2,... (3 Replies)
Discussion started by: Subbeh
3 Replies

7. Shell Programming and Scripting

Compare multiple files, and extract items that are common to ALL files only

I have this code awk 'NR==FNR{a=$1;next} a' file1 file2 which does what I need it to do, but for only two files. I want to make it so that I can have multiple files (for example 30) and the code will return only the items that are in every single one of those files and ignore the ones... (7 Replies)
Discussion started by: castrojc
7 Replies

8. Shell Programming and Scripting

Compare multiple files, identify common records and combine unique values into one file

Good morning all, I have a problem that is one step beyond a standard awk compare. I would like to compare three files which have several thousand records against a fourth file. All of them have a value in each row that is identical, and one value in each of those rows which may be duplicated... (1 Reply)
Discussion started by: nashton
1 Replies

9. Shell Programming and Scripting

Compare and Differentiate Multiple Files

Im using code below comparing and getting the diff of two files. But what about multiple files? can you help me guys diff -u file1 file2|sed -n '/^---/!{/^-/s/-//p}' diff -ui file1 file2| sed -n '/^++/!{/^+/s/+//p}' Example File1: aaa bbb ccc ddd File2: bbb eee (5 Replies)
Discussion started by: kenshinhimura
5 Replies

10. Shell Programming and Scripting

How to compare multiple files in UNIX?

Hi, I have below query related to multiple file comparing I have four files i want to compare it and contents of one file will not be presence in 3other files and if any content found then it will print the execution. Can you please help me how to achieve it. (2 Replies)
Discussion started by: soumyamihp
2 Replies
genhash(1)						      General Commands Manual							genhash(1)

NAME
genhash - md5 hash generation tool for remote web pages SYNOPSIS
genhash [options] [-s server-address] [-p port] [-u url] DESCRIPTION
genhash is a tool used for generating md5sum hashes of remote web pages. genhash can use HTTP or HTTPS to connect to the web page. The output by this utility includes the HTTP header, page data, and the md5sum of the data. This md5sum can then be used within the keepalived(8) program, for monitoring HTTP and HTTPS services. OPTIONS
--use-ssl, -S Use SSL to connect to the server. --server <host>, -s Specify the ip address to connect to. --port <port>, -p Specify the port to connect to. --url <url>, -u Specify the path to the file you want to generate the hash of. --use-virtualhost <host>, -u Specify the virtual host to send along with the HTTP headers. --verbose, -v Be verbose with the output. --help, -h Display the program help screen and exit. --release, -r Display the release number (version) and exit. SEE ALSO
keepalived(8), keepalived.conf(5) AUTHOR
genhash was written by Alexandre Cassen <acassen@linux-vs.org>. This man page was contributed by Andres Salomon <dilinger@voxel.net> for the Debian GNU/Linux system (but may be used by others). Feb 2004 genhash(1)
All times are GMT -4. The time now is 09:43 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy