Sponsored Content
Full Discussion: Q: Howto compare 2 files
Top Forums UNIX for Dummies Questions & Answers Q: Howto compare 2 files Post 302695753 by Sekullos on Monday 3rd of September 2012 06:05:29 PM
Old 09-03-2012
Q: Howto compare 2 files

Greetings,

I made an extraction on 2 different databases. What I need to do is to compare those extractions to know what is on database1 which is not on database2 and vice versa.

In those files, there are only numbers. So each line is just a number witch should be present on both file. If it's not, I want to know which number is not present on X file.

Working on Linux (Red Hat) I tried compare / sdiff etc but all those tools seems to compare line number X from file 1 to line number X on file 2 instead of checking in the whole file Smilie


Here is my output :

File1 :
Code:
123456123
234561234
345612345
456123456

File2 :
Code:
234561234
345612345
123456123

So here I'd like to know that 456123456 is not present in File2 (and get the output in a third file)

Note : I ve got 15 millions lines to deal with so a simple cat | while read and grep script is too slow Smilie


Big thanks to anyone who can help me with this Smilie

Last edited by Scott; 09-03-2012 at 07:12 PM.. Reason: Code tags
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Howto Archive Including Hidden Files?

Hi I want to archive the following all the files and directory like listed below: $ ls -a . .. .bash_history .bash_logout .bash_profile .bashrc .emacs .mysql_history public_html .viminfo What I tried is to use the following command $ gtar cvzf allmyfiles.tar.gz * ... (1 Reply)
Discussion started by: monkfan
1 Replies

2. UNIX for Dummies Questions & Answers

Howto removing files with the same inode

Dear all, # ls -li total 16 2623392 drwxrwxrwx 2 root root 512 Apr 10 01:57 10HPA- 8447490 drwxr-xr-x 3 root root 512 Apr 14 05:29 118OQ- 8447490 drwxr-xr-x 3 root root 512 Apr 14 05:29 118OQ-.old 1925572 drwxrwxrwx 2 root root 512... (3 Replies)
Discussion started by: fu4d
3 Replies

3. Shell Programming and Scripting

compare files in two directories and output changed files to third directory

I have searched about 30 threads, a load of Google pages and cannot find what I am looking for. I have some of the parts but not the whole. I cannot seem to get the puzzle fit together. I have three folders, two of which contain different versions of multiple files, dist/file1.php dist/file2.php... (4 Replies)
Discussion started by: bkeep
4 Replies

4. Shell Programming and Scripting

How to compare 2 files & get only few columns based on a condition related to both files?

Hiiiii friends I have 2 files which contains huge data & few lines of it are as shown below File1: b.dat(which has 21 columns) SSR 1976 8 12 13 10 44.00 39.0700 70.7800 7.0 0 0.00 0 2.78 0.00 0.00 0 0.00 2.78 0 NULL ISC 1976 8 12 22 32 37.39 36.2942 70.7338... (6 Replies)
Discussion started by: reva
6 Replies

5. Shell Programming and Scripting

HowTo translate KSH Scripts to DOS Batch Files ?

Hi there, in near future I have to change my work surrounding from HP UNIX to Windows Vista (great to get rid of old hardware :), but bad to loose UNIX :( ). As I heavily use KSH scripts to do my job, I was wondering, if there is any HowTo available, supporting me in re-writing the scripts to... (4 Replies)
Discussion started by: Joe-K7
4 Replies

6. Shell Programming and Scripting

Compare 2 folders to find several missing files among huge amounts of files.

Hi, all: I've got two folders, say, "folder1" and "folder2". Under each, there are thousands of files. It's quite obvious that there are some files missing in each. I just would like to find them. I believe this can be done by "diff" command. However, if I change the above question a... (1 Reply)
Discussion started by: jiapei100
1 Replies

7. Shell Programming and Scripting

Require compare command to compare 4 files

I have four files, I need to compare these files together. As such i know "sdiff and comm" commands but these commands compare 2 files together. If I use sdiff command then i have to compare each file with other which will increase the codes. Please suggest if you know some commands whcih can... (6 Replies)
Discussion started by: nehashine
6 Replies

8. Shell Programming and Scripting

howto monitor a directory for files then sftp them

Morning all I hope I have put this in the correct forum. I have a requirement to monitor a directory on a server for files being sftp'ed in and then to sftp them of to another server. The issues I have though of are making sure the files have completely transferred onto the server before they... (6 Replies)
Discussion started by: ltodd2
6 Replies

9. Shell Programming and Scripting

Compare multiple files, and extract items that are common to ALL files only

I have this code awk 'NR==FNR{a=$1;next} a' file1 file2 which does what I need it to do, but for only two files. I want to make it so that I can have multiple files (for example 30) and the code will return only the items that are in every single one of those files and ignore the ones... (7 Replies)
Discussion started by: castrojc
7 Replies

10. Shell Programming and Scripting

Howto compare the columns of 2 diff tables of 2 different schemas in UNIX shell script

HI All, I am new to Unix shell scripts.. Could you please post the unix shell script for for the below request., There are two different tables(sample1, sample2) in different schemas(s_schema1, s_schema2). Unix shell script to compare the columns of two different tables of two... (2 Replies)
Discussion started by: Rajkumar Gopal
2 Replies
DIFF3(1)							     GNU Tools								  DIFF3(1)

NAME
diff3 - find differences between three files SYNOPSIS
diff3 [options] mine older yours DESCRIPTION
The diff3 command compares three files and outputs descriptions of their differences. The files to compare are mine, older, and yours. At most one of these three file names may be -, which tells diff3 to read the standard input for that file. Options Below is a summary of all of the options that GNU diff3 accepts. Multiple single letter options (unless they take an argument) can be com- bined into a single command line argument. -a Treat all files as text and compare them line-by-line, even if they do not appear to be text. -A Incorporate all changes from older to yours into mine, surrounding all conflicts with bracket lines. -B Old behavior of -A. Shows non-conflicts. -e Generate an ed script that incorporates all the changes from older to yours into mine. -E Like -e, except bracket lines from overlapping changes' first and third files. With -e, an overlapping change looks like this: <<<<<<< mine lines from mine ======= lines from yours >>>>>>> yours --ed Generate an ed script that incorporates all the changes from older to yours into mine. --easy-only Like -e, except output only the nonoverlapping changes. -i Generate w and q commands at the end of the ed script for System V compatibility. This option must be combined with one of the -AeExX3 options, and may not be combined with -m. --initial-tab Output a tab rather than two spaces before the text of a line in normal format. This causes the alignment of tabs in the line to look normal. -L label --label=label Use the label label for the brackets output by the -A, -E and -X options. This option may be given up to three times, one for each input file. The default labels are the names of the input files. Thus diff3 -L X -L Y -L Z -m A B C acts like diff3 -m A B C , except that the output looks like it came from files named X, Y and Z rather than from files named A, B and C. -m --merge Apply the edit script to the first file and send the result to standard output. Unlike piping the output from diff3 to ed, this works even for binary files and incomplete lines. -A is assumed if no edit script option is specified. --overlap-only Like -e, except output only the overlapping changes. --show-all Incorporate all unmerged changes from older to yours into mine, surrounding all overlapping changes with bracket lines. --show-overlap Like -e, except bracket lines from overlapping changes' first and third files. -T Output a tab rather than two spaces before the text of a line in normal format. This causes the alignment of tabs in the line to look normal. --text Treat all files as text and compare them line-by-line, even if they do not appear to be text. -v --version Output the version number of diff3. -x Like -e, except output only the overlapping changes. -X Like -E, except output only the overlapping changes. In other words, like -x, except bracket changes as in -E. -3 Like -e, except output only the nonoverlapping changes. SEE ALSO
cmp(1), comm(1), diff(1), ed(1), patch(1), sdiff(1). DIAGNOSTICS
An exit status of 0 means diff3 was successful, 1 means some conflicts were found, and 2 means trouble. GNU Tools 22sep1993 DIFF3(1)
All times are GMT -4. The time now is 07:50 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy