Sponsored Content
Full Discussion: Q: Howto compare 2 files
Top Forums UNIX for Dummies Questions & Answers Q: Howto compare 2 files Post 302695753 by Sekullos on Monday 3rd of September 2012 06:05:29 PM
Old 09-03-2012
Q: Howto compare 2 files

Greetings,

I made an extraction on 2 different databases. What I need to do is to compare those extractions to know what is on database1 which is not on database2 and vice versa.

In those files, there are only numbers. So each line is just a number witch should be present on both file. If it's not, I want to know which number is not present on X file.

Working on Linux (Red Hat) I tried compare / sdiff etc but all those tools seems to compare line number X from file 1 to line number X on file 2 instead of checking in the whole file Smilie


Here is my output :

File1 :
Code:
123456123
234561234
345612345
456123456

File2 :
Code:
234561234
345612345
123456123

So here I'd like to know that 456123456 is not present in File2 (and get the output in a third file)

Note : I ve got 15 millions lines to deal with so a simple cat | while read and grep script is too slow Smilie


Big thanks to anyone who can help me with this Smilie

Last edited by Scott; 09-03-2012 at 07:12 PM.. Reason: Code tags
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Howto Archive Including Hidden Files?

Hi I want to archive the following all the files and directory like listed below: $ ls -a . .. .bash_history .bash_logout .bash_profile .bashrc .emacs .mysql_history public_html .viminfo What I tried is to use the following command $ gtar cvzf allmyfiles.tar.gz * ... (1 Reply)
Discussion started by: monkfan
1 Replies

2. UNIX for Dummies Questions & Answers

Howto removing files with the same inode

Dear all, # ls -li total 16 2623392 drwxrwxrwx 2 root root 512 Apr 10 01:57 10HPA- 8447490 drwxr-xr-x 3 root root 512 Apr 14 05:29 118OQ- 8447490 drwxr-xr-x 3 root root 512 Apr 14 05:29 118OQ-.old 1925572 drwxrwxrwx 2 root root 512... (3 Replies)
Discussion started by: fu4d
3 Replies

3. Shell Programming and Scripting

compare files in two directories and output changed files to third directory

I have searched about 30 threads, a load of Google pages and cannot find what I am looking for. I have some of the parts but not the whole. I cannot seem to get the puzzle fit together. I have three folders, two of which contain different versions of multiple files, dist/file1.php dist/file2.php... (4 Replies)
Discussion started by: bkeep
4 Replies

4. Shell Programming and Scripting

How to compare 2 files & get only few columns based on a condition related to both files?

Hiiiii friends I have 2 files which contains huge data & few lines of it are as shown below File1: b.dat(which has 21 columns) SSR 1976 8 12 13 10 44.00 39.0700 70.7800 7.0 0 0.00 0 2.78 0.00 0.00 0 0.00 2.78 0 NULL ISC 1976 8 12 22 32 37.39 36.2942 70.7338... (6 Replies)
Discussion started by: reva
6 Replies

5. Shell Programming and Scripting

HowTo translate KSH Scripts to DOS Batch Files ?

Hi there, in near future I have to change my work surrounding from HP UNIX to Windows Vista (great to get rid of old hardware :), but bad to loose UNIX :( ). As I heavily use KSH scripts to do my job, I was wondering, if there is any HowTo available, supporting me in re-writing the scripts to... (4 Replies)
Discussion started by: Joe-K7
4 Replies

6. Shell Programming and Scripting

Compare 2 folders to find several missing files among huge amounts of files.

Hi, all: I've got two folders, say, "folder1" and "folder2". Under each, there are thousands of files. It's quite obvious that there are some files missing in each. I just would like to find them. I believe this can be done by "diff" command. However, if I change the above question a... (1 Reply)
Discussion started by: jiapei100
1 Replies

7. Shell Programming and Scripting

Require compare command to compare 4 files

I have four files, I need to compare these files together. As such i know "sdiff and comm" commands but these commands compare 2 files together. If I use sdiff command then i have to compare each file with other which will increase the codes. Please suggest if you know some commands whcih can... (6 Replies)
Discussion started by: nehashine
6 Replies

8. Shell Programming and Scripting

howto monitor a directory for files then sftp them

Morning all I hope I have put this in the correct forum. I have a requirement to monitor a directory on a server for files being sftp'ed in and then to sftp them of to another server. The issues I have though of are making sure the files have completely transferred onto the server before they... (6 Replies)
Discussion started by: ltodd2
6 Replies

9. Shell Programming and Scripting

Compare multiple files, and extract items that are common to ALL files only

I have this code awk 'NR==FNR{a=$1;next} a' file1 file2 which does what I need it to do, but for only two files. I want to make it so that I can have multiple files (for example 30) and the code will return only the items that are in every single one of those files and ignore the ones... (7 Replies)
Discussion started by: castrojc
7 Replies

10. Shell Programming and Scripting

Howto compare the columns of 2 diff tables of 2 different schemas in UNIX shell script

HI All, I am new to Unix shell scripts.. Could you please post the unix shell script for for the below request., There are two different tables(sample1, sample2) in different schemas(s_schema1, s_schema2). Unix shell script to compare the columns of two different tables of two... (2 Replies)
Discussion started by: Rajkumar Gopal
2 Replies
Test::Regression(3pm)					User Contributed Perl Documentation				     Test::Regression(3pm)

NAME
Test::Regression - Test library that can be run in two modes; one to generate outputs and a second to compare against them VERSION
Version 0.05 SYNOPSIS
use Test::Regression; ok_regression(sub {return "hello world"}, "t/out/hello_world.txt"); DESCRIPTION
Using the various Test:: modules you can compare the output of a function against what you expect. However if the output is complex and changes from version to version, maintenance of the expected output could be costly. This module allows one to use the test code to generate the expected output, so that if the differences with model output are expected, one can easily refresh the model output. EXPORT
ok_regression FUNCTIONS
ok_regression This function requires two arguments: a CODE ref and a file path. The CODE ref is expected to return a SCALAR string which can be compared against previous runs. If the TEST_REGRESSION_GEN is set to a true value, then the CODE ref is run and the output written to the file. Otherwise the output of the file is compared against the contents of the file. There is a third optional argument which is the test name. ENVIRONMENT VARIABLES
TEST_REGRESSION_GEN If the TEST_REGRESSION_GEN environment file is unset or false in a perl sense, then the named output files must exist and be readable and the test will run normally comparing the outputs of the CODE refs against the contents of those files. If the environment variable is true in a perl sense, then model output files will be overwritten with the output of the CODE ref. AUTHOR
Nicholas Bamber, "<nicholas at periapt.co.uk>" BUGS
Please report any bugs or feature requests to "bug-test-regression at rt.cpan.org", or through the web interface at <http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Test-Regression>. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes. testing of STDERR The testing of stderr from this module is not as thorough as I would like. Test::Builder::Tester allows turning off of stderr checking but not matching by regular expression. Handcrafted efforts currently fall foul of Test::Harness. Still it is I believe adequately tested in terms of coverage. SUPPORT
You can find documentation for this module with the perldoc command. perldoc Test::Regression You can also look for information at: o RT: CPAN's request tracker <http://rt.cpan.org/NoAuth/Bugs.html?Dist=Test-Regression> o AnnoCPAN: Annotated CPAN documentation <http://annocpan.org/dist/Test-Regression> o CPAN Ratings <http://cpanratings.perl.org/d/Test-Regression> o Search CPAN <http://search.cpan.org/dist/Test-Regression/> ACKNOWLEDGEMENTS
Some documentation improvements have been suggested by toolic (http://perlmonks.org/?node_id=622051). Thanks to Filip GraliXski for pointing out I need to test against output of zero length and providing a patch. COPYRIGHT &; LICENSE Copyright 2009-10 Nicholas Bamber. This program is free software; you can redistribute it and/or modify it under the terms of either: the GNU General Public License as published by the Free Software Foundation; or the Artistic License. See http://dev.perl.org/licenses/ for more information. perl v5.10.1 2010-08-29 Test::Regression(3pm)
All times are GMT -4. The time now is 10:54 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy