Sponsored Content
Full Discussion: Diff on 1gb files
Top Forums Shell Programming and Scripting Diff on 1gb files Post 302719507 by abhisheksunkari on Monday 22nd of October 2012 02:48:03 PM
Old 10-22-2012
Diff on 1gb files

Hey Guys,
I have a scenario to compare two different files which are of size 1gb each. I need to get the uncommon lines. I planned to use sdiff command, which generally works perfect for me. But in this case am facing a error saying

"diff: memory exhausted"
Can anyone please explain this.

Also, can anyone suggest a unix command to compare huge files, to get the common and the uncommon lines between the two files.

Thanks,
Abhishek S.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

diff 2 files; output diff's to 3rd file

Hello, I want to compare two files. All records in file 2 that are not in file 1 should be output to file 3. For example: file 1 123 1234 123456 file 2 123 2345 23456 file 3 should have 2345 23456 I have looked at diff, bdiff, cmp, comm, diff3 without any luck! (2 Replies)
Discussion started by: blt123
2 Replies

2. UNIX for Advanced & Expert Users

ZIP file with 1GB or more

Hi, I have to do a search on a zip files whose sizes vary from 1GB to 1.5GB. I dont want to unzip it since if it goes beyond 2GB....... also will unzip -p filename | grep create any problems, will it unzip the whole file or will it unzip it piece by piece?? I appreciate your inputs... ... (1 Reply)
Discussion started by: baanprog
1 Replies

3. Shell Programming and Scripting

Find duplicates from multuple files with 2 diff types of files

I need to compare 2 diff type of files and find out the duplicate after comparing each types of files: Type 1 file name is like: file1.abc (the extension abc could any 3 characters but I can narrow it down or hardcode for 10/15 combinations). The other file is file1.bcd01abc (the extension... (2 Replies)
Discussion started by: ricky007
2 Replies

4. Shell Programming and Scripting

Diff b/w 2 files

Hi Masters, I have two files named file1 and file2. Both the files contains the same contents with some difference in comments,space.But no content change. I tried to find the diff between the two files to make sure that contents are same. For that i tried diff -ibw file1 file2 But... (1 Reply)
Discussion started by: ecearund
1 Replies

5. Red Hat

1Gb+ memory problem

Hi, I have a Linux distribution ( Oralce Enterprise Linux 5.3 i.e. Redhat ) that I have installed. It works fine when I used 2*512Mb dimms or replace them with a single 1Gb dimm. However when I try to go above 1 Gb the bootup and general performance deteriorates badly. The BIOS picks up the memory... (3 Replies)
Discussion started by: jimthompson
3 Replies

6. Shell Programming and Scripting

diff of files

Hi, I have 2 files.I want to check if file1 is contained in file2. A.txt: ----- AAA BBB B.txt: ------ CCC AAA BBB DDD I want to check if A.txt is contained in B.txt. Can it be done using SED ? (12 Replies)
Discussion started by: giri_luck
12 Replies

7. Shell Programming and Scripting

diff bw two files

Hi All, I have two files which look as below File1 serial="1" name="abc" type="employee" field="IT" serial="2" name="cde" type="intern" field="Marketing" serial="3" name="pqr" type="contractor" field="IT" serial="4" name="xyz" type="employee" field="Sales" File2 serial="1"... (3 Replies)
Discussion started by: grajp002
3 Replies

8. Shell Programming and Scripting

.procmailrc and uudeview (put attachments from diff senders to diff folders)

Moderator, please, delete this topic (1 Reply)
Discussion started by: optik77
1 Replies

9. UNIX for Dummies Questions & Answers

How do I find files which are older than 30 days and greater than 1GB

Hi All, I know the separate commands for finding files greater than 30 days and finding files greater than 1GB. How do I combine these two commands? Meaning how do I find files which are > 1GB and older than 30 days? ;) (4 Replies)
Discussion started by: Hangman2
4 Replies

10. Shell Programming and Scripting

Diff 3 files, but diff only their 2nd column

Guys i have 3 files, but i want to compare and diff only the 2nd column path=`/home/whois/doms` for i in `cat domain.tx` do whois $i| sed -n '/Registry Registrant ID:/,/Registrant Email:/p' > $path/$i.registrant whois $i| sed -n '/Registry Admin ID:/,/Admin Email:/p' > $path/$i.admin... (10 Replies)
Discussion started by: kenshinhimura
10 Replies
sdiff(1)						      General Commands Manual							  sdiff(1)

NAME
sdiff - Compares two files and displays the differences in a side-by-side format SYNOPSIS
sdiff [-l | -s] [-w number] [-o output_file] file1 file2 The sdiff command reads file1 and file2, uses diff to compare them, and writes the results to standard output in a side-by-side format. OPTIONS
Displays only the left side when lines are identical. Creates a third file, output_file, by a controlled interactive line-by-line merging of file1 and file2. The following subcommands govern the creation of this file: Adds the left side to output_file. Adds the right side to output_file. Stops displaying identical lines. Begins displaying identical lines. Enters ed with the left side, the right side, both sides, or an empty file, respectively. Each time you exit from ed, sdiff writes the resulting edited file to the end of output_file. If you fail to save the changes before exiting, sdiff writes the initial input to output_file. Exits the interactive session. Suppresses display of identical lines. Sets the width of the output line to number (130 characters by default). DESCRIPTION
The sdiff command displays each line of the two files with a series of spaces between them if the lines are identical, a < (left angle bracket) in the field of spaces if the line only exists in file1, a > (right angle bracket) if the line only exists in file2, and a | (ver- tical bar) for lines that are different. When you specify the -o option, sdiff produces a third file by merging file1 and file2 according to your instructions. Note that the sdiff command invokes the diff -b command to compare two input files. The -b option causes the diff command to ignore trail- ing spaces, tab characters, and consider other strings of spaces as equal. EXAMPLES
To print a comparison of two files, enter: sdiff chap1.bak chap1 This displays a side-by-side listing that compares each line of chap1.bak and chap1. To display only the lines that differ, enter: sdiff -s -w 80 chap1.bak chap1 This displays the differences at the tty. The -w 80 sets page width to 80 columns. The -s option tells sdiff not to display lines that are identical in both files. To selectively combine parts of two files, enter: sdiff -s -w 80 -o chap1.combo chap1.bak chap1 This combines chap1.bak and chap1 into a new file called chap1.combo. For each group of differing lines, sdiff asks you which group to keep or whether you want to edit them using ed. SEE ALSO
Commands: diff(1), ed(1) sdiff(1)
All times are GMT -4. The time now is 02:32 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy