Sponsored Content
Full Discussion: Compare 2 Large files
Top Forums Shell Programming and Scripting Compare 2 Large files Post 302260223 by bezudar on Thursday 20th of November 2008 01:26:25 AM
Old 11-20-2008
Compare 2 Large files

Hi,
i want to compare 2 large files both around 300 mb. They are text files having only one column of numbers. I wish to get the unique values in file2. I tried using diff but it gave an error of memory exhausted. Both files are sorted and i am running on a 1gb ram core 2 duo 2ghz. Help!!
Thanks in advance
AC
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Large files

I am trying to understand the webserver log file for an error which has occured on my live web site. The webserver access file is very big in size so it's not possible to open this file using vi editor. I know the approximate time the error occured, so i am interested in looking for the log file... (4 Replies)
Discussion started by: sehgalniraj
4 Replies

2. UNIX for Dummies Questions & Answers

Need to gzip LARGE files

The windows version of gzip supports pretty much unlimited file sizes while the one we have in solaris only goes up to a set size, one or two gigs I think. Is there a new version of gzip I can put on our systems that supports massive file sizes? (2 Replies)
Discussion started by: LordJezo
2 Replies

3. UNIX for Dummies Questions & Answers

large files?

How do we check 'large files' is enabled on a Unix box -- HP-UX B11.11 (2 Replies)
Discussion started by: ranj@chn
2 Replies

4. Shell Programming and Scripting

Script to Compare a large number of files.

I have a large Filesystem on an AIX server and another one on a Red Hat box. I have syncd the two filesystems using rsysnc. What Im looking for is a script that would compare to the two filesystems to make sure the bits match up and the number of files match up. its around 2.8 million... (5 Replies)
Discussion started by: zippdawg2001
5 Replies

5. UNIX for Dummies Questions & Answers

compare 2 very large lists of different length

I have two very large datasets (>100MB) in a simple vertical list format. They are of different size and with different order and formatting (e.g. whitespace and some other minor cruft that would thwart easy regex). Let's call them set1 and set2. I want to check set2 to see if it contains... (2 Replies)
Discussion started by: uiop44
2 Replies

6. Shell Programming and Scripting

Divide large data files into smaller files

Hello everyone! I have 2 types of files in the following format: 1) *.fa >1234 ...some text... >2345 ...some text... >3456 ...some text... . . . . 2) *.info >1234 (7 Replies)
Discussion started by: ad23
7 Replies

7. Shell Programming and Scripting

Require compare command to compare 4 files

I have four files, I need to compare these files together. As such i know "sdiff and comm" commands but these commands compare 2 files together. If I use sdiff command then i have to compare each file with other which will increase the codes. Please suggest if you know some commands whcih can... (6 Replies)
Discussion started by: nehashine
6 Replies

8. Solaris

How to safely copy full filesystems with large files (10Gb files)

Hello everyone. Need some help copying a filesystem. The situation is this: I have an oracle DB mounted on /u01 and need to copy it to /u02. /u01 is 500 Gb and /u02 is 300 Gb. The size used on /u01 is 187 Gb. This is running on solaris 9 and both filesystems are UFS. I have tried to do it using:... (14 Replies)
Discussion started by: dragonov7
14 Replies

9. Shell Programming and Scripting

Compare large file and identify difference in separate file

I have a very large system generated file containing around 500K rows size 100MB like following HOME|ALICE STREET|3||NEW LISTING HOME|NEWPORT STREET|1||NEW LISTING HOME|KING STREET|5||NEW LISTING HOME|WINSOME AVENUE|4||MODIFICATION CAR|TOYOTA|4||NEW LISTING CAR|FORD|4||NEW... (9 Replies)
Discussion started by: jubaier
9 Replies

10. Shell Programming and Scripting

Script to compare files in 2 folders and delete the large file

Hello, my first thread here. I've been searching and fiddling around for about a week and I cannot find a solution.:confused: I have been converting all of my home videos to HEVC and sometimes the files end up smaller and sometimes they don't. I am currently comparing all the video files... (5 Replies)
Discussion started by: Josh52180
5 Replies
BZDIFF(1)						      General Commands Manual							 BZDIFF(1)

NAME
bzcmp, bzdiff - compare bzip2 compressed files SYNOPSIS
bzcmp [ cmp_options ] file1 [ file2 ] bzdiff [ diff_options ] file1 [ file2 ] DESCRIPTION
Bzcmp and bzdiff are used to invoke the cmp or the diff program on bzip2 compressed files. All options specified are passed directly to cmp or diff. If only 1 file is specified, then the files compared are file1 and an uncompressed file1.bz2. If two files are specified, then they are uncompressed if necessary and fed to cmp or diff. The exit status from cmp or diff is preserved. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +--------------------+-----------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +--------------------+-----------------+ |Availability | SUNWbzip | +--------------------+-----------------+ |Interface Stability | External | +--------------------+-----------------+ SEE ALSO
cmp(1), diff(1), bzmore(1), bzless(1), bzgrep(1), bzip2(1) BUGS
Messages from the cmp or diff programs refer to temporary filenames instead of those specified. BZDIFF(1)
All times are GMT -4. The time now is 12:47 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy