|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
Problem comparing 2 files with lot of data
Post 302128627 by rafisha on Wednesday 25th of July 2007 07:56:47 PM
07-25-2007
Solution
Thanks to porter lorcan and ygor
Porter, your solution was not possible to perform because it needs a lot of memory (diff uses 6 times the size of the file in memory), and bdiff didnīt get exact results because the fragmentation of the files.
Lorcan: your solution worked great, it takes a few minutes
Ygor: after the problem with diff i was afraid to use greo so i didn't try it
The solution i found was this: first i had to take all the blank spaces out with awk, then i sort the files and then use COMM to get the diferences, comm worked great and get the results in a few seconds
Thanks again for your help
Porter, your solution was not possible to perform because it needs a lot of memory (diff uses 6 times the size of the file in memory), and bdiff didnīt get exact results because the fragmentation of the files.
Lorcan: your solution worked great, it takes a few minutes
Ygor: after the problem with diff i was afraid to use greo so i didn't try it
The solution i found was this: first i had to take all the blank spaces out with awk, then i sort the files and then use COMM to get the diferences, comm worked great and get the results in a few seconds
Thanks again for your help
|
|
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
here I go again...kinda hard to explain so I apologize.
I need to rename a bunch of files in a directory. I need to remove the first three characters of the filename, and then toward the end of the filename there is constant text inside of brackets. here is a demo (not for real) 'ls -1' of the... (11 Replies)
Discussion started by: ajp7701
11 Replies
2. Shell Programming and Scripting
Hi All,
I've two .csv files as below
file1.csv
abc, tdf, 223, tpx
jgsd, tex, 342, rpy
a, jdjdsd, 423, djfkld
Where as file2.csv is the new version of file1.csv with some added fields in the end of each line and some additional lines.
lfj, eru, 98, jkldj, 39, jdkj9
abc, tdf, 223, tpx,... (3 Replies)
Discussion started by: ganapati
3 Replies
3. UNIX for Dummies Questions & Answers
Hi power user,
if I have this file:
file1.txt:
1111
1111
2222
2222
3333
3333
3333
4444
4444
4444
when I run the
sort file1.txt | uniq > data1.txt
the result is (2 Replies)
Discussion started by: anjas
2 Replies
4. Shell Programming and Scripting
So i have about 600gb of data.. in which there are alot of directories and alot of files.. Im trying to put this on a ftp server.. So i want to set the permissions on the directories to be 755 and the permission on the files to be 644. So i used:
find . -type d -exec chmod 755 {}\;
and
find .... (6 Replies)
Discussion started by: supermiguel
6 Replies
5. Shell Programming and Scripting
Hi
This is the list file that i have :
The files is more than this.
I will rename one by one file become like this :
So just change the time stamp 200906 become 200905.
Is it possible using script ?
Thanks (3 Replies)
Discussion started by: justbow
3 Replies
6. Shell Programming and Scripting
I've 2 files. Need to compare File1.Field1,File1.Field2 with File2.Field1,File2.Field2. If matches then create a new file.
File1
10 A|ADB|967143.24|1006101.5
3E HK|DHB|24294.76|242513.89
ABN ACU|ADB|22104.69|51647.14
ABN BU|DBA|39137.14|109128.38
ABN|ADB|64466.89|167936.55
ABOC... (2 Replies)
Discussion started by: buster
2 Replies
7. Shell Programming and Scripting
Hello, I have about 3400 files in a tree structure (about 80% are html files).
1. I need to modify every html file to remove <p> style and old things like font attribute and add another style.
2. I need to change the root of all links that are in the html. e.g. change /old/path/ to /new/path... (1 Reply)
Discussion started by: Yaazkal
1 Replies
8. UNIX for Dummies Questions & Answers
hi guys
I have suse 11 sp1 and I have a lot of warn file filling / these are under /var/log
there's this big one
-rw-r----- 1 root root 3.9G Feb 1 10:28 warn
warn: ASCII text
and the others that are about 2.5 to 3MB - they are about 130 warn-*.bz2
-rw-r----- 1 root root 3.9G Feb... (2 Replies)
Discussion started by: karlochacon
2 Replies
9. Shell Programming and Scripting
Hi Friends,
I have a file 1
CREATE MULTISET TABLE TEYT_Q9_T.TEST ,NO FALLBACK ,
NO BEFORE JOURNAL,
NO AFTER JOURNAL,
CHECKSUM = DEFAULT,
DEFAULT MERGEBLOCKRATIO
(
XYZ DECIMAL(10,0),
ABC VARCHAR(5) CHARACTER SET LATIN NOT CASESPECIFIC,
PQR... (3 Replies)
Discussion started by: i150371485
3 Replies
10. Shell Programming and Scripting
Hi,
I have a huge structure of directories and subdirectories contsining some data. The lowest folders contain a file "image.png" which need to be converted to "folder.jpg". But how can I do that for all these files automatically? That's what I alredy have
find /path -type f -name... (1 Reply)
Discussion started by: KarlKarpfen
1 Replies
LEARN ABOUT DEBIAN
phones
phones(4) File Formats phones(4) NAME
phones - remote host phone number database SYNOPSIS
/etc/phones DESCRIPTION
The file /etc/phones contains the system-wide private phone numbers for the tip(1) program. /etc/phones is normally unreadable, and so may contain privileged information. The format of /etc/phones is a series of lines of the form: <system-name>[ ]*<phone-number>. The system name is one of those defined in the remote(4) file and the phone number is constructed from [0123456789-=*%]. The `=' and `*' characters are indicators to the auto call units to pause and wait for a second dial tone (when going through an exchange). The `=' is required by the DF02-AC and the `*' is required by the BIZCOMP 1030. Comment lines are lines containing a `#' sign in the first column of the line. Only one phone number per line is permitted. However, if more than one line in the file contains the same system name tip(1) will attempt to dial each one in turn, until it establishes a connection. FILES
/etc/phones SEE ALSO
tip(1), remote(4) SunOS 5.10 14 Jan 1992 phones(4)