06-10-2010
We have a similar problem. Are you running diff? That would take forever.
Use something that has associative (hashed) arrays like awk or perl. Assuming you have several files, and an "old" one and a "new" one, that should take less than an hour.
You can search here for examples of both types of code on how to find file differences.
You need a lot of virtual memory, we run on a Solaris 9 sparc v440 with 32GB of memory.
We complete comparing 1.5GB (250K lines) files in about 5 minutes. We do them 12 at a time: 6 old vs 6 new.
I hope this is what you were asking....
This User Gave Thanks to jim mcnamara For This Post:
5 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
- I am looking for different kind of awk solution which I don't think is mentioned before in these forums.
Number of rows in the file are fixed
Their are two columns in file1.txt
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8
9 9
10 10
I am looking for 3... (1 Reply)
Discussion started by: softwarekids23
1 Replies
2. Shell Programming and Scripting
I have a txt file with several columns and i want to peform an operation on two columns and output it to a new txt file .
file.txt
900.00000 1 1 1
500.00000
500.00000
100000.000
4
4
1.45257346E-07 899.10834 ... (4 Replies)
Discussion started by: shashi792
4 Replies
3. Homework & Coursework Questions
Hi all! I need help to do a few things with a .txt file using egrep.
1. I need to list all sequences where the vowel letters 'a, e, i, o, u' occur in that order, possibly separated by characters other than a, e, i, o, u; consisting of one or more complete words, possibly including punctuation.
... (1 Reply)
Discussion started by: dindiqotu
1 Replies
4. Shell Programming and Scripting
I have a script that I am using to copy around 40-70k files to a NFS NAS.
I have posted my code below in hopes that someone can help me figure out a faster way of achieving this.
At the end of the script i need to have all the files in the list, copied over to the nas with source directory... (8 Replies)
Discussion started by: nitrobass24
8 Replies
5. Shell Programming and Scripting
friends
good morning
FTP works perfect but I have a doubt
if I want to transport 10 files, I imagine that I should not open 10 connections as I can transfer more than 1 file?
ftp -n <<!EOF
open caburga
user ephfact ephfact
cd /users/efactura/docONE/entrada
bin
mput EPH`date... (16 Replies)
Discussion started by: tricampeon81
16 Replies
DIFF(1) General Commands Manual DIFF(1)
NAME
diff - print differences between two files
SYNOPSIS
diff [-c | -e | -C n] [-br]file1 file2
OPTIONS
-C n Produce output that contains n lines of context
-b Ignore white space when comparing
-c Produce output that contains three lines of context
-e Produce an ed-script to convert file1 into file2
-r Apply diff recursively to files and directories of
EXAMPLES
diff file1 file2 # Print differences between 2 files
diff -C 0 file1 file2
# Same as above
diff -C 3 file1 file2
# Output three lines of context with every
diff -c file1 file2 # Same
diff /etc /dev # Compares recursively the directories /etc and /dev
diff passwd /etc # Compares ./passwd to /etc/passwd
DESCRIPTION
the same name, when file1 and file2 are both directories" difference encountered"
Diff compares two files and generates a list of lines telling how the two files differ. Lines may not be longer than 128 characters. If
the two arguments on the command line are both directories, diff recursively steps through all subdirectories comparing files of the same
name. If a file name is found only in one directory, a diagnostic message is written to stdout. A file that is of either block special,
character special or FIFO special type, cannot be compared to any other file. On the other hand, if there is one directory and one file
given on the command line, diff tries to compare the file with the same name as file in the directory directory.
SEE ALSO
cdiff(1), cmp(1), comm(1), patch(1).
DIFF(1)