11-03-2005
How to compare data in two flat files and update them?
Hi All,
I am giving an example similar to the problem I have.
I have two data files of 10 columns each in which fields are delimited by comma(,). I need to compare compare the two files using the uniq col(col3). If there are any records in file1 and are not in file2 then I have check the value for col4 of those records from a database table, if its success then I have to include those records in file2 and have to change the value of col6 to "updated".
Can you please give me an idea how to implement in a shell script.
your help will be greatly appreciated.
Thanks in Advance.
Raju
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi,
does anyone know of a way to compare files update time (not only days - also hours and minutes) (command? scripts? perl scripts?)
Dori (8 Replies)
Discussion started by: dorilevy
8 Replies
2. Shell Programming and Scripting
I would like to compare a data file (before and after a process has run) to identify if there are any differences. Can anyone help !! (1 Reply)
Discussion started by: ithomp
1 Replies
3. Shell Programming and Scripting
Hi,
I need to compare two flat files (yesterday & today's data) and get only the changed data from flat files. In flat file i dont have data column or anything its just a string data in flat file.Can any one please let me know the script
With Regds
Shashi (3 Replies)
Discussion started by: jtshashidhar
3 Replies
4. Shell Programming and Scripting
Hi Gurus,
I searched the forum but didnt get much info.
I want to compare 2 files.
1)Newfile comes today with
2)Old file of previous day.
The files are same ,just the new files might have new records sometimes.
So I want to capture these new records in another file.
Can anyone help... (5 Replies)
Discussion started by: ganesh123
5 Replies
5. Shell Programming and Scripting
Hi Frnds,
I have a flat file with millions of records. .
Now I on this. (I prefer for AWK as its gives good performance.)
Old_file.txt
------------------
1 gopi ase ....
2 arun pl ...
3 jack sutha ..
4 peter pm ..
...
New_file.txt
---------------
4 peter pm ..
..
... (12 Replies)
Discussion started by: Gopal_Engg
12 Replies
6. Shell Programming and Scripting
Hello,
I am strugling from quite a some time to compare flat files with over 1 million records could anyone please help me.
I want to compare two pipe delimited flat files, file1 with file2 and output the unmatched rows from file2 in file3
Sample File1:
... (9 Replies)
Discussion started by: suhaeb
9 Replies
7. UNIX for Dummies Questions & Answers
Hi,
I'm a newbie to scripting and am trying to compare two files using awk.
The files are exactly the same dimensions. Where the first file has 0's I would like to create an updated version of the second file which has the corresponding elements set to zero also.
eg:
file1:
12345 1 2 0... (3 Replies)
Discussion started by: kasan0
3 Replies
8. Shell Programming and Scripting
compare to flat files using awk .but in 4th field contains non ordered substring. how to do that.
file1.txt
john|0.0|4|**:25;JP:50;UY:25
file2.txt
andy|0.0|4|JP:50;**:25;UY:25 (4 Replies)
Discussion started by: veeruasu
4 Replies
9. Shell Programming and Scripting
Hi,
I have got 2 files which i need to compare and append based on the below conditions.
file 1:
File 1 has data in the following format
4300 2356
C234 5689
5500 2345
File 2 has data in the same fomat
4300 49
5555 12345
Now i need to compare the first 4 bytes in each line... (4 Replies)
Discussion started by: roy121
4 Replies
10. Shell Programming and Scripting
Gents,
Can you help please
I have a data base with lot information (file2) and I have some data in (file1) to compare.
Then the comparison should be done using the following keys:
Example ( values from file1 )
key1 = columns from 20-34 substr($0,20,15)
66705.00 19793
key2 = columns... (5 Replies)
Discussion started by: jiam912
5 Replies
LEARN ABOUT DEBIAN
file::dircompare
DirCompare(3pm) User Contributed Perl Documentation DirCompare(3pm)
NAME
File::DirCompare - Perl module to compare two directories using callbacks.
SYNOPSIS
use File::DirCompare;
# Simple diff -r --brief replacement
use File::Basename;
File::DirCompare->compare($dir1, $dir2, sub {
my ($a, $b) = @_;
if (! $b) {
printf "Only in %s: %s
", dirname($a), basename($a);
} elsif (! $a) {
printf "Only in %s: %s
", dirname($b), basename($b);
} else {
print "Files $a and $b differ
";
}
});
# Version-control like Deleted/Added/Modified listing
my (@listing, @modified); # use closure to collect results
File::DirCompare->compare('old_tree', 'new_tree', sub {
my ($a, $b) = @_;
if (! $b) {
push @listing, "D $a";
} elsif (! $a) {
push @listing, "A $b";
} else {
if (-f $a && -f $b) {
push @listing, "M $b";
push @modified, $b;
} else {
# One file, one directory - treat as delete + add
push @listing, "D $a";
push @listing, "A $b";
}
}
});
DESCRIPTION
File::DirCompare is a perl module to compare two directories using a callback, invoked for all files that are 'different' between the two
directories, and for any files that exist only in one or other directory ('unique' files).
File::DirCompare has a single public compare() method, with the following signature:
File::DirCompare->compare($dir1, $dir2, $sub, $opts);
The first three arguments are required - $dir1 and $dir2 are paths to the two directories to be compared, and $sub is the subroutine
reference called for all unique or different files. $opts is an optional hashref of options - see OPTIONS below.
The provided subroutine is called for all unique files, and for every pair of 'different' files encountered, with the following signature:
$sub->($file1, $file2)
where $file1 and $file2 are the paths to the two files. For 'unique' files i.e. where a file exists in only one directory, the subroutine
is called with the other argument 'undef' i.e. for:
$sub->($file1, undef)
$sub->(undef, $file2)
the first indicates $file1 exists only in the first directory given ($dir1), and the second indicates $file2 exists only in the second
directory given ($dir2).
OPTIONS
The following optional arguments are supported, passed in using a hash reference after the three required arguments to compare() e.g.
File::DirCompare->compare($dir1, $dir2, $sub, {
cmp => $cmp_sub,
ignore_unique => 1,
});
cmp By default, two files are regarded as different if their contents do not match (tested with File::Compare::compare). That default
behaviour can be overridden by providing a 'cmp' subroutine to do the file comparison, returning zero if the two files are equal, and
non-zero if not.
E.g. to compare using modification times instead of file contents:
File::DirCompare->compare($dir1, $dir2, $sub, {
cmp => sub { -M $_[0] <=> -M $_[1] },
});
ignore_cmp
If you want to see all corresponding files, not just 'different' ones, set the 'ignore_cmp' flag to tell File::DirCompare to skip its
file comparison checks i.e.
File::DirCompare->compare($dir1, $dir2, $sub,
{ ignore_cmp => 1 });
ignore_unique
If you want to ignore files that only exist in one of the two directories, set the 'ignore_unique' flag i.e.
File::DirCompare->compare($dir1, $dir2, $sub,
{ ignore_unique => 1 });
SEE ALSO
File::Dircmp, which provides similar functionality (and whose directory walking code I've adapted for this module), but a simpler
reporting-only interface, something like the first example in the SYNOPSIS above.
AUTHOR AND CREDITS
Gavin Carr <gavin@openfusion.com.au>
Thanks to Robin Barker for a bug report and fix for glob problems with whitespace.
COPYRIGHT AND LICENSE
Copyright 2006-2007 by Gavin Carr.
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
perl v5.10.1 2010-03-02 DirCompare(3pm)