Really appreciate your time in writing the awk script.
But each of these files have 500K records.
But I have managed to write a perl script for the diff file. i.e.
Code:
#!/usr/local/bin/perl
$self = $0;
$self =~ s!^.*/!!;
#
$[ = 1; # = number of first index into arrays and strings
#
$FIELD_SEPARATOR = '\t';
$FIELD_NUMBER_LIST =('38','82');
$field_separator = $FIELD_SEPARATOR;
$field_number_list = $FIELD_NUMBER_LIST;
#
while (@ARGV)
{
$_ = shift;
if (/^-F$/) { $field_separator = shift; }
elsif (/^-L$/) { $field_number_list = shift; }
elsif (/^-F.+$/) { $field_separator = substr($_,$[+2); }
elsif (/^-L.+$/) { $field_number_list = substr($_,$[+2); }
#else { push(@filename, $_); }
}
#
$file_a = 'file1';
$file_b = 'file2';
#
unless (($file_a ne "") && (-f $file_a))
{
die "Error: Can't find file '$file_a'!\n";
}
unless (($file_b ne "") && (-f $file_b))
{
die "Error: Can't find file '$file_b'!\n";
}
#
@index_list = split(/,/, $field_number_list);
#
# Scan first file, Pass 1:
open(FILE_A, "<$file_a") || die "Can't open '$file_a': $!\n";
#
while (<FILE_A>)
{
chop if /\n$/;
undef $key;
undef @field;
@field = split(/$field_separator/o);
foreach $index (@index_list)
{
if (defined $key)
{
$key .= "\n" . $field[$index];
}
else
{
$key = $field[$index];
}
}
$intersection{$key} = 1;
}
#
close(FILE_A);
# Scan second file, Pass 1:
#
$empty_intersection = 1;
#
open(FILE_B, "<$file_b") || die "Can't open '$file_b': $!\n";
#
while (<FILE_B>)
{
chop if /\n$/;
undef $key;
undef @field;
@field = split(/$field_separator/o);
foreach $index (@index_list)
{
if (defined $key)
{
$key .= "\n" . $field[$index];
}
else
{
$key = $field[$index];
}
}
$code = $intersection{$key};
if ($code == 1)
{
$intersection{$key} = 3;
$empty_intersection = 0;
}
else
{
if ($code != 3) { $intersection{$key} = 2; }
}
}
#
close(FILE_B);
#
# Prepare output file names:
$file_a_1 = $file_a . '.1';
#
# Scan first file, Pass 2:
#
open(FILE_A, "<$file_a") || die "Can't open '$file_a': $!\n";
open(FILE_A_1, ">$file_a_1") || die "Can't write '$file_a_1': $!\n";
#
while (<FILE_A>)
{
chop if /\n$/;
undef $key;
undef @field;
@field = split(/$field_separator/o);
foreach $index (@index_list)
{
if (defined $key)
{
$key .= "\n" . $field[$index];
}
else
{
$key = $field[$index];
}
}
if ($intersection{$key} == 3)
{
#
}
else
{
print FILE_A_1 $_, "\n";
}
}
#
close(FILE_A);
close(FILE_A_1);
#
# Scan second file, Pass 2:
#
open(FILE_B, "<$file_b") || die "Can't open '$file_b': $!\n";
open(FILE_A_1, ">>$file_a_1") || die "Can't write '$file_a_1': $!\n";
#
while (<FILE_B>)
{
chop if /\n$/;
undef $key;
undef @field;
@field = split(/$field_separator/o);
foreach $index (@index_list)
{
if (defined $key)
{
$key .= "\n" . $field[$index];
}
else
{
$key = $field[$index];
}
}
if ($intersection{$key} == 3)
{
# }
else
{
print FILE_A_1 $_, "\n";
}
}
#
close(FILE_B);
close(FILE_A_1);
#
# Display results:
#
printf("The Diff file created '%s'\n\n", $file_a_1);
#
The above code works perfectly for generating the diff file i.e. depending upon the primary keys (here taken 2) the outfile contains the records that exists in file1 but not in file2 and the records that exists in file2 but not in file1.
Now,
I need to compare the whole record(line) if the primary keys in file1 matches with the primary keys in file2. If both the lines are equal then discard else write to the outfile.
Could someone please help me out in order to the above step.
File2.diff --> Records that exists in File2 but not in File1
same1.txt --> Records that exists in both the files'
Here,
The file size of the same.txt and same1.txt is different.
--> 506108009 Jun 15 01:50 same.txt
--> 505878904 Jun 15 01:52 same1.txt
So, is there anyway that we can capture the records if the Primary Keys ($38 and $82) of both the files matches but the data in the other columns is not the same.
To solve the above issue, I thought once we have the files same.txt and same1.txt we should compare the number of characters in the a line in File1 with number of characters in a line in file2 , if there is a change then write to a file else discard (do nothing or leave it)
Could you please share any ideas to solve the above issue. It would be really grateful.
Hello all,
I have 2 text files.
For example:
File1.txt contains data
A
B
C
D
****NEXT****
X
Y
Z
****NEXT****
L
M
N
and File2.txt contains data (13 Replies)
I need to take 2 input files and create 1 output based on matches from each file. I am looking to match field #1 in both files (Userid) and create an output file that will be a combination of fields from
both file1 and file2 if there are any differences in the fields 2,3,4,5,or 6.
Below is an... (5 Replies)
Hi
I am writing a perl script which checks for the specific column values from a file and writes to the OUT file.
So the feed file has a header information and footer information.
I header information isaround107 lines i.e.
Starts with
START-OF-FILE
.......
so on ....
... (11 Replies)
cat file1.txt
field1 "user1":
field2:"data-cde"
field3:"data-pqr"
field4:"data-mno"
field1 "user1":
field2:"data-dcb"
field3:"data-mxz"
field4:"data-zul"
field1 "user2":
field2:"data-cqz"
field3:"data-xoq"
field4:"data-pos"
Now i need to have the date like below.
i have just... (7 Replies)
Hi Guys,
I have two input files and I want to combine them and get the unique values and differences and put them into one file. See below desired output file.
Inputfile1:
1111111
2222222
3333333
7860068
7860069
7860071
7860072
Inputfile2:
4444444 (4 Replies)
Hi,
I have several text files. One main file contains the detail data, other have some information to extract data from the main file, and some are empty files. Examples are shown below:
The main file look like:
MainFile.txt
>Header1
data1...data1...
>Header2
data2...data2...
...
...... (2 Replies)
Hi,
I have two text files. Need to create a third text file extracting specific data from first two existing files..
Text File 1: Format contains:
SQL*Loader: Release 10.2.0.1.0 - Production on Wed Aug 4 21:06:34 2010
some text ............so on...and somwhere text like:
Record 1:... (1 Reply)
ehe may i know what are the difference between Unix & Linux, and what are the advantages of having Unix as well as disadvantages of having Unix or if u dun mind i am dumb do pls tell me what are the advantages as well as the disadvantages of having linux as well. thanks (1 Reply)
06-14-2011
