Perl 2 files compares


 
Thread Tools Search this Thread
Top Forums Programming Perl 2 files compares
# 1  
Old 07-12-2012
Perl 2 files compares

What do i need to do have the below perl program load 205 million record files into the hash. It currently works on smaller files, but not working on huge files. Any idea what i need to do to modify to make it work with huge files:


Code:
#!/usr/bin/perl
$ot1=$ARGV[2];
$ot2=$ARGV[3];
open(mfileot1, ">$ot1");
open(mfileot2, ">$ot2");
use strict;
#----------------
# Hash Definition
#----------------
my %HashArray;
my @file1Line;
my @file2Line;
#--------------------
# Subroutine
#--------------------
sub comp_file{
   my ($FILE1, $FILE2) = @_;
   open (R, $FILE1) or die ("Can't open file $FILE1");
     foreach my $FP1(<R>){
        chomp($FP1);
        my ($k, $l) = split(/\s+/,$FP1);
        push @{$HashArray{'$FP1'}{$k}},$l;
     }
   close (R);
   open (P, $FILE2) or die ("Can't open file $FILE2");
     foreach my $FP2(<P>){
        chomp($FP2);
        my ($k, $l) = split(/\s+/,$FP2);
        push @{$HashArray{'$FP2'}{$k}},$l;
     }
   close (P);
   foreach my $key(keys %{$HashArray{'$FP1'}}){
     if (!exists $HashArray{'$FP2'}{$key}){
         foreach my $last(@{$HashArray{'$FP1'}{$key}}){
           push (@file1Line,"$key$last");
         }
     }
   }
   print mfileot1 "$_\n" for (sort @file1Line);
   close(mfileot1);
   foreach my $key(keys %{$HashArray{'$FP2'}}){
     if (!exists $HashArray{'$FP1'}{$key}){
         foreach my $last(@{$HashArray{'$FP2'}{$key}}){
           push (@file2Line,"$key$last");
         }
     }
   }
   print mfileot2 "$_\n" for (sort @file2Line);
   close(mfileot2);
}
############MAIN MENU####################################
# Pre-check Condition
# if the input doesn't contain two(2) files, return help
# USAGE: hash2files.pl FILE1 FILE2 FILE3 FILE4
#########################################################

if ($#ARGV != 3){
    print "USAGE: $0 <FILE1> <FILE2> <FILE3> <FILE4>\n";
    exit;
}
else {
    my ($FILE1, $FILE2, $OT1, $OT2)= @ARGV;
    &comp_file($FILE1, $FILE2);
}

Moderator's Comments:
Mod Comment Code tags for code, please.

Last edited by Corona688; 07-18-2012 at 01:58 PM..
# 2  
Old 07-18-2012
Try the Unix utility "diff" for comparing files. Or use a module from CPAN if Perl must be used.

tyler_durden
# 3  
Old 07-18-2012
Bumping up posts or double posting is not permitted in these forums.

Please read the rules, which you agreed to when you registered, if you have not already done so.

You may receive an infraction for this. If so, don't worry, just try to follow the rules more carefully. The infraction will expire in the near future

Thank You.

The UNIX and Linux Forums.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compares not working past the 4th char

There probably is a simple explanation, and maybe I haven’t searched enough for it, or I've looked at this so long I can't see my own mistake, but here’s my issue. My code is such #!/bin/ksh awk ' {if ($1 == $2) {print $1,$2} } ' file1.txt File1.txt is this 000EEB2A 000EEB2A... (8 Replies)
Discussion started by: jgreene42
8 Replies

2. Shell Programming and Scripting

Perl : Delete all files except few

I have a directory like below. Need help with Perl to delete all files under test1 except the one passed as parameters.The parameters will always be the directories under test1 in the format below. EX: dir1/abc.txt,dir2/test.xml,dir3/dfb.txt,dir4/text.xml test1 ... (5 Replies)
Discussion started by: gaurav99
5 Replies

3. Shell Programming and Scripting

Diff of 2 files using perl

Assitance needed in checking file difference using a perl script Below is the example I wanted a report saying file2 has an update of new value. File1 old date Alpha 156 Beta 255 Gama 6987 segma 4578 File2 new date Alpha 156 Beta 255 Gama 1000 (3 Replies)
Discussion started by: ajayram_arya
3 Replies

4. Shell Programming and Scripting

How to ftp the files in perl.?

i want to upload two different files in to two different directories(path) on ftp server.but i am able to ftping only one at a time :(.. not both simultaneously. here i have connectd the ftp and use the binary mode to transfer the files $ftp = Net::FTP->new($ftphost, Debug => 0) or... (2 Replies)
Discussion started by: aarts
2 Replies

5. Shell Programming and Scripting

using perl config files

Hi, I have 2 perl SubRoutines (sub 1 and sub 2). I created two perl modules for these (Sub1.pm and Sub2.pm). How can I include these in a config file (say Config.cfg) and call the config file in my main script Rename.pl to use the 2 subroutines? Right now here are the content of Config.cfg... (1 Reply)
Discussion started by: CCFP
1 Replies

6. Shell Programming and Scripting

perl script for listing files and mailing the all files

Hi, I am new to perl: I need to write perl script to list all the files present in directory and mail should be come to my inbox with all the files present in that directory. advanced thanks for valuable inputs. Thanks Prakash GR (1 Reply)
Discussion started by: prakash.gr
1 Replies

7. Shell Programming and Scripting

perl with two files and print

Suppose u have two files one file >hi|23433|sp|he is RAJ<space>>hi|23333|df|He is HUMAN<space>>hi|222|gi|howru|just WOWHEISWONDERFUL >hi|25559|gs|heisANUJ<space>>hi|2232|sp|he is fool SKSIKSIKSLKSSLLS Another file HUMAN so output wil be ...if the list contain HUMAN only take it... (1 Reply)
Discussion started by: cdfd123
1 Replies

8. Shell Programming and Scripting

ftp files in perl

Hi, I am using NET::FTP in perl to tranfer files. i have an array of files and then I m getting one bye one using foreach. the problem is I m not gettng the complete list of files. the program terminates after 2 files. when I execute again it will transfer the remaining 2 files. (... (3 Replies)
Discussion started by: shellwell
3 Replies

9. UNIX for Dummies Questions & Answers

Download files using perl

What is the easiest way to download some file using perl for it. (2 Replies)
Discussion started by: mirusnet
2 Replies

10. UNIX for Advanced & Expert Users

Perl Uploading Files

Using perl 5.8.0, Linux 2.4.20-30.9, RedHat 9.0. We have many .cgi's that allow privileged users to upload files to the server through a web browser. We've had these .cgi's for years and have never had any problems with them. Recently the files being uploaded are sometimes being given 600... (16 Replies)
Discussion started by: sstevens
16 Replies
Login or Register to Ask a Question