Comparison of fields in Files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Comparison of fields in Files
# 1  
Old 07-18-2011
Comparison of fields in Files

Hello,
I have two files with tab delimited data. The file will contain details something like below:

FILENAME.A.B.C. 3 5 VALID PROCESSED

I would have a bench mark file. I would be getting new files of the same format. My requirement is to compare a particular field for a particular filename and give out whether it matches to the benchn mark or not.

For Instance, I may need to check the validity count for a filename xyz. First I need to search for the filename xyz in the bench mark file and find the valid count, then search the filename xyz in the input file and see with the valid count matches.

I hope I am clear on what is required. Please let me know the best way this can be achieved using shell scripting. Is there any UNIX commands to handle this kind of comparsion??

Thanks a lot in advance.
# 2  
Old 07-18-2011
for this kind of text manipulation, i would suggest you to use perl scripting - instead of shell script or unix commands.
# 3  
Old 07-19-2011
It would be great if someone could help me with that on how to achieve it with Perl
# 4  
Old 07-19-2011
Bug Comparison of fields.

Quote:
Originally Posted by Praveenkulkarni
It would be great if someone could help me with that on how to achieve it with Perl
Consider Two Files that is 1.csv and 2.csv
Perl Script:

Code:
#! /usr/bin/perl
use strict;
my $rc = 0;
my %File1hash = ();
my %File2hash = ();
my @UnMatchFieldsDet = ();
my $mLineno = 0;
my $mUnMatchFieldCount = 0;
my $mMatchFieldCount = 0;
my $mFieldCount = 0;
open(FILE1,"<1.csv") or $rc = 1;
open(FILE2,"<2.csv") or $rc = 2;
while ( <FILE1> )
{
   $mLineno++;
   my $mFieldno = 0;
   my @LineInfo = split ( "\t",$_ );
   foreach my $mFieldVal ( @LineInfo )
   {
      $mFieldno++;
      chomp ( $mFieldVal );
      $File1hash{$mLineno}{$mFieldno} = $mFieldVal;
   }
}
$mLineno = 0;
while ( <FILE2> )
{
   $mLineno++;
   my $mFieldno = 0;
   my @LineInfo = split ( "\t",$_ );
   foreach my $mFieldVal ( @LineInfo )
   {
      $mFieldno++;
      chomp ( $mFieldVal );
      $File2hash{$mLineno}{$mFieldno} = $mFieldVal;
   }
}
foreach my $mLineNoKey ( sort keys %File1hash )
{
   foreach my $mFieldNoKey ( sort keys %{$File1hash{$mLineNoKey}} )
   {
      $mFieldCount++;
      if ( $File1hash{$mLineNoKey}{$mFieldNoKey} eq $File2hash{$mLineNoKey}{$mFieldNoKey} )
      {
         $mMatchFieldCount++;
      }
      else
      {
         $mUnMatchFieldCount++;
         push( @UnMatchFieldsDet, "Line:$mLineNoKey Field:$mFieldNoKey" );
      }
   }
}
print "Total Field Count: $mFieldCount\n";
print "Match Field Count : $mMatchFieldCount\n";
print "UnMatch Field Count : $mUnMatchFieldCount\n";
print "\nUnMatch Details\n" if ( $mUnMatchFieldCount >= 1 );
foreach my $msg ( @UnMatchFieldsDet )
{
   print "$msg\n";
}


You can also use Array. For you information hash is more faster that array.Smilie

Regards,
RangaSmilie

Last edited by Franklin52; 07-19-2011 at 08:20 AM.. Reason: Please use code tags for code and data samples, thank you
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Is there a UNIX command that can compare fields of files with differing number of fields?

Hi, Below are the sample files. x.txt is from an Excel file that is a list of users from Windows and y.txt is a list of database account. $ head -500 x.txt y.txt ==> x.txt <== TEST01 APP_USER_PROFILE USER03 APP_USER_PROFILE TEST02 APP_USER_EXP_PROFILE TEST04 APP_USER_PROFILE USER01 ... (3 Replies)
Discussion started by: newbie_01
3 Replies

2. Shell Programming and Scripting

Comparison of files

I have the requirement I have two files cat fileA something anythg nothing everythg cat fileB everythg anythg Now i shld use fileB and compare every line at fileA and get the output as something nothing (3 Replies)
Discussion started by: Priya Amaresh
3 Replies

3. Shell Programming and Scripting

XML Fields comparison using awk script

Hello All, I have many zipped XMLs (example file name in tgz formate - file_rec.trx.2016-01-23.000123.exc.85sesdzd45wsds5299c8f2994f7.tgz) looks following and I need to verify two numbers, they are RecordNumber and EnrolData (only sequence number, NOT hole). for all the records, both should be... (5 Replies)
Discussion started by: VasuKukkapalli
5 Replies

4. Shell Programming and Scripting

Comparison of fields then increment a counter reading line by line in a file

Hi, i have a scenario were i should compare a few fields from each line then increment a variable based on that. Example file 989878|8999|Y|0|Y|N|V 989878|8999|Y|0|N|N|V 989878|8999|Y|2344|Y|N|V i have 3 conditions to check and increment a variable on every line condition 1 if ( $3... (4 Replies)
Discussion started by: selvankj
4 Replies

5. Shell Programming and Scripting

Comparison of two files

Hi all I have two files which I have to compare that whetehr there is soemthing common or not body, div, table, thead, tbody, tfoot, tr, th, td, p { font-family: "Liberation Sans"; font-size: x-small; } body, div, table, thead, tbody, tfoot,... (2 Replies)
Discussion started by: manigrover
2 Replies

6. Shell Programming and Scripting

Add fields in different files only if some fields between them match

Hi everybody (first time posting here) I have a file1 that looks like > 1,101,0.1,0.1 1,26,0.1,0.1 1,3,0.1,0.1 1,97,0.5,0.5 1,98,8.1,0.218919 1,99,6.2,0.248 2,101,0.1,0.1 2,24,3.1,0.147619 2,25,23.5,0.559524 2,26,34,0.723404with 762 lines.. I have another 'similar' file2 > ... (10 Replies)
Discussion started by: murpholinox
10 Replies

7. Shell Programming and Scripting

numbers comparison in fields of a file and print least value of them

Hi , I'm trying to compare fields in the file, I want compare the numbers in each column and get the least value of it. > cat input_file 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 -0.2050 -0.6629 -0.6407 -0.6599 -0.4085 -0.3959 -0.2526 -0.3597 0.3439 0.2275 0.2780 ... (5 Replies)
Discussion started by: novice_man
5 Replies

8. Shell Programming and Scripting

Comparison of two files (sh)

Hi, I have a problem with comparison of two files file1 20100101 20090101 20080101 20071001 20121229 file2 19990112 12 456 7 20011131 19 20100101 2 567 1 987 17890709 123 555 and, sh script needs to compare of these two files and give out to me result: 20100101 2 567 1 987 it... (5 Replies)
Discussion started by: shizik
5 Replies

9. UNIX for Dummies Questions & Answers

Comparison between binary files

Hello, I would like to know if we can compare binary(non text) files in unix eg. exes Thanks Saurabh (3 Replies)
Discussion started by: smehra_20
3 Replies

10. Shell Programming and Scripting

comparison of 2 files

Kindly help on follows. I have 2 files. One file contains only one column of mobile numbers. And total records in a file 12 million. Second file contains 2 columns mobile numbers and balance. and total records 30 million. I want to find out balance of each data in file 1 corresponding to file 2.... (2 Replies)
Discussion started by: kamal_418
2 Replies
Login or Register to Ask a Question