Sponsored Content
Top Forums Shell Programming and Scripting Compare columns of 2 files based on condition defined in a different file Post 302469547 by newtoawk on Saturday 6th of November 2010 11:33:55 PM
Old 11-07-2010
Compare columns of 2 files based on condition defined in a different file

I have a control file which tells me which are the fields in the files I need to compare and based on the values I need to print the exact value if key =Y and output is Y , or if output is Y/N then I need to print only Y if it matches or N if it does not match and if output =N , then skip the feild to compare and write it to a output file
For ex:
my control file
Code:
key|compare_field|output
Y|Field_1|Y
N|Filed_2|Y/N
Y|Field_3|Y
N|Field_4|Y/N
N|Field_5|N
N|Field_6|Y/N
file1
field_1|feld_2|field_3|field_4|field_5|field_6
000|adbc|edfr|hjkl|890|jlk|ioy
678|jfjd|djla|uopp|678|jyh|jkl
file2
field_1|feld_2|field_3|field_4|field_5|field_6
000|adbc|edfr|hjkl|890|jlk|ioy
678|juio|djla|uopu|678|jyh|jkl
my output should be
field_1|feld_2|field_3|field_4|field_6
000|Y|edfr|Y|Y
678|N|djla|N|Y

I was trying to do it in 2 parts and then combine, but I am lost, need your help to combine this logic.
Code:
# to copy the field names as the header in the report file.
nawk -F\| 'END {print x } $NF =="Y" || $NF == "Y\/N" { printf "%s",$2 FS >> "report_file" }' control_file

To compare the 2 files and print the output as Y or N
Code:
nawk -F'|' '{ getline x <f; split(x,F,"|")}
NR >1 {for(i=2;i<=NF;i++) $i=(F[i]==$i)?"Y";"N"}1' OFS="|" f=file2 file1

I can do then seperately, but I am not able to read the control file and compare the files based on the control file.

Please help me.
Thanks in Advance
newtoawk

Last edited by Scott; 11-07-2010 at 05:57 AM.. Reason: Please use code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

compare two columns of different files and print the matching second file..

Hi, I have two tab separated files; file1: S.No ddi fi cu o/l t+ t- 1 0.5 0.6 o 0.1 0.2 2 0.2 0.3 l 0.3 0.4 3 0.5 0.8 l 0.1 0.6 ... (5 Replies)
Discussion started by: vasanth.vadalur
5 Replies

2. Shell Programming and Scripting

How to compare 2 files & get only few columns based on a condition related to both files?

Hiiiii friends I have 2 files which contains huge data & few lines of it are as shown below File1: b.dat(which has 21 columns) SSR 1976 8 12 13 10 44.00 39.0700 70.7800 7.0 0 0.00 0 2.78 0.00 0.00 0 0.00 2.78 0 NULL ISC 1976 8 12 22 32 37.39 36.2942 70.7338... (6 Replies)
Discussion started by: reva
6 Replies

3. Shell Programming and Scripting

compare 2 files based on columns

Hi Experts, Is there a way to compare 2 files by columns and print matching cases. I have 2 files as below, I want cases where col1 and col2 in f1 matches col1 and col2 in f2 to be printed as output. The separator is space. I want the output to have col1 col2 col 3 from both files printed... (7 Replies)
Discussion started by: novice_man
7 Replies

4. Shell Programming and Scripting

compare 2 files and extract the data which is not present in other file with condition

I have 2 files whose data's are as follows : fileA 00 lieferungen 00 attractiop 01 done 02 forness 03 rasp 04 alwaysisng 04 funny 05 done1 fileB alwayssng dkhf fdgdfg dfgdg sdjkgkdfjg funny rasp (7 Replies)
Discussion started by: rajniman
7 Replies

5. Shell Programming and Scripting

compare 2 files and return unique lines in each file (based on condition)

hi my problem is little complicated one. i have 2 files which appear like this file 1 abbsss:aa:22:34:as akl abc 1234 mkilll:as:ss:23:qs asc abc 0987 mlopii:cd:wq:24:as asd abc 7866 file2 lkoaa:as:24:32:sa alk abc 3245 lkmo:as:34:43:qs qsa abc 0987 kloia:ds:45:56:sa acq abc 7805 i... (5 Replies)
Discussion started by: anurupa777
5 Replies

6. Shell Programming and Scripting

Compare files in a folder based on another file

I have a file named file.txt that looks as follows //class1.txt 45 234 67 89 90 //class2.txt 456 34 78 89 120 class1 and class2.txt are the names of files in a folder named folder1. The content of class1.txt file in folder1 67 9 89 5 234 9The content of class2.txt file in... (1 Reply)
Discussion started by: jaff rufus
1 Replies

7. Shell Programming and Scripting

Extracting rows and columns in a matrix based on condition

Hi I have a matrix with n rows and m columns like below example. i want to extract all the pairs with values <200. Input A B C D A 100 206 51 300 B 206 100 72 48 C 351 22 100 198 D 13 989 150 100 Output format A,A:200 A,C:51 B,B:100... (2 Replies)
Discussion started by: anurupa777
2 Replies

8. Shell Programming and Scripting

Convert rows to columns based on condition

I have a file some thing like this: GN Name=YWHAB; RC TISSUE=Keratinocyte; RC TISSUE=Thymus; CC -!- FUNCTION: Adapter protein implicated in the regulation of a large CC spectrum of both general and specialized signaling pathways GN Name=YWHAE; RC TISSUE=Liver; RC ... (13 Replies)
Discussion started by: raj_k
13 Replies

9. Shell Programming and Scripting

[Solved] awk compare two different columns of two files and print all from both file

Hi, I want to compare two columns from file1 with another two column of file2 and print matched and unmatched column like this File1 1 rs1 abc 3 rs4 xyz 1 rs3 stu File2 1 kkk rs1 AA 10 1 aaa rs2 DD 20 1 ccc ... (2 Replies)
Discussion started by: justinjj
2 Replies

10. Shell Programming and Scripting

I have this list of files . Now I will have to pick the latest file based on some condition

3679 Jul 21 23:59 belk_rpo_error_**po9324892**_07212014.log 0 Jul 22 23:59 belk_rpo_error_**po9324892**_07222014.log 3679 Jul 23 23:59 belk_rpo_error_**po9324892**_07232014.log 22 Jul 22 06:30 belk_rpo_error_**po9324267**_07012014.log 0 Jul 20 05:50... (5 Replies)
Discussion started by: LoneRanger
5 Replies
Array::Compare(3pm)					User Contributed Perl Documentation				       Array::Compare(3pm)

NAME
Array::Compare - Perl extension for comparing arrays. SYNOPSIS
use Array::Compare; my $comp1 = Array::Compare->new; $comp->Sep('|'); $comp->Skip({3 => 1, 4 => 1}); $comp->WhiteSpace(0); $comp->Case(1); my $comp2 = Array::Compare->new(Sep => '|', WhiteSpace => 0, Case => 1, Skip => {3 => 1, 4 => 1}); my @arr1 = 0 .. 10; my @arr2 = 0 .. 10; $comp1->compare(@arr1, @arr2); $comp2->compare(@arr1, @arr2); DESCRIPTION
If you have two arrays and you want to know if they are the same or different, then Array::Compare will be useful to you. All comparisons are carried out via a comparator object. In the simplest usage, you can create and use a comparator object like this: my @arr1 = 0 .. 10; my @arr2 = 0 .. 10; my $comp = Array::Compare->new; if ($comp->compare(@arr1, @arr2)) { print "Arrays are the same "; } else { print "Arrays are different "; } Notice that you pass references to the two arrays to the comparison method. Internally the comparator compares the two arrays by using "join" to turn both arrays into strings and comparing the strings using "eq". In the joined strings, the elements of the original arrays are separated with the "^G" character. This can cause problems if your array data contains "^G" characters as it is possible that two different arrays can be converted to the same string. To avoid this, it is possible to override the default separator character, either by passing and alternative to the "new" function my $comp = Array::Compare->new(Sep => '|'); or by changing the seperator for an existing comparator object $comp->Sep('|'); In general you should choose a separator character that won't appear in your data. You can also control whether or not whitespace within the elements of the arrays should be considered significant when making the comparison. The default is that all whitespace is significant. The alternative is for all consecutive white space characters to be converted to a single space for the pruposes of the comparison. Again, this can be turned on when creating a comparator object: my $comp = Array::Compare->new(WhiteSpace => 0); or by altering an existing object: $comp->WhiteSpace(0); You can also control whether or not the case of the data is significant in the comparison. The default is that the case of data is taken into account. This can be changed in the standard ways when creating a new comparator object: my $comp = Array::Compare->new(Case => 0); or by altering an existing object: $comp->Case(0); In addition to the simple comparison described above (which returns true if the arrays are the same and false if they're different) there is also a full comparison which returns a list containing the indexes of elements which differ between the two arrays. If the arrays are the same it returns an empty list. In scalar context the full comparison returns the length of this list (i.e. the number of elements that differ). You can access the full comparision in two ways. Firstly, there is a "DefFull" attribute. If this is "true" then a full comparison if carried out whenever the "compare" method is called. my $comp = Array::Compare->new(DefFull => 1); $comp->compare(@arr1, @arr2); # Full comparison $comp->DefFull(0); $comp->compare(@arr1, @arr2); # Simple comparison $comp->DefFull(1); $comp->compare(@arr1, @arr2); # Full comparison again Secondly, you can access the full comparison method directly $comp->full_compare(@arr1, @arr2); For symmetry, there is also a direct method to use to call the simple comparison. $comp->simple_compare(@arr1, @arr2); The final complication is the ability to skip elements in the comparison. If you know that two arrays will always differ in a particular element but want to compare the arrays ignoring this element, you can do it with Array::Compare without taking array slices. To do this, a comparator object has an optional attribute called "Skip" which is a reference to a hash. The keys in this hash are the indexes of the array elements and the values should be any true value for elements that should be skipped. For example, if you want to compare two arrays, ignoring the values in elements two and four, you can do something like this: my %skip = (2 => 1, 4 => 1); my @a = (0, 1, 2, 3, 4, 5); my @b = (0, 1, X, 3, X, 5); my $comp = Array::Compare->new(Skip => \%skip); $comp->compare(@a, @b); This should return true, as we are explicitly ignoring the columns which differ. Of course, having created a comparator object with no skip hash, it is possible to add one later: $comp->Skip({1 => 1, 2 => 1}); or: my %skip = (1 => 1, 2 => 2); $comp->Skip(\%skip); To reset the comparator so that no longer skips elements, set the skip hash to an empty hash. $comp->Skip({}); You can also check to see if one array is a permutation of another, i.e. they contain the same elements but in a different order. if ($comp->perm(@a, @b) { print "Arrays are perms "; else { print "Nope. Arrays are completely different "; } In this case the values of "WhiteSpace" and "Case" are still used, but "Skip" is ignored for, hopefully, obvious reasons. METHODS
new [ %OPTIONS ] Constructs a new comparison object. Takes an optional hash containing various options that control how comparisons are carried out. Any omitted options take useful defaults. Sep This is the value that is used to separate fields when the array is joined into a string. It should be a value which doesn't appear in your data. Default is '^G'. WhiteSpace Flag that indicates whether or not whitespace is significant in the comparison. If this value is false then all multiple whitespace characters are changed into a single space before the comparison takes place. Default is 1 (whitespace is significant). Case Flag that indicates whther or not the case of the data should be significant in the comparison. Default is 1 (case is significant). Skip a reference to a hash which contains the numbers of any columns that should be skipped in the comparison. Default is an empty hash (all columns are significant). DefFull Flag which indicates whether the default comparison is simple (just returns true if the arrays are the same or false if they're not) or full (returns an array containing the indexes of the columns that differ). Default is 0 (simple comparison). compare_len @ARR1, @ARR2 Very simple comparison. Just checks the lengths of the arrays are the same. compare @ARR1, @ARR2 Compare the values in two arrays and return a data indicating whether the arrays are the same. The exact return values differ depending on the comparison method used. See the descriptions of simple_compare and full_compare for details. Uses the value of DefFull to determine which comparison routine to use. simple_compare @ARR1, @ARR2 Compare the values in two arrays and return a flag indicating whether or not the arrays are the same. Returns true if the arrays are the same or false if they differ. Uses the values of 'Sep', 'WhiteSpace' and 'Skip' to influence the comparison. full_compare @ARR1, @ARR2 Do a full comparison between two arrays. Checks each individual column. In scalar context returns the number of columns that differ (zero if the arrays are the same). In list context returns an list containing the indexes of the columns that differ (an empty list if the arrays are the same). Uses the values of 'Sep' and 'WhiteSpace' to influence the comparison. Note: If the two arrays are of different lengths then this method just returns the indexes of the elements that appear in one array but not the other (i.e. the indexes from the longer array that are beyond the end of the shorter array). This might be a little counter-intuitive. perm @ARR1, @ARR2 Check to see if one array is a permutation of the other (i.e. contains the same set of elements, but in a different order). We do this by sorting the arrays and passing references to the assorted versions to simple_compare. There are also some small changes to simple_compare as it should ignore the Skip hash if we are called from perm. AUTHOR
Dave Cross <dave@mag-sol.com> SEE ALSO
perl(1). COPYRIGHT AND LICENSE
Copyright (C) 2000-2005, Magnum Solutions Ltd. All Rights Reserved. This script is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.14.2 2011-12-18 Array::Compare(3pm)
All times are GMT -4. The time now is 12:48 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy