Sponsored Content
Top Forums Shell Programming and Scripting Need help regarding comparison between two files through UNIX script Post 302948286 by STCET22 on Sunday 28th of June 2015 08:24:28 AM
Old 06-28-2015
Need help regarding comparison between two files through UNIX script

Hi All ,
I am aware of unix command ,but not comforable in putting together in script level.I came to situation where I need to compare between two .txt files fieldwise and need a mismatch report. As I am new to unix script arena ,if anyone can help in the below scenario that will be really helpful.

We have one source mainframe .txt file (readble pipe dililimited format) and also have one target hdfs .txt file (pipe dilimited format).I need to compare two files field by field and not by whole line.Need to compare like
f1.first field with f2.first field
f1.second field with f2.second field
and so on .Please find below sample source & target file.

f1 :
Code:
000001|     1|AQWWW|234,456.00  |     | 123456|     |41|abC| 0|xyZ|     |99900|0.00999|      |1|c|6|S|
000002|     2|11  4|1,234,456.99|     |      0|     |23|   |99|!  |     |00000|0.00000|      |2| |4|#|
000003|     3|!!@#$|0,000,001.10|     |      9|     | 0|XSW|12|  7|     |00100|0.00001|      |0|S|0| |
000004|     4|     |            |     |   3400|     | 2|r7!|72|xY1|     |01200|0.00045|      |9|W|3|2|

f2 :
Code:
000001|000001|AQWWW|234,456.00  |     |123456|     |41|abC|00|xyZ|     |99900|0.00999|      |1|c|6|S
000002|000002|11  4|1,234,456.99|     |0|     |23|   |99|!  |     |0|0|      |2| |4|#
000003|000003|!!@#$|0,000,001.10|     |9|     |00|XSW|12|  7|     |100|0.00001|      |0|S|0| 
000004|000004|     |            |     |3400|     |02|r7!|72|xY1|     |1200|0.00045|      |9|W|3|2

Once the comparison between two files are complete fieldwise ,we need a mismatch report which will contain source/target count validation ,field level src & target mismatches and their corresponding mismatch details .

It will be helpful if mismatch report like below :
Code:
<<Mismatch Report >>

Src & tgt Count Validation
Source Count :4
Target Count :4

Field lebel Src & Tgt Mismatches
No of Mismatches :0

Mismatch Details :
Key Column Value	Column Name / Index	Source Data	Target Data

Source data might contain leading spaces/zero ,precision and target data might not have those.We can ignore these cases in the mismatch report.
If anyone can help me in the above scenario that will really beneficial for me.Thanks !
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Unix comparison

I am very new to Unix. What are the similiarities and differences between ScoUnix and AIX5 if any? Where might i find the information? Which is better? (1 Reply)
Discussion started by: NewGuy100
1 Replies

2. UNIX for Dummies Questions & Answers

Comparison of 2 files in UNIX

Hi, There are two files in UNIX system with some lines are exactly the same, some lines are not. I want to compare these two files.The 2 files (both the files have data in Column format )should be compared row wise and any difference in data for a particular row should lead to storage of data of... (32 Replies)
Discussion started by: Dana Evans
32 Replies

3. Shell Programming and Scripting

comparison of 2 files

Kindly help on follows. I have 2 files. One file contains only one column of mobile numbers. And total records in a file 12 million. Second file contains 2 columns mobile numbers and balance. and total records 30 million. I want to find out balance of each data in file 1 corresponding to file 2.... (2 Replies)
Discussion started by: kamal_418
2 Replies

4. Shell Programming and Scripting

Comparison of two files (sh)

Hi, I have a problem with comparison of two files file1 20100101 20090101 20080101 20071001 20121229 file2 19990112 12 456 7 20011131 19 20100101 2 567 1 987 17890709 123 555 and, sh script needs to compare of these two files and give out to me result: 20100101 2 567 1 987 it... (5 Replies)
Discussion started by: shizik
5 Replies

5. Shell Programming and Scripting

comparison of 2 files using unix or awk

Hello, I have 2 files and I want them to be compared in a specific fashion file1: A_1200_1250 A_1251_1300 B_1301_1350 B_1351_1400 B_1401_1450 C_1451_1500 and so on... file2: 1210 1305 1260 1295 1400 1500 1450 1495 Now The script should look for "1200" from A_1200_1250 of... (8 Replies)
Discussion started by: Diya123
8 Replies

6. Solaris

Unix script Unix script which counts no. of files/sub-files

Hi All, For past some days iam trying, which not able to get to..so please help me on this.. My exact requirement is... Step1: Find how many files/sub files exist in /some/path (maybe in multiple path) Step2: Count the no. of files/sub files with their respective size. Step3: Then a file... (1 Reply)
Discussion started by: sam09
1 Replies

7. Homework & Coursework Questions

Unix script Unix script which counts no. of files/sub-files

Hi All, For past some days iam trying, which not able to get to..so please help me on this.. My exact requirement is... Step1: Find how many files/sub files exist in /some/path (maybe in multiple path) Step2: Count the no. of files/sub files with their respective size. Step3: Then a file... (0 Replies)
Discussion started by: sam09
0 Replies

8. Shell Programming and Scripting

Comparison between two files through UNIX script

Hi All , As I am new to unix scripting ,I need a help regarding unix scripting .I have two .txt files .One is source file and another is target file.I need a script through which I can compare those two files.I need a automated comparison report in a directory after comparing between source &... (2 Replies)
Discussion started by: STCET22
2 Replies

9. Shell Programming and Scripting

UNIX file comparison

I have two files which has component name and version number separated by a space cat file1 com.acc.invm:FNS_PROD 94.0.5 com.acc.invm:FNS_TEST_DCCC_Mangment 94.1.6 com.acc.invm:FNS_APIPlat_BDMap 100.0.9 com.acc.invm:SendEmail 29.6.113 com.acc.invm:SendSms 12.23.65 cat file2 ... (8 Replies)
Discussion started by: rakeshtomar82
8 Replies

10. Shell Programming and Scripting

Comparison of files

I have the requirement I have two files cat fileA something anythg nothing everythg cat fileB everythg anythg Now i shld use fileB and compare every line at fileA and get the output as something nothing (3 Replies)
Discussion started by: Priya Amaresh
3 Replies
Data::Compare(3)					User Contributed Perl Documentation					  Data::Compare(3)

NAME
Data::Compare - compare perl data structures SYNOPSIS
use Data::Compare; my $h1 = { 'foo' => [ 'bar', 'baz' ], 'FOO' => [ 'one', 'two' ] }; my $h2 = { 'foo' => [ 'bar', 'barf' ], 'FOO' => [ 'one', 'two' ] }; my @a1 = ('one', 'two'); my @a2 = ('bar', 'baz'); my %v = ( 'FOO', @a1, 'foo', @a2 ); # simple procedural interface print 'structures of $h1 and \%v are ', Compare($h1, \%v) ? "" : "not ", "identical. "; print 'structures of $h1 and $h2 are ', Compare($h1, $h2, { ignore_hash_keys => [qw(foo)] }) ? '' : 'not ', "close enough to identical. "; # OO usage my $c = new Data::Compare($h1, \%v); print 'structures of $h1 and \%v are ', $c->Cmp ? "" : "not ", "identical. "; # or my $c = new Data::Compare; print 'structures of $h and \%v are ', $c->Cmp($h1, \%v) ? "" : "not ", "identical. "; DESCRIPTION
Compare two perl data structures recursively. Returns 0 if the structures differ, else returns 1. A few data types are treated as special cases: Scalar::Properties objects This has been moved into a plugin, although functionality remains the same as with the previous version. Full documentation is in Data::Compare::Plugins::Scalar::Properties. Compiled regular expressions, eg qr/foo/ These are stringified before comparison, so the following will match: $r = qr/abc/i; $s = qr/abc/i; Compare($r, $s); and the following won't, despite them matching *exactly* the same text: $r = qr/abc/i; $s = qr/[aA][bB][cC]/; Compare($r, $s); Sorry, that's the best we can do. CODE and GLOB references These are assumed not to match unless the references are identical - ie, both are references to the same thing. You may also customise how we compare structures by supplying options in a hashref as a third parameter to the "Compare()" function. This is not yet available through the OO-ish interface. These options will be in force for the *whole* of your comparison, so will apply to structures that are lurking deep down in your data as well as at the top level, so beware! ignore_hash_keys an arrayref of strings. When comparing two hashes, any keys mentioned in this list will be ignored. CIRCULAR STRUCTURES
Comparing a circular structure to itself returns true: $x = $y; $y = $x; Compare([$x, $y], [$x, $y]); And on a sort-of-related note, if you try to compare insanely deeply nested structures, the module will spit a warning. For this to affect you, you need to go around a hundred levels deep though, and if you do that you have bigger problems which I can't help you with ;-) PLUGINS
The module takes plug-ins so you can provide specialised routines for comparing your own objects and data-types. For details see Data::Compare::Plugins. Plugins are *not* available when running in "taint" mode. You may also make it not load plugins by providing an empty list as the argument to import() - ie, by doing this: use Data::Compare (); A couple of functions are provided to examine what goodies have been made available through plugins: plugins Returns a structure (a hash ref) describing all the comparisons made available through plugins. This function is *not* exported, so should be called as Data::Compare::plugins(). It takes no parameters. plugins_printable Returns formatted text EXPORTS
For historical reasons, the Compare() function is exported. If you don't want this, then pass an empty list to import() as explained under PLUGINS. If you want no export but do want plugins, then pass the empty list, and then call the register_plugins class method: use Data::Compare (); Data::Compare->register_plugins; or you could call it as a function if that floats your boat. SOURCE CODE REPOSITORY
<git://github.com/DrHyde/perl-modules-Data-Compare.git> BUGS
Plugin support is not quite finished (see the TODO file for details) but is usable. The missing bits are bells and whistles rather than core functionality. Please report any other bugs either by email to David Cantrell (see below for address) or using rt.cpan.org: <https://rt.cpan.org/Ticket/Create.html?Queue=Data-Compare> AUTHOR
Fabien Tassin <fta@sofaraway.org> Portions by David Cantrell <david@cantrell.org.uk> COPYRIGHT and LICENCE Copyright (c) 1999-2001 Fabien Tassin. All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. Some parts copyright 2003 - 2013 David Cantrell. Seeing that Fabien seems to have disappeared, David Cantrell has become a co-maintainer so he can apply needed patches. The licence, of course, remains the same. As the "perl licence" is "Artistic or GPL, your choice", you can find them as the files ARTISTIC.txt and GPL2.txt in the distribution. SEE ALSO
perl(1), perlref(1) perl v5.18.2 2013-09-26 Data::Compare(3)
All times are GMT -4. The time now is 06:41 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy