Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Change in Input feed based on condition file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Change in Input feed based on condition file
# 1  
Old 08-20-2010
Change in Input feed based on condition file
Sorry Guys for not being able to explain in one of my earlier post.

I am now putting my requirement with the input file and desired output file.


In the below input file -

Transaction code is at position 31:40.
Business code is from position 318:321

Code:
TSCM00000005837               CM0002N  -0000000001906.072010-12-10XML MM 201002081000000   YORK
 003007XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045
TSCM00000008112               CM0002N  -0000000001906.072010-12-10XML MM 201002081000000   YORK
 007777XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045
TSCM80000005282               CM0019NM +0000000002254.982010-12-10XML MM 201002081000001   YORK
 000440XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045
TSCM00000000215               CM0019NM +0000400002254.982010-12-10XML MM 201002081000001   YORK
 000500XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045
TSCM00000000215               CN0001N  -0000400002254.982010-12-10XML MM 201002081000001   YORK
 000292XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045
TSCM80000005282               CN0001N  -0000400002254.982010-12-10XML MM 201002081000001   YORK
 007843XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045
TSCM00000008107               CN0001N  -0000400002254.982010-12-10XML MM 201002081000001   YORK
 000012XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045
TSCM00000008093               CN0001P  -0000400002254.982010-12-10XML MM 201002081000001   YORK
 000379XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045
TSCM00000002646               CN0001P  -0000400002254.982010-12-10XML MM 201002081000001   YORK
 007847XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045
TSCM00000002646               CN0001P  -0000400002254.982010-12-10XML MM 201002081000001   YORK
 003400XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045


For record 1-10 the combination of transaction code and business code is as following-

Code:
CM0002N  -               3007
CM0002N  -               7777
CM0019NM +            0440
CM0019NM +            0500
CN0001N  -                0292
CN0001N  -                7843
CN0001N  -                0012
CN0001P  -                 0379
CN0001P  -                 7847
CN0001P  -                 3400


Further we have got a condition file which is having 3 values (comma separated file)

The first value is business code, second is transaction code and third is derived transaction code. The file is given below

Code:
3007,CM0002N  -,CMCAMTN  -
3037,CM0002N  -,CMCAMTN  -
3059,CM0002N  -,CMCAMTN  -
3007,CM0002N  +,CMCAMTN  +
3037,CM0002N  +,CMCAMTN  +
3059,CM0002N  +,CMCAMTN  +
3007,CM0002P  -,CMCAMTP  -
3037,CM0002P  -,CMCAMTP  -
3059,CM0002P  -,CMCAMTP  -
3007,CM0002P  +,CMCAMTP  +
3037,CM0002P  +,CMCAMTP  +
3059,CM0002P  +,CMCAMTP  +
0440,CM0019N  -,CMDDPTN  -
0440,CM0019N  +,CMDDPTN  +
0440,CM0019NM -,CMDDPTNM -
0440,CM0019NM +,CMDDPTNM +
0292,CN0001N  -,CNQCSHN  -
0379,CN0001N  -,CNQCSHN  -
1038,CN0001N  -,CNQCSHN  -
7810,CN0001N  -,CNQCSHN  -
7811,CN0001N  -,CNQCSHN  -
7812,CN0001N  -,CNQCSHN  -
7842,CN0001N  -,CNQCSHN  -
7843,CN0001N  -,CNQCSHN  -
0292,CN0001N  +,CNQCSHN  +
0379,CN0001N  +,CNQCSHN  +
1038,CN0001N  +,CNQCSHN  +
7810,CN0001N  +,CNQCSHN  +
7811,CN0001N  +,CNQCSHN  +
7812,CN0001N  +,CNQCSHN  +
7842,CN0001N  +,CNQCSHN  +
7843,CN0001N  +,CNQCSHN  +
0292,CN0001P  -,CNQCSHP  -
0379,CN0001P  -,CNQCSHP  -
1038,CN0001P  -,CNQCSHP  -
7810,CN0001P  -,CNQCSHP  -
7811,CN0001P  -,CNQCSHP  -
7812,CN0001P  -,CNQCSHP  -
7842,CN0001P  -,CNQCSHP  -
7843,CN0001P  -,CNQCSHP  -
7719,CN0001P  -,CNCSHBP  -
7800,CN0001P  -,CNCSHBP  -
7801,CN0001P  -,CNCSHBP  -
7802,CN0001P  -,CNCSHBP  -
7830,CN0001P  -,CNCSHBP  -
7831,CN0001P  -,CNCSHBP  -
7846,CN0001P  -,CNCSHBP  -
7847,CN0001P  -,CNCSHBP  -



The requirement is-

Check the business code and transaction code from the input file.

Compare the values with the conditions given in condition file. If the business code (position 31:40) and transaction code (position 318:321) matches with the first and second value of any record in condition file, replace the transaction code with the third value of condition sheet.


After running the command/script the output file should be like this (Change in transaction code, position 318:321 wherever condition is met)

Code:
TSCM00000005837               CMCAMTN  -0000000001906.072010-12-10XML MM 201002081000000   YORK
 003007XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045
TSCM00000008112               CM0002N  -0000000001906.072010-12-10XML MM 201002081000000   YORK
 007777XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045
TSCM80000005282               CMDDPTNM +0000000002254.982010-12-10XML MM 201002081000001   YORK
 000440XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045
TSCM00000000215               CM0019NM +0000400002254.982010-12-10XML MM 201002081000001   YORK
 000500XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045
TSCM00000000215               CNQCSHN  -0000400002254.982010-12-10XML MM 201002081000001   YORK
 000292XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045
TSCM80000005282               CNQCSHN  -0000400002254.982010-12-10XML MM 201002081000001   YORK
 007843XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045
TSCM00000008107               CN0001N  -0000400002254.982010-12-10XML MM 201002081000001   YORK
 000012XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045
TSCM00000008093               CNQCSHP  -0000400002254.982010-12-10XML MM 201002081000001   YORK
 000379XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045
TSCM00000002646               CNCSHBP  -0000400002254.982010-12-10XML MM 201002081000001   YORK
 007847XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045
TSCM00000002646               CN0001P  -0000400002254.982010-12-10XML MM 201002081000001   YORK
 003400XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045
~



My Input file is having around 1 million records and these needs to be checked against all the conditions in condition sheet.
# 2  
Old 08-20-2010
Code:
TSCM00000005837               CMCAMTN  -0000000001906.072010-12-10XML MM 201002081000000   YORK
 003007XML MM 2010000000000*                   00000000*
                                                                                  2010-12-10-00.00.00123456789123450 GB 5045

in your input file, they are three lines, but in your description, 3007 is at 318:321. So I guess they should be in one line.
# 3  
Old 08-20-2010
1 record in 3 lines
One single record is taking 3 lines. All the records are starting with TSCM. There are 10 records in total.
# 4  
Old 08-20-2010
perl should be ok
Code:
# first get your condition mapping into a hash
open FH,"<yourconditionfile";
while(<FH>){
  chomp;
  my @tmp = split(",",$_);
  $hash{$tmp[0]."-".$tmp[1]}=$tmp[2];
}
close FH;

# then modify your first file, first let's seperate it into record per 3 lines
local $/="\n\n";
open FH,"awk '{print;if(NR %3 == 0) print \"\"}' youroriginalfile|";
while(<FH>){
  if(/\S*\s*(.*?[-+]).*?^\s*\d{2}(\d{4})/sm){
    #print $1,"  ",$2,"\n";
    my $tmp = $2."-".$1;
    if(exists $hash{$tmp}){
      s/$2/$tmp/;
    }
    s/\n$//;
    print;
  }
}

# 5  
Old 08-20-2010
1 record in single line without any newline character
Sorry for my earlier message. In Unix File 1 record is in 1 line only. There is no newline character in the single record. However while displaying it takes 3 lines.

Also the above solution seems to be in Perl, can't it be done using sed or awk ?
# 6  
Old 08-20-2010
Code:
awk '
NR==FNR {split($1,a,","); split($2,b,",");c[a[2] FS "00" a[1] "XML"]=b[2];next} 
(c[$2 FS $7]){gsub($2,c[$2 FS $7])}1
' condition.txt input.txt

# 7  
Old 08-20-2010
Just for fun ( with the original requirements "3 lines" ) ....

Code:
# cat gen.pl
#!/usr/bin/perl

use strict;

my $meta=shift;
my $dat=shift;
my $destination=shift;

open (META,"<",$meta ) || die "cant't open file $meta \n";

my (%metadata,@line);
while (<META>) {
   /^(\d+),(\S+)\s+(\S),(\S+)\s+(\S)/;
   $metadata{$1.$2.$3} = $4.":".$5;
   }
close(META);


open (DAT,"<",$dat ) || die "cant't open file $dat \n";
open (DEST,">",$destination ) || die "cant't open file $destination \n";

while (<DAT>) {
   /^\S+\s+(\S+)\s+(\S)\S+\s+\S+\s+\S+\s+\S+$/;
   if ($1 && $2 ) { 
      $line[0]=$_;
      my ($tCode,$sig)=($1,$2);
      $_=<DAT>;
      $line[1]=$_;
      /\s+\S+(\d{4})\S+\s+.*$/;
      if ( $1) {
         my $bCode=$1;
         if ($metadata{$bCode.$tCode.$sig}) {
            $metadata{$bCode.$tCode.$sig}=~/(\S+):(\S)/;
            my ($newTcode,$newSig)=($1,$2);
            $_=$line[0];
            s/^(\S+\s+)(\S+)(\s+)(\S)(\S+\s+\S+\s+\S+\s+\S+)$/\1$newTcode\3$newSig\5/;
            print DEST;
            print DEST $line[1];
            $_=<DAT>;
            print DEST ;
            }
         else {
            print DEST $line[0].$line[1];
            $_=<DAT>;
            print DEST;
            }
         }
      }
   }
close(DAT);
close(DEST);

Usage:

Code:
gen.pl  condFile inputFile newFile

Last edited by Klashxx; 08-20-2010 at 08:30 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk to add +1 to value based on condition in input

In the awk below I am trying to add a | that will adjust $2 in the ouput by adding +1 if the original value from file that was used in $3 had a - in it. Line 3 of file is an example of this. In my current awk I just subtract one but I am not sure how to only apply this to those values without a -.... (5 Replies)
Discussion started by: cmccabe
5 Replies

2. UNIX for Beginners Questions & Answers

Change the field color based on condition in email

Request your help to change the field color based on condition , if it is otherthan 0. using html in unix. Here is my condition for(i=1;i<=NF;i++) { print "<td> "$i"</td> } Please use CODE tags when displaying sample input, output, and code segments. (17 Replies)
Discussion started by: CatchMe
17 Replies

3. Shell Programming and Scripting

awk to change contents of field based on condition in same file

In the awk below I am trying to copy the entire contents of $6 there may be multiple values seperated by a ;, to $8, if $8 is . (lines 1 and 3 are examples). If that condition $8 is not . (line2 is an example) then that line is skipped and printed as is. The awk does execute but prints the output... (3 Replies)
Discussion started by: cmccabe
3 Replies

4. Shell Programming and Scripting

Script to select the rows from the feed file based on the input value provided

Hi Folks, I have the below feed file named abc1.txt in which you can see there is a title and below is the respective values in the rows and it is completely pipe delimited file ,. ... (3 Replies)
Discussion started by: punpun66
3 Replies

5. UNIX for Dummies Questions & Answers

Feed Input to a script running on bash

Hi Sir, I am just learning bash scripting and I came across a challenge. I need to input F11 to a script among many text inputs. For all the text inputs i did following. # sh test.sh < input.txt where input.txt contains all the text inputs in new lines. This worked fine until i... (1 Reply)
Discussion started by: gaurav kumar
1 Replies

6. Shell Programming and Scripting

How can I change file value based on condition

Hi, Gurus, I got a problem to resolve following issue: I have one file file1as following: start_dt=2010-01-01 12:00:02 start_dt=2011-01-01 09:00:02 start_dt=2009-01-01 11:00:02I have another file file2 as following: title1, 2010-01-03 10:00:02 title2, 2011-01-04 11:00:02 title3,... (5 Replies)
Discussion started by: ken002
5 Replies

7. Shell Programming and Scripting

Move input file based on condition

Hello, I have File1 in a directory A, a File2 in a directory B. If the File2 is not empty Then I have to move File1 from directory A to a directory archive Else no action. Is it possible to do this from one command line? Thank you in advance for your answers. Madi (2 Replies)
Discussion started by: AngelMady
2 Replies

8. Shell Programming and Scripting

Copy input file based on condition

Hi, I am new to unix shell programming. I want to write a shell script for a functionality existing in mainframe system. I have one file as below as input 123456 &__987 &12yuq abcdef _ referes to blank condition:whenever the input file is having &__ ,it should be replaced... (4 Replies)
Discussion started by: charan0703
4 Replies

9. Shell Programming and Scripting

Awk to print on condition in input file

I need only those records which has $2 equal to "DEF" independent of case (i.e upper or lower) nawk -F"," '$2 ~ //{print $0}' file This returns 3rd record also which i dont want I tried this but this doesnt work as expected. nawk -F"," '$2 == ""{print $0}' file i dont... (3 Replies)
Discussion started by: pinnacle
3 Replies

10. Shell Programming and Scripting

search file, change existing value based on input (awk help)

I have a file (status.file) of the form: valueA 3450 valueB -20 valueC -340 valueD 48 I am tailing a data.file, and need to search and modify a value in status.file...the tail is: tail -f data.file | awk '{ print $3, ($NF - $(NF-1)) }' which will produce lines that look like this: ... (3 Replies)
Discussion started by: nortonloaf
3 Replies
Login or Register to Ask a Question