Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Appending ErrorCodes to the corresponding error record

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Appending ErrorCodes to the corresponding error record
# 15  
Old 06-23-2014
Error after excecuting the above code is :
Code:
awk: cmd. line:19:      OUT==$0 {print OUT > GOOD; g++;next}
awk: cmd. line:19:              ^ syntax error
awk: cmd. line:21:      {print OUT > BAD: b++}
awk: cmd. line:21:                      ^ syntax error

Last edited by Scrutinizer; 06-24-2014 at 04:49 AM.. Reason: quote -> code tags
# 16  
Old 06-23-2014
Try to copy this as is into your script. The OUT==$0 is a pattern so it needs to be outside the braces, and you need a semicolon, not a colon after BAD:
Code:
                         if (L < MN[i])                OUT = OUT ",nullerr|" hh[i]
                        }
        OUT==$0         {print OUT > GOOD; g++;next}
                        {print OUT > BAD; b++}

# 17  
Old 06-23-2014
Hi,

I don't know what is going wrong here. Code is working fine but everything is written on to the good file.No bad file is getting created.
The script is :
Code:
#!/bin/bash
awk -F, -vDT="$(date +%m%d%Y%H%M)" 'BEGIN {    
#Adding timestamp into a GOOD and BAD file
GOOD = "good_" DT;   
BAD = "bad_" DT;     
#Loading validated good and bad records into HDFS
putG = "hadoop fs -put /home/user/sapthashree/galileo/validation/" GOOD " /user/user/dataparse/good/";       
putG1 = "hadoop fs -put /home/user/sapthashree/galileo/validation/" GOOD " /user/user/dataparse/hdfs_good/";  
putB = "hadoop fs -put /home/user/sapthashree/galileo/validation/" BAD " /user/user/dataparse/bad/"}         

NR==FNR {h[NR]=$1
                         
split ($2,a,"[)(]")
d[NR]=toupper(a[1])
l[NR]=a[2]
n[NR]=$3!=""
FLDS  =NR
next}
{
OUT=$0
for (i=1; i<=FLDS; i++)
{
L=length($i)
N=$i==$i+0
if (d[i] == "INTEGER" && !N) OUT = OUT ",dattyperr|" h[i]
if (d[i] == "STRING" &&   N) OUT = OUT ",dattyperr|" h[i]
if (L > l[i])                OUT = OUT ",colwiderr|" h[i]
if (L < n[i])                OUT = OUT ",nullerr|" h[i]
}
OUT==$0 
{print OUT > GOOD; g++;next}
{print OUT > BAD; b++}
}
END {      
print "parsing sucess!!";
print "Count of Bad Records : " b;
print "Count of Good Records : " g;
#system(putB);
    }' a.txt b.txt

On console i'm seeing:
Code:
parsing sucess!!
Count of Bad Records :
Count of Good Records : 20

Contents of good_062320142336 is :
Code:
201,,MI,4589,M
202,Lilly,FL,589,F
203,Richard,CA,2212,M
204,Cruse,VA,2222,M
20w,Taylor,,5888,M,dattyperr|id,nullerr|state
206,Merry,TX,6969,F
207,,CO,5656,F
208,Tom,AL,5555,M
209,Sam,FL,2586,M
210,8888,OK,456,dattyperr|name
211,George11,MI,5555,M
212,Reet,MI,4589,M
213,8888a,FL,5899,F
214,Meera,NY,2546,F
215,Madav,,4454,M,nullerr|state
216,ram,MI,4589,M
217,Leena,FL,589,F
218,Rohan,CA,2212,M
219,Raj,VA,2222,M
TaylorSmith,NY,5888,M,dattyperr|id,colwiderr|id,dattyperr|state,dattyperr|phone_no

Hope you are alos referring to the same code.I know something silly mistake i'm doing here but not getting where it's going wrong!

Thanks
Last edited by Scrutinizer; 06-24-2014 at 04:49 AM.. Reason: quote -> code tags
# 18  
Old 06-23-2014
Code:
}
OUT==$0 
{print OUT > GOOD; g++;next}
{print OUT > BAD; b++}
}                                        # put this brace up three lines; before OUT==$0
END {

# 19  
Old 06-24-2014
Hi, I did the way you suggested but still it's not working fine. It's displaying good records on console and displaying all good and bad records with errorcode attached to the bad records into a single good file. Bad file is not getting created.

Code i used :
Code:
if (L < n[i]) OUT = OUT ",nullerr|" h[i]
}
}
OUT==$0 
{print OUT > GOOD; g++;next}
{print OUT > BAD; b++}
END {

# 20  
Old 06-24-2014
Quote:
Originally Posted by shree11
Hi, I did the way you suggested but still it's not working fine. It's displaying good records on console and displaying all good and bad records with errorcode attached to the bad records into a single good file. Bad file is not getting created.

Code i used :
Code:
if (L < n[i]) OUT = OUT ",nullerr|" h[i]
}
}
OUT==$0 
{print OUT > GOOD; g++;next}
{print OUT > BAD; b++}
END {

There is a HUGE difference between the code RudiC suggested:
Code:
        OUT==$0         {print OUT > GOOD; g++;next}
                        {print OUT > BAD; b++}

and the code you're using:
Code:
OUT==$0 
{print OUT > GOOD; g++;next}
{print OUT > BAD; b++}

The condition OUT==$0 must be placed on the same line as the opening brace for the actions to be performed when that condition is true.

Otherwise, as you have seen, you get the default action for the good lines (print to standard output) and write every line (the default when no condition is specified) to the file named by GOOD and skip remaining lines in the script (next) (so nothing is written to the file named by BAD on the next line of your script.
# 21  
Old 06-24-2014
Hi,
Yes you are right. I was using the OUT==$0 above the print statement. So because of that i was not getting teh proper output.This small mistake took most of my time.And thanks to explain it briefly.
Now it's working fine

Thanks,
Shree

---------- Post updated at 02:35 AM ---------- Previous update was at 01:41 AM ----------

Hi RudiC ,

I have one more query.

In my bad record file i'm supposed to get 20 records but i'm getting 23 records.
The below 3 records given below are coming as a part of bad records but they are not a bad records.
Code:
22,Meet,CO,,F,datatyperr|phone_no
52,Meet,CO,,F,datatyperr|phone_no
82,Meet,CO,,F,datatyperr|phone_no

Here above, 4th field(phone_no) can be a null fileld. And it should not throw a datatyperror.They should come as apart of good file.

---------- Post updated at 04:48 AM ---------- Previous update was at 02:35 AM ----------

Hi,I'm able to solve the above issue by adding the below code:
Code:
N=$i==$i+0 
M=$i==""
if (d[i] == "INTEGER" && (!N && !M)) OUT = OUT ",datatyperr|" h[i]

Thanks,
Shree
Last edited by Scrutinizer; 06-24-2014 at 04:49 AM.. Reason: quote -> code tags
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Getting error not able remove the record

export PATH=$PATH:/opt/IBM/db2/V9.5/95fp4/bin DB2INSTANCE=bancbcb1 export DB2INSTANCE db2 "connect to dyaxprpt user xyz using byx" echo "Load lastest data start" Row=$(db2 –x “delete from xyz.ROL_yxpwhere axl_ING_ID in(50127,50455,503458,175748)”) If ; then echo “no data deleted” elif ;... (2 Replies)
Discussion started by: jagu
2 Replies

2. Shell Programming and Scripting

Appending CRLF to end of record

I need to append |\r\n (a pipe character and CRLF) at end of each record in Unix to all records where they are not already present. So first check for the presence of |\r\n and if absent append it else do nothing (3 Replies)
Discussion started by: abhilashnair
3 Replies

3. Shell Programming and Scripting

Extract timestamp from first record in xml file and it checks if not it will replace first record

I have test.xml <emp><id>101</id><name>AAA</name><date>06/06/14 1811</date></emp> <Join><id>101</id><city>london</city><date>06/06/14 2011</date></join> <Join><id>101</id><city>new york</city><date>06/06/14 1811</date></join> <Join><id>101</id><city>sydney</city><date>06/06/14... (2 Replies)
Discussion started by: vsraju
2 Replies

4. Shell Programming and Scripting

How to compare current record,with next and previous record in awk without using array?

Hi! all can any one tell me how to compare current record of column with next and previous record in awk without using array my case is like this input.txt 0 32 1 26 2 27 3 34 4 26 5 25 6 24 9 23 0 32 1 28 2 15 3 26 4 24 (7 Replies)
Discussion started by: Dona Clara
7 Replies

5. Shell Programming and Scripting

Splitting record into multiple records by appending values from an input field (AWK)

Hello, For the input file, I am trying to split those records which have multiple values seperated by '|' in the last input field, into multiple records and each record corresponds to the common input fields + one of the value from the last field. I was trying with an example on this forum... (4 Replies)
Discussion started by: imtiaz99
4 Replies

6. Shell Programming and Scripting

Appending error messages from log file next to the corresponding error record

Hi Everyone, I have an issue and trying to get a solution but was not succesful yet. Any help is greatly appreciated. I am using ksh to inoke sql loader to load data from txt file into two oracle tables based on the condition written in the control file. If an error occurs while loading into... (8 Replies)
Discussion started by: vpv0002
8 Replies

7. UNIX for Dummies Questions & Answers

Appending error

Hi All, I just want to append the value in variable at the end of the file. var=1234 sed -e "$a $var" file1 > file 2. But I get this error sed: -e expression #1, char 4: unknown command: `1' Kindly let m know how can I do that... (5 Replies)
Discussion started by: waqar1
5 Replies

8. Shell Programming and Scripting

Error while appending records to a file

Hi, I have a sample file which contains records. Input File : 1 user1 username1\password@database-name 2 user2 username2\password@database-name 3 user3 username1\password@database-name I should search for a 'username1\' in those records. If 'username1\' is found in those records, that record... (7 Replies)
Discussion started by: siri_886
7 Replies

9. AIX

pax error on appending data to LTO3

I have problem when I use the command "pax -awvf /dev/rmt0 ./data1" in AIX 5.3.0.0. The command with parameter -a allow me to append the tape but when I try to retrieve the data that I append, it will show me error. I would like to know if anyone have the same problem and any solution found? Tq. (0 Replies)
Discussion started by: kwliew999
0 Replies

10. Shell Programming and Scripting

appending spaces to first line based on second record.

Hi, I have a situation to append spaces to end of first record (header)and last record (footer) based on second record length. The first record length is always 20.The second record will be different for different files.I have to append spaces for the first line based on second record... (2 Replies)
Discussion started by: ammu
2 Replies
Login or Register to Ask a Question