Traverse a flatfile and check for errors


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Traverse a flatfile and check for errors
# 1  
Old 04-17-2008
Data Check records for errors

Hi,

I need to check a flatfile for various parameters like length of the record, format of record, any tab character present in the record etc.,
for checking presence of tab character, i'm trying to use the following code and i'm not sure if the same is right. Pls Help.
Code:
nawk '{print length($0)}' $fdate > filelength.txt
totrec=`cat filelength.txt | wc -l`
i=1
while [ $i -le $totrec ]
if [ $i -gt 1 ] && [ $i -lt $totrec ]; then
    if [ D -eq `nawk "(NR==${i}){print}" $fdate | cut -b1` ]; then
        if [ `nawk "(NR==${i}){print}" filelength.txt` -eq 3168 ]; then
        tabrec=`nawk '/[\t]/ {if (NR==\$i) {print $0} }' $fdate`
        echo $tabrec
            if [ `nawk '/[\t]/ {if (NR==\$i) {print NR} else {print 0} }' $fdate` ]; then
            echo "The Detail record $i has a tab character inside it"
            nawk "(NR==${i}){print} $fdate" >> ./error/$fdate.err
            else
            echo "The Detail record $i looks fine"
            nawk "(NR==${i}){print} $fdate" >> ./output/$fdate.dat
            fi
        else
        echo "The Detail record $i length is not proper"
        nawk "(NR==${i}){print} $fdate" >> ./error/$fdate.err
        fi
    else
    echo "The Detail record $i format is not proper"
    nawk "(NR==${i}){print} $fdate" >> ./error/$fdate.err
    fi
fi
i=`expr $i + 1`
done

I check for Record to start with 'D' and total record length to be equal to 3168 and if tab is present in the record.

Thanks in Advance.

Regards,
Aravind. C

Last edited by aravindc; 04-17-2008 at 04:03 AM.. Reason: Meaningful subjectline
# 2  
Old 04-17-2008
First take a look at Useless Use of Cat and corect your script, second use code tag.
Note: You can edit your original post.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Need in for a script that should check for errors in multiple log file (approx 2500) and should mail

hello everyone, I am new to linux and got this deliverable to write a script that should check for error in multiple log file (count is approx 2500 log files on single server) and once error is found, it should mail that error My logic says: we can put all log files path/location in one... (2 Replies)
Discussion started by: Pratik_CTS
2 Replies

2. Shell Programming and Scripting

Check for “errors” or “ORA-”

I want to check for "errors" or "ORA-" in Y.if there is an error then exit Y=`sqlplus -s user/passwd<< EOF exec test_Proc; exit; EOF` if ; then exit 1 fi but this doesnt work (6 Replies)
Discussion started by: haadiya
6 Replies

3. Shell Programming and Scripting

Generating XML from a flatfile

Hi all, I am trying to generate an XML file from a flatfile in ksh/bash (could also use perl at a pinch, but out of my depth there!). I have found several good solutions on this very forum for cases where the header line in the file forms the XML tags, however my flatfile is as follows:... (5 Replies)
Discussion started by: ianmrid
5 Replies

4. Shell Programming and Scripting

sending output to flatfile

Hi, I am writing one unix script to get row count of few tables into one sequential file my script is like this $ORACLE_HOME/bin/sqlplus -s <<EOF >output.txt userid/password@databasename set verify off set heading off set feedback off select count(*) count from tablel where ; select... (4 Replies)
Discussion started by: spmsarada
4 Replies

5. UNIX for Advanced & Expert Users

Delimeters Count in a FlatFile

Hi, I have the below script to check the count of delimeters for a file (here is File : test.csv Delimeter is ",") awk '{gsub(/"*"/,x);print gsub(/,/,x)}' test.csv And it return the output for each line as: 2 2 cat test.csv: abc,xyz "abc,zxyz",1 I need help one the below things: - IS... (8 Replies)
Discussion started by: venkatajay_18
8 Replies

6. Shell Programming and Scripting

Deleting column from a flatfile with delimiter

I have a set of flatfiles which have columns delimited by #. How can a particular column be deleted in all the flatfiles. All flatfiles have same number of columns. (5 Replies)
Discussion started by: rsprabha
5 Replies

7. What is on Your Mind?

Check this out. Find out the errors as much as possible. thanks

Sorry guys, busy working on a new project these days so I am not able to keep this topic update frequently. Few days earlier I talked to some of my friends in China and understand more about the current situation of China's IT industry. From what they told me and considering my experience and... (6 Replies)
Discussion started by: CULM
6 Replies

8. UNIX for Dummies Questions & Answers

Check my crontab for possible errors.

I'm posting the line I just added to my crontab because it's my first and I want to be sure it's doing what I'm intending it to do. 59 23 0 * 1-5 /export/<many_directories_later...>/scripts/get_clientkpi.sh Supposed to do: Run script at 11:59pm every night, except weekends. Question: If I wish... (4 Replies)
Discussion started by: yongho
4 Replies

9. UNIX for Dummies Questions & Answers

encrypting Unix flatfile

Hi, I am new to unix. I have a flat file that needs to be pgp encyrpted in ASCII format and ftped in Ascii format to remote location. Can any one tell me the steps involved in the pgp encryption of the unix file. I will really appreciate if any one can help me with the pgp encryption shell... (1 Reply)
Discussion started by: rkumar28
1 Replies

10. UNIX for Dummies Questions & Answers

Finding a column in a flatfile

I have a file which is fixed width columns. This is an offset buffer - rather than space or tab delimited. There are upto about 8 columns and I need to get all of the column 5's values into another file. The problem is that because the delimiter is a space - and some fields are blank - the 5th... (3 Replies)
Discussion started by: peter.herlihy
3 Replies
Login or Register to Ask a Question