Test File Reading & Validation using Shell script


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Test File Reading & Validation using Shell script
# 8  
Old 02-24-2009
Hi Dip,

"Every record between the header and trailer is considered as a Detail record. I was not able to check the consistancy of that as u did not give me any standard on that."

Can you please let me know what you need to know about consistency ?

All the records between header and Trailer must be detail record. If there are more than one header or trailer record than file is corrupt.

Regards
Niraj
# 9  
Old 02-24-2009
Try this :

$ validate.sc your_file_name

validate.sc is the following:
Code:
#!/bin/ksh
today=`date +%Y%m%d`
TRAILER=false
HEADER=false
DETAIL=0
while read LINE
do
#
#  Validating 1st record is Header + header's date
#
   FIRST_CAR=`echo $LINE | cut -c1`
   if [ $HEADER = "false" -a $FIRST_CAR = "H" ]
   then
      HEADER=true
      echo $LINE | grep $today > /dev/null
      if [ $? -eq 1 ]
         echo "Header record not from today"
         exit 1
      fi
   else
#
#  Validating detail record or trailer recors
#
      if [ $HEADER = "true" ]
      then
         if [ $FIRST_CAR = "D" ]
         then
            if [ $TRAILER = "false" ]
            then
               DETAIL=`expr $DETAIL + 1`
            else
               echo "File is corrupt - Detail record found after a trailer record"
               exit 1
            fi
         else
            if [ $FIRST_CAR = "T" -a $TRAILER = "false" ]
            then
               TRAILER=true
               TOT=`echo $LINE | cut -c2-`
               if [ $DETAIL -ne $TOT ]
               then
                  echo "File is corrupt - Number of detail record mismatch"
                  exit 1
               fi
            else
               echo "File is corrupt - Invalid record or more than one header/trailer record"
               exit 1
            fi
         fi
      else
          echo "File is corrupt - No header record found"
          exit 1
      fi
   fi
done < $1
if [ "$TRAILER" = "false" ]
then
   echo "File is corrupt - No trailer record found"
   exit 1
else
   echo "File seems to be Ok"
fi


Last edited by ce9888; 02-24-2009 at 02:15 PM..
# 10  
Old 02-24-2009
Quote:
Originally Posted by ce9888
Try this :

$ validate.sc your_file_name

validate.sc is the following:
Code:
#!/bin/ksh
today=`date +%Y%m%d`
TRAILER=false
HEADER=false
DETAIL=0
while read LINE
do
#
#  Validating 1st record is Header + header's date
#
   FIRST_CAR=`echo $LINE | cut -c1`


First UEC (Unnecessary External Command). External commands slow a script considerably, especially when in a loop.

Code:
FIRST_CAR=${LINE%"${LINE#?}"}

Quote:
Code:
   if [ $HEADER = "false" -a $FIRST_CAR = "H" ]
   then
      HEADER=true
      echo $LINE | grep $today > /dev/null


Second UEC.

Code:
case $LINE in
     *"$today"*) ;;
     *) echo "Header record not from today" 
         exit 1 ;;
esac

Quote:
Code:
      if [ $? -eq 1 ]
         echo "Header record not from today"
         exit 1
      fi
   else
#
#  Validating detail record or trailer recors
#
      if [ $HEADER = "true" ]
      then
         if [ $FIRST_CAR = "D" ]
         then
            if [ $TRAILER = "false" ]
            then
               DETAIL=`expr $DETAIL + 1`


Third UEC.

Code:
DETAIL=$(( $DETAIL + 1 ))

Quote:
Code:
            else
               echo "File is corrupt - Detail record found after a trailer record"
               exit 1
            fi
         else
            if [ $FIRST_CAR = "T" -a $TRAILER = "false" ]
            then
               TRAILER=true
               TOT=`echo $LINE | cut -c2-`


Fourth UEC.
Quote:
Code:
               if [ $DETAIL -ne $TOT ]
               then
                  echo "File is corrupt - Number of detail record mismatch"
                  exit 1
               fi
            else
               echo "File is corrupt - Invalid record or more than one header/trailer record"
               exit 1
            fi
         fi
      else
          echo "File is corrupt - No header record found"
          exit 1
      fi
   fi
done < $1
if [ "$TRAILER" = "false" ]
then
   echo "File is corrupt - No trailer record found"
   exit 1
else
   echo "File seems to be Ok"
fi

# 11  
Old 02-24-2009
I presume that would be the fourth UEC

Quote:
TOT=`echo $LINE | cut -c2-`
What should it be ?

thx
# 12  
Old 02-24-2009
Quote:
Originally Posted by ce9888
I presume that would be the fourth UEC


Quote:
TOT=`echo $LINE | cut -c2-`
What should it be ?

thx
Code:
 TOT=${LINE#?}

Regards
# 13  
Old 02-28-2009
Hi Dip,

I am also getting error as below at TOT variable

./validate.sh[9]: syntax error at line 47 : ``' unmatched

Regards
# 14  
Old 02-28-2009
Hi ce9888

I am getting below eeror with example I gave.

File is corrupt - Invalid record or more than one Header/Trailer record"

There is only one Header and trailer still I am getting that error.

Can you please advise ?

Regards
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script reading file slow

I have shell program as below #!/bin/sh echo ======= LogManageri start ========== #This directory is getting the raw data from remote server Raw_data=/opt/ftplogs # This directory is ready for process the data Processing_dir=/opt/processing_dir # This directory is prcoessed files and... (4 Replies)
Discussion started by: Chenchireddy
4 Replies

2. UNIX for Dummies Questions & Answers

C-Shell script help reading from txt file

I need to write a C-Shell script with these properties: It should accept two arguments on the command line. The first argument is the name of a file which contains a list of names, and the second argument is the name of a directory. For each file in the directory, the script should print the... (1 Reply)
Discussion started by: cerce
1 Replies

3. Shell Programming and Scripting

Reading arguments for a shell script from file

I have a shell script that takes 2 arguments. I will have to execute this script multiple times with different values for the arguments. for example, ./shscript env1 value1 ./shscript env1 value2 ./shscript env2 value3 ./shscript env3 value4 ./shscript env1 value5 ./shscript env3... (24 Replies)
Discussion started by: goddevil
24 Replies

4. Shell Programming and Scripting

Error while reading from a file in shell script

Hi All, I'm writing a script to read a file line by line and then perform awk function on it. I am getting an error . My file has one name in it "James". I'm expecting my o/p to be youareJamesbond James ./users.sh: line 7: =: command not found #script to read file line by line #adding... (5 Replies)
Discussion started by: Irishboy24
5 Replies

5. Shell Programming and Scripting

Reading a property file through shell script???

Hi! i need a script that can read a property file. i.e., A script to read a "property" from property file. Read the property value and based on value of property, decide whether to start the some dataload activity or not. Its urngent. Can anyone help me out???:( (7 Replies)
Discussion started by: sukhdip
7 Replies

6. Shell Programming and Scripting

shell script data & time validation

How to validate a date and optionly a time in shell scripting when i get the date and time as pararmeters that sent out with the call of the file? (in my case sh union.sh `first parameter ,second parameter...` (4 Replies)
Discussion started by: tal
4 Replies

7. Shell Programming and Scripting

file reading through shell script

For reading a file through shell script I am using yhe code : while read line do echo $line done<data.txt It reads all the line of that file data.txt. Content of data.txt looks like: code=y sql=y total no of sql files=4 a.sql b.sql c.sql d.sql cpp=n c=y total no of c files=1 (4 Replies)
Discussion started by: Dip
4 Replies

8. Shell Programming and Scripting

Reading data from a file through shell script

There is one Text file data.txt. Data within this file looks like: a.sql b.sql c.sql d.sql ..... ..... want to write a shell script which will access these values within a loop, access one value at a time and store into a variable. can anyone plz help me. (2 Replies)
Discussion started by: Dip
2 Replies

9. Shell Programming and Scripting

Need help in file validation by shell script

Hi I am new to this forum.I need a help in the following: We receve pipe delimited file with transaction ID,tran_date,Quest_cd,Ans_cd,ans_value. Same transaction ID can be repeated with different quest_cd and ans_cd. Basically I need to check if a perticular pair of quest_cd and ans_cd... (1 Reply)
Discussion started by: srichakra
1 Replies

10. Shell Programming and Scripting

Reading file names from a file and executing the relative file from shell script

Hi How can i dynamically read files names from a list file and execute them from a single shell script. Please help its urgent Thanks in Advance (4 Replies)
Discussion started by: anushilrai
4 Replies
Login or Register to Ask a Question