Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Test File Reading & Validation using Shell script

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Test File Reading & Validation using Shell script
# 1  
Old 02-15-2009
Test File Reading & Validation using Shell script
Please help develop script for below requirement
Code:
-------Sample file-------------------------------
HSVSHOSTRECON 20090115011817BP            
DARMAR60064966247003504720000000000000000000066626000000000000133000003D003463001332    
DARMAR60064966247008334930000000000000000000066626000000000000191000003D006888001911    
DARMAR60064966247023430040000000000000000000066626000000000000237000003D010451002371    
DARMAR60064966247018096900000000000000000000066626000000000000358000003D000069003581    
DARMAR60064966247026650830000000000000000000066626000000000000358000003D011186003581    
DARMAR60064966247028148300000000000000000000066626000000000000358000003D009154003581    
DARMAR60064966247019083510000000000000000000066626000000000000374000004D030784003741    
DARMAR60064966247018409190000000000000000000066626000000000000428000003D010969004281    
DARMAR60064966247021076490000000000000000000066626000000000000442000003D005373004421    
DARMAR60064966247028097310000000000000000000066626000000000000442000003D008266004421
DARMAR60064966247028097310000000000000000000066626000000000000442000003D008266004421
T00000011                                        
-----------------------------------------------

Here is the List of Validation. We need to send mail for each validation failure. First Char is Type of Record Say Header/Detail/Trailer
  • 1. The first record in the file must be the header record.
  • 2. The file can have at most one header record.
  • 3. The date on the header should match the date received (Note: If a file in error is retransmitted the next day, this may not be so. The purpose of this edit is to make sure that we are not picking up a duplicate of an earlier file.)
  • 4. The last record in the file must be a trailer record. (This means that the file can have at most one trailer record, and that there should be no header or detail records following the trailer record.)
  • 5. Every record between the header record and the trailer record must be a detail record.
  • 6. The trailer count should match the calculated number of detail records in the file.
Regards
Niraj Bhatt
# 2  
Old 02-15-2009
Tools I have the following queries..
I have the following queries...

1. Is the file name same Smilie i ment for the one validation has to be done?

2. The date feild in the first line can be validated with system date Smilie?

3. The first line i ment header is always with time stamp and footer is always with no. of records?

4. Will ther be a common patern matcher for the record... in teh below case like "DARMAR" Smilie

Regards
Bips...
# 3  
Old 02-16-2009
Hi Bips,

Please see below answers

1. Is the file name same Smilie i ment for the one validation has to be done?
it will be <filename><receivedate>.txt format.

2. The date feild in the first line can be validated with system date Smilie?
Yes

3. The first line i ment header is always with time stamp and footer is always with no. of records?
Yes...Header will be date & time stamp and footer will be with total records

4. Will ther be a common patern matcher for the record... in teh below case like "DARMAR" Smilie

D stands for detail...after that records will be changed.
# 4  
Old 02-16-2009
Some more inputs.

First Char is Type of Record Say Header/Detail/Trailer
Last records contains number of detail records i.e. 11 it can be 105 or 1230 also but position will be same always.
First record has date which is 20090115011817 in yyyymmddhhmmss format position in header will be same always
# 5  
Old 02-16-2009
Hey Niraj,

Kindly execute the below script in the following pattern:

cmd$>sh script.name <file.nameto be validated>

below is script.name
---------
Code:
#!/bin/sh
X=`date +%Y%m%d`
#W=`cat X`
head -1 $1 > Y
Z=`cat Y | grep -i $X | wc -l`
if [ "$Z" -eq 1 ]
then
         echo "Record generated today found"
          temp1=`tail -1 $1 | wc -c`
          length=`expr $temp1 - 1`
          t=0
          Target_Record=0
          s=1
          while [ $length -gt $s ]
          do
                  len=`expr $length - $t`
                  x=`tail -1 $1| cut -c $len`
                  if [ $t -le 0 ]
                  then
                         y=0
                         Target_Record=`expr $y + $Target_Record + $x`
                  else
                         y=`expr $x \* 10`
                         Target_Record=`expr $y + $Target_Record `
                  fi
                  s=`expr $s + 1`
                  t=`expr $t + 1`
           done
           echo "Expected enties as per footer:"$Target_Record
           temp_1=`cat $1|wc -l`
           temp_5=`expr $temp_1 - 2`
           if [ $temp_5 -eq $Target_Record ]
           then
               echo "record is perfect"
           else
               echo "record is corrept"
           fi
else
           echo "record not of todays date"
fi
exit 1

---------------

i have assumed that the total count of individual records is equal to total lines present in the record minus first and teh last line....

i guess this will help you....

Regards,
Bips Smilie.
Last edited by Franklin52; 02-24-2009 at 03:16 PM.. Reason: adding code tags
# 6  
Old 02-17-2009
Hi Bips.

Can I request you to please add some comments so that It is more clear to understand ?

Did you cover all 6 Validation ?

Regards
Niraj Bhatt
# 7  
Old 02-18-2009
Hi Niraj,

The script does the following:
1. Checks if the files header corresponds to the current date.
else the output is "record not of todays date"
2. Checks if the file has more than one header it wil fail with error "record is corrept"
3. Checks the date in the header with the system date if ther is a missmatch it wil fail with "record not of todays date"
4. If there are more than one trailer record it wil give output "record is corrept"
5. Every record between the header and trailer is considered as a Detail record. I was not able to check the consistancy of that as u did not give me any standard on that.
6. The trailer count will match the records found if sucessful it wil give message "record is perfect" else fails with the message its corrept.

I have not considered output to be redirected to a file.
Example:
Code:
Bips> cat test
HSVSHOSTRECON 20090219011817BP
DARMAR60064966247003504720000000000000000000066626000000000000133000003D003463001332
DARMAR60064966247008334930000000000000000000066626000000000000191000003D006888001911
DARMAR60064966247023430040000000000000000000066626000000000000237000003D010451002371
DARMAR60064966247018096900000000000000000000066626000000000000358000003D000069003581
DARMAR60064966247026650830000000000000000000066626000000000000358000003D011186003581
T00000005
Bips> sh sample test
Record generated today found
Expected enties as per footer:5

record is perfect
----
2nd case :
---
Code:
Bips> cat test
HSVSHOSTRECON 20090219011817BP
DARMAR60064966247003504720000000000000000000066626000000000000133000003D003463001332
DARMAR60064966247008334930000000000000000000066626000000000000191000003D006888001911
DARMAR60064966247023430040000000000000000000066626000000000000237000003D010451002371
DARMAR60064966247018096900000000000000000000066626000000000000358000003D000069003581
DARMAR60064966247026650830000000000000000000066626000000000000358000003D011186003581
T00000005
T00000005
Bips> sh sample test
Record generated today found
Expected enties as per footer:5

record is corrupt
---
I guess its clear now...

Regards,
Bips.Smilie
Last edited by Franklin52; 02-24-2009 at 03:20 PM.. Reason: adding code tags
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script reading file slow

I have shell program as below #!/bin/sh echo ======= LogManageri start ========== #This directory is getting the raw data from remote server Raw_data=/opt/ftplogs # This directory is ready for process the data Processing_dir=/opt/processing_dir # This directory is prcoessed files and... (4 Replies)
Discussion started by: Chenchireddy
4 Replies

2. UNIX for Dummies Questions & Answers

C-Shell script help reading from txt file

I need to write a C-Shell script with these properties: It should accept two arguments on the command line. The first argument is the name of a file which contains a list of names, and the second argument is the name of a directory. For each file in the directory, the script should print the... (1 Reply)
Discussion started by: cerce
1 Replies

3. Shell Programming and Scripting

Reading arguments for a shell script from file

I have a shell script that takes 2 arguments. I will have to execute this script multiple times with different values for the arguments. for example, ./shscript env1 value1 ./shscript env1 value2 ./shscript env2 value3 ./shscript env3 value4 ./shscript env1 value5 ./shscript env3... (24 Replies)
Discussion started by: goddevil
24 Replies

4. Shell Programming and Scripting

Error while reading from a file in shell script

Hi All, I'm writing a script to read a file line by line and then perform awk function on it. I am getting an error . My file has one name in it "James". I'm expecting my o/p to be youareJamesbond James ./users.sh: line 7: =: command not found #script to read file line by line #adding... (5 Replies)
Discussion started by: Irishboy24
5 Replies

5. Shell Programming and Scripting

Reading a property file through shell script???

Hi! i need a script that can read a property file. i.e., A script to read a "property" from property file. Read the property value and based on value of property, decide whether to start the some dataload activity or not. Its urngent. Can anyone help me out???:( (7 Replies)
Discussion started by: sukhdip
7 Replies

6. Shell Programming and Scripting

shell script data & time validation

How to validate a date and optionly a time in shell scripting when i get the date and time as pararmeters that sent out with the call of the file? (in my case sh union.sh `first parameter ,second parameter...` (4 Replies)
Discussion started by: tal
4 Replies

7. Shell Programming and Scripting

file reading through shell script

For reading a file through shell script I am using yhe code : while read line do echo $line done<data.txt It reads all the line of that file data.txt. Content of data.txt looks like: code=y sql=y total no of sql files=4 a.sql b.sql c.sql d.sql cpp=n c=y total no of c files=1 (4 Replies)
Discussion started by: Dip
4 Replies

8. Shell Programming and Scripting

Reading data from a file through shell script

There is one Text file data.txt. Data within this file looks like: a.sql b.sql c.sql d.sql ..... ..... want to write a shell script which will access these values within a loop, access one value at a time and store into a variable. can anyone plz help me. (2 Replies)
Discussion started by: Dip
2 Replies

9. Shell Programming and Scripting

Need help in file validation by shell script

Hi I am new to this forum.I need a help in the following: We receve pipe delimited file with transaction ID,tran_date,Quest_cd,Ans_cd,ans_value. Same transaction ID can be repeated with different quest_cd and ans_cd. Basically I need to check if a perticular pair of quest_cd and ans_cd... (1 Reply)
Discussion started by: srichakra
1 Replies

10. Shell Programming and Scripting

Reading file names from a file and executing the relative file from shell script

Hi How can i dynamically read files names from a list file and execute them from a single shell script. Please help its urgent Thanks in Advance (4 Replies)
Discussion started by: anushilrai
4 Replies
Login or Register to Ask a Question