Help With UNIX Shell Scripting For Data Validation
Hi All,
I am completely new to Unix Shell Scripting.
I m validating(Basic File Validation) an .HHT file in TIBCO.
After that i need to do Data Validation through UNIX Shell scripting.
Rules in DataValidation:
1.) Need to Check Every field wheather it is a Char or Number?(Fields are seperated bya Delimeter ||)
2.) Need to check the DateFormat. It should be (dd/mm/yyyy hh:mm:ss)
3.) Need to check the Nullabality(null or notNull)
4.) Need to Check Length --> Should be within the range
Now i need to validate all the rows/Records in that File and the
Passed records which satisfies the above criteria should write into another File
the failed records should Write into different File.
Sample File:
Note: 1st Record(row)--> List of Columns
Last edited by jim mcnamara; 05-24-2013 at 11:19 AM..
Sounds like a PERL, C, C++, JAVA job. First, a file is just an array of char, and you need to parse/validate it sequentially to discover fields, separators, and whatever passes in there for null.
If you take a light weight approach, bash can read your fields into a previously typeset indexed array with "while read -a flds ; do ... done <file". You have to add '|' to $IFS to make it an additional field separator, or "tr '|' ' '" to translate the pipes to spaces (since your data normally has none and space is already in $IFS, along with tab and linefeed). You get to write the ..., and bash has regex tools in the "if [[ ... ]]" form of pattern examination, so you can ensure the field has legal length and the right type of character in the right place. You can capture and save header if you use a form like "tr '|' ' ' <file | ( read -a hdr ; while read -a flds ; do ... done )". A typical regex might be "^[a-zA-Z]\{1,22\}$" to pattern match a field with one to 22 letters only. I think the \{\} is limited to 0-99 bytes, but you can stack them.
Last edited by DGPickett; 05-24-2013 at 05:11 PM..
I have input data looks like this which is a part of a csv file
7,1265,76548,"0102:04"
8,1266,76545,"0112:04"
I need to make the output data should look like this and the output data will be part of text file:
7|1265000 |7654899 |A|
8|12660000 |76545999 |B|
The logic behind the... (6 Replies)
Hi ,
I have input file as XML. following are input data
#complex.xml
Code:
<?xml version="1.0" encoding="UTF-8"?><TEST_doc xmlns="http://www.w3.org/2001/XMLSchema-instance"> <ENTRY uid="123456"> <protein> <name>PROT001</name> <organism>Human</organism> ... (1 Reply)
Hi evry1,
This is my 1st post in this forum.Pls help me
I want to extract some data froma xml file which has 2000 lines using shell scripting. Actually my xml file has some "audio and video codes" which i need to arrange in a column wise format after extracting it using shell scripting.I... (4 Replies)
hello friends,
my file is like
123 |asd|asd|asd
123_1|awd|asw|asw
121 |wer|qwe|wee
124 |weq|qwe|iop
1_23 |bla|blh|bha
145 |ghj|jkl|ghj
146 |qwe|qwe|wer
154 |asd|wer|qw_e
134_5|qwe|wer|qw_e
is their any solution to retrive only those lines which are having only 3 numerical letters... (20 Replies)
Hi,
i want to fetch the data from the alert log file, for a particular time interval.
Example :
Alert log content :
Thu Mar 18 08:47:36 2010
Completed: alter database open
Thu Mar 18 19:13:38 2010
MMNL absent for 6390 secs; Foregrounds taking over
Fri Mar 19 08:30:52 2010... (1 Reply)
How to validate a date and optionly a time in shell scripting when i get the date and time as pararmeters that sent out with the call of the file? (in my case sh union.sh `first parameter ,second parameter...` (4 Replies)
Hi All,
Currently i have to write a script.
For which i need to cut a block from .txt file.
I know the specific word that starts the block and ends the block.
Can we do it in shell scripting..?
Please suggest.... (6 Replies)