Sponsored Content
Top Forums Shell Programming and Scripting Help With UNIX Shell Scripting For Data Validation Post 302811965 by DGPickett on Friday 24th of May 2013 04:05:14 PM
Old 05-24-2013
Sounds like a PERL, C, C++, JAVA job. First, a file is just an array of char, and you need to parse/validate it sequentially to discover fields, separators, and whatever passes in there for null.

If you take a light weight approach, bash can read your fields into a previously typeset indexed array with "while read -a flds ; do ... done <file". You have to add '|' to $IFS to make it an additional field separator, or "tr '|' ' '" to translate the pipes to spaces (since your data normally has none and space is already in $IFS, along with tab and linefeed). You get to write the ..., and bash has regex tools in the "if [[ ... ]]" form of pattern examination, so you can ensure the field has legal length and the right type of character in the right place. You can capture and save header if you use a form like "tr '|' ' ' <file | ( read -a hdr ; while read -a flds ; do ... done )". A typical regex might be "^[a-zA-Z]\{1,22\}$" to pattern match a field with one to 22 letters only. I think the \{\} is limited to 0-99 bytes, but you can stack them.

Last edited by DGPickett; 05-24-2013 at 05:11 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

difference between AIX shell scripting and Unix shell scripting.

please give the difference between AIX shell scripting and Unix shell scripting. (2 Replies)
Discussion started by: haroonec
2 Replies

2. Shell Programming and Scripting

data formatting in Unix shell Scripting

#! /bin/ksh ############################ # AFI Monitor Script ############################ . /db2/uszlad48/sqllib/db2profile export mondir=/home/script/krishna export monlog=$mondir/Error_Report_`date +%Y%m%d`.log echo "connect to database r2pdev" >>$monlog db2 connect to r2pdev user... (1 Reply)
Discussion started by: regnumber
1 Replies

3. UNIX for Dummies Questions & Answers

How to cut data block from .txt file in shell scripting

Hi All, Currently i have to write a script. For which i need to cut a block from .txt file. I know the specific word that starts the block and ends the block. Can we do it in shell scripting..? Please suggest.... (6 Replies)
Discussion started by: pank29
6 Replies

4. Shell Programming and Scripting

shell script data & time validation

How to validate a date and optionly a time in shell scripting when i get the date and time as pararmeters that sent out with the call of the file? (in my case sh union.sh `first parameter ,second parameter...` (4 Replies)
Discussion started by: tal
4 Replies

5. Shell Programming and Scripting

Shell scripting to extract data from file

Hi, i want to fetch the data from the alert log file, for a particular time interval. Example : Alert log content : Thu Mar 18 08:47:36 2010 Completed: alter database open Thu Mar 18 19:13:38 2010 MMNL absent for 6390 secs; Foregrounds taking over Fri Mar 19 08:30:52 2010... (1 Reply)
Discussion started by: Pinki018
1 Replies

6. Shell Programming and Scripting

Extracting data from file-shell scripting--please help

hello friends, my file is like 123 |asd|asd|asd 123_1|awd|asw|asw 121 |wer|qwe|wee 124 |weq|qwe|iop 1_23 |bla|blh|bha 145 |ghj|jkl|ghj 146 |qwe|qwe|wer 154 |asd|wer|qw_e 134_5|qwe|wer|qw_e is their any solution to retrive only those lines which are having only 3 numerical letters... (20 Replies)
Discussion started by: PankajChawla
20 Replies

7. Shell Programming and Scripting

How to extract data from xml file using shell scripting?

Hi evry1, This is my 1st post in this forum.Pls help me I want to extract some data froma xml file which has 2000 lines using shell scripting. Actually my xml file has some "audio and video codes" which i need to arrange in a column wise format after extracting it using shell scripting.I... (4 Replies)
Discussion started by: arun_kohan
4 Replies

8. Shell Programming and Scripting

How to extract data from XML file using shell scripting?

Hi , I have input file as XML. following are input data #complex.xml Code: <?xml version="1.0" encoding="UTF-8"?><TEST_doc xmlns="http://www.w3.org/2001/XMLSchema-instance"> <ENTRY uid="123456"> <protein> <name>PROT001</name> <organism>Human</organism> ... (1 Reply)
Discussion started by: arun_kohan
1 Replies

9. UNIX for Dummies Questions & Answers

Shell script to read lines in a text file and filter user data Shell Programming and Scripting

sxsaaas (3 Replies)
Discussion started by: VikrantD
3 Replies

10. Shell Programming and Scripting

Read csv file, convert the data and make one text file in UNIX shell scripting

I have input data looks like this which is a part of a csv file 7,1265,76548,"0102:04" 8,1266,76545,"0112:04" I need to make the output data should look like this and the output data will be part of text file: 7|1265000 |7654899 |A| 8|12660000 |76545999 |B| The logic behind the... (6 Replies)
Discussion started by: RJG
6 Replies
bufsplit(3GEN)					     String Pattern-Matching Library Functions					    bufsplit(3GEN)

NAME
bufsplit - split buffer into fields SYNOPSIS
cc [ flag ... ] file ... -lgen [ library ... ] #include <libgen.h> size_t bufsplit(char *buf, size_t n, char **a); DESCRIPTION
bufsplit() examines the buffer, buf, and assigns values to the pointer array, a, so that the pointers point to the first n fields in buf that are delimited by TABs or NEWLINEs. To change the characters used to separate fields, call bufsplit() with buf pointing to the string of characters, and n and a set to zero. For example, to use colon (:), period (.), and comma (,), as separators along with TAB and NEWLINE: bufsplit (":., ", 0, (char**)0 ); RETURN VALUES
The number of fields assigned in the array a. If buf is zero, the return value is zero and the array is unchanged. Otherwise the value is at least one. The remainder of the elements in the array are assigned the address of the null byte at the end of the buffer. EXAMPLES
Example 1: Example of bufsplit() function. /* * set a[0] = "This", a[1] = "is", a[2] = "a", * a[3] = "test" */ bufsplit("This is a test ", 4, a); NOTES
bufsplit() changes the delimiters to null bytes in buf. When compiling multithreaded applications, the _REENTRANT flag must be defined on the compile line. This flag should only be used in mul- tithreaded applications. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |MT-Level |MT-Safe | +-----------------------------+-----------------------------+ SEE ALSO
attributes(5) SunOS 5.10 29 Dec 1996 bufsplit(3GEN)
All times are GMT -4. The time now is 07:43 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy