Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Help With UNIX Shell Scripting For Data Validation
Top Forums Shell Programming and Scripting Help With UNIX Shell Scripting For Data Validation Post 302811965 by DGPickett on Friday 24th of May 2013 04:05:14 PM
Old 05-24-2013
Sounds like a PERL, C, C++, JAVA job. First, a file is just an array of char, and you need to parse/validate it sequentially to discover fields, separators, and whatever passes in there for null.

If you take a light weight approach, bash can read your fields into a previously typeset indexed array with "while read -a flds ; do ... done <file". You have to add '|' to $IFS to make it an additional field separator, or "tr '|' ' '" to translate the pipes to spaces (since your data normally has none and space is already in $IFS, along with tab and linefeed). You get to write the ..., and bash has regex tools in the "if [[ ... ]]" form of pattern examination, so you can ensure the field has legal length and the right type of character in the right place. You can capture and save header if you use a form like "tr '|' ' ' <file | ( read -a hdr ; while read -a flds ; do ... done )". A typical regex might be "^[a-zA-Z]\{1,22\}$" to pattern match a field with one to 22 letters only. I think the \{\} is limited to 0-99 bytes, but you can stack them.
Last edited by DGPickett; 05-24-2013 at 05:11 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

difference between AIX shell scripting and Unix shell scripting.

please give the difference between AIX shell scripting and Unix shell scripting. (2 Replies)
Discussion started by: haroonec
2 Replies

2. Shell Programming and Scripting

data formatting in Unix shell Scripting

#! /bin/ksh ############################ # AFI Monitor Script ############################ . /db2/uszlad48/sqllib/db2profile export mondir=/home/script/krishna export monlog=$mondir/Error_Report_`date +%Y%m%d`.log echo "connect to database r2pdev" >>$monlog db2 connect to r2pdev user... (1 Reply)
Discussion started by: regnumber
1 Replies

3. UNIX for Dummies Questions & Answers

How to cut data block from .txt file in shell scripting

Hi All, Currently i have to write a script. For which i need to cut a block from .txt file. I know the specific word that starts the block and ends the block. Can we do it in shell scripting..? Please suggest.... (6 Replies)
Discussion started by: pank29
6 Replies

4. Shell Programming and Scripting

shell script data & time validation

How to validate a date and optionly a time in shell scripting when i get the date and time as pararmeters that sent out with the call of the file? (in my case sh union.sh `first parameter ,second parameter...` (4 Replies)
Discussion started by: tal
4 Replies

5. Shell Programming and Scripting

Shell scripting to extract data from file

Hi, i want to fetch the data from the alert log file, for a particular time interval. Example : Alert log content : Thu Mar 18 08:47:36 2010 Completed: alter database open Thu Mar 18 19:13:38 2010 MMNL absent for 6390 secs; Foregrounds taking over Fri Mar 19 08:30:52 2010... (1 Reply)
Discussion started by: Pinki018
1 Replies

6. Shell Programming and Scripting

Extracting data from file-shell scripting--please help

hello friends, my file is like 123 |asd|asd|asd 123_1|awd|asw|asw 121 |wer|qwe|wee 124 |weq|qwe|iop 1_23 |bla|blh|bha 145 |ghj|jkl|ghj 146 |qwe|qwe|wer 154 |asd|wer|qw_e 134_5|qwe|wer|qw_e is their any solution to retrive only those lines which are having only 3 numerical letters... (20 Replies)
Discussion started by: PankajChawla
20 Replies

7. Shell Programming and Scripting

How to extract data from xml file using shell scripting?

Hi evry1, This is my 1st post in this forum.Pls help me I want to extract some data froma xml file which has 2000 lines using shell scripting. Actually my xml file has some "audio and video codes" which i need to arrange in a column wise format after extracting it using shell scripting.I... (4 Replies)
Discussion started by: arun_kohan
4 Replies

8. Shell Programming and Scripting

How to extract data from XML file using shell scripting?

Hi , I have input file as XML. following are input data #complex.xml Code: <?xml version="1.0" encoding="UTF-8"?><TEST_doc xmlns="http://www.w3.org/2001/XMLSchema-instance"> <ENTRY uid="123456"> <protein> <name>PROT001</name> <organism>Human</organism> ... (1 Reply)
Discussion started by: arun_kohan
1 Replies

9. UNIX for Dummies Questions & Answers

Shell script to read lines in a text file and filter user data Shell Programming and Scripting

sxsaaas (3 Replies)
Discussion started by: VikrantD
3 Replies

10. Shell Programming and Scripting

Read csv file, convert the data and make one text file in UNIX shell scripting

I have input data looks like this which is a part of a csv file 7,1265,76548,"0102:04" 8,1266,76545,"0112:04" I need to make the output data should look like this and the output data will be part of text file: 7|1265000 |7654899 |A| 8|12660000 |76545999 |B| The logic behind the... (6 Replies)
Discussion started by: RJG
6 Replies

LEARN ABOUT CENTOS

cpanplus::shell

CPANPLUS::Shell(3)					User Contributed Perl Documentation					CPANPLUS::Shell(3)

NAME

       CPANPLUS::Shell - base class for CPANPLUS shells

SYNOPSIS

	   use CPANPLUS::Shell; 	    # load the shell indicated by your
					    # config -- defaults to
					    # CPANPLUS::Shell::Default

	   use CPANPLUS::Shell qw[Classic]  # load CPANPLUS::Shell::Classic;

	   my $ui      = CPANPLUS::Shell->new();
	   my $name    = $ui->which;	    # Find out what shell you loaded

	   $ui->shell;			    # run the ui shell

DESCRIPTION

       This module is the generic loading (and base class) for all "CPANPLUS" shells. Through this module you can load any installed "CPANPLUS"
       shell.

       Just about all the functionality is provided by the shell that you have loaded, and not by this class (which merely functions as a generic
       loading class), so please consult the documentation of your shell of choice.

BUG REPORTS

       Please report bugs or other issues to <bug-cpanplus@rt.cpan.org<gt>.

AUTHOR

       This module by Jos Boumans <kane@cpan.org>.

COPYRIGHT

       The CPAN++ interface (of which this module is a part of) is copyright (c) 2001 - 2007, Jos Boumans <kane@cpan.org>. All rights reserved.

       This library is free software; you may redistribute and/or modify it under the same terms as Perl itself.

SEE ALSO

       CPANPLUS::Shell::Default, CPANPLUS::Shell::Classic, cpanp

perl v5.16.3							    2013-05-20							CPANPLUS::Shell(3)
All times are GMT -4. The time now is 09:46 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy