Sponsored Content
Full Discussion: Data validation engine
Top Forums Shell Programming and Scripting Data validation engine Post 302785909 by DGPickett on Tuesday 26th of March 2013 02:16:05 PM
Old 03-26-2013
Shouldn't that be '123,4565,FGJG,Rule1;Rule3'?
 

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

awk data validation

Hi , This is a general doubt.... Is there any way to do data validation inside an awk script.. Let me make it more clear.... I have a string variable inside awk script .. Is there any way to check whether it is number or string etc... Thanks in advance. Shihab (1 Reply)
Discussion started by: shihabvk
1 Replies

2. UNIX for Dummies Questions & Answers

Data Validation

Hello, I am trying to use data validation with a program. I have everything else working fine. I just can't figure out what I am doing wrong with the data validation in one of my files. Here is the code: # data validation loop while : do # get input from keyboard ... (2 Replies)
Discussion started by: ericelysia
2 Replies

3. Shell Programming and Scripting

validation of data using filter (awk or other that works...) in csv files

Hello People I have the following file.csv: date,string,float,number,boolean 20080303,abc,1.5,123,Y 20080304,abc,1.2,345,N 20080229,nvh,1.4,098,Y 20080319,ugy,1.9,586,N 20080315,gyh,2.4,345,Y 20080316,erf,3.1,932,N I need to filter the date field where I have a data bigger than I... (1 Reply)
Discussion started by: Rafael.Buria
1 Replies

4. Shell Programming and Scripting

shell script data & time validation

How to validate a date and optionly a time in shell scripting when i get the date and time as pararmeters that sent out with the call of the file? (in my case sh union.sh `first parameter ,second parameter...` (4 Replies)
Discussion started by: tal
4 Replies

5. Shell Programming and Scripting

Data Validation

I have a comma delimited file that I need to validate the data in one two columns in. Sample File: 1234,1234,1234,DESCRIPTION,1,1,2 1234,1234,1234,DESCRIPTION,1,1,2 1234,1234,1234,DESCRIPTION,1,1,2 1234,1234,1234,DESCRIPTION,1,1,2 I need to make sure that the second column's entries are... (3 Replies)
Discussion started by: hmnetonline
3 Replies

6. Shell Programming and Scripting

File and Data Validation.

Hello, I am working on an interface between a legacy system and an ERP system.The format of the data extracted into the staging folder from the legacy system is a follows. One control file named ExtractDataControl.txt Multiple entity files eg R1001.txt, R1002.txt, R1020.txt The control... (5 Replies)
Discussion started by: Savio_Saldanha
5 Replies

7. Shell Programming and Scripting

basic data validation

hpux. older version. don't have alot of the newer features in some utilities. How do I verify that a variable starts with the letter A. I would like to make it case insensitive. Also, if I have a variable that has letters and numbers. I want to change all the lower case letters to upper case.... (2 Replies)
Discussion started by: guessingo
2 Replies

8. Shell Programming and Scripting

Help With UNIX Shell Scripting For Data Validation

Hi All, I am completely new to Unix Shell Scripting. I m validating(Basic File Validation) an .HHT file in TIBCO. After that i need to do Data Validation through UNIX Shell scripting. Rules in DataValidation: 1.) Need to Check Every field wheather it is a Char or Number?(Fields are... (1 Reply)
Discussion started by: Chaitanya K
1 Replies

9. Shell Programming and Scripting

Source data validation

I am using below logic to validate whether i am expecting the correct data from source,if not logic should give which column has error.i am running below logic in linux awk -F, ' NR==1{next} {f=" "} $1!~/^{0,5}$|^$/{f=f?f" emp_id-error":"emp_id-error"} $4!~/^{0,6}$|^$/{f=f?f"... (4 Replies)
Discussion started by: katakamvivek
4 Replies
Fields(3pm)						User Contributed Perl Documentation					       Fields(3pm)

NAME
Sort::Fields - Sort lines containing delimited fields SYNOPSIS
use Sort::Fields; @sorted = fieldsort [3, '2n'], @lines; @sorted = fieldsort '+', [-1, -3, 0], @lines; $sort_3_2n = make_fieldsort [3, '2n'], @lines; @sorted = $sort_3_2n->(@lines); DESCRIPTION
Sort::Fields provides a general purpose technique for efficiently sorting lists of lines that contain data separated into fields. Sort::Fields automatically imports two subroutines, "fieldsort" and "make_fieldsort", and two variants, "stable_fieldsort" and "make_sta- ble_fieldsort". "make_fieldsort" generates a sorting subroutine and returns a reference to it. "fieldsort" is a wrapper for the "make_fieldsort" subroutine. The first argument to make_fieldsort is a delimiter string, which is used as a regular expression argument for a "split" operator. The delimiter string is optional. If it is not supplied, make_fieldsort splits each line using "/s+/". The second argument is an array reference containing one or more field specifiers. The specifiers indicate what fields in the strings will be used to sort the data. The specifier "1" indicates the first field, "2" indicates the second, and so on. A negative specifier like "-2" means to sort on the second field in reverse (descending) order. To indicate a numeric rather than alphabetic comparison, append "n" to the specifier. A specifier of "0" means the entire string ("-0" means the entire string, in reverse order). The order in which the specifiers appear is the order in which they will be used to sort the data. The primary key is first, the secondary key is second, and so on. "fieldsort [1, 2], @data" is roughly equivalent to "make_fieldsort([1, 2])->(@data)". Avoid calling fieldsort repeatedly with the same sort specifiers. If you need to use a particular sort more than once, it is more efficient to call "make_fieldsort" once and reuse the subroutine it returns. "stable_fieldsort" and "make_stable_fieldsort" are like their "unstable" counterparts, except that the items that compare the same are maintained in their original order. EXAMPLES
Some sample data (in array @data): 123 asd 1.22 asdd 32 ewq 2.32 asdd 43 rewq 2.12 ewet 51 erwt 34.2 ewet 23 erww 4.21 ewet 91 fdgs 3.43 ewet 123 refs 3.22 asdd 123 refs 4.32 asdd # alpha sort on column 1 print fieldsort [1], @data; 123 asd 1.22 asdd 123 refs 3.22 asdd 123 refs 4.32 asdd 23 erww 4.21 ewet 32 ewq 2.32 asdd 43 rewq 2.12 ewet 51 erwt 34.2 ewet 91 fdgs 3.43 ewet # numeric sort on column 1 print fieldsort ['1n'], @data; 23 erww 4.21 ewet 32 ewq 2.32 asdd 43 rewq 2.12 ewet 51 erwt 34.2 ewet 91 fdgs 3.43 ewet 123 asd 1.22 asdd 123 refs 3.22 asdd 123 refs 4.32 asdd # reverse numeric sort on column 1 print fieldsort ['-1n'], @data; 123 asd 1.22 asdd 123 refs 3.22 asdd 123 refs 4.32 asdd 91 fdgs 3.43 ewet 51 erwt 34.2 ewet 43 rewq 2.12 ewet 32 ewq 2.32 asdd 23 erww 4.21 ewet # alpha sort on column 2, then alpha on entire line print fieldsort [2, 0], @data; 123 asd 1.22 asdd 51 erwt 34.2 ewet 23 erww 4.21 ewet 32 ewq 2.32 asdd 91 fdgs 3.43 ewet 123 refs 3.22 asdd 123 refs 4.32 asdd 43 rewq 2.12 ewet # alpha sort on column 4, then numeric on column 1, then reverse # numeric on column 3 print fieldsort [4, '1n', '-3n'], @data; 32 ewq 2.32 asdd 123 refs 4.32 asdd 123 refs 3.22 asdd 123 asd 1.22 asdd 23 erww 4.21 ewet 43 rewq 2.12 ewet 51 erwt 34.2 ewet 91 fdgs 3.43 ewet # now, splitting on either literal period or whitespace # sort numeric on column 4 (fractional part of decimals) then # numeric on column 3 (whole part of decimals) print fieldsort '(?:.|s+)', ['4n', '3n'], @data; 51 erwt 34.2 ewet 43 rewq 2.12 ewet 23 erww 4.21 ewet 123 asd 1.22 asdd 123 refs 3.22 asdd 32 ewq 2.32 asdd 123 refs 4.32 asdd 91 fdgs 3.43 ewet # alpha sort on column 4, then numeric on the entire line # NOTE: produces warnings under -w print fieldsort [4, '0n'], @data; 32 ewq 2.32 asdd 123 asd 1.22 asdd 123 refs 3.22 asdd 123 refs 4.32 asdd 23 erww 4.21 ewet 43 rewq 2.12 ewet 51 erwt 34.2 ewet 91 fdgs 3.43 ewet # stable alpha sort on column 4 (maintains original relative order # among items that compare the same) print stable_fieldsort [4], @data; 123 asd 1.22 asdd 32 ewq 2.32 asdd 123 refs 3.22 asdd 123 refs 4.32 asdd 43 rewq 2.12 ewet 51 erwt 34.2 ewet 23 erww 4.21 ewet 91 fdgs 3.43 ewet BUGS
Some rudimentary tests now. Perhaps something should be done to catch things like: fieldsort '.', [1, 2], @lines; '.' translates to "split /./" -- probably not what you want. Passing blank lines and/or lines containing the wrong kind of data (alphas instead of numbers) can result in copious warning messages under "-w". If the regexp contains memory parentheses ("(...)" rather than "(?:...)"), split will function in "delimiter retention" mode, capturing the contents of the parentheses as well as the stuff between the delimiters. I could imagine how this could be useful, but on the other hand I could also imagine how it could be confusing if encountered unexpectedly. Caveat sortor. Not really a bug, but if you are planning to sort a large text file, consider using sort(1). Unless, of course, your operating system doesn't have sort(1). AUTHOR
Joseph N. Hall, joseph@5sigma.com SEE ALSO
perl(1). perl v5.8.8 2008-03-25 Fields(3pm)
All times are GMT -4. The time now is 10:50 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy