Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Data Validation

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Data Validation
# 1  
Old 07-15-2009
Data Validation
I have a comma delimited file that I need to validate the data in one two columns in.

Sample File:

1234,1234,1234,DESCRIPTION,1,1,2
1234,1234,1234,DESCRIPTION,1,1,2
1234,1234,1234,DESCRIPTION,1,1,2
1234,1234,1234,DESCRIPTION,1,1,2

I need to make sure that the second column's entries are numeric. If they have an alpha character, I need to delete that line of data and write it to another file.

I'm not sure where to get started, and any advice would be appreciated.

Thanks!
# 2  
Old 07-15-2009
Code:
#!/bin/ksh

validate()
{
    val="$2"
    # remove all numbers = some left = not numbers
    # new=$( echo "$val" | tr -d "[0-9]" )
    # without external command:
    new=${val//[0-9]/}
    [ "$new" != "" ] && echo "$*" >&2 && return 1
    echo "$*"
}

oifs="$IFS"
while read line
do
     IFS=","
     validate $line
     IFS="$oifs"
done

And use this script:
Code:
cat file | ./thisscript   > ok.txt  2>err.txt

# 3  
Old 07-15-2009
You can try alaso awk solution:
Code:
awk -v FS=',' -v OFS=',' '{ 
	if ($2 ~ /^[0-9]*$/)
		print $0
	else
		print $0 >> "err_file"
}' input_file

# 4  
Old 07-15-2009
try :

Code:
sed -n '/.*,[0-9]*,.*,.*,.*,.*,.*/p' filename

Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Source data validation

I am using below logic to validate whether i am expecting the correct data from source,if not logic should give which column has error.i am running below logic in linux awk -F, ' NR==1{next} {f=" "} $1!~/^{0,5}$|^$/{f=f?f" emp_id-error":"emp_id-error"} $4!~/^{0,6}$|^$/{f=f?f"... (4 Replies)
Discussion started by: katakamvivek
4 Replies

2. Shell Programming and Scripting

Help With UNIX Shell Scripting For Data Validation

Hi All, I am completely new to Unix Shell Scripting. I m validating(Basic File Validation) an .HHT file in TIBCO. After that i need to do Data Validation through UNIX Shell scripting. Rules in DataValidation: 1.) Need to Check Every field wheather it is a Char or Number?(Fields are... (1 Reply)
Discussion started by: Chaitanya K
1 Replies

3. Shell Programming and Scripting

Data validation engine

Generic Data validator Data file: Name,Sal,Dept ABC,1234,D1 AYX,12356,D2 DHF,345,ED3 123,4565,FGJG Config File: Delimiter-"," Rule1-Name- Rule2-Sal- Rule3-Dept-* Can be used to match any regex including date different format and numbers. (3 Replies)
Discussion started by: dikesm
3 Replies

4. Shell Programming and Scripting

basic data validation

hpux. older version. don't have alot of the newer features in some utilities. How do I verify that a variable starts with the letter A. I would like to make it case insensitive. Also, if I have a variable that has letters and numbers. I want to change all the lower case letters to upper case.... (2 Replies)
Discussion started by: guessingo
2 Replies

5. Shell Programming and Scripting

File and Data Validation.

Hello, I am working on an interface between a legacy system and an ERP system.The format of the data extracted into the staging folder from the legacy system is a follows. One control file named ExtractDataControl.txt Multiple entity files eg R1001.txt, R1002.txt, R1020.txt The control... (5 Replies)
Discussion started by: Savio_Saldanha
5 Replies

6. Shell Programming and Scripting

shell script data & time validation

How to validate a date and optionly a time in shell scripting when i get the date and time as pararmeters that sent out with the call of the file? (in my case sh union.sh `first parameter ,second parameter...` (4 Replies)
Discussion started by: tal
4 Replies

7. Shell Programming and Scripting

validation of data using filter (awk or other that works...) in csv files

Hello People I have the following file.csv: date,string,float,number,boolean 20080303,abc,1.5,123,Y 20080304,abc,1.2,345,N 20080229,nvh,1.4,098,Y 20080319,ugy,1.9,586,N 20080315,gyh,2.4,345,Y 20080316,erf,3.1,932,N I need to filter the date field where I have a data bigger than I... (1 Reply)
Discussion started by: Rafael.Buria
1 Replies

8. UNIX for Dummies Questions & Answers

Data Validation

Hello, I am trying to use data validation with a program. I have everything else working fine. I just can't figure out what I am doing wrong with the data validation in one of my files. Here is the code: # data validation loop while : do # get input from keyboard ... (2 Replies)
Discussion started by: ericelysia
2 Replies

9. UNIX for Advanced & Expert Users

awk data validation

Hi , This is a general doubt.... Is there any way to do data validation inside an awk script.. Let me make it more clear.... I have a string variable inside awk script .. Is there any way to check whether it is number or string etc... Thanks in advance. Shihab (1 Reply)
Discussion started by: shihabvk
1 Replies
Login or Register to Ask a Question