Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Source data validation
Top Forums Shell Programming and Scripting Source data validation Post 302952098 by katakamvivek on Thursday 13th of August 2015 03:09:59 PM
Old 08-13-2015
Source data validation
I am using below logic to validate whether i am expecting the correct data from source,if not logic should give which column has error.i am running below logic in linux

Code:
awk -F, '
NR==1{next}
{f=" "}
$1!~/^[0-9. -]{0,5}$|^$/{f=f?f" emp_id-error":"emp_id-error"}
$4!~/^[0-9 -]{0,6}$|^$/{f=f?f" tot_amt-error":"tot_amt-error"}
$5!~/^[0-9" "]{0,4}-[0-9" "]{0,2}-[0-9" "]{0,2} [0-9" "]{0,2}:[0-9" "]{0,2}:[0-9" "]{0,2}{0,29}$|^$/{f=f?f" current_time-error":"current_time-error"}
{print f,$0}
' /home/vivek/Documents/datagentest_10.txt

datagentest_10.txt file

Code:
emp_id,emp_name,market_name,tot_amt,current_time
1,Brandon Buckner,avil,526,2015-08-11 00:47:00
11,ram,avil,633,2015-08-11 00:47:00
11,raffm,avil,633,2015-08-11 00:47:00
11,ramrtt,avil,633,2015-08-11 00:47:00

data looks good but i am getting error for all columns and for all rows.please help me.


Code:
error message:

emp_id-error tot_amt-error current_time-error 1,Brandon Buckner,avil,526,2015-08-11 00:47:00

Moderator's Comments:
Mod Comment Note that after receiving ten infractions for not using CODE tags (as required by forum rules) and a few other infractions as well, the submitter has been placed in read-only mode for two weeks.
Last edited by Don Cragun; 08-13-2015 at 04:51 PM.. Reason: Add CODE and ICODE tags.
 

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

awk data validation

Hi , This is a general doubt.... Is there any way to do data validation inside an awk script.. Let me make it more clear.... I have a string variable inside awk script .. Is there any way to check whether it is number or string etc... Thanks in advance. Shihab (1 Reply)
Discussion started by: shihabvk
1 Replies

2. UNIX for Dummies Questions & Answers

Data Validation

Hello, I am trying to use data validation with a program. I have everything else working fine. I just can't figure out what I am doing wrong with the data validation in one of my files. Here is the code: # data validation loop while : do # get input from keyboard ... (2 Replies)
Discussion started by: ericelysia
2 Replies

3. Shell Programming and Scripting

validation of data using filter (awk or other that works...) in csv files

Hello People I have the following file.csv: date,string,float,number,boolean 20080303,abc,1.5,123,Y 20080304,abc,1.2,345,N 20080229,nvh,1.4,098,Y 20080319,ugy,1.9,586,N 20080315,gyh,2.4,345,Y 20080316,erf,3.1,932,N I need to filter the date field where I have a data bigger than I... (1 Reply)
Discussion started by: Rafael.Buria
1 Replies

4. Shell Programming and Scripting

shell script data & time validation

How to validate a date and optionly a time in shell scripting when i get the date and time as pararmeters that sent out with the call of the file? (in my case sh union.sh `first parameter ,second parameter...` (4 Replies)
Discussion started by: tal
4 Replies

5. Shell Programming and Scripting

Data Validation

I have a comma delimited file that I need to validate the data in one two columns in. Sample File: 1234,1234,1234,DESCRIPTION,1,1,2 1234,1234,1234,DESCRIPTION,1,1,2 1234,1234,1234,DESCRIPTION,1,1,2 1234,1234,1234,DESCRIPTION,1,1,2 I need to make sure that the second column's entries are... (3 Replies)
Discussion started by: hmnetonline
3 Replies

6. Shell Programming and Scripting

File and Data Validation.

Hello, I am working on an interface between a legacy system and an ERP system.The format of the data extracted into the staging folder from the legacy system is a follows. One control file named ExtractDataControl.txt Multiple entity files eg R1001.txt, R1002.txt, R1020.txt The control... (5 Replies)
Discussion started by: Savio_Saldanha
5 Replies

7. Shell Programming and Scripting

basic data validation

hpux. older version. don't have alot of the newer features in some utilities. How do I verify that a variable starts with the letter A. I would like to make it case insensitive. Also, if I have a variable that has letters and numbers. I want to change all the lower case letters to upper case.... (2 Replies)
Discussion started by: guessingo
2 Replies

8. Shell Programming and Scripting

Data validation engine

Generic Data validator Data file: Name,Sal,Dept ABC,1234,D1 AYX,12356,D2 DHF,345,ED3 123,4565,FGJG Config File: Delimiter-"," Rule1-Name- Rule2-Sal- Rule3-Dept-* Can be used to match any regex including date different format and numbers. (3 Replies)
Discussion started by: dikesm
3 Replies

9. Shell Programming and Scripting

Help With UNIX Shell Scripting For Data Validation

Hi All, I am completely new to Unix Shell Scripting. I m validating(Basic File Validation) an .HHT file in TIBCO. After that i need to do Data Validation through UNIX Shell scripting. Rules in DataValidation: 1.) Need to Check Every field wheather it is a Char or Number?(Fields are... (1 Reply)
Discussion started by: Chaitanya K
1 Replies

LEARN ABOUT MOJAVE

locale::codes::langfam

Locale::Codes::LangFam(3pm)				 Perl Programmers Reference Guide			       Locale::Codes::LangFam(3pm)

NAME

       Locale::Codes::LangFam - standard codes for language extension identification

SYNOPSIS

	  use Locale::Codes::LangFam;

	  $lext = code2langfam('apa');		       # $lext gets 'Apache languages'
	  $code = langfam2code('Apache languages');    # $code gets 'apa'

	  @codes   = all_langfam_codes();
	  @names   = all_langfam_names();

DESCRIPTION

       The "Locale::Codes::LangFam" module provides access to standard codes used for identifying language families, such as those as defined in
       ISO 639-5.

       Most of the routines take an optional additional argument which specifies the code set to use. If not specified, the default ISO 639-5
       language family codes will be used.

SUPPORTED CODE SETS

       There are several different code sets you can use for identifying language families. A code set may be specified using either a name, or a
       constant that is automatically exported by this module.

       For example, the two are equivalent:

	  $lext = code2langfam('apa','alpha');
	  $lext = code2langfam('apa',LOCALE_LANGFAM_ALPHA);

       The codesets currently supported are:

       alpha
	   This is the set of three-letter (lowercase) codes from ISO 639-5 such as 'apa' for Apache languages.

	   This is the default code set.

ROUTINES

       code2langfam ( CODE [,CODESET] )
       langfam2code ( NAME [,CODESET] )
       langfam_code2code ( CODE ,CODESET ,CODESET2 )
       all_langfam_codes ( [CODESET] )
       all_langfam_names ( [CODESET] )
       Locale::Codes::LangFam::rename_langfam  ( CODE ,NEW_NAME [,CODESET] )
       Locale::Codes::LangFam::add_langfam  ( CODE ,NAME [,CODESET] )
       Locale::Codes::LangFam::delete_langfam  ( CODE [,CODESET] )
       Locale::Codes::LangFam::add_langfam_alias  ( NAME ,NEW_NAME )
       Locale::Codes::LangFam::delete_langfam_alias  ( NAME )
       Locale::Codes::LangFam::rename_langfam_code  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Codes::LangFam::add_langfam_code_alias  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Codes::LangFam::delete_langfam_code_alias  ( CODE [,CODESET] )
	   These routines are all documented in the Locale::Codes::API man page.

SEE ALSO

       Locale::Codes
	   The Locale-Codes distribution.

       Locale::Codes::API
	   The list of functions supported by this module.

       http://www.loc.gov/standards/iso639-5/id.php
	   ISO 639-5 .

AUTHOR

       See Locale::Codes for full author history.

       Currently maintained by Sullivan Beck (sbeck@cpan.org).

COPYRIGHT

	  Copyright (c) 2011-2013 Sullivan Beck

       This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

perl v5.18.2							    2013-11-04					       Locale::Codes::LangFam(3pm)
All times are GMT -4. The time now is 09:11 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy