Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Datatype,structure and dateformat checking.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Datatype,structure and dateformat checking.
# 1  
Old 06-10-2013
Datatype,structure and dateformat checking.
I have a sourcefile which contains data as below.I want to check whether datatype,structure and date format looks good as mentioned.

Data is delemited by cydila Ç.

Source file-Emp.txt
Code:
 
snoÇnameÇphonenoÇdeptÇjoineddate
1ÇvivekÇ0861ÇCSEÇ2013-05-29 00:00:00
2ÇdineshÇ123456ÇECEÇ2013-05-29 00:00:00
3ÇsaiÇ78945ÇEEEÇ2013-05-29 00:00:00
4ÇsridarÇ45612ÇITÇ2013-05-29 00:00:00

Defination of emp.txt file.
Code:
sno number(2)
name varchar(20)
phoneno number(10)
dept varchar(20)
joineddate date

Please suggest me which command should use to check whether datatype,structure and date format is arrived as mentioned.
Last edited by Scott; 06-10-2013 at 05:39 AM.. Reason: Fixed code tags
# 2  
Old 06-10-2013
Simple test:
Code:
awk -FÇ  '
	NR==1{next} 
	{f="ok"} 
	length($1)>2{f="err"} 
	length($2)>20{f="err"} 
	length($3)>10{f="err"} 
	length($4)>20{f="err"} 
	length($5)>19{f="err"} 
	{print f,$0}
	' file-Emp.txt

A full date check is more complicate, but possible.
# 3  
Old 06-10-2013
Code:
 
awk -FÇ '    NR==1{next}     {f="ok"}     length($1)>2{f="err"}     length($2)>20{f="err"}     length($3)>10{f="err"}     length($4)>20{f="err"}     length($5)>19{f="err"}     {print f,$0}    ' file-Emp.txt

which file i should place in above highlighted area.data file(Emp.txt) or defination file(emp_def.txt)

---------- Post updated at 02:18 AM ---------- Previous update was at 02:12 AM ----------

I want to check datatype also,i think above code will only lengh not datatype of all fields .correct me if i am wrong.
Last edited by Scott; 06-10-2013 at 05:40 AM.. Reason: Fixed code tags
# 4  
Old 06-10-2013
File that have the data you like to validate.
I have all definition in awk, not a separate file.

---------- Post updated at 09:27 ---------- Previous update was at 09:18 ----------

Some like this:

Code:
awk -FÇ  '
	NR==1{next}
	{f="ok"}
	length($1)>2{f="err"}
	$1~!/[0-9]+/{f="err"}
	length($2)>20{f="err"}
	$2~!/[0-9a-zA-Z]+/{f="err"}
	length($3)>10{f="err"}
	$3~!/[0-9]+/{f="err"}
	length($4)>20{f="err"}
	$4~!/[0-9a-zA-Z]+/{f="err"}
	length($5)>19{f="err"}
	{print f,$0}
	' file

---------- Post updated at 10:03 ---------- Previous update was at 09:27 ----------

With date test
NB this uses POSIX functions.
Code:
awk --posix -FÇ  '
	NR==1{next}
	{f="ok"}
	$1!~/^[0-9]{1,2}$/{f="s-err"}
	$2!~/^[a-zA-Z0-9]{1,20}$/{f="n-err"}
	$3!~/^[0-9]{1,10}$/{f="p-err"}
	$4!~/^[a-zA-Z0-9]{1,20}$/{f="de-err"}
	$5!~/^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}$/{f="da-err"}
	{print f,$0}
	' t
ok 1ÇvivekÇ0861ÇCSEÇ2013-05-29 00:00:00
ok 2ÇdineshÇ123456ÇECEÇ2013-05-29 00:00:00
ok 3ÇsaiÇ78945ÇEEEÇ2013-05-29 00:00:00
ok 4ÇsridarÇ45612ÇITÇ2013-05-29 00:00:00

This User Gave Thanks to Jotne For This Post:
katakamvivek
# 5  
Old 06-13-2013
Quote:
length=2
awk '{print $length}' file
awk does not read variable this way, you need to do like this
awk -v var=$length '{print var}' file
or
awk '{print var}' var=$length file
# 6  
Old 06-13-2013
Like above file i have 10 different files which contains more that 700+ columns so it is very tuff define lenght and datatype for all columns so i want to get details from source defination file file then compare with source file emp.txt.

emp.txt(Source file)

Code:
snoÇnameÇphonenoÇdeptÇjoineddate
1ÇvivekÇ0861ÇCSEÇ2013-05-29 00:00:00
2ÇdineshÇ123456ÇECEÇ2013-05-29 00:00:00
3ÇsaiÇ78945ÇEEEÇ2013-05-29 00:00:00
4ÇsridarÇ45612ÇITÇ2013-05-29 00:00:00

souce defination file
Code:
sno number(2)
name varchar(20)
phoneno number(10)
dept varchar(20)
joineddate date

Last edited by Scott; 06-13-2013 at 05:40 AM.. Reason: Fixed code tags again
# 7  
Old 06-13-2013
You can use match to create what you whant.
Code:
sno="^[0-9]{1,2}$"
awk --posix 'match($1,p1){print}' p1=$sno

You can then make a script that reads variable from a file and use it in awk
This is just for you to expand to your script.

---------- Post updated at 11:24 ---------- Previous update was at 10:28 ----------

Here is one way to do it. I have changed to tab separator to make it more easy.
Errors in red.
Code:
cat data
1       vivek   0861    CSE     2013-05-29 00:00:00
2       dinesh  1234A   ECE     2013-05-29 00:00:00
352     sai     78945   EEE     2013-05-29 00:00:00
4       sridar  45612   IT      201A-05-29 00:00:00

Code:
cat pat
sno     ^[0-9]{1,2}$
name    ^[a-zA-Z0-9]{1,20}$
phoneno ^[0-9]{1,10}$
dept    ^[a-zA-Z0-9]{1,20}$
date    ^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}$

Code:
awk --posix -F"\t" '
	FNR==NR {pat[$1]=$2;next}
	match($1,pat["sno"])==0 {f=f FNR "-sno "}
	match($2,pat["name"])==0 {f=f FNR "-name "}
	match($3,pat["phoneno"])==0 {f=f FNR "-phoneno "}
	match($4,pat["dept"])==0 {f=f FNR "-dept "}
	match($5,pat["date"])==0 {f=f FNR "-date "}
	END {print f}
	' pat data
2-phoneno 3-sno 4-date

It then list what line and what error it has.
This User Gave Thanks to Jotne For This Post:
katakamvivek
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Datatype and length validation

I have sourcefile and structure of source file,i want to check whether datatype and length mention in emp.txt is same as source file. Example: in emp.txt first row contains sno number so in source file also first column should contain only number if data is other than number then that... (1 Reply)
Discussion started by: katakamvivek
1 Replies

2. Shell Programming and Scripting

Datatype file validation

I have a sourcefile which contains data as below.I want to check whether datatype,structure and date format looks good as mentioned. Data is delemited by cydila . Source file-Emp.txt sno name phoneno dept joineddate 1 vivek 0861 CSE 2013-05-29 00:00:00 2 dinesh 123456 ECE ... (2 Replies)
Discussion started by: katakamvivek
2 Replies

3. Programming

Understanding C++ template partial specialization with pointer datatype arguments

When I compile the below code, I am getting error as template<typename T> T AddFun(T i, T j) { return i + j; } template<> T* AddFun<T*>(T* i, T* j) { return new T(*i + *j); } int main() { int n = AddFun<int>(10, 20); int i = 10, j = 20; int* p = AddFun<int*>(&i,... (1 Reply)
Discussion started by: royalibrahim
1 Replies

4. Shell Programming and Scripting

How do I rename list of files with dateformat

Hello, I have a list of files like -rw-rw-r-- 1 rails rails 8463005 Jul 27 04:02 find_or.log.3.gz -rw-rw-r-- 1 rails rails 33786339 Jul 27 04:02 pro.log.10.gz -rw-rw-r-- 1 rails rails 44815467 Aug 3 04:02 pro.log.9.gz -rw-rw-r-- 1 rails rails 81562896 Aug 4 04:02 pro.log.8.gz... (7 Replies)
Discussion started by: ashokvpp
7 Replies

5. Shell Programming and Scripting

Help with separating datatype, column name

Hi All, I am new to unix but have a requirement wherein I need to separate datatype,length, and column name from input file which is of below format -- record integer(10) empid; string(25) name; date("YYYY-MM-DD") dob; decimal(10) salary; end now after getting datatype,its length and... (4 Replies)
Discussion started by: phoenix09
4 Replies

6. Shell Programming and Scripting

Oracle DataBase DateFormat

I am currently writing a script in AIX I connected to the database using SQL *PLUS I need to retrive data of Number of rows inserted today in the table(TABLE_NAME) which has column DATETIME(DDMMYYYY) I tried using select TO_CHAR(sysdate, 'DDMMYYYY') CHRDATE from dual select count (*)... (2 Replies)
Discussion started by: PhAnT0M
2 Replies

7. Programming

How to get the size of the datatype passed as the command line argumet?

#include <stdio.h> int main(int argc, char *argv) { printf("%d\n", sizeof(argv)); return 0; } when I run the executable a.out after compiling the above program as: a.out short (or) a.out "long double", I expected to get the output as 2 and 12, but I am always getting the size of... (2 Replies)
Discussion started by: royalibrahim
2 Replies

8. Programming

Search attributes in one structure using the values from another structure

Hello Groups I am trying to find out ways of comparing a value from a 'c' structure to a value in another 'C' structure. the 'C' structure can be a List or liked list as it contains lot many records. if we loop it in both the structures it is going to consume time. I am looking for a simple... (3 Replies)
Discussion started by: dhanamurthy
3 Replies

9. UNIX for Advanced & Expert Users

MV files from one directory structure(multiple level) to other directory structure

Hi, I am trying to write a script that will move all the files from source directory structure(multiple levels might exist) to destination directory structure. If a sub folder is source doesnot exist in destination then I have to skip and goto next level. I also need to delete the files in... (4 Replies)
Discussion started by: srmadab
4 Replies

10. UNIX for Dummies Questions & Answers

Copying a Directory Structure to a new structure

Hi all Is it possible to copy a structure of a directory only. e.g. I have a file with the following entries that is a result of a find :- /dir1/dir2/file.dbf /dir1/dir2/dir3/file1.dbf /dir1/file.dbf I want to copy these to a directory and keep the structure however starting at a new dir... (8 Replies)
Discussion started by: jhansrod
8 Replies
Login or Register to Ask a Question