Date validity check


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Date validity check
# 1  
Old 11-11-2013
Date validity check

hi All,

i have file in which it has 2000 records like,

test.txt
====
Code:
2011-03-01
2011-03-01
2011-03-01
2011-03-01
2011-03-01
2011-03-02
2011/03/02

previously i used for loop to find the date check like below,
Code:
for i in `cat test.txt`
do
d=`echo $i | cut -c9-10| sed 's/^0*//'`;
m=`echo $i | cut -c6-7`;
Y=`echo $i | cut -c1-4`;
if cal $m $Y| tr -s " " "|" | tr -s "\n" "|" | grep $d > /dev/null 2>&1;
then 
a=1;
else 
echo "N" ; 
fi
done


but it is taking so much time when the file has 7000 records,i need a command to find the whetehr any invalid date is there in the file.if any one of the date is invalid i need to return a flag.

please help asap.

Last edited by Franklin52; 11-11-2013 at 05:57 PM.. Reason: Please use code tags
# 2  
Old 11-11-2013
With GNU date:
Code:
while read d
do
    date -d "$d" >/dev/null 2>&1
    [ $? -ne 0 ] && echo $d
done < test.txt

# 3  
Old 11-11-2013
I'm assuming the bad date format is the last one in your test.txt output above. This will print the line number and the bad date format.

Code:
perl -ne 'print "Line #$.: $1\n" if(/(\d+\/\d+\/\d+)/)' test.txt
Line #9: 2011/03/02

# 4  
Old 11-11-2013
You call many external commands (cut, sed etc.) which all take take to start a sub process and this is what is costing you.

I'm aware that date -d is not available in all implementations. What OS are you using?

Trying to avoid being OS specific (and not the neatest code) could you consider this:-
Code:
$ cat test.txt
2011-03-01
2011-03-01
2011-03-01
2011-03-01
2011-03-01
2011-03-02
2011/03/02
$ (IFS=-;tr "\/" "-" < test.txt | while read d m Y
> do
>    echo "I got \$d=\"$d\", \$m=\"$m\" and \$Y=\"$Y\""
> done)
I got $d="2011", $m="03" and $Y="01"
I got $d="2011", $m="03" and $Y="01"
I got $d="2011", $m="03" and $Y="01"
I got $d="2011", $m="03" and $Y="01"
I got $d="2011", $m="03" and $Y="01"
I got $d="2011", $m="03" and $Y="02"
I got $d="2011", $m="03" and $Y="02"
$

If you can be sure that you don't actually have any / as date separators in your input (as you posted, possibly in error) then it simplifies to just:-
Code:
$ (IFS=- ; while read d m Y
> do
>    echo "I got \$d=\"$d\", \$m=\"$m\" and \$Y=\"$Y\""
> done <test.txt)


I'm not quite sure what you are trying with the remainder. Are you trying to validate that it is an acceptable date?

If you are sure you are getting just numerics, that might be better as a case statement like this:-
Code:
case $m in
   01) mxd=31 ;;
   02) ((a=($Y%4)/$Y)) 2>/dev/null ;;        # Handles leap year
   03) mxd=31 ;;
   04) mxd=30 ;;
   05) mxd=31 ;;
   06) mxd=30 ;;
   07) mxd=31 ;;
   08) mxd=31 ;;
   09) mxd=30 ;;
   10) mxd=31 ;;
   11) mxd=30 ;;
   12) mxd=31 ;;
   *) echo "Invalid month" ; exit 99
esac

if [ $d -gt $mxd -o $d -lt 1 ]
then
   echo "Invalid day" ; exit 99
fi


Of course, if you are just looking for the right format, then:-
Code:
grep -v "^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]$" test.txt




I hope that this helps,
Robin
Liverpool/Blackburn
UK
# 5  
Old 11-12-2013
hi all,

my requirement is that in a file only dates will be there, in that i have to find whether any invalid date is there. if any invalid date is there i have to just get the result the file is invalid.

the correct format of date is "2011-03-13"(YYYY-MM-DD).

the version and os i am using is

SunOS upp21n 5.10 Generic_148888-05 sun4u sparc SUNW,SPARC-Enterprise
# 6  
Old 11-12-2013
Python makes this task insanely simple:
Code:
>>> import time
>>> mask = '%Y-%m-%d'
>>> if time.strptime('2013-12-12', mask):                                                       
...    print "date ok"                                                                          ... 
date ok

Complete script:
Code:
#!/usr/bin/python
import time 
def check_mask ( datec,mask='%Y-%m-%d' ):
    try: 
        time.strptime(datec, mask)
    except ValueError:
        return False
    return True

f = open('./test.txt','rb')
for d in f:
   if not check_mask(d.rstrip()):
      print "Invalid: %s" % d,
f.close()


Last edited by Klashxx; 11-12-2013 at 07:00 AM..
# 7  
Old 11-12-2013
Okay, so I can refine simple grep then apply the other parts in sequence. The first test is for basic structural validation, then it follows with a date check without calling date or cal which will slow down processing on a large file too.

Code:
#!/bin/ksh
echo "Basic invalid formatted dates found:-"
grep -v "^[0-9][0-9][0-9][0-9]-[0-1][0-9]-[0-3][0-9]$" test.txt | while read line
do
   echo "\t$line"
done

echo "\nLooking for invalid date values:-"
(IFS=-
typeset -i d m Y mxd L
grep "^[0-9][0-9][0-9][0-9]-[0-1][0-9]-[0-3][0-9]$" test.txt | while read Y m d
do
   case $m in
      1)  mxd=31 ;;
      2)  ((L=($Y%4)/($Y%4))) 2>/dev/null ; ((mxd=29-$L)) ;; # Handles leap year
      3)  mxd=31 ;;
      4)  mxd=30 ;;
      5)  mxd=31 ;;
      6)  mxd=30 ;;
      7)  mxd=31 ;;
      8)  mxd=31 ;;
      9)  mxd=30 ;;
      10) mxd=31 ;;
      11) mxd=30 ;;
      12) mxd=31 ;;
      *)  mxd=0  ;;
   esac

   if [ ${d} -gt ${mxd} -o ${d} -lt 1 ]
   then
      echo "\tInvalid date found \"${Y}-${m}-${d}\""
   fi
done)

The echo statements man need to be replaced with printf depending on your OS.

I must say that I like the Python way, if you have that.

Additionally, if you have a database, that may have similar tools.



I hope that this helps,
Robin
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Confirming validity of programming language tools

so i have scripts that get run in ways similar to this: cat script.pl | perl - $1 $2 $3 cat script.rb | ruby - $1 $ 2 $3 my question is, how can i verify that that the "perl" or "ruby" or "python" tool being run on the box is actually a legit tool? meaning, someone may move the tool from... (2 Replies)
Discussion started by: SkySmart
2 Replies

2. Shell Programming and Scripting

Check, if date is not today

hello, in a file exist entries in date format YYYYMMDD. i want to find out, if there are dates, which isn't today's date. file: date example text 20140714 <= not today's date 20140715 <= not today's date 20140716 <= today's date my idea is to use Perderabo's datecalc ... (2 Replies)
Discussion started by: bora99
2 Replies

3. Shell Programming and Scripting

Perl code to check date and check files in particular dir

Hi Experts, I am checking how to get day in Perl. If it is “Monday” I need to process…below is the pseudo code. Can you please prove the code for below condition. if (today=="Monday" ) { while (current_time LESS THAN 9:01 AM) ... (1 Reply)
Discussion started by: ajaypatil_am
1 Replies

4. Shell Programming and Scripting

Check if a date field has date or timestamp or date&timestamp

Hi, In a field, I should receive the date with time stamp in a particular field. But sometimes the vendor sends just the date or the timestamp or correctl the date&timestamp. I have to figure out the the data is a date or time stamp or date&timestamp. If it is date then append "<space>00:00:00"... (1 Reply)
Discussion started by: machomaddy
1 Replies

5. Shell Programming and Scripting

finding date numeral from file and check the validity of date format

hi there I have file names in different format as below triss_20111117_fxcb.csv triss_fxcb_20111117.csv xpnl_hypo_reu_miplvdone_11172011.csv xpnl_hypo_reu_miplvdone_11-17-2011.csv xpnl_hypo_reu_miplvdone_20111117.csv xpnl_hypo_reu_miplvdone_20111117xfb.csv... (10 Replies)
Discussion started by: manas_ranjan
10 Replies

6. Shell Programming and Scripting

Script to check the validity of password

Hi, I have to validate the passwords for 100s of unix users across several servers. I have the list of unix users and servers with passwrods. How can I check whether a password is correct or not using a single shell script? Note : I do not have root privileges on any server. All the... (1 Reply)
Discussion started by: Pupil
1 Replies

7. Shell Programming and Scripting

Check for date

How to validate the first line from 1-8 position of audit file that contains the script run date... script could run in random dates. head -1 file1 20090516 100034837SHDHSHE (9 Replies)
Discussion started by: ford2020
9 Replies

8. Shell Programming and Scripting

How to check date variable according to the current date?

Hi folks, I need to write a script that should activate a process according to the current hour. The process should be activatet only if the hour is between midnight (00:00) and 07:00. How should I create the condition? Thanks in advance, Nir (2 Replies)
Discussion started by: nir_s
2 Replies

9. Shell Programming and Scripting

checking for command output validity

hi, i'm trying to write a script to check if the home directories of users are set correctly. below is an extract of the script here, i am trying to put the name of the owner of the home directory into the variable dirperm (by reading lines in /etc/passwd). however, it seems that when the... (1 Reply)
Discussion started by: roddo90
1 Replies

10. Shell Programming and Scripting

Validity

Hey, I was wondering how I could write a bash script which accepts: cat <<% | bash ./results06 ---------------------------------------------------------- Exam Results 2006 ---------------------------------------------------------- ... (1 Reply)
Discussion started by: Xerobeat
1 Replies
Login or Register to Ask a Question