UNIX for Dummies Questions & Answers

my script:
# grep -v ^# letsgo.sh
DateStart="2012/01/01"
DateEnd="2012/04/15"
start=`date +%s -d "$DateStart"`
end=`date +%s -d "$DateEnd"`
gawk -F '^"|","|"$' -v start=$start '{cmd="date +%s -d \"" $10 "\""; cmd | getline current; close(cmd)} current > start ' file.csv > temp.file
# grep -v ^# letsgo.sh

my basic file contains very similiar records like below:

Code:

# cat lol.tmp
"aaaaa","bbbbbb","ccccc","ddddd,eeeee","ffffff","ggggg","3","2012/02/22 12:25:21","2012/02/22 00:00:00","8","hhhh","iiii"
#

I need to count records which have second date (in above example it is "2012/02/22 00:00:00") between given dates (variables $end and $start) and redirect result to temporary file for future modifications.

I saw ready scripts for compare dates (loop in loop), but I thought it will be faster to change dates into unix timestamp and compare. I guess awk is a best tool to such operation, but now I have doubts ;/

If you have better idea - I am willing to listen

Thanks in advance for any tips.

Kind regards

Thanks for your reply but I'm afraid you failed to answer to the questions I asked.
Here they are again:

1. this might be sample input file:

Code:

"aaaaa","bbbbbb","ccccc","ddddd,eeeee","ffffff","ggggg","3","2012/02/22 12:25:21","2012/02/22 00:00:00","8","hhhh","iiii"
"aaaaa","bbbbbb","ccccc","ddddd,eeeee","ffffff","ggggg","2","2012/02/22 12:25:21","2011/02/22 00:00:00","8","hhhh","iiii"
"aaaaa","bbbbbb","ccccc","ddddd,eeeee","ffffff","ggggg","2","","2010/02/22 00:00:00","8","hhhh","iiii"
"aaaaa","bbbbbb","ccccc","ddddd,eeeee","ffffff","ggggg","1","2012/02/22 12:25:21","2009/02/22 00:00:00","8","hhhh","iiii"

2. no order. only a lots of records (at least 40 MB)

Is there always a colon inside field 4 ?
Is the HH:MM:SS always 00:00:00 in field 9 ?

1. yes, colon might appear in field 4 ( generally this field contain input from user, so here can be everything.... thats why I have weird separator in awk)
2. field 9 always contains "00:00:00". however if it is possible - please do not skip this part. In my next script I have to work with fields where this part is different then "00:00:00"

Here is something probably much faster than your previous attempts:

Code:

startDate="2012/01/01 00:00:00"
endDate="2012/04/15 00:00:00"
gawk -F'","' \
    -v start=${startDate//[\/ :]/} \
    -v end=${endDate//[\/ :]/} '
{
    a=gensub("[/ :]","","g",$9);
    if((a>=start)&&(a<=end))
        print
}' file.csv > extract.csv

LOL ...its working....it is ultra-fast !

Many thanks for help jlliagre ! ( now I have to slowly read and understand your code ;p )

Best regards,
Paweł