find formatted filename with date time


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users find formatted filename with date time
# 1  
Old 01-21-2008
find formatted filename with date time

Hi,

I operate and use HF radars along the California coast for ocean surface currents. The devices use Mac OS as the control and logging software. The software generates thousands of files a week and while I've used PERL in the past to solve the problems of finding files I come to realize some inherent inefficiencies with this type of search method. Ultimately I'd like to find files using the unix find command, however, my knowledge of regular expressions is in its infancy and I'm learning. So in my pursuit of this knowledge I'm turning to this forum to hopefully give me some hints.

The files fortunately are all fairly standardize -- i.e. the filename format is:

TYPE_SITE_[YY|YYYY]_MO_MD_[HHMM|HHMMSS].[xx|xxx]

where,
TYPE is a three or four character string depicting the type of data (CSS, CSQ, RDLi, STAT, etc.)
SITE is a four character string depicting the name of the radar site (GCYN, NPGS, BML1, etc.)
YYYY is a two or four digit year
MO is a two digit month
MD is a two digit month day
HH is a two digit hour
MM is a two digit minute
SS is a two digit second
xxx is a two or three character string for a file extension

The main difficulty I have is in crossing years -- i.e. if I wanted to say find CSS files for site BML1 between Dec 29th 2007 and Jan. 2nd 2008. Any advice / pointers on how to attack this using the find or grep or egrep commands would be greatly appreciated.

Thanks,
dpath2o
# 2  
Old 01-21-2008
Are the files' modification date/time the same as the filename? If so, use find with the -mtime +<days> -mtime -<days> flags to set the min and max age to look between.
# 3  
Old 01-21-2008
No unfortunately not ... File creation, access, and modification times can and often differ from the files stat ... In fact ultimately I should search for the "%TimeStamp: ..." string inside the text files to be most accurate, but once I figure this regex I think I'll be able to apply it to awk or maybe sed to read this line inside the file ... Long answer to your short question!
# 4  
Old 01-21-2008
assuming all files end in *.xtx . Using GNUawk
Code:
ls *.xtx | awk 'BEGIN { FS="[:_]"
 printf "Enter from date [yyyy mm dd]: "
 getline startymd < "/dev/tty"
 printf "Enter to date [yyyy mm dd]: "
 getline endymd < "/dev/tty"
 s=startymd " 00 00 00"
 stdate = mktime(s)
 e=endymd " 00 00 00"
 edate = mktime(e)
}
{
 find = $3" "$4" "$5" 00 00 00" 
 founddate=mktime(find)
 if ( (founddate > stdate) &&  (founddate < edate) ) {
   print "Found file: " $0
 } 
}'

output:
Code:
# ls -1 *xtx
CSQ_NPGS_2008_01_22_120012.xtx
CSS_GCYN_2008_01_18_130022.xtx
STAT_BML1_2007_11_22_110012.xtx
STAT_BML1_2007_12_22_110012.xtx
# ./test.sh
Enter from date [yyyy mm dd]: 2007 12 11
Enter to date [yyyy mm dd]: 2008 01 18
Found file: STAT_BML1_2007_12_22_110012.xtx

pls do more testing on your own
# 5  
Old 01-22-2008
This looks GREAT!

I'm working on it, but at present I get error message:
"awk: calling undefined function mktime source line number 7"
# 6  
Old 01-22-2008
Quote:
Originally Posted by dpath2o
This looks GREAT!

I'm working on it, but at present I get error message:
"awk: calling undefined function mktime source line number 7"
that's a gnu awk feature.
# 7  
Old 02-05-2008
OK so here's the regex that I came up with that meets my needs and I use it in a perl script to find files and then parse out file name with the parts I need.
Code:
if ($_ =~ m/^($type)[-_\s](\w{4})[-_\s](\d{4}|\d{2})[-_\s](\d{2})[_\s](\d{2})[-_\s]?(\d{2})?(\d{2})?(\d{2})?[.]?(\w{3}|\w{4})?/i) { #SITE_TYPE_YYYY_MO[_HRMN|_HRMNSC]?.[xx|xxx]
        $Ftype   = $1;
        $Fsite   = $2;
	$yyyy    = $3;
        $mm      = $4;
        $dd      = $5;
        $HH      = $6;
        $MM      = $7;
        $SS      = $8;
        $Fsuffix = $9;
        $yyyy += 1900 if $yyyy >= 90 && $yyyy <= 100; #only good to 2090 
        $yyyy += 2000 if $yyyy < 90 && $yyyy;
        $tmpFile = $_;
        $tmpFILE = $File::Find::name;
        $tmpDir  = $File::Find::dir; 
        if ($yyyy < 1980 || ($DirsDont =~ m/$tmpDir/) ) { #don't attempt to perform work on these files
            print "!!! $_ is not understood ... skipping\n";
            return; 
        }
        $tmpT = Mktime($yyyy,$mm,$dd,$HH,$MM,$SS);
}

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Extract date and time part from filename

Hi, I am facing one scenario in which I need to extract exact position of date and time from the name of the files. For example, Below is the record in which I need to extract position of YYYYMMDD,HHMISS and YYMMDD. Date and time variables can come more than once. I need to use these position... (13 Replies)
Discussion started by: Prathmesh
13 Replies

2. UNIX for Beginners Questions & Answers

How to extract date and time from filename?

Hi, I'm totally new in sell script and working with a shell code. I want to extract the date and time from the filenames. The filenames are different but all of them begins with WI_ SCOPE_: WI_SCOPE_DATA_CHANGE_2017-09-12_15-30-40.txt WI_SCOPE_BACK_COMPLETE_QUEUE_2017-09-12_15-31-40.txt... (5 Replies)
Discussion started by: Home
5 Replies

3. Shell Programming and Scripting

Validate date and time in filename by awk

hi i want to validate the date and time in filename filename : mohan.moh.ccyymmdd.ccyymmdd.hhmmss.txt mohan_moh.20151222.20151222.122442.txt i want code that check that date given in filename 20151222 in this format ccyymmdd else it mark file is not valid used in my OS detail is AIX 6... (12 Replies)
Discussion started by: MOHANP12
12 Replies

4. Shell Programming and Scripting

Find files greater than a particular date in filename.

I need a unix command which will find all the files greater that a particular date in the file name. say for example I have files like(filenaming cov : filename.YYDDMMSSSS.txt) abc.201206015423.txt abc.201207013456.txt abc.201202011234.txt abc.201201024321.txt efg.201202011234.txt... (11 Replies)
Discussion started by: lijjumathew
11 Replies

5. Shell Programming and Scripting

append a filename with system date and time

Hi, There are similar kind of posts, but none seems like working for me. Please correct me if I'm wrong. I need append/rename file abc.txt with file processed date and time like abc_systemdatetimestamp.txt and move it to different folder. for example I have /source/data/abc.txt ... (1 Reply)
Discussion started by: amsn08
1 Replies

6. UNIX for Dummies Questions & Answers

Shell Scripts - Append a filename with date and time....

Hello, I need to create a shell script that appends a filename to create a name with the date and time appended that is guaranteed to not exist. That is, the script insures you will not overwrite a file with the same name. I am lost with this one. I know I need to use date but after that I am... (3 Replies)
Discussion started by: citizencro
3 Replies

7. Shell Programming and Scripting

unixtime to formatted date time

Hi, I need to take the unix time and format it to a date/time string like this yyyymmdd,hhmmss I'm wrting in shell but have tried calling perl, but all the perl options I found on here puts output to Thu Jan 1 00:00:00 1970 format. Any help? Cheers Neil (4 Replies)
Discussion started by: nhatch
4 Replies

8. UNIX for Dummies Questions & Answers

Adding Date & time stamps to filename

I need to edit the file name with date and time while writing the script. please help. (1 Reply)
Discussion started by: manish.s
1 Replies

9. UNIX for Dummies Questions & Answers

Insert date/time within a filename

Hi Guys, I need to script the renaming of files as followins: files: firstjd secondjo thirdjv My script needs to insert the date/time infront of the last 2 characters of the filenames above, any ideas greatly received :) the letters before the last 2 characters could change, I'm only... (7 Replies)
Discussion started by: cooperman
7 Replies

10. UNIX for Dummies Questions & Answers

Renaming files to have date/time in filename

I have a program that will export my data to a single file, but it assigns a file name that is overridden every time I run the program. I need to change the file name to have a sequential number in the filename. How do I rename a file so that the filename contains the system date and time. I want... (5 Replies)
Discussion started by: wayneb
5 Replies
Login or Register to Ask a Question