Find Files with a input in directories

11-09-2012

Registered User

21, 0

Join Date: Nov 2012

Last Activity: 30 November 2012, 10:41 AM EST

Posts: 21

Thanks Given: 4

Thanked 0 Times in 0 Posts

Find Files with a input in directories

So i have directories that store logs, 1 directorie for each day, with the name like this :

2012_07_01/
2012_07_02/

and for each directorie we have the logs, inside them lives the logs for that day, and every log have this name pattern :

sensor1_log_from_2012_07_01_10_:_30_to_2012_07_01_10_:_50

and every file have 100 mb

now that the introduction about the system is done i can tell the problem, i do have a script that is on the bottom to show you how i do things right now but, question is if i have a leap in days or hours or anything between my $startdate and $enddate (the logs are created with timestamps for each 100 mb collected )value that i use to search for the files that i want to see in the directorie logs, then a problem comes because we can have logs that came like this :

sensor1_log_from_2012_07_01_10_:_30_to_2012_07_03_10_:_30

and think on this what if i want to see logs from 2012_07_02 ??? and this can happen because if a sensor dont collect enough traffic for the 100 mb then he goona leaps the end date until it reachs the 100mb but, for this i'm thinking or if i can search for the last know file before the date i search for $startdate ? that would do the trick but i dont know how to do it right now i have this :

Code:

Code:
cmd={ls | sort | sed -n "/$stardate/"}

if [cmd eq True]:
then 

 for dir in `ls | sort | sed -n "/$startdate/,/$enddate/p"` ; do
 
    if [ `ls $dir | wc -l` -gt 0 ] ; then
         ls $dir > list.txt
     else
         rm $dir
     fi
 done

else 


done

soo with that said can anyone give me a hand , i know some people use find like:

Code:

find ${DIR} \ (-name $startdate and $enddate) -ctime > /tmp/list.txt

i just dont know how to combine that in my script in the right way or even to say in a simple if to search for the last file recorded before the $startdate.

Last edited by jim mcnamara; 11-09-2012 at 08:56 PM.. Reason: the else is the key for the solution that is were im stuck

drd0spt

View Public Profile for drd0spt

Find all posts by drd0spt

11-09-2012

Registered User

64, 4

Join Date: Jul 2011

Last Activity: 11 September 2014, 8:50 AM EDT

Posts: 64

Thanks Given: 9

Thanked 4 Times in 4 Posts

What exactly are you looking for? Do you want to find files which are older than a particular file or you want to find it using name?
If you are looking for files older than your reference file (let's say ref_file) you can use this :

Code:

find . ! -newer ref_file print

(! for not newer than or rather older than your reference file)

Moderator's Comments:

Please use code tags

Last edited by jim mcnamara; 11-09-2012 at 08:55 PM..

aksijain

View Public Profile for aksijain

Find all posts by aksijain

11-09-2012

Registered User

316, 33

Join Date: Sep 2008

Last Activity: 13 September 2020, 12:21 AM EDT

Location: US

Posts: 316

Thanks Given: 66

Thanked 33 Times in 31 Posts

I understood the problem lays in the fact that a single file can span several days worth of data, such as

Quote:

Code:

sensor1_log_from_2012_07_01_10_:_30_to_2012_07_03_10_:_30

You need a method to find appropriate file for a given date, if I am not mistaken.

I propose to use gawk capacity to manipulate dates and use unix epoch date form, which can be compared arithmetically.

1) convert given date/time to epoch form
2) go through files and for each extract start_time and end_time from the file name and using a mktime function convert both into the unix epoch time
3) if given_time >= start_time and given_time <= end_time then you got the file needed.

Note: I'm not sure if awk/nawk has mktime function, that is why emphasis on gawk.

To demonstrate I put a simple script:

Code:

#!/bin/ksh
s="sensor1_log_from_2012_07_01_10_:_30_to_2012_07_03_10_:_30"
echo $s|
gawk '
BEGIN{
        qry1_time = mktime("2012 06 20 18 30 00");
        qry2_time = mktime("2012 07 02 10 00 00");
        qry3_time = mktime("2012 07 03 22 45 00");
}
{
n=split($0, arr, "_");
str_fr_time = arr[4]" "arr[5]" "arr[6]" "arr[7]" "arr[9]" 00"
str_to_time = arr[11]" "arr[12]" "arr[13]" "arr[14]" "arr[16]" 00"
fr_time = mktime(str_fr_time);
to_time = mktime(str_to_time);
print "FR:"str_fr_time
print "TO:"str_to_time
if(qry1_time >= fr_time && qry1_time <= to_time)  
        print "OK for " strftime("%x %X", qry1_time);
else 
        print "NO for " strftime("%x %X", qry1_time);
 
if(qry2_time >= fr_time && qry2_time <= to_time)  
        print "OK for " strftime("%x %X", qry2_time);
else 
        print "NO for " strftime("%x %X", qry2_time);
if(qry3_time >= fr_time && qry3_time <= to_time)  
        print "OK for " strftime("%x %X", qry3_time);
else 
        print "NO for " strftime("%x %X", qry3_time);
 
}'

And here is the output

Code:

 
FR:2012 07 01 10 30 00
TO:2012 07 03 10 30 00
NO for 06/20/12 18:30:00
OK for 07/02/12 10:00:00
NO for 07/03/12 22:45:00

migurus

View Public Profile for migurus

Find all posts by migurus

Shell Programming and Scripting