Selecting files between a user inputed date range


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Selecting files between a user inputed date range
# 1  
Old 07-28-2008
Question Selecting files between a user inputed date range

Hi all!

I'm working on a KSH script to select files between a user inputed date range (stored in a variable) and then move them and unzip them. I'm stuck at how to get the files between the user inputed date range selected. Any help would be greatly appreciated!

The files are as such:

1112858279 May 2 08:00 20080502_0800.zip
2226093319 Jul 1 15:16 20080502_1230.zip
2362831536 May 2 17:28 20080502_1700.zip
418436718 May 3 08:00 20080503_0800.zip

Code:
# User Prompting for email, start date and end date
print " Welcome to the MRR User Recovery Interface"
print ""
print ""
print " Please enter the email address to search for:"
read user
print ""
print " Enter start date to search (Format: YYYY/MM/DD):"
read sdate
print ""
print "Enter end date of search (Format: YYYY/MM/DD):"
read edate
print ""
print "I will now search for the email from $user"
print "starting at $sdate and ending at $edate."
print ""
print ""
print "${DATE} Search context email $user" >> logs/recovery.log
print "${DATE} Starting date $sdate" >> logs/recovery.log
print "${DATE} Ending date $edate" >> logs/recovery.log
# Eventually there will be some stuff here to sort the zip files by the sdate and edate and then copy them off to a work directory.
#sorting the files by sdate and edate
# find . -xdev -type f -newer /tmp/start -a ! -newer /tmp/finish -exec mv {} /newdir \;
 
# grab a list of the zip files. 
print "Now Getting a file list..."
print ""
# The following needs to be changed when putting into production. These file paths
# will *NOT* work. Make sure they are correct! You've been warned.
cd /nmltest/cbpnear/data/mrr/mrr
ls *.zip > /nmltest/cbpnear/data/mrr/ziplist
print "${DATE} ziplist created" >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log
 
# Unzipping the files
# To do this, we need to feed the file into an array otherwise ksh will choke.
print "Now unzipping files into the recovery directory..."
print ""
    for zip in *.zip
    do
        dir=`echo $zip | sed 's/.\{4\}$//'`
        unzip -o -qq -d recovery/$dir $zip >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log
    done
# moving old usermail.log to something else...
print "Creating a log of the user's emails..."
print ""
mv /nmltest/cbpnear/data/mrr/scripts/logs/usermail.log /nmltest/cbpnear/data/mrr/scripts/logs/$user-$DATE2.log
# Let's grep through the dir's to find our user...
print "Now searching for ${user}."
print ""
print "${DATE} Started search for ${user} in unzipped directories" >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log
cd /nmltest/cbpnear/data/mrr/mrr/recovery/ >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log
find . -type f -print |xargs grep -l $user >> /nmltest/cbpnear/data/mrr/scripts/logs/user.log
# Just going to remove the first two characters of the grep results and move them to a new file.
cat /nmltest/cbpnear/data/mrr/scripts/logs/user.log | sed 's/^..//' >> /nmltest/cbpnear/data/mrr/scripts/logs/usermail.log
rm /nmltest/cbpnear/data/mrr/scripts/logs/user.log >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log
# let's move the files to the reprocessing directory.
print "${DATE} Moving files to the recovery folder" >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log
print "Now moving the email messages to the recovery directory"
print ""
rm -rf /nmltest/cbpnear/data/mrr/mrr/reprocess/* >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log
cd /nmltest/cbpnear/data/mrr/mrr/recovery/ >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log
cat /nmltest/cbpnear/data/mrr/scripts/logs/usermail.log | while read line; do cp /nmltest/cbpnear/data/mrr/mrr/recovery/$line ../reprocess/. ; done >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log
print "${DATE} Recovery of ${user} is complete." >> /nmltest/cbpnear/data/mrr/scripts/logs/recovery.log


Last edited by kelldan; 07-28-2008 at 11:00 AM.. Reason: Forgot to add the file formats. Duh, I'm stupid sometimes.
# 2  
Old 07-28-2008
You can filter the files out between the given dates with something like:

Code:
output_of_files | awk -F" |_" '$5 >= '$sdate' && $5 <= '$edate

If you get errors use nawk, gawk or /usr/xpg4/bin/awk on Solaris.

Regards
# 3  
Old 07-28-2008
Franklin

Thanks for the help. Will this select on the system date/time stamp or the file name as it is in a YYMMDD format?

Thanks!
# 4  
Old 07-28-2008
Sorry, you're asking for a date of this format YYYY/MM/DD in your script so first we have to remove the slashes.
The script select the timestamp from the lines. We use 2 field separators, a space or a underscore so the date is the bold portion (the 5th field) and we compare it with the 2 dates on the last line:

1112858279 May 2 08:00 20080502_0800.zip

Code:
output_of_files |
awk -F" |_" -v s=$sdate -v e=$edate '
BEGIN{gsub("/","",s);gsub("/","",e)}
$5 >= s && $5 <= e'

Regards
# 5  
Old 07-28-2008
Ah ok, I'm following your now. Would it be easier to have the user input the dates with out the slashes?

If so, that would remove the needs for the

{gsub("/","",s);gsub("/","",e)}

Portion, right?

Thanks again!
# 6  
Old 07-28-2008
Quote:
Originally Posted by kelldan
Ah ok, I'm following your now. Would it be easier to have the user input the dates with out the slashes?

If so, that would remove the needs for the

{gsub("/","",s);gsub("/","",e)}

Portion, right?

Thanks again!
Right, in that case you can remove this portion:

Code:
BEGIN{gsub("/","",s);gsub("/","",e)}

or you can use the first solution.

Regards
# 7  
Old 07-28-2008
Awesome, many thanks Franklin!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Find the count of files by last created date based on the given date range

My unix version is IBM AIX Version 6.1 I tried google my requirement and found the below answer, find . -newermt “2012-06-15 08:13" ! -newermt “2012-06-15 18:20" But newer command is not working in AIX version 6.1 unix I have given my requirement below: Input: atr files: ... (1 Reply)
Discussion started by: yuvaa27
1 Replies

2. UNIX for Dummies Questions & Answers

Deleted files between date range

Dear Friends, I have HP_ux 11.31 and want to delete some unwanted very old log files between two date range. Please help in the matter. Regards, Bhagawati Pandey (6 Replies)
Discussion started by: BPANDEY
6 Replies

3. UNIX for Advanced & Expert Users

Help with ksh script to list, then cp files from a user input date range

Hi, I'm quite new to ksh scripting, can someone help me with this. Requirements: I need to create a script that list the files from a user input date range. e. g. format of file: *c1*log.2012-12-22-14-00* *c1*log.2012-12-22-14-00* *c1*log.2012-12-22-14-00*... (1 Reply)
Discussion started by: chococrunch6
1 Replies

4. UNIX for Dummies Questions & Answers

Selecting highest value within a range

Within millions of lines of data, there are perhaps 20 "spikes" that are very narrow. I want only the highest value from each spike within a range of 10 rows. Possible? My data looks like this, 8 columns of integers, millions of rows. There are clear spikes when you graph columns. ... (6 Replies)
Discussion started by: markymarkg123
6 Replies

5. Shell Programming and Scripting

display Range of date depend on user input php

Hi, i am very new to php Is it possible to display Range of date depend on user input day example: user input 2 day start from 28/4/12 it will add 2 day from date of input so display should look like this 28/4/12 to 30/4/12 then from 30/412 user add another 4 date so will... (0 Replies)
Discussion started by: guidely
0 Replies

6. Shell Programming and Scripting

Search on date range of file based on user input

Hello I would like to ask for help with a script to search a directory that contains many log files and based on a users input after being prompted, they enter a date range down to the hour which searches the files that contain that range. I dont know how to go about this. I am hoping that the... (5 Replies)
Discussion started by: lostincashe
5 Replies

7. Shell Programming and Scripting

Selecting a range of Lines

Hi All, Is there a way to get a range of lines from a file??? I want to search through a set of scripts and need to select the group of lines which do the FTP. Say, Line1 Line2 ftp SERVER user UNAME PASS send FILE_TO_BE_SENT close Line3 Line4 Line5 ftp SERVER1 user USER1 PASS1... (6 Replies)
Discussion started by: beinthemiddle
6 Replies

8. Shell Programming and Scripting

PERL - Selecting specific files based on 'date stamp' values

Hi, I've list of files in a directory, which have date stamp value in their names. ex: abc_data_20071102.csv, abc_data_20091221.csv, abc_data_20100110.csv, abc_data_20100222.csv, abc_data_20080620.csv,... etc., I need to select and process only files, within the given date... (4 Replies)
Discussion started by: ganapati
4 Replies

9. Shell Programming and Scripting

Log File date compare for user defined range

:confused: Hi i am a noob and need a little help to finish my shell script. I am learning as i go but hit a problem. I am search thorugh logs(*.rv) files to find entires between two user defined dates, The script so far looks for the "START" and "END" of each entry at sees if it belongs To... (0 Replies)
Discussion started by: mojo24
0 Replies

10. UNIX for Dummies Questions & Answers

cp only files in certain date range

hi all, I'm trying to do a cp only on files I created on a given day or within a certain date range. What's the best way to do this? Cheers, KL (1 Reply)
Discussion started by: ee7klt
1 Replies
Login or Register to Ask a Question