Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

How to move files between 2 dates from one directory to another

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers How to move files between 2 dates from one directory to another
# 1  
Old 06-14-2012
How to move files between 2 dates from one directory to another
Hi All,

I am coding for a requirement where I need to move files (filename.yymmdd) from one directory(A) to another(B) based on 2 date fields in a paramtere file. (Paramfile.txt)

For e.g: In Paramfile.txt,

Code:
BUS_DT =20120612
SUB_DT =20120602

In this case, i need to move all the files from directory A to directory B which falls between these 2 dates.

Also, I need to capture these dates(In a comma delimited format) into a .done file.

Can you pls shed some light on this.

Any thoughts are appreciated. Pls note i am trying to accomplish this in LINUX based system.

Thanks
Freddie
Last edited by methyl; 06-14-2012 at 06:44 PM.. Reason: please use code tags
# 2  
Old 06-14-2012
Quote:
Originally Posted by dsfreddie
Hi All,

I am coding for a requirement where I need to move files (filename.yymmdd) from one directory(A) to another(B) based on 2 date fields in a paramtere file. (Paramfile.txt)

For e.g: In Paramfile.txt,

BUS_DT =20120612
SUB_DT =20120602

In this case, i need to move all the files from directory A to directory B which falls between these 2 dates.
I'm not sure whether I got it or I didn't.
Could you please clarify what has to fall between the two dates?
The date in the name of the files (filename.yymmdd), or the files modification time, or...?

--
Bye
# 3  
Old 06-14-2012
Thanks Lem for your reply. Let me clarify your queres.

Could you please clarify what has to fall between the two dates?

The filename is as (yymmdd_filename). the date value in the filename should fall between the BUS_DT & SUB_DT values.

Hope this is clear now.

Thanks Much
Freddie
# 4  
Old 06-14-2012
Please post what Operating System and version you have. There are umpteen Linuxes.
Please post what Shell you use.
Please post whether you have the GNU version of the date command and whether you have Perl installed.

And to labour the point. Is your range inclusive or exclusive or some combination of the two. Please give a simple example showing real filenames in the form of an abbreviated ls -la listing (I hope your filenames do not actually have brackets in them).

Can we lose the space character in the parameter file design to match Shell syntax, so we can make the parameter file executable and import the parameters into the Shell environment?

Very important: Does the directory timestamp on every file match the date in the name of the file?
# 5  
Old 06-15-2012
Untested:

Code:
#!/bin/bash

PARAMFILE="/path/to/paramfile.txt" # Please set the right absolute path
#example: PARAMFILE="/home/johh/paramfile.txt"

DIRA="/path/to/dirA" # Please set the right absolute path
#example: DIRA="/home/john/things/stuff"

DIRB="/path/to/dirB" # Please set the right absolute path
#example: DIRB="/home/john/things/otherstuff"

SINCE=$(grep SUB_DT "$PARAMFILE" 2>/dev/null | egrep -o "[0-9]{8}$" 2>/dev/null)
TILL=$(grep BUS_DT "$PARAMFILE" 2>/dev/null | egrep -o "[0-9]{8}$" 2>/dev/null)

find "$DIRA" -maxdepth 1 -type f | while IFS= read -r NOMEFILE; do
     FILEDATE=$(egrep -o "[0-9]{8}$" <<<"$NOMEFILE" 2>/dev/null)
     [[ $FILEDATE != "" ]] && (( $FILEDATE >= $SINCE )) && (( $FILEDATE <= $TILL )) && cp -t "$DIRB" "$NOMEFILE"
     done
exit 0

And I used cp instead of mv: files are copied, not moved, so you can test the script and prevent any damage.

It's a bit late, here. See you tomorrow for everything.
--
Bye

---------- Post updated at 10:24 AM ---------- Previous update was at 12:04 AM ----------

I realize now that I missed the last part of your request (the .done file).

So we need to put $SINCE and $TILL in this file.done, right?

Something like:

[...]
20120318,20120325
20120326,20120406
[...]

Am I right?
Last edited by Lem; 06-15-2012 at 11:33 AM.. Reason: Added some quotes, just to be safer
# 6  
Old 06-15-2012
Thanks Methyl for your reply. Pls find the answers to your queries below,

LINUX version :
Quote:
uname -r
2.6.18-274.el5

what Shell you use : ksh
whether you have the GNU version of the date command and whether you have Perl installed - the date command gives me an output as below,
Quote:
date
Fri Jun 15 08:49:28 CDT 2012

Is your range inclusive or exclusive or some combination of the two : Yes, if i pass the BUS_DATE=20120615 & SUB_DATE=20120611, it should even include files for these 2 dates as well.

Please give a simple example showing real filenames in the form of an abbreviated ls -la listing (I hope your filenames do not actually have brackets in them).

filename is as below,
20120615_filename.dat
20120614_filename.dat etc

Very important: Does the directory timestamp on every file match the date in the name of the file?
Need not be the same. If we run the run the process that generates the file multiple times a day, the timestamp will be the same, but the date value in the file will be different.

Hope I answered your questions.

Thanks much,
Freddie

---------- Post updated at 10:26 AM ---------- Previous update was at 09:53 AM ----------

Hi Lam,

Thanks for the solution you provided. I tried running it, but it throws error msg, Here is what I tried

Quote:
PARAMFILE=/dev/tmp/DynamicDateFile.txt
DIRA=/dev/cmg/ctl/
DIRB=/dev/cmg/ctl/inbox/

SINCE=$(grep BUS_DT $PARAMFILE 2>/dev/null | egrep -o "[0-9]{8}$" 2>/dev/null)
TILL=$(grep SUB_DT $PARAMFILE 2>/dev/null | egrep -o "[0-9]{8}$" 2>/dev/null)
find $DIRA -maxdept 1 -type f | while IFS= read -r filenameA; do
FILEDATE=$(egrep -o "[0-9]{8}$" <<<$filenameA 2>/dev/null)
[[ $FILEDATE != "" ]] && (( $FILEDATE >= $SINCE )) && (( $FILEDATE <= $TILL )) && cp -t $DIRB $NOMEFILE
done
exit
Error msg :

Quote:
PARAMFILE=/dev/tmp/scripts/DynamicDateFile.txt
+ DIRA=/dev/cmg/ctl/
+ DIRB=/dev/cmg/ctl/inbox/
++ grep BUS_DT //dev/tmp/scripts/DynamicDateFile.txt
++ egrep -o '[0-9]{8}$'
+ SINCE=
++ grep SUB_DT /dev/tmp/scripts/DynamicDateFile.txt
++ egrep -o '[0-9]{8}$'
+ TILL=
+ find /dev/cmg/ctl/ -maxdept 1 -type f
+ IFS=
+ read -r filename
find: invalid predicate `-maxdept'
+ exit
Also,regarding the .done file, if I pass the BUS_DATE=20120615 & SUB_DATE=20120613,
the .done should have values as below,

20120615,20120614,20120613

Hope i answered your questions.

Thanks Much for your help,
Freddie
# 7  
Old 06-15-2012
Quote:
Originally Posted by dsfreddie
Hi Lam,

Thanks for the solution you provided. I tried running it, but it throws error msg, Here is what I tried



Error msg :
Please, double check where your parameter file is: /dev/tmp or /dev/tmp/scripts?

Besides, i have to apologize for my typo: it's maxdepth, with the h, not maxdept.



Quote:
Also,regarding the .done file, if I pass the BUS_DATE=20120615 & SUB_DATE=20120613,
the .done should have values as below,

20120615,20120614,20120613
20120614 should be present even if no file has this value in its name?
If so, let's try with:

Code:
#!/bin/ksh

PARAMFILE="/path/to/paramfile.txt" # Please set the right absolute path
#example: PARAMFILE="/home/johh/paramfile.txt"

DIRA="/path/to/dirA" # Please set the right absolute path
#example: DIRA="/home/john/things/stuff"

DIRB="/path/to/dirB" # Please set the right absolute path
#example: DIRB="/home/john/things/otherstuff"

DONEFILE="/path/to/file.done" #Please set the righ absolute path

SINCE=$(grep SUB_DT "$PARAMFILE" 2>/dev/null | egrep -o "[0-9]{8}$" 2>/dev/null)
TILL=$(grep BUS_DT "$PARAMFILE" 2>/dev/null | egrep -o "[0-9]{8}$" 2>/dev/null)

find "$DIRA" -maxdepth 1 -type f | while IFS= read -r NOMEFILE; do
     FILEDATE=$(egrep -o "[0-9]{8}$" <<<"$NOMEFILE" 2>/dev/null)
     [[ $FILEDATE != "" ]] && (( $FILEDATE >= $SINCE )) && (( $FILEDATE <= $TILL )) && cp -t "$DIRB" "$NOMEFILE"
     done
for z in {$SINCE..$TILL}; do
    /bin/echo -n $z"," >> "$DONEFILE"
    done
exit 0

 
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How Create new directory and move files to that directory.?

Hi All, We have main directory called "head" under this we have several sub directories and under these directories we have sub directories. My requirement is I have to find the SQL files which are having the string "procedure" under "head" directory and sub directories as well. And create... (14 Replies)
Discussion started by: ROCK_PLSQL
14 Replies

2. UNIX for Dummies Questions & Answers

How to move gz files from one source directory to destination directory?

Hi All, Daily i am doing the house keeping in one of my server and manually moving the files which were older than 90 days and moving to destination folder. using the find command . Could you please assist me how to put the automation using the shell script . ... (11 Replies)
Discussion started by: venkat918
11 Replies

3. Shell Programming and Scripting

List files with date, create directory, move to the created directory

Hi all, i have a folder, with tons of files containing as following, on /my/folder/jobs/ some_name_2016-01-17-22-38-58_some name_0_0.zip.done some_name_2016-01-17-22-40-30_some name_0_0.zip.done some_name_2016-01-17-22-48-50_some name_0_0.zip.done and these can be lots of similar files,... (6 Replies)
Discussion started by: charli1
6 Replies

4. UNIX for Dummies Questions & Answers

Need to Move files of different dates

Hi, Currently I'm moving the files based on date like below. "mv *20150901* backup_folder" - Limitation: can move only 1 day files to backup folder. I want to move the files of different dates like 20150901,02, 03, 04..... Is there any single command to do it. Thanks in advance!! (2 Replies)
Discussion started by: prakashs1218
2 Replies

5. UNIX for Dummies Questions & Answers

Rename files in a directory and move them

I have a directory e2e_ms_xfer/cent01 this contains the multiple files some of which will be named below with unique date time stamps e2e_ms_edd_nom_CCYYMMDD_HHMM.csv What I want to do is in a loop 1) Get the oldest file 2) Rename 3) Move it up one level from e2e_ms_xfer/cent01 to... (1 Reply)
Discussion started by: andymay
1 Replies

6. Shell Programming and Scripting

Process files in a directory and move them

I have a directory e2e_ms_xfer/cent01 this contains the multiple files some of which will be named below with unique date time stamps e2e_ms_edd_nom_CCYYMMDD_HHMM.csv What I want to do is in a loop 1) Get the oldest file 2) Rename 3) Move it up one level from e2e_ms_xfer/cent01 to... (1 Reply)
Discussion started by: andymay
1 Replies

7. UNIX for Dummies Questions & Answers

Zip all files in a directory and move to another directory

Hi, need to zip all files in a directory and move to another directory after the zip.. i am using this one but didnt help me... zip -r my_proj_`date +%Y%m%d%H%MS`.zip /path/my_proj mv in_proj_`date +%Y%m%d%H%M%S`.zip /path/source/ i am trying to zip all the files in my_proj... (0 Replies)
Discussion started by: dssyadav
0 Replies

8. UNIX for Dummies Questions & Answers

Reading the dates from a file & moving the files from a directory

Hi All, I am coding for a requirement where I need to read a file & get the values of SUB_DATE. Once the dates are found, i need to move the files based on these dates from one directory to another. ie, this is how it will be in the file, SUB_DATE = 20120608,20120607,20120606,20120606... (5 Replies)
Discussion started by: dsfreddie
5 Replies

9. UNIX for Dummies Questions & Answers

Move all files in a directory tree to a signal directory?

Is this possible? Let me know If I need specify further on what I am trying to do- I just want to spare you the boring details of my personal file management. Thanks in advance- Brian- (2 Replies)
Discussion started by: briandanielz
2 Replies
Login or Register to Ask a Question