Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

How to move files between 2 dates from one directory to another

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers How to move files between 2 dates from one directory to another
# 8  
Old 06-15-2012
Quote:
Originally Posted by dsfreddie
Code:
PARAMFILE=/dev/tmp/DynamicDateFile.txt
DIRA=/dev/cmg/ctl/
DIRB=/dev/cmg/ctl/inbox/

Somehow i find that hard to believe. the directory /dev is the directory for devices and everything related to their handling. Nobody in his right mind would put files in there just to work on them.

Are you really, really sure you want to move files around in this sensitive part of a system? Chances are you might crash the system beyond any posibility of repair.

I hope this helps.

bakunin
# 9  
Old 06-15-2012
Hi Lam,

Thanks again.

I made the changes you suggested & here is the error log,
Couple of observations i had :
1. It is not fetching the BUS-DATE/SUB_DATE values
2. Can you pls tell what maxdepth do ?

Quote:
++ cat /devdata/cpmg/tmp/DynamicDateFile.txt
+ PARAMFILE=/devdata/cpmg/tmp/DynamicDateFile.txt
+ DIRA=/devdata/cpmg/tmp/ctl/
+ DIRB=/devdata/cpmg/tmp/inbox/
++ grep BUS_DT /devdata/cpmg/tmp/DynamicDateFile.txt
++ egrep -o '[0-9]{8}$'
+ SINCE=
++ grep SUB_DT /devdata/cpmg/tmp/DynamicDateFile.txt
++ egrep -o '[0-9]{8}$'
+ TILL=
+ find /devdata/cpmg/tmp/ctl/ -maxdepth 1 -type f
+ IFS=
+ read -r filename
++ egrep -o '[0-9]{8}$'
+ FILEDATE=
+ [[ '' != '' ]]
+ IFS=
+ read -r filename
++ egrep -o '[0-9]{8}$'
+ FILEDATE=
+ [[ '' != '' ]]
+ IFS=
+ read -r filename
++ egrep -o '[0-9]{8}$'
+ FILEDATE=
+ [[ '' != '' ]]
+ IFS=
+ read -r filename
++ egrep -o '[0-9]{8}$'
+ FILEDATE=
+ [[ '' != '' ]]
+ IFS=
+ read -r filename
++ egrep -o '[0-9]{8}$'
+ FILEDATE=
+ [[ '' != '' ]]
+ IFS=
+ read -r filename
+ exit
# 10  
Old 06-15-2012
Quote:
Originally Posted by dsfreddie
Couple of observations i had :
1. It is not fetching the BUS-DATE/SUB_DATE values
You're right.
But my humble opinion is that it should.

Try to run directly:
Code:
ls /devdata/cpmg/tmp/DynamicDateFile.txt     #the file is there?
cat /devdata/cpmg/tmp/DynamicDateFile.txt     #let's look at its text
grep BUS_DT /devdata/cpmg/tmp/DynamicDateFile.txt     #see whether grep works this time, or what errors it produces

What happens?

Quote:
2. Can you pls tell what maxdepth do ?
Option "-maxdepth N", where N>=0 is an integer, tells find to go down in the directory tree for N levels at most. N=1 means: search only in the specified directory, do not search in any subdirectory.

From the man:

Quote:
-maxdepth levels
Descend at most levels (a non-negative integer) levels of directories below the
command line arguments. -maxdepth 0
means only apply the tests and actions to the command line arguments.
# 11  
Old 06-18-2012
Hi Lem,

How are you ? I tried the following commands. Here is what I found,

ls /devdata/cpmg/tmp/DynamicDateFile.txt #the file is there? File is present
cat /devdata/cpmg/tmp/DynamicDateFile.txt #let's look at its text It showed the dates correctly.
grep BUS_DT /devdata/cpmg/tmp/DynamicDateFile.txt #see whether grep works this time, or what errors it produces
grep command gave the correct dates

But, the commands given by you somehow is not taking values from the file, i guess that s the issue. I will work on it today & let you know in-case of any progress.

Pls share your thoughts if you have any.

Thanks,
Freddie

---------- Post updated at 12:06 PM ---------- Previous update was at 12:03 PM ----------

Hello Bakunin,

I am not using the /tmp in my code. I gave it in the post as I didnt want to paste the actual directory (which is pretty lengthy). Thanks for pointing it out & i am sure your observation will help others who might visit this thread.

Thanks
Freddie

---------- Post updated at 12:16 PM ---------- Previous update was at 12:06 PM ----------

Hi Lem,

Adding one more point. While reading about egrep, i didnt find -o function for it. Pls correct me if I am wrong.

Thanks
Freddie
# 12  
Old 06-18-2012
Quote:
Originally Posted by dsfreddie
grep BUS_DT /devdata/cpmg/tmp/DynamicDateFile.txt #see whether grep works this time, or what errors it produces
grep command gave the correct dates

But, the commands given by you somehow is not taking values from the file, i guess that s the issue.
Most probably the next grep is the guilty one:
Code:
egrep -o '[0-9]{8}$'

Let's see if the syntax above works well with your grep, your system and your shell - as it works in mine:

In console:
Code:
echo aaa20120304 | egrep -o '[0-9]{8}$'

If it works, the problem in the script is surely the trailing $, so remove it:
Code:
egrep -o '[0-9]{8}'

If it doesn't work, I think you should double check which is the right syntax for a regexp with your version of grep, your system and your shell.

I read now your last edit: -o tells grep to print only the matched (non-empty) parts of a matching line. If your grep doesn't have the -o option, you can't use grep for that task. Try this modified version of the script, in which we use this kind of shell parameter expansion: ${parameter##pattern}.

Code:
#!/bin/ksh

PARAMFILE="/path/to/paramfile.txt" # Please set the right absolute path
#example: PARAMFILE="/home/johh/paramfile.txt"

DIRA="/path/to/dirA" # Please set the right absolute path
#example: DIRA="/home/john/things/stuff"

DIRB="/path/to/dirB" # Please set the right absolute path
#example: DIRB="/home/john/things/otherstuff"

DONEFILE="/path/to/file.done" #Please set the righ absolute path

SINCEa=$(grep SUB_DT "$PARAMFILE" 2>/dev/null)
TILLa=$(grep BUS_DT "$PARAMFILE" 2>/dev/null)

SINCE=${SINCEa##*=}
TILL=${TILLa##*=}

find "$DIRA" -maxdepth 1 -type f | while IFS= read -r NOMEFILE; do
     FILEDATE=$(egrep -o "[0-9]{8}" <<<"$NOMEFILE" 2>/dev/null)
     [[ $FILEDATE != "" ]] && (( $FILEDATE >= $SINCE )) && (( $FILEDATE <= $TILL )) && cp -t "$DIRB" "$NOMEFILE"
     done
for z in {$SINCE..$TILL}; do
    /bin/echo -n $z"," >> "$DONEFILE"
    done
exit 0

Last edited by Lem; 06-18-2012 at 05:31 PM..
# 13  
Old 06-18-2012
Ok. I think we made some progress Smilie

I could now see it is reading the dates. But it doesnt seem to pick the files & copy it to destination directory.
I am pasting the actual log itself (with the actual directories etc).

Quote:
+ PARAMFILE=/iis_dev_data3/wcc/cpmg/tmp/CPMGPERDynamicDateFile.txt
+ DIRA=/iis_dev_data3/wcc/cpmg/ctl/
+ DIRB=/iis_dev_data3/wcc/cpmg/inbox/
++ grep P_BUS_DATE /iis_dev_data3/wcc/cpmg/tmp/CPMGPERDynamicDateFile.txt
+ SINCE=P_BUS_DATE=20120615
++ grep P_SUB_DATE /iis_dev_data3/wcc/cpmg/tmp/CPMGPERDynamicDateFile.txt
+ TILL=P_SUB_DATE=20120603
+ find /iis_dev_data3/wcc/cpmg/ctl/ -maxdepth 1 -type f
+ IFS=
+ read -r filename
++ egrep -o '[0-9]{8}'
+ FILEDATE=20120529
+ [[ 20120529 != '' ]]
+ (( 20120529 >= 20120615 ))
+ IFS=
+ read -r filename
++ egrep -o '[0-9]{8}'
+ FILEDATE=20120530
+ [[ 20120530 != '' ]]
+ (( 20120530 >= 20120615 ))
+ IFS=
+ read -r filename
++ egrep -o '[0-9]{8}'
+ FILEDATE=20120616
+ [[ 20120616 != '' ]]
+ (( 20120616 >= 20120615 ))
+ (( 20120616 <= 20120603 ))
+ IFS=
+ read -r filename
++ egrep -o '[0-9]{8}'
+ FILEDATE=
+ [[ '' != '' ]]
+ IFS=
+ read -r filename
++ egrep -o '[0-9]{8}'
+ FILEDATE=20120615
+ [[ 20120615 != '' ]]
+ (( 20120615 >= 20120615 ))
+ (( 20120615 <= 20120603 ))
+ IFS=
+ read -r filename
++ egrep -o '[0-9]{8}'
+ FILEDATE=20120614
+ [[ 20120614 != '' ]]
+ (( 20120614 >= 20120615 ))
+ IFS=
+ read -r filename
++ egrep -o '[0-9]{8}'
+ FILEDATE=20120531
+ [[ 20120531 != '' ]]
+ (( 20120531 >= 20120615 ))
+ IFS=
+ read -r filename
++ egrep -o '[0-9]{8}'
+ FILEDATE=20120613
+ [[ 20120613 != '' ]]
+ (( 20120613 >= 20120615 ))
+ IFS=
+ read -r filename
++ egrep -o '[0-9]{8}'
+ FILEDATE=
+ [[ '' != '' ]]
+ IFS=
+ read -r filename
++ egrep -o '[0-9]{8}'
+ FILEDATE=20120612
+ [[ 20120612 != '' ]]
+ (( 20120612 >= 20120615 ))
+ IFS=
+ read -r filename
+ exit 0
Files in CTL directory,
$ pwd
/iis_dev_data3/wcc/cpmg/ctl
filename.20120531.dat
filename.20120530.dat
filename.20120529.dat
filename.20120614.dat
filename.20120613.dat
filename.20120612.dat
filename.20120615.dat
filename.20120616.dat

I think we are almost close to the solution. Thanks for your prompt reply & thoughts Lem. I appreciate it.

Thanks,
Freddie
# 14  
Old 06-18-2012
Quote:
Originally Posted by dsfreddie

+ SINCE=P_BUS_DATE=20120615
++ grep P_SUB_DATE /iis_dev_data3/wcc/cpmg/tmp/CPMGPERDynamicDateFile.txt
+ TILL=P_SUB_DATE=20120603
LOL! That's easy. Smilie
You switched SINCE and TILL definitions. SINCE must go with SUB_DATE (easy way to avoid confusion: Since-Sub), and TILL must go with BUS_DATE.

Glad to see that egrep -o works extracting FILEDATE from FILENAME. So the problem was only the trailing $ in the regexp.
BTW: I see now that FILENAME ends with ".dat".

Code:
#!/bin/ksh

PARAMFILE="/iis_dev_data3/wcc/cpmg/tmp/CPMGPERDynamicDateFile.txt" 

DIRA="/iis_dev_data3/wcc/cpmg/ctl"

DIRB="/iis_dev_data3/wcc/cpmg/inbox"

#########################################
DONEFILE="/path/to/file.done" ###PLEASE SET THE RIGHT PATH!!!
#########################################

SINCE=$(grep SUB_DT "$PARAMFILE" 2>/dev/null | egrep -o "[0-9]{8}" 2>/dev/null)
TILL=$(grep BUS_DT "$PARAMFILE" 2>/dev/null | egrep -o "[0-9]{8}" 2>/dev/null)

find "$DIRA" -maxdepth 1 -type f | while IFS= read -r FILENAME; do
     FILEDATE=$(egrep -o "[0-9]{8}" <<<"$FILENAME" 2>/dev/null)
     [[ $FILEDATE != "" ]] && (( $FILEDATE >= $SINCE )) && (( $FILEDATE <= $TILL )) && cp -t "$DIRB" "$FILENAME"
     done
for z in {$SINCE..$TILL}; do
    /bin/echo -n $z"," >> "$DONEFILE"
    done
exit 0

 
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How Create new directory and move files to that directory.?

Hi All, We have main directory called "head" under this we have several sub directories and under these directories we have sub directories. My requirement is I have to find the SQL files which are having the string "procedure" under "head" directory and sub directories as well. And create... (14 Replies)
Discussion started by: ROCK_PLSQL
14 Replies

2. UNIX for Dummies Questions & Answers

How to move gz files from one source directory to destination directory?

Hi All, Daily i am doing the house keeping in one of my server and manually moving the files which were older than 90 days and moving to destination folder. using the find command . Could you please assist me how to put the automation using the shell script . ... (11 Replies)
Discussion started by: venkat918
11 Replies

3. Shell Programming and Scripting

List files with date, create directory, move to the created directory

Hi all, i have a folder, with tons of files containing as following, on /my/folder/jobs/ some_name_2016-01-17-22-38-58_some name_0_0.zip.done some_name_2016-01-17-22-40-30_some name_0_0.zip.done some_name_2016-01-17-22-48-50_some name_0_0.zip.done and these can be lots of similar files,... (6 Replies)
Discussion started by: charli1
6 Replies

4. UNIX for Dummies Questions & Answers

Need to Move files of different dates

Hi, Currently I'm moving the files based on date like below. "mv *20150901* backup_folder" - Limitation: can move only 1 day files to backup folder. I want to move the files of different dates like 20150901,02, 03, 04..... Is there any single command to do it. Thanks in advance!! (2 Replies)
Discussion started by: prakashs1218
2 Replies

5. UNIX for Dummies Questions & Answers

Rename files in a directory and move them

I have a directory e2e_ms_xfer/cent01 this contains the multiple files some of which will be named below with unique date time stamps e2e_ms_edd_nom_CCYYMMDD_HHMM.csv What I want to do is in a loop 1) Get the oldest file 2) Rename 3) Move it up one level from e2e_ms_xfer/cent01 to... (1 Reply)
Discussion started by: andymay
1 Replies

6. Shell Programming and Scripting

Process files in a directory and move them

I have a directory e2e_ms_xfer/cent01 this contains the multiple files some of which will be named below with unique date time stamps e2e_ms_edd_nom_CCYYMMDD_HHMM.csv What I want to do is in a loop 1) Get the oldest file 2) Rename 3) Move it up one level from e2e_ms_xfer/cent01 to... (1 Reply)
Discussion started by: andymay
1 Replies

7. UNIX for Dummies Questions & Answers

Zip all files in a directory and move to another directory

Hi, need to zip all files in a directory and move to another directory after the zip.. i am using this one but didnt help me... zip -r my_proj_`date +%Y%m%d%H%MS`.zip /path/my_proj mv in_proj_`date +%Y%m%d%H%M%S`.zip /path/source/ i am trying to zip all the files in my_proj... (0 Replies)
Discussion started by: dssyadav
0 Replies

8. UNIX for Dummies Questions & Answers

Reading the dates from a file & moving the files from a directory

Hi All, I am coding for a requirement where I need to read a file & get the values of SUB_DATE. Once the dates are found, i need to move the files based on these dates from one directory to another. ie, this is how it will be in the file, SUB_DATE = 20120608,20120607,20120606,20120606... (5 Replies)
Discussion started by: dsfreddie
5 Replies

9. UNIX for Dummies Questions & Answers

Move all files in a directory tree to a signal directory?

Is this possible? Let me know If I need specify further on what I am trying to do- I just want to spare you the boring details of my personal file management. Thanks in advance- Brian- (2 Replies)
Discussion started by: briandanielz
2 Replies
Login or Register to Ask a Question