Process files in a directory and move them


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Process files in a directory and move them
# 1  
Old 09-13-2013
Process files in a directory and move them

I have a directory e2e_ms_xfer/cent01
this contains the multiple files some of which will be named below with unique date time stamps
e2e_ms_edd_nom_CCYYMMDD_HHMM.csv
What I want to do is in a loop
1) Get the oldest file
2) Rename
3) Move it up one level from e2e_ms_xfer/cent01 to /e2e_ms_xfer
3) Process it
4) Move it down 2 levels to e2e_ms_xfer/cent01/done
4) Exit loop

This is what I have below but it does not do step 2 it says it cannot find files of that name although I have put 5 unique
e2e_ms_edd_nom_CCYYMMDD_HHMM.csv in the directory.




#
#
# Control the Unix processing cycle for uploading
# M+S Nomiated Delivery Days into the E2E database.
#
# 1.0 AM 13/09/2013 Initial Version
#
set +xv

# Execute the .profile (coz cron doesn't)
. ~/.profile

# Library functions
. $( dirname $0 )/common_functions.ksh

THIS=$( basename $0 )

fn_banner "Process M+S Nominated Delivery Days files"

echo "Starting: $(date)"
fn_writeTrace "starting"


# Local variables
bookmark="10"
TODAY=$(date "+%Y%m%d")
D_EXTTAB=/e2e_ms_xfer
MASK=e2e_ms_edd_nom*
D_WORK=$XFMSC
D_DONE=$XFMSZ
JOBNAME=$( echo $THIS | cut -f1 -d. )


fileCount=0

# Oracle
bookmark="20"
fn_oraenv

# Good to go
bookmark="30"
runControl=1

while [ $runControl -eq 1 ]
do
cd $D_WORK


# Are there any files?
bookmark="35"
fileFound=0
ls -1 $MASK > /dev/null 2>&1
if [ $? -eq 0 ]
then
# Oldest file
bookmark="37"
fileName=$( ls -1 $MASK | head -n 1 )
mv filename e2e_ms_edd_nom.csv

let fileCount=fileCount+1
fileFound=1

# Move the file to where the ext.tables expect
bookmark="40"
mv $XFMSC/e2e_ms_edd_nom.csv $XFMS

# Invoke the Oracle process to upload the data
#
bookmark="50"
fn_oraenv
sqlplus $OCS @$SQL/exec_pull_nom_del_date.sql

# Move the file to 'done'
#
bookmark="60"
mv $XFMS/e2e_ms_edd_nom.csv $XFMSZ

fi # file found


fn_setHeartbeat $JOBNAME

done # while

# All done - Cleanup and exit
#
bookmark="70"
fn_cleanup
echo "Complete: $(date)"
fn_writeTrace "complete"

exit 0

# 2  
Old 09-13-2013
Below is your post, with all the formatting displayed. May I ask you please to create a new thread without all the cruft. Please also use CODE tags for CODE blocks, not ICODE tags. Thanks.


I have a directory e2e_ms_xfer/cent01
this contains the multiple files some of which will be named below with unique date time stamps
[COLOR=navy][FONT=Arial]e2e_ms_edd_nom_CCYYMMDD_HHMM.csv[/FONT][/COLOR]
[FONT=Calibri]What I want to do is in a loop[/FONT]
[FONT=Calibri]1) Get the oldest file[/FONT]
[FONT=Calibri]2) Rename [/FONT]
[FONT=Calibri]3) Move it up one level from e2e_ms_xfer/cent01 to /e2e_ms_xfer[/FONT]
[FONT=Calibri]3) Process it[/FONT]
[FONT=Calibri]4) Move it down 2 levels to e2e_ms_xfer/cent01/done[/FONT]
[FONT=Calibri]4) Exit loop[/FONT]

[FONT=Calibri]This is what I have below but it does not do step 2 it says it cannot find files of that name although I have put 5 unique [/FONT]
[FONT=Calibri][COLOR=navy][FONT=Arial]e2e_ms_edd_nom_CCYYMMDD_HHMM.csv[/FONT][/COLOR][FONT=Calibri] in the directory.[/FONT]



[FONT=Calibri][ICODE]
[FONT=Calibri]#[/FONT]
[FONT=Calibri]#[/FONT]
[FONT=Calibri]# Control the Unix processing cycle for uploading[/FONT]
[FONT=Calibri]# M+S Nomiated Delivery Days into the E2E database.[/FONT]
[FONT=Calibri]#[/FONT]
[FONT=Calibri]# 1.0 AM 13/09/2013 Initial Version[/FONT]
[FONT=Calibri]#[/FONT]
[FONT=Calibri]set +xv[/FONT]

[FONT=Calibri]# Execute the .profile (coz cron doesn't)[/FONT]
[FONT=Calibri]. ~/.profile[/FONT]

[FONT=Calibri]# Library functions[/FONT]
[FONT=Calibri]. $( dirname $0 )/common_functions.ksh[/FONT]

[FONT=Calibri]THIS=$( basename $0 )[/FONT]

[FONT=Calibri]fn_banner "Process M+S Nominated Delivery Days files"[/FONT]

[FONT=Calibri]echo "Starting: $(date)"[/FONT]
[FONT=Calibri]fn_writeTrace "starting"[/FONT]


[FONT=Calibri]# Local variables[/FONT]
[FONT=Calibri]bookmark="10"[/FONT]
[FONT=Calibri]TODAY=$(date "+%Y%m%d")[/FONT]
[FONT=Calibri]D_EXTTAB=/e2e_ms_xfer[/FONT]
[FONT=Calibri]MASK=e2e_ms_edd_nom*[/FONT]
[FONT=Calibri]D_WORK=$XFMSC[/FONT]
[FONT=Calibri]D_DONE=$XFMSZ[/FONT]
[FONT=Calibri]JOBNAME=$( echo $THIS | cut -f1 -d. )[/FONT]


[FONT=Calibri]fileCount=0[/FONT]

[FONT=Calibri]# Oracle[/FONT]
[FONT=Calibri]bookmark="20"[/FONT]
[FONT=Calibri]fn_oraenv[/FONT]

[FONT=Calibri]# Good to go[/FONT]
[FONT=Calibri]bookmark="30"[/FONT]
[FONT=Calibri]runControl=1[/FONT]

[FONT=Calibri]while [ $runControl -eq 1 ][/FONT]
[FONT=Calibri]do[/FONT]
[FONT=Calibri]cd $D_WORK[/FONT]


[FONT=Calibri]# Are there any files?[/FONT]
[FONT=Calibri]bookmark="35"[/FONT]
[FONT=Calibri]fileFound=0[/FONT]
[FONT=Calibri]ls -1 $MASK > /dev/null 2>&1[/FONT]
[FONT=Calibri]if [ $? -eq 0 ][/FONT]
[FONT=Calibri]then[/FONT]
[FONT=Calibri]# Oldest file[/FONT]
[FONT=Calibri]bookmark="37"[/FONT]
[FONT=Calibri]fileName=$( ls -1 $MASK | head -n 1 )[/FONT]
[FONT=Calibri]mv filename e2e_ms_edd_nom.csv[/FONT]

[FONT=Calibri]let fileCount=fileCount+1[/FONT]
[FONT=Calibri]fileFound=1[/FONT]

[FONT=Calibri]# Move the file to where the ext.tables expect[/FONT]
[FONT=Calibri]bookmark="40"[/FONT]
[FONT=Calibri]mv $XFMSC/e2e_ms_edd_nom.csv $XFMS[/FONT]

[FONT=Calibri]# Invoke the Oracle process to upload the data[/FONT]
[FONT=Calibri]#[/FONT]
[FONT=Calibri]bookmark="50"[/FONT]
[FONT=Calibri]fn_oraenv[/FONT]
[FONT=Calibri]sqlplus $OCS @$SQL/exec_pull_nom_del_date.sql[/FONT]

[FONT=Calibri]# Move the file to 'done'[/FONT]
[FONT=Calibri]#[/FONT]
[FONT=Calibri]bookmark="60"[/FONT]
[FONT=Calibri]mv $XFMS/e2e_ms_edd_nom.csv $XFMSZ[/FONT]

[FONT=Calibri]fi # file found[/FONT]


[FONT=Calibri]fn_setHeartbeat $JOBNAME[/FONT]

[FONT=Calibri]done # while[/FONT]

[FONT=Calibri]# All done - Cleanup and exit[/FONT]
[FONT=Calibri]#[/FONT]
[FONT=Calibri]bookmark="70"[/FONT]
[FONT=Calibri]fn_cleanup[/FONT]
[FONT=Calibri]echo "Complete: $(date)"[/FONT]
[FONT=Calibri]fn_writeTrace "complete"[/FONT]

[FONT=Calibri]exit 0[/FONT]
[/ICODE][/FONT]
[/FONT]
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Move files from one directory to another in chunks

All, I have an application that is not working properly and the company is 'in the process' of fixing it. In the meantime, I want to write a bash script work-around. However, what I thought was going to be simple is seemingly not. Need: - Move files from one directory to another in... (3 Replies)
Discussion started by: hburnswell
3 Replies

2. Shell Programming and Scripting

How Create new directory and move files to that directory.?

Hi All, We have main directory called "head" under this we have several sub directories and under these directories we have sub directories. My requirement is I have to find the SQL files which are having the string "procedure" under "head" directory and sub directories as well. And create... (14 Replies)
Discussion started by: ROCK_PLSQL
14 Replies

3. UNIX for Dummies Questions & Answers

How to move gz files from one source directory to destination directory?

Hi All, Daily i am doing the house keeping in one of my server and manually moving the files which were older than 90 days and moving to destination folder. using the find command . Could you please assist me how to put the automation using the shell script . ... (11 Replies)
Discussion started by: venkat918
11 Replies

4. Shell Programming and Scripting

List files with date, create directory, move to the created directory

Hi all, i have a folder, with tons of files containing as following, on /my/folder/jobs/ some_name_2016-01-17-22-38-58_some name_0_0.zip.done some_name_2016-01-17-22-40-30_some name_0_0.zip.done some_name_2016-01-17-22-48-50_some name_0_0.zip.done and these can be lots of similar files,... (6 Replies)
Discussion started by: charli1
6 Replies

5. Shell Programming and Scripting

Process 2 lists at the same time and move files.

I have this while loop, that works but is quite slow to process though. I'm hopping there might be a faster/better way to find what I'm looking for. I have 2 lists of numbers, and want to only find files where a file name has both values present. each list has about 100 values. while... (10 Replies)
Discussion started by: whegra
10 Replies

6. UNIX for Dummies Questions & Answers

Zip all files in a directory and move to another directory

Hi, need to zip all files in a directory and move to another directory after the zip.. i am using this one but didnt help me... zip -r my_proj_`date +%Y%m%d%H%MS`.zip /path/my_proj mv in_proj_`date +%Y%m%d%H%M%S`.zip /path/source/ i am trying to zip all the files in my_proj... (0 Replies)
Discussion started by: dssyadav
0 Replies

7. UNIX for Dummies Questions & Answers

How to move files between 2 dates from one directory to another

Hi All, I am coding for a requirement where I need to move files (filename.yymmdd) from one directory(A) to another(B) based on 2 date fields in a paramtere file. (Paramfile.txt) For e.g: In Paramfile.txt, BUS_DT =20120612 SUB_DT =20120602 In this case, i need to move all the files... (14 Replies)
Discussion started by: dsfreddie
14 Replies

8. UNIX for Dummies Questions & Answers

How to move all files in a directory and subdirectory?

I'm trying to organize my MB Pro by moving all my jpeg files to a single folder from the desktop. There are some on the desktop that are not in any folder. I was at the command line and typed mv *.jpg "Jpeg files" but it only moved the files that were on the desktop, not any of the ones that... (3 Replies)
Discussion started by: Straitsfan
3 Replies

9. UNIX for Dummies Questions & Answers

Move all files in a directory tree to a signal directory?

Is this possible? Let me know If I need specify further on what I am trying to do- I just want to spare you the boring details of my personal file management. Thanks in advance- Brian- (2 Replies)
Discussion started by: briandanielz
2 Replies
Login or Register to Ask a Question