How archive the older than 30 day files to another unix server


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How archive the older than 30 day files to another unix server
# 8  
Old 08-07-2012
At first, you should not ignore mod comments or PMs about using code tags. Because when you hit 20 points, your account will be set to "read-only" automatically. In worst cases it can result in a ban.

Iterating through a list of files and then opening a ftp connection for each of them, I would suggest, you just move the files to be transferred into a different directory and just ftp them with mput * all at once which is much more effective, if space on disk permits this.
Or if possible, think about using scp or rsync and move them by this. ftp is not encrypted and does not offer as much options.
Also 2 times EOF is not needed - once is enough. i<="$FILE” does not make sense to me as you are comparing a number with a string.
# 9  
Old 08-07-2012
you may use
Code:
basename $FILE # This will give you the filename onle

Code:
 
for LOOK_DIR in /TempFiles
do
for FILE in `find ${LOOK_DIR} -mtime -30 -exec ls {} \;`
do
echo ${FILE} >> file_list ## This file will have the list of files copied and removed
ArchFiles_ftp_put ${FILE}
done
done
#*********************************************************#
# Copy the files from share drive to Unix server #
#*********************************************************#
#ArchFiles_ftp_put
ArchFiles_ftp_put()
{
/usr/bin/ftp -v -n << EOF > ${FTP_LOG_FILE}
open ${ARCH_SERVER}
quote USER ${ARCH_USERID}
quote PASS ${ARCH_PASSWORD}
passive
ascii
prompt off
cd ${REMOTE_DIR}
lcd ${DATA_DIR}
mput $1
bye
EOF
EOF
}

# 10  
Old 08-07-2012
as per the above code i am getting error as can not open the file

Code:
archive_file.ksh[9]: : cannot open
+ echo test_sample.txt
+ 1>> file_list
+ ArchFiles_ftp_put test_sample.txt

---------- Post updated at 10:14 AM ---------- Previous update was at 09:58 AM ----------

Quote:
Originally Posted by murari83.ds
as per the above code i am getting error as can not open the file

archive_file.ksh[9]: : cannot open
+ echo test_sample.txt
+ 1>> file_list
+ ArchFiles_ftp_put test_sample.txt
as per your suggesion next time on wards i will use code tags.

Last edited by zaxxon; 08-08-2012 at 05:16 AM.. Reason: code tags
# 11  
Old 08-07-2012
Hi ,
Have made some changes in the code and below is working fine for me.

Code:
#!/bin/ksh
ARCH_SERVER=xxxxxx
ARCH_USERID=xxxxxxx
ARCH_PASSWORD="xxxxxxx"
REMOTE_DIR=xxxxxxx
FTP_LOG_FILE=xxxxxxx/TEST/FTPLOGa
echo  > $FTP_LOG_FILE

#*********************************************************#
# Copy the files from share drive to Unix server #
#*********************************************************#
ArchFiles_ftp_put()
{
/usr/bin/ftp -v -n << EOF >> ${FTP_LOG_FILE}
open ${ARCH_SERVER}
quote USER ${ARCH_USERID}
quote PASS ${ARCH_PASSWORD}
passive
ascii
prompt off
cd ${REMOTE_DIR}
lcd ${DATA_DIR}
mput ${FILE_NAME}
bye
EOF
}
for LOOK_DIR in /cb/home/n421522/TEST
do
for FILE in `find ${LOOK_DIR} -mtime -30 -exec ls {} \;`
do
echo ${FILE} >> file_list ## This file will have the list of files copied and removed
DATA_DIR=`dirname ${FILE}`
FILE_NAME=`basename ${FILE}`
ArchFiles_ftp_put ${FILE}
done
done

You can also add date to the FTP_LOG_FILE generated to retrive it for longer time.

Please let me know if you get struck somewhere.

Rgds,
Aashish
# 12  
Old 08-08-2012
Quote:
Originally Posted by aashish.sharma8
Hi ,
Have made some changes in the code and below is working fine for me.

Code:
#!/bin/ksh
ARCH_SERVER=xxxxxx
ARCH_USERID=xxxxxxx
ARCH_PASSWORD="xxxxxxx"
REMOTE_DIR=xxxxxxx
FTP_LOG_FILE=xxxxxxx/TEST/FTPLOGa
echo  > $FTP_LOG_FILE
 
#*********************************************************#
# Copy the files from share drive to Unix server #
#*********************************************************#
ArchFiles_ftp_put()
{
/usr/bin/ftp -v -n << EOF >> ${FTP_LOG_FILE}
open ${ARCH_SERVER}
quote USER ${ARCH_USERID}
quote PASS ${ARCH_PASSWORD}
passive
ascii
prompt off
cd ${REMOTE_DIR}
lcd ${DATA_DIR}
mput ${FILE_NAME}
bye
EOF
}
for LOOK_DIR in /cb/home/n421522/TEST
do
for FILE in `find ${LOOK_DIR} -mtime -30 -exec ls {} \;`
do
echo ${FILE} >> file_list ## This file will have the list of files copied and removed
DATA_DIR=`dirname ${FILE}`
FILE_NAME=`basename ${FILE}`
ArchFiles_ftp_put ${FILE}
done
done

You can also add date to the FTP_LOG_FILE generated to retrive it for longer time.

Please let me know if you get struck somewhere.

Rgds,
Aashish
Thanks Aashish for the replay

Code:
I wrote the scipt as below:
#!/bin/ksh
set -x
ArchFiles_ftp_put()
{
        /usr/bin/ftp -v -n << EOF > ${FTP_LOG_FILE}
        open ${ARCH_SERVER}
        quote USER ${ARCH_USERID}
        quote PASS ${ARCH_PASSWORD}
         passive
        ascii
        prompt off
        cd ${ARCH_REMOTE_DIR}
        lcd ${DATA_DIR}
        mput ${FILE_NAME}
        bye
 EOF
EOF
}
#**************************************************#
# Main                                             #
#**************************************************#
if [ $# -ne 5 ]
then
echo "Invalid number of parameters::"
echo "Please pass 5 parameters in following order:Host Name, UserID, Password, RemotDir and project dir::"
exit 1
fi
#***************************************************************#
# Set "FTP" variables                                           #
#***************************************************************#
ARCH_SERVER=$1
ARCH_USERID=$2
ARCH_PASSWORD=$3
ARCH_REMOTE_DIR=$4
PROJ_NAME=$5
#**************************************************#
# Set directory values                             #
#**************************************************#

for LOOK_DIR in /tmp/arch_files/
do
for FILE in `find ${LOOK_DIR} -mtime -30 -exec ls {} \;`
do
echo ${FILE}>> file_list ## This file will have the list of files copied and removed
DATA_DIR=`dirname ${FILE}`
FILE_NAME=`basename ${FILE}`
ArchFiles_ftp_put ${FILE}
done
done
FTP_LOG_FILE="/tmp/Log/FTP_put_Archive_Files.log"
echo "Starting...">$FTP_LOG_FILE

When i ran the above scipt i am geeting below error message, can you please check the script.
Code:
"
archive_file.ksh[5]: : cannot open
+ echo test_sample.txt
+ 1>> file_list
+ dirname test_sample.txt
+ DATA_DIR=.
+ basename test_sample.txt
+ FILE_NAME=test_sample.txt
+ ArchFiles_ftp_put test_sample.txt
+ /usr/bin/ftp -v -n
+ 0<< \EOF
        open usmanasdsbul01
        quote USER testuser
        quote PASS testuser
         passive
        ascii
        prompt off
        cd /proj_name/Archive
        lcd .
        mput test_sample.txt
        bye
        EOF
EOF
archive_file.ksh[5]: : cannot open
+ echo tmp.txt
+ 1>> file_list
+ dirname tmp.txt
+ DATA_DIR=.
+ basename tmp.txt
+ FILE_NAME=tmp.txt
+ ArchFiles_ftp_put tmp.txt
+ /usr/bin/ftp -v -n
+ 0<< \EOF
        open usmanasdsbul01
        quote USER testuser
        quote PASS testuser
         passive
        ascii
        prompt off
        cd /proj_name/Archive
        lcd .
        mput tmp.txt
        bye
        EOF
EOF
"


Moderator's Comments:
Mod Comment Please use code tags next time for your code and data. If you continue to ignore this, you will be set to "read-only" soon or it can result in a ban - up to you.


---------- Post updated at 07:21 AM ---------- Previous update was at 05:05 AM ----------

Thanks zaxxon and aashish
i have resolved the problem.. as per the zaxxon suggestion I moved the older than the 30 day file to another folder and I called mput option it's working fine.
Thanks you so much for your suggestion s and code.

Last edited by zaxxon; 08-08-2012 at 07:40 AM.. Reason: code tags
# 13  
Old 08-08-2012
Thanks for update...Rather than using ftp for each file why don't you zip all the files together and ftp the compressed file only.
This will be much more effective !
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need command/script to archive files older than

I need to find a way to archive all files older than a given date but there are some conditions that are making it difficult for me to find the correct command: Linux based system (RH5) there are multiple layers of directory depth I need to search each file should be tar'd in it's original... (1 Reply)
Discussion started by: KaosJedi
1 Replies

2. Shell Programming and Scripting

Grep files older than 1 day

I thought that this would work for grep'ing files older than 1 day. ps -o etime,pid,user,args -e|awk '/^+-/'|sort -t- -n -k 1,1 |grep qdaemon |grep /usr/bin/ksh But, it is not grep'ing any of files (i.e. below) older than 1 day. d_prod 33757970 61999560 0 Oct 27 - 0:00... (8 Replies)
Discussion started by: Daniel Gate
8 Replies

3. Shell Programming and Scripting

Sftp - 1 day older files count

Need to write a shell script on AIX box which will connect to different servers using SFTP and get the file count of only 1 day older files. (purging list) How to achieve this? On local server we can use: find <path> -type f -mtime +1 But how to do it in case of SFTP? Please advise. Thanks... (9 Replies)
Discussion started by: vegasluxor
9 Replies

4. UNIX for Advanced & Expert Users

Help with get/mget from FTP server with files older than 10 minutes

Hi! I am new to unix and this forum as well.. Can someone please help me : I want to "get/mget" files which are older than 10 minutes from a remote FTP server like "ftp.com". After getting the files to local unix server say "Prod.com" , i need to delete only those files from ftp.com which... (4 Replies)
Discussion started by: SravsJaya
4 Replies

5. Shell Programming and Scripting

scp files that are 3 days older from remote server-

hello, i am trying to get a list of files to be scped from the remote server by running the below in my local unix server ( note - there is a passwordless connectivity setup made between the local and remote server) and, we use KSH. --- ssh $scp_host "find /a/b/c/*/ -iname "$remote_file"" >... (4 Replies)
Discussion started by: billpeter3010
4 Replies

6. Shell Programming and Scripting

Find and delete files and folders which are n days older from one unix server to another unix server

Hi All, Let me know how can i find and delete files from one unix server to another unix server which are 'N' days older. Please note that I need to delete files on remote unix server.So, probably i will need to use sftp, but question is how can i identify files and folders which are 'N'... (2 Replies)
Discussion started by: sachinkl
2 Replies

7. Shell Programming and Scripting

Find the number of files older than 1 day from a dir

Hello All, I need to write a script/command which can find out the number of .csv files residing in a directory older than 1 day. The output should come with datewise (means for each date how many files are there). I've this command, but this command gives the total number of files. It's... (10 Replies)
Discussion started by: NARESH1302
10 Replies

8. UNIX for Dummies Questions & Answers

sample script to archive & move previous day syslog files

hi all. Please help me with archiving previous day syslog files. the files have no extension and have the format YYYY-MM-DD. I want to archive the file then move it to some other machine. thanks. (2 Replies)
Discussion started by: coolatt
2 Replies

9. Shell Programming and Scripting

Deleting / finding files older than X days missess a day

Hi When trying to find and delete files which are, say, 1 day, the find command misses a day. Please refer the following example. xxxd$ find . -type f -ctime +1 -exec ls -ltr {} \; total 64 -rw-rw-r-- 1 oracle xxxd 81 Apr 30 11:25 ./ful_cfg_tmp_20080429_7.dat -rw-rw-r-- 1... (4 Replies)
Discussion started by: guruparan18
4 Replies

10. Shell Programming and Scripting

Archive files older than 30days

I have files with the names BARE01_DLY_MKT_20060724 in the source directory /biddf/ab6498/dev/ctl. I need to archive folders older than 30days. Like if i have a file named BARE01_DLY_MKT_20060622 I need to move this to /biddf/ab6498/dev/ctl/archive. How can I do this. One more thing is that I... (8 Replies)
Discussion started by: dsravan
8 Replies
Login or Register to Ask a Question