Hi. I am attempting to write a script that copies files from a filesystem shared between two linux boxes (gfs2) to a windows server. The ftp client is installed on the windows box. I can create an ftp connection without issue (netrc is used for authentication)
This is the business need I am addressing: For the next several months a new reporting system is rolling out. It is hosted on Red Hat. There is a temporary business need for certain users to be able to drop reports run on these linux boxes to shared folder on the windows box.
The script should go into the shared drive, ftp files to the windows share, then delete the files on the share drive. They do not have to stay there.
I have a good portion of the script working. My difficulty is with the names of the reports. As this is end user controlled, they can include spaces, special characters, etc. It is not possible to enforce a naming convention through the application.
The script I have so far handles the names with out issue, but, once ftp is invoked, special characters are causing transfers to fail.
The script and output from the problematic area appear below. Any suggestions would be greatly appreciated. Thanks for your help.
Here is the script
Code:
# Script specific variables
destserver="example.windows.file.server"
ftpuploaddir="/share/share/reports/'A) Example folder name'/Test"
monitordir="/accept/file/reports/shared/"
# Switch to appropriate directory
cd ${monitordir}
#Start FTP Process
for f in ${monitordir};
do
echo "Uploading " $f
ftp -inv ${destserver} << EOF
binary
cd ${ftpuploaddir}
put $f
bye
EOF
#Remove transferred files
#rm -f "/accept/file/reports/shared/*"
done
On the linux box, the names seem to be handled initially, but are causing issues when in ftp:
Code:
+ for f in '${monitordir}/*'
+ echo 'Uploading ' '/accept/file/reports/shared/Last Saturday'\''s Date2013-02-25 17.03.35.169.pdf'
Uploading /accept/file/mstr/store-dashboard/Last Saturday's Date2013-02-25 17.03.35.169.pdf
+ ftp -inv example.windows.file.server
Connected to example.windows.file.server.
220 Microsoft FTP Service
500 'AUTH GSSAPI': command not understood
500 'AUTH KERBEROS_V4': command not understood
KERBEROS_V4 rejected as an authentication type
530 Please login with USER and PASS.
530 Please login with USER and PASS.
local: /accept/file/reports/shared/Last remote: Saturday's
local: /accept/file/reports/shared/Last: No such file or directory
221
+ for f in '${monitordir}/*'
+ echo 'Uploading ' '/accept/file/reports/shared/Today'\''s Date2013-02-25 17.04.06.380.xlsx'
Uploading /accept/file/reports/shared/Today's Date2013-02-25 17.04.06.380.xlsx
+ ftp -inv example.windows.file.server
Connected to example.windows.file.server.
220 Microsoft FTP Service
500 'AUTH GSSAPI': command not understood
500 'AUTH KERBEROS_V4': command not understood
KERBEROS_V4 rejected as an authentication type
530 Please login with USER and PASS.
530 Please login with USER and PASS.
local: /accept/file/reports/shared/Today's remote: Date2013-02-25
local: /accept/file/reports/shared/Today's: No such file or directory
221
DGPickett, thanks for the suggestion. I tried about 10 different iterations of "{f}", but was unsuccessful in getting the full file names to pass to ftp.
As such, I decided to try a different approach. In lieu of looping through the source directory, I placed the filenames in a file using sed. This works fine. The full file paths are listed in the file (though I am as yet unsure if they need to be surrounded by quotes).
Code:
# Script specific variable
destserver="ksms455b.cp.ad.kohls.com"
ftpuploaddir="/share/share/reports/'O) EDW Reports'/Test"
monitordir="/accept/file/mstr/store-dashboard"
tempfile="/tmp/temptransfer.txt"
# Switch to appropriate directory
cd ${monitordir}
#Create temporary file
find . -type f | sed "s#^.#$(pwd)#" > ${tempfile}
I would then like to read each line in the file, and have each line enter the ftp portion of the script so the files are transferred.
The problem I am having is that the script keeps erroring out with syntax error: unexpected end of file with the loops I try to do this with.
I have checked for unclosed parentheses, brackets, quotes, etc. but did not see any.
Here are a few attempts I made at writing a script that would read each line and pass to ftp. I am relatively new to shell scripting so any suggestions would be greatly appreciated. Thanks in advance for your help.
Try 1:
Code:
IFS="
"
for line in `cat ${tempfile}`;do
echo "Uploading " $line
ftp -inv ${destserver} << EOF
binary
cd ${ftpuploaddir}
put $line
bye
EOF
done
Try 2:
Code:
while read line
do
echo "Uploading " $line
ftp -inv ${destserver} << EOF
binary
cd ${ftpuploaddir}
put $f
bye
EOF
done <${tempfile}
Try 3:
Code:
for line in $(< /tmp/temptransfer.txt);
do
#Start FTP session
ftp -inv <<EOF
#change to the correct directory
cd ${ftpuploaddir}
#change to binary transfer mode
bin
mput $line
bye
EOF
done
The {} on variables are just for name separation and other optional shell behaviors. The "" around them ensure that when unpacked they are 'one word'. Two separate suggestions/ Try "$line" and "${ftpuploaddir}" and "$f".
Also, if changing $IFS, to keep your shell normal elsewhere, do it in a subshell (...)
The hot tip is to have an ssh client on the windows box, and then you can shuffle between two remote hosts:
Code:
scp -rp user1@host1:place1 user2@host:place2
or locally:
Code:
scp -rp user1@host1:place1place2
scp -rp place1user2@host:place2
-r is recursive (target should be parent directory of moved subtree)
-p preserves permissions and timestamps.
Thanks again for your help DGPickett. I realize that ftp is not that robust, but I have limited options in terms of permissible technologies where I am.
I nearly have it working. I managed to address the issues with metacharacters and spaces, etc. The for loop below works in that sense. The only issue I have no is that the loop only does one iteration (i.e. if there are 5 files in the source directory only one will be transferred to the windows server).
I would appreciate any other suggestions anyone might have. Thanks again for all your help.
Code:
# Switch to appropriate directory
cd ${monitordir}
+ cd /accept/file/mstr/store-dashboard
#Start FTP Process
for i in "`ls *`";
#while read i in "`ls *`";
do
echo "Uploading " "${i}"
ftp -i -v ${destserver} << EOF
bin
prompt
cd ${ftpuploaddir}
mput "${i}"
bye
EOF
done
ls *
++ ls 'Last Saturday'\''s Date2013-02-25 17.03.35.169.pdf' 'Test #12013-03-01 13.05.42.628.pdf' 'Test #22013-03-01 13.07.29.063.pdf' 'Test #32013-03-01 13.08.33.566.pdf' 'Today'\''s Date2013-02-25 17.04.06.380.xlsx'
+ for i in '"`ls *`"'
+ echo 'Uploading ' 'Last Saturday'\''s Date2013-02-25 17.03.35.169.pdf
Test #12013-03-01 13.05.42.628.pdf
Test #22013-03-01 13.07.29.063.pdf
Test #32013-03-01 13.08.33.566.pdf
Today'\''s Date2013-02-25 17.04.06.380.xlsx'
Uploading Last Saturday's Date2013-02-25 17.03.35.169.pdf
Test #12013-03-01 13.05.42.628.pdf
Test #22013-03-01 13.07.29.063.pdf
Test #32013-03-01 13.08.33.566.pdf
Today's Date2013-02-25 17.04.06.380.xlsx
+ ftp -i -v windows.file.server.com
Connected to windows.file.server.com.
220 Microsoft FTP Service
500 'AUTH GSSAPI': command not understood
500 'AUTH KERBEROS_V4': command not understood
KERBEROS_V4 rejected as an authentication type
331 Password required for cp\ftpusr.
230 User cp\ftpusr logged in.
Remote system type is Windows_NT.
200 Type set to I.
Interactive mode on.
250 CWD command successful.
mput Last Saturday's Date2013-02-25 17.03.35.169.pdf? 227 Entering Passive Mode (10,1,36,198,14,85).
125 Data connection already open; Transfer starting.
226 Transfer complete.
6498 bytes sent in 3.6e-05 seconds (1.8e+05 Kbytes/s)
?Invalid command
?Invalid command
?Invalid command
221
I do a lot less "for var in ". You are foundering on for i in "..." which is one word. How about "ls ... | while read i" instead. I will have the whole line, including any embedded meta, as each read gets one line, and your files are linefeed not white space delimited. For cases like "find ... | while read i" you also get pipeline parallelism, as the loop starts ASAP not after all as in "for i in `find ...`".
Hi,
I am new to shell scripting,and i was planning to write a script that will FTP files to destination folder.
I was planning that All configuration should be done through a properties files. and finally the output should be
Files are transferred
I have developed a properties file named... (4 Replies)
hello ,
I am working on a FTP script which navigates to specific directory and pulls the file
Issue which i am facing ::
----------------------------
i) When there is a file it pulls the file , but when there is not file in the directory it fails in the same statement and it is not... (3 Replies)
Hi all,
I'm using the following script to automated ftp files to 1 ftp servers
host=192.168.0.1
/usr/bin/ftp -vi >> $bkplog 2>&1 <<ftp
open $host
bin
cd ${directory}
put $files
quit
ftp
and the .netrc file contain
machine 192.168.0.1
login abc... (4 Replies)
Hi,
I have a ftp script which first gets all the file names and echo's the latest file.
I'm using another ftp command sets to get the file name given by first ftp. The problem is the parameter is not accepted by second ftp. The error message i'm getting is
> Rename Temp File calloc:ICMP... (5 Replies)
I have the following ftp script to get files from a remote location.
However, on running the script I find that I am not even able to connect to ftp server.
I am able to connect to ftp server using other GUI ftp tools like WS_FTP using the same IP.
IP used here is a dummy IP.
What can go... (3 Replies)
Hello,
A couple of times per week, i receive emails notifications when files are available for processing.
Currently i read these eamails with a java program and store the attachement on my C: drive and would now like to generate a PC script to send this file name up to UNIX-Solaris and... (3 Replies)
I have my ftp script as below..
This is logging the messages into ftp.log file ,
But I want the same output(what ever messages are going into ftp.log) to be printed on the console too for the user to show the status.. Tried with "tee" .. unable to get the solution..
Can some one help me... (3 Replies)
Novice here...
I need help with excuting a shell script on a flat file that I've transfered over from a Windows XP machine for manipulation through an auto FTP script...
so that after it is transfers it excutes the shell script and then returns it back to XP machine...
Any ideas... (2 Replies)
Hello ,
I am trying to make a shell script (Unix) for a ftp connection to another server and to get a file.
So I have no knowledge about ftp and my script must do automaticly the connection with the user and passwd.
Can you help us about that...
Christian... (2 Replies)
