Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Ftp with bash, append file where left off

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Ftp with bash, append file where left off
# 8  
Old 02-01-2017
Quote:
Originally Posted by Corona688
Check if your FTP client has a batch mode, that may avoid it prompting for local file names. It might be -b, not all ftp clients are the same.
I used the prompt command to stop it from prompting.

Thanks everyone for your help.
It's solved.

I'll get some sample code up soon for others searching with this same need.

Wayne Sallee
Wayne@WayneSallee.com
This User Gave Thanks to WayneSallee For This Post:
Corona688
# 9  
Old 02-01-2017
Hi, Corona688.
Quote:
Originally Posted by Corona688
I don't think rsync searches for differences smaller than an entire file, just compares checksums and replaces.
I use rsync as part of rsnaphot to do backups 6 times daily. It is very efficient:
Quote:
Originally Posted by https://en.wikipedia.org/wiki/Rsync
Once connected, it will invoke the remote host's rsync and then the two programs will determine what parts of the file need to be transferred over the connection.
Best wishes ... cheers, drl
This User Gave Thanks to drl For This Post:
Corona688
# 10  
Old 02-01-2017
Here is the working code simplified:

Code:
ftp -n <<EOF
open ftp.$server
user $user@$server $password
prompt
ls -l sizeonserver.txt
EOF
myappendsize=$(awk '/security.mpg/ {print $5;}' sizeonserver.txt)

ftp -n <<EOF
open ftp.$server
user $user@$server $password
restart $myappendsize
put $myfile
EOF

The code that I was working on is a bash file that uploads a file while it is being created.


So for anyone looking for some code to upload to ftp a file while it is being created, here is the code.

Code:
#!/bin/sh

thisdir=`dirname $0`
cd $thisdir

user=youruser
server=yoursever.com
password=yourpassword
myfile=yourfile.extention
progcreatingfi=yourprogramcreatingthefile

touch sizeonserver.txt
echo "Ready to upload..."
echo "Buffering...."
sleep 30
echo "Uploading..."
ftp -n <<EOF
open ftp.$server
user $user@$server $password
put $myfile
prompt
ls -l sizeonserver.txt
EOF
myappendsize=$(awk '/security.mpg/ {print $5;}' sizeonserver.txt)

echo "Done first part"
echo "Buffering...."
sleep 30
while pgrep "$progcreatingfi" >/dev/null; do
echo "Appending"
ftp -n <<EOF
open ftp.$server
user $user@$server $password
restart $myappendsize
put $myfile
prompt
ls -l sizeonserver.txt
EOF
myappendsize=$(awk '/security.mpg/ {print $5;}' sizeonserver.txt)

echo "Buffering...."
sleep 30
done

echo "Finalizing upload"
ftp -n <<EOF
open ftp.$server
user $user@$server $password
restart $myappendsize
put $myfile
EOF
echo done
sleep 3

I noticed that there is also a throttle command for ftp, so that might also be useful.

Wayne Sallee
Wayne@WayneSallee.com
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Bash to append array value to file before copying

The bash stores each uniqueid in an array and then passes them to %q to get the unique path. That seems to work what I am having trouble with is renaming each .png with the unique value in %q. I thought it was working but upon closer inspection, a .png file is being sent to scp.... but only 1 and... (21 Replies)
Discussion started by: cmccabe
21 Replies

2. Shell Programming and Scripting

Bash - proper way to append variable to stderr

Hello, Can you please if the bellow is the proper way of appending a variable to the stderr: The easiest way to test this,I was able to imagine, was by touching 5 files and afterwards looping trough to the results: -rw-r--r-- 1 ab owner 0 Sep 14 13:45 file1 -rw-r--r-- 1 ab owner 0 Sep... (7 Replies)
Discussion started by: alex2005
7 Replies

3. Shell Programming and Scripting

Bash append values if keywords are present in the file

Hi Team, i have a web ui where user will be passing values and the output will be saved to a file say test with the following contents . These below mentioned values will change according to the user_input Just gave here one example Contents of file test is given below Gateway... (7 Replies)
Discussion started by: venkitesh
7 Replies

4. Shell Programming and Scripting

Ftp bash script appends to file using cron instead of coping new file

I have a bash script that is running a mysql query and creating a csv file with a time stamp. It then uploads that to a ftp server. Everything works great when I manually run it. But then I have a cron job set to run every monday, wednesday and friday at 5am est. When the cron job runs, it... (7 Replies)
Discussion started by: akallenberger
7 Replies

5. Shell Programming and Scripting

File count from where it left??

i have this code num=1 dat10=`date "+%m/%d/%Y" -d "+10 days"` dat=`date "+%m/%d/%Y"` set -x grep "text=.&" /root/Softwares/apache-tomcat-5.5.30/logs/catalina.out|awk -F"from=" '{print $2}'|awk -F"opid" '{print $1}'|sort|uniq > pnos while read line do psql -U poss -d emsver -c "insert into... (3 Replies)
Discussion started by: nikhil jain
3 Replies

6. Shell Programming and Scripting

Bash script for new file in ftp folder

Hello, I'm trying to make a bash script that send me e-mail if there is any new file in my ftp folder. cat inotify.sh #!/bin/sh /usr/bin/inotifywait -e create \ -mrq /home/mrowcp | while read line; do echo -n "$line " >> /var/log/inotify.log echo `date | cut -d " " -f1-4` >>... (3 Replies)
Discussion started by: mrowcp
3 Replies

7. Shell Programming and Scripting

end of file error bash ftp script

Hello kind programmers :) I am a newbie and running into an error "line 28: syntax error: unexpected end of file" on the script shown below. Any help would be greatly appreciated. Thanks! #! /bin/bash if ($#argv <3) then echo 'Usage get_modis_snow ' echo 'ftp script for MYD10A2... (2 Replies)
Discussion started by: cmshreve
2 Replies

8. Shell Programming and Scripting

append date time stamp via ftp script

I have searched several thread and not found my solution, so I am posting a new qustion. I have a very simple script on an AIX server that FTPs 2 files to a MS FTP server. These 2 files are created on the AIX server every hour, with a static name. I need to FTP the files to the MS server, but... (1 Reply)
Discussion started by: sknisely
1 Replies

9. UNIX and Linux Applications

ftp append question

Quick question. Will append act like 'put' if the file I'm telling it to append to doesn't exist? (2 Replies)
Discussion started by: Lindarella
2 Replies

10. UNIX for Dummies Questions & Answers

ftp append

What is the syntax for an ftp that is appending to the target file? I understand to use the append command, but how do you tell it which source file to append to which target file. ftpEngine() { ftp -inv ${FTP_SERVER} << EOF_FTP >> ${FTP_LOG} user ${FTP_USER} ${FTP_PSWD} cd ${FTP_DIR1}... (1 Reply)
Discussion started by: brdholman
1 Replies
Login or Register to Ask a Question