FTP in shell script and selecting files for upload


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting FTP in shell script and selecting files for upload
# 1  
Old 06-20-2015
FTP in shell script and selecting files for upload

Hi,

Im a newbie with programming and shell scripting. Im running OSX/Mac and Darwin.

I would like to create a shell script that would :

1. Search a Volume and directory (including subdirectories) for a file that :

* filename ends with ”_Highres.pdf” and
* the file creation date of that file is no older than 2 days

2. Upload this file to a FTP server.


I managed to figure out the part 2 my self, like this:
Code:
#!/bin/sh
USER=userid
PASSWD=userpw
HOST=ftphost
ftp -n $HOST <<SCRIPT
user $USER $PASSWD
binary
put some.file
quit
SCRIPT

But Im stuck at the above ”some.file” that matches the ”1.” above


I hope someone can give me some hints in the right direction.

Thanks,

Niklas

Moderator's Comments:
Mod Comment Please use code tags for your code and data, thanks

Last edited by vbe; 06-20-2015 at 09:43 AM..
# 2  
Old 06-20-2015
Using the utility find might help you search for that file.

Code:
find /path/to/directory -name "_Highres.pdf" -type f -ctime -2

find: the program to search for files, links and directories

/path/to/directory: location to start the search

-name "_Highres.pdf": searches for by name

-type f: it has to be a file

-ctime -2: it has to be less than two days old

Now, that might return one, several or no file meeting those conditions. It all depends of what you have in /path/to/directory. Something to think about when trying to upload.
# 3  
Old 06-20-2015
Aia's suggestion is close, but -name "_Highres.pdf" will only look for files named _Highres.pdf; not all files with names ending with _Highres.pdf.

Note also that most UNIX filesystems do not have a file creation date timestamp. The
Code:
-ctime -2

find primary will limit the results to files whose status has changed in the last two days. If you use
Code:
-mtime -2

instead, it will limit the results to files whose contents have changed in the last two days. If you are using a system that has a filesystem that does have file creation date timestamps, the man page for find on your system may list a different primary that can be used to check those file creation timestamps.

Try:
Code:
find /path/to/directory -name '*_Highres.pdf' -type f -mtime -2

Presumably, you'll want to read the output from find in a loop and call ftp for each file found. Or, gather a list and if the list is not empty invoke ftp using mput instead of put.

Last edited by Don Cragun; 06-20-2015 at 04:32 PM.. Reason: Edit usage note.
# 4  
Old 06-20-2015
Thanks, I will try using this one:

find /path/to/directory -name '*_Highres.pdf' -type f -mtime -2

the files that I want to send will end with "*_Highres.pdf". I'll see the script to run each night at 00.05 and usually there will be just one file found each night. So I don't have to worry about looping I guess?

I could also change the: -mtime -2 to just going back one day, as in -mtime -1 since running the script with cron every night. It doesn't matter If I send duplicate files though, its better to do that than to miss any files, so maybe its better to keep it at -2

After some searching I will give up on the ftp command and go with cURL instead, that seemed more simple and I could get the whole command in just a single line.

So my full script will be:

Code:
#!/bin/sh
FILE=$(find /path/to/directory -name "*_HIGHRES.pdf" -type f -mtime -2)
curl --upload-file $FILE ftp://user:password@ftp.someserver.com

# 5  
Old 06-20-2015
Quote:
Originally Posted by NickeZ28
Thanks, I will try using this one:

find /path/to/directory -name '*_Highres.pdf' -type f -mtime -2

the files that I want to send will end with "*_Highres.pdf". I'll see the script to run each night at 00.05 and usually there will be just one file found each night. So I don't have to worry about looping I guess?

I could also change the: -mtime -2 to just going back one day, as in -mtime -1 since running the script with cron every night. It doesn't matter If I send duplicate files though, its better to do that than to miss any files, so maybe its better to keep it at -2

After some searching I will give up on the ftp command and go with cURL instead, that seemed more simple and I could get the whole command in just a single line.

So my full script will be:

Code:
#!/bin/sh
FILE=$(find /path/to/directory -name "*_HIGHRES.pdf" -type f -mtime -2)
curl --upload-file $FILE ftp://user:password@ftp.someserver.com

Usually in the context above (especially when you expect one file per day and are looking for files for two days) seems like a disaster waiting to happen... Try:
Code:
#!/bin/sh
find /path/to/directory -name "*_HIGHRES.pdf" -type f -mtime -2 |
while read -r FILE
do      curl --upload-file "$FILE" ftp://user:password@ftp.someserver.com
done

which should work correctly when there are no files, when there is one file, when there are two files, and even if there are more files.

Or, if you really want a 1-liner:
Code:
find /path/to/directory -name "*_HIGHRES.pdf" -type f -mtime -2 -exec curl --upload-file "{}" ftp://user:password@ftp.someserver.com \;


Last edited by Don Cragun; 06-20-2015 at 05:31 PM.. Reason: Add 1-liner.
This User Gave Thanks to Don Cragun For This Post:
# 6  
Old 06-20-2015
Quote:
Originally Posted by Don Cragun
Usually in the context above (especially when you expect one file per day and are looking for files for two days) seems like a disaster waiting to happen... Try:
Code:
#!/bin/sh
find /path/to/directory -name "*_HIGHRES.pdf" -type f -mtime -2 |
while read -r FILE
do      curl --upload-file "$FILE" ftp://user:password@ftp.someserver.com
done

which should work correctly when there are no files, when there is one file, when there are two files, and even if there are more files.

Or, if you really want a 1-liner:
Code:
find /path/to/directory -name "*_HIGHRES.pdf" -type f -mtime -2 -exec curl --upload-file "{}" ftp://user:password@ftp.someserver.com \;

Perfect! thanks!

hmm. I would have also liked if there was a way to find files (pdf-documents) that contained 5 pages or more. But I guess that doesnt really qualify as a question in here, since there isn't such a shell command?

Niklas
# 7  
Old 06-20-2015
You could try some heuristic based on assuming that a PDF file containing 5 pages will be in a regular file larger than some number and that smaller files will contain fewer pages, but that would depend on very specific knowledge of the content and structure of the PDF files. One PDF page containing a high-res image could be much larger than a 100 pages text document saved in PDF format.

There may also be PDF processing utilities on your system that can return the number of pages in a PDF file, but there aren't any such utilities in the standards, I'm afraid I can't make any specific suggestions on a way to do that.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find files and upload to FTP Server

HI, I need a script to find new files that created after 6:00 from /home/ugh /demo/conn /UAT/d01 and upload them into ftp server according to system date: Please help me (1 Reply)
Discussion started by: refra
1 Replies

2. Shell Programming and Scripting

Script to upload latest file to other server via FTP

Hello, I have a script that finds the latest version of a file in a folder on my Minecraft server. I'm trying to come up with something that will then FTP that file over to my fileserver. Here's what I have that finds the newest file: find /home/mc/archive/sbhouse -type f -mtime +45 -exec... (7 Replies)
Discussion started by: nbsparks
7 Replies

3. Emergency UNIX and Linux Support

Shell script to get all the files from FTP server

Hi Guru's, I am new to UNIX. my requirement is to log on to FTP server and get all the .txt files. i have developed one script by searching some forums but getting error and not able to fix them. pls see below code. ftp -i-n<<EOF open $FTP_HOST... (30 Replies)
Discussion started by: arund_01
30 Replies

4. Shell Programming and Scripting

bash script for ftp-upload is not working

Hello everyone, sorry for the title, most of you must getting sick of reading something like this, but I haven't found a solution, although I found many threads according to it. I'm working on a bash script that connects to a network printer with ftp where I want to upload a pdf created... (3 Replies)
Discussion started by: le_mae
3 Replies

5. Shell Programming and Scripting

Shell Script for Upload/download files using cURL

hi please help me out here, i want to use curl command in shell script to test web pages, what i have is an opening page, when i click on a button on opening page, the next page comes up and then i have to upload a file n then click another button to submit and then comes the output page,... (2 Replies)
Discussion started by: Olivia
2 Replies

6. Shell Programming and Scripting

Shell Script to monitor folder and upload found files via FTP

Hi everyone! I'm in a need of a shell script that search for all files in a folder, move all those files to a temp folder, and upload those files via FTP. When the file transfer via FTP completes successfully, the file is moved to a completed folder. In case any of those files fails, the file... (4 Replies)
Discussion started by: pulsorock
4 Replies

7. UNIX for Dummies Questions & Answers

ftp files from one server to another using shell script

Hi Guys Any Help I have created a spool file that i need to copy onto another server using FTP in a shell script both servers are linux (3 Replies)
Discussion started by: itai
3 Replies

8. Shell Programming and Scripting

Upload files from desktop to unix through FTP

Hi , I want to upload some files from my desktop to unix server through FTP . The list of files are in a text file .Please suggest how to do it through scrip . Thanks in advance .. Anupam (8 Replies)
Discussion started by: anupamhalder
8 Replies

9. Shell Programming and Scripting

ftp put in shell script -- whole file doesn't upload

Hi I'm having some trouble with a bash shell script that I'm writing. In the script, I'm trying to upload a file to a backup repository using ftp, but the whole file doesn't get uploaded. This is the file's properties at the start (I've highlighted the file size in red): -rw-r--r-- 1 root... (2 Replies)
Discussion started by: Viola
2 Replies

10. Shell Programming and Scripting

Random files do not FTP in the shell script

The following script is used to loop through files in the /tmp directory and transfer those files onto another server. However, some of the files do not transfer. It is very random when the transferring is done (i.e. one of the files won't transfer then next time, that one will transfer and... (1 Reply)
Discussion started by: RLatham20
1 Replies
Login or Register to Ask a Question