Visit Our UNIX and Linux User Community


FTP "get" on files using date/time


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users FTP "get" on files using date/time
# 1  
Old 07-30-2002
Network FTP "get" on files using date/time

hi!

I want to "get/mget" files from a FTP server,
selecting on age of existing files at remote host.

example in natural lang:
"get all files from FTP-server newer than 2 days".

I'm presently trying to "output - grep - awk on date/time - mget".
Doesn't work very good.

Has any1 stubled of this before and like 2 share?

/Jonas
# 2  
Old 07-30-2002
Smilie I propose to handle this in 2 steps (whithin 1 script off course).
First make a connection and execute the ftp command dir. Make sure u use the option -v and capture your ftp output in a temp file.
After that you are going to read that file and create dynamicly a new ftp script to get only the files in which you're intrested.
You'll probably need also a script to calculate whith dates. If you haven't got one, just ask.
# 3  
Old 07-30-2002
Thanks for your quick reply.

Your last statement IS my REAL problem - calculating dates from the file output. Doesn't output differ among servers?

Please - what's your solution?

//greatful Jonas
# 4  
Old 07-30-2002
yeah,

Can you post a small example of the output you got
# 5  
Old 07-30-2002
extract from output of ftp "ls" command:

-r-xr-xr-x 1 owner group 254082 Dec 27 2001 KD2.D48
-r-xr-xr-x 1 owner group 36690 Jan 2 12:10 KD2.D49
-r-xr-xr-x 1 owner group 954175 Jun 10 9:39 KD2.D72
-r-xr-xr-x 1 owner group 475550 Jun 17 11:09 KD2.D73
-r-xr-xr-x 1 owner group 267741 Jun 24 10:00 KD2.D74
/
# 6  
Old 07-30-2002
This script seems to work on my system. But as you say, the exact format of the ouput from "dir" can vary. So you may need to tweak it for your system. But anyway, it should get you real close. It relies on datecalc, another script that I wrote. You can find it by searching this site.
Code:
#! /usr/bin/ksh

#
#  Setup a few variables

SYSTEM=somehost
USER=joeblow
PASSWORD=*******
DIRECTORY=tmp

tmpfile=/tmp/ftpdir$$
thisyear=$(date +%Y)
today=$(datecalc -j $(date "+%Y %m %d"))
Jan=01 Feb=02 Mar=03 Apr=04 May=05 Jun=06 
Jul=07 Aug=08 Sep=09 Oct=10 Nov=11 Dec=12

#
#  Get a copy of the remote dir output in a local file
#  called $tmpfile and attach that file to stdin for reading

exec 5>&1 >$tmpfile 4>&1
ftp -nv  >&4 2>&4 |&
print -p open $SYSTEM
print -p user $USER $PASSWORD
print -p cd $DIRECTORY
print -p dir
print -p bye
wait
exec >&5 2>&1 < $tmpfile

#
#  Ignore lines until we get a line that starts with 150

IFS=""
looking=1
while ((looking)) ; do
        read line
        word1=${line%%${line##+([! ])}}
        [[ $word1 = 150 ]] && looking=0
done

#
# Loop getting lines that start with a "-"
# Then delete some leading fields

{
IFS="" 
while read line ; do
        char1=${line%%${line#?}}
        [[ $char1 != - ]] && continue
        line="${line##+([! ])}"
        line="${line##+([ ])}"
        line="${line##+([! ])}"
        line="${line##+([ ])}"
        line="${line##+([! ])}"
        line="${line##+([ ])}"
        line="${line##+([! ])}"
        line="${line##+([ ])}"
        line="${line##+([! ])}"
        line="${line##+([ ])}"
        echo "$line"
done ; } | {

#
# Second loop reads selected and shortened lines,
# Computes the date and may get the file

IFS=" "

exec 4>&1
ftp -nv  >&4 2>&4 |&
print -p open $SYSTEM
print -p user $USER $PASSWORD
print -p cd $DIRECTORY
while read month day swing name ; do
        eval month=\$$month
        if [[ $swing = *:* ]] ; then
                year=$thisyear
        else
                year=$swing
        fi
        julian=$(datecalc -j $year $month $day)
        if ((julian > (today-2))) ; then
                print -p get $name
        fi
done
print -p bye
wait
}
rm $tmpfile
exit 0

# 7  
Old 07-31-2002
Thanks Perderabo,


I'll get back to you with a report on my implementation,
and hopefully a few tips for the community.

//Jonas

Previous Thread | Next Thread
Test Your Knowledge in Computers #841
Difficulty: Medium
Push technology, or server push, is a style of Internet-based communication where the request for a given transaction is initiated by the publisher or central server.
True or False?

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash script - Print an ascii file using specific font "Latin Modern Mono 12" "regular" "9"

Hello. System : opensuse leap 42.3 I have a bash script that build a text file. I would like the last command doing : print_cmd -o page-left=43 -o page-right=22 -o page-top=28 -o page-bottom=43 -o font=LatinModernMono12:regular:9 some_file.txt where : print_cmd ::= some printing... (1 Reply)
Discussion started by: jcdole
1 Replies

2. Shell Programming and Scripting

Delete all log files older than 10 day and whose first string of the first line is "MSH" or "<?xml"

Dear Ladies & Gents, I have a requirement to delete all the log files in /var/log/test directory that are older than 10 days and their first line begin with "MSH" or "<?xml" or "FHS". I've put together the following BASH script, but it's erroring out: for filename in $(find /var/log/test... (2 Replies)
Discussion started by: Hiroshi
2 Replies

3. Shell Programming and Scripting

awk "date" and "system" command

Hello experts! I need your help please I have a file.txt of which I want to extract 3rd and 4th columns with date with the form e.g.: 2016-11-25 03:14:50and pass them to "date" command, but also append the 9th column in a file as well. So I want to execute date -d '2016-11-25 03:14:50' ... (2 Replies)
Discussion started by: phaethon
2 Replies

4. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

5. Shell Programming and Scripting

"Join" or "Merge" more than 2 files into single output based on common key (column)

Hi All, I have working (Perl) code to combine 2 input files into a single output file using the join function that works to a point, but has the following limitations: 1. I am restrained to 2 input files only. 2. Only the "matched" fields are written out to the "matched" output file and... (1 Reply)
Discussion started by: Katabatic
1 Replies

6. AIX

xx=`date +"%a %b %d"`;rsh xxx grep "^$XX" zzz ?

AIX 4.2 I am trying to do an rsh grep to search for date records inside server logs by doing this : xx=`date +"%a %b %d"` rsh xxx grep "^$XX" zzz gives : grep: 0652-033 Cannot open Jun. grep: 0652-033 Cannot open 11. But if I do : xx=`date +"%a %b %d"` grep "^$XX" zzz it works... (2 Replies)
Discussion started by: Browser_ice
2 Replies

7. HP-UX

FTP large files - Getting "Connection Refused"

Hello Friends, When i trying to transfer a huge amount of files via FTP to a HP-Unx server, I am getting an error "Connection Refused"...! How can i avoid this error. Regards, Prakash K:b: (4 Replies)
Discussion started by: bullz26
4 Replies

8. Shell Programming and Scripting

How to remove "New line characters" and "spaces" at a time

Dear friends, following is the output of a script from which I want to remove spaces and new-line characters. Example:- Line1 abcdefghijklmnopqrstuvwxyz Line2 mnopqrstuvwxyzabcdefghijkl Line3 opqrstuvwxyzabcdefdefg Here in above example, at every starting line there is a “tab” &... (4 Replies)
Discussion started by: anushree.a
4 Replies

9. UNIX for Advanced & Expert Users

add seconds to: date"|"time"|"HHMMSS

Hey all, I have a shell that invokes a AWK. In this AWK i want invoke a function that receives 3 parameters: date: 20080831 time: 235901 duration: 00023 that function receive this 3 parameters and sum to this value two more seconds: 2008083123590100025 Remember that in case that... (3 Replies)
Discussion started by: anaconga
3 Replies

Featured Tech Videos