Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Latest file through SFTP

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Latest file through SFTP
# 15  
Old 03-26-2010
Create a list of files that you have received. Lets call it received_files.
Using the filelist file from the previous posts.
Code:
#!/bin/ksh
while read a b c d e f g h file
do
  grep $file received_files >/dev/null
  if [ $?  ! = 0 ]
    echo get $file  >>list_of_new_files_to_get
  fi
done<filelist
echo quit >>list_of_new_files_to_get

Maintain received_files by appending the names of the newly received files to it.
Use the list_of_new_files_to_get as input to sftp.
# 16  
Old 03-26-2010
I will get the complete list everytime whenever I fetch the filenames. I am not sure whether it's possible to get only latest files from the server. That's what is the actual requirement is.
# 17  
Old 03-26-2010
Quote:
Originally Posted by ssachins
I will get the complete list everytime whenever I fetch the filenames. I am not sure whether it's possible to get only latest files from the server. That's what is the actual requirement is.
Yes. The filelist will have more entries in it than the current_received_list.
The script lists the names of files that are in filelist, but not the current_received_list. In other words, only the files you haven't got.
# 18  
Old 03-26-2010
Quote:
Originally Posted by jgt
Yes. The filelist will have more entries in it than the current_received_list.
The script lists the names of files that are in filelist, but not the current_received_list. In other words, only the files you haven't got.
I think jgt's script will work, nice one jgt!

ssachins, you would run this after jgt's script: sftp -b list_newfiles username@host

you could then run your find -newer touched file command locally on your received files and append those to the received_files list per jgt's intructions
Last edited by markdjones82; 03-26-2010 at 01:49 PM.. Reason: append
# 19  
Old 03-26-2010
I will try it tomorrow and confirm.
# 20  
Old 03-29-2010
Hello All,

This is what the complete script I have written.

Code:
#!/bin/ksh

sftp  user@hostname >filelist <<EOF
cd /home/data/
ls -l "ABC*"
quit

while read a b c d e f g h file
do
   grep $file Files_Receievd_ABC.txt >/dev/null
  if [ $?  != 0 ] ; then
    echo get /home/data/$file  >> list_of_new_files_to_get
  fi
done < filelist
echo quit >>list_of_new_files_to_get

list_of_new_files_to_get >> Files_Receievd_ABC.txt

sftp -v -b list_of_new_files_to_get user@hostname

### the filelist file contains first (extra sftp) record as follow due to which the script is failing.

sftp> sftp> -rw-r--r-- 0 13106 7745 0 Mar 29 05:47 ABC29054723
-rw-r--r-- 0 13106 7745 0 Mar 29 05:47 ABC29054729
-rw-r--r-- 0 13106 7745 0 Mar 29 05:47 ABC29054731
-rw-r--r-- 0 13106 7745 0 Mar 29 05:47 ABC29054732
-rw-r--r-- 0 13106 7745 0 Mar 29 05:47 ABC29054733
sftp>
# 21  
Old 03-29-2010
#!/bin/ksh

sftp user@hostname >filelist <<EOF
cd /home/data/
ls -l "ABC*"
quit
Add a couple of lines here to get rid of sftp> prompts in filelist
while read a b c d e f g h file
do
grep $file Files_Receievd_ABC.txt >/dev/null
if [ $? != 0 ] ; then
echo get /home/data/$file >> list_of_new_files_to_get
fi
done < filelist
echo quit >>list_of_new_files_to_get

#list_of_new_files_to_get >> Files_Receievd_ABC.txt
I would move this line to after the new files are received.

sftp -v -b list_of_new_files_to_get user@hostname
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Retrieve the Latest file in a folder using SFTP

HI Guys, Can anyone help me to retrieve the latest file from below example using SFTP I have below list of files in a folder v403t.lstprgms.sortin1 val027.d099.fwest.oct2711 xcelrptd.d1400sqp.dec1713.t040459.@02del xcelrptd.d1400sqp.dec1713.t073308.@02del... (3 Replies)
Discussion started by: heye18
3 Replies

2. Shell Programming and Scripting

Picking the latest file based on a timestamp for a Dynamic file name

Hi , I did the initial search but could not find what I was expecting for. 15606Always_9999999997_20160418.xml 15606Always_9999999998_20160418.xml 15606Always_9999999999_20160418.xml 9819Always_99999999900_20160418.xml 9819Always_99999999911_20160418.xmlAbove is the list of files I... (4 Replies)
Discussion started by: chillblue
4 Replies

3. Shell Programming and Scripting

Need to automate sftp and always get the latest file

hi guys the script is working but i need to change the file that i want to get to the latest.. i will use this as cron so i can get the latest sftp file evryday. pls help.. set user "big_user" set IP "XX.XX.XX" set dir "/export/home/oracle/REF/INPUT" spawn sftp $user@$IP expect sftp> ... (7 Replies)
Discussion started by: JONATHAN0919
7 Replies

4. Shell Programming and Scripting

Shell script to get the latest file from the file list and move

Hi, Anybody help me to write a Shell Script Get the latest file from the file list based on created and then move to the target directory. Tried with the following script: got error. A=$(ls -1dt $(find "cveit/local_ftp/reflash-parts" -type f -daystart -mtime -$dateoffset) | head... (2 Replies)
Discussion started by: saravan_an
2 Replies

5. UNIX for Dummies Questions & Answers

Find the latest file on remote sftp

Hi All, I need your help in finding the latest files in remote sftp and get those files to local server and process them. Please let me know I appreciate your valuable inputs. Thanks raj (7 Replies)
Discussion started by: rajeevm
7 Replies

6. Shell Programming and Scripting

Need to copy latest file using SFTP

Hi All, In my unix server, I have the following files: h1.txt h2.txt h3.txt and through SFTP i need to copy only the latest file to another unix server. Can you please let me know what command i need to use. Thanks in Advance, (2 Replies)
Discussion started by: HemaV
2 Replies

7. Shell Programming and Scripting

SFTP latest file from Windows server to Unix host

Hi All, I am SFTP ing file from Windows server (Sydney) to Unix host in London (both servers are on different timezones). It appears the upstream (Windows server team) does not do a file housekeeping activity and henceforth there are multiple files i.e. each week the script is scheduled to run... (1 Reply)
Discussion started by: vigdmab
1 Replies

8. Shell Programming and Scripting

get latest file

i have the following in my directory x: 3 files with the word "LIST" inside the files 2 files without the word "LIST" 1 folder (sudirectory) i want to get the filename of the latest file (timestamp) with the word "LIST". by the way the script and the list of files are in seperate... (4 Replies)
Discussion started by: inquirer
4 Replies

9. Shell Programming and Scripting

how to get the latest file

I am trying to scp the latest file which ends with "_abc.log". Can some help me figure out how can do that? (3 Replies)
Discussion started by: shehzad_m
3 Replies

10. Shell Programming and Scripting

How do I get the name of latest file?

1) How do I get the name of latest file in a variable? 2) Is it safe to delete all files from a dir I am doing cd $dir_name if return_code > 0 rm * fi what are other alternates to delete all files from a dir in a shell script? :) (5 Replies)
Discussion started by: Hangman2
5 Replies
Login or Register to Ask a Question