Login or Register to Ask a Question and Join Our Community


Linux

Get all the files from a FTP location with previous week's dates in the file names using Linux

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Operating Systems Linux Get all the files from a FTP location with previous week's dates in the file names using Linux
# 1  
Old 09-08-2013
Get all the files from a FTP location with previous week's dates in the file names using Linux
I have a weird requirement where I have to get the files from a FTP(Lets say FTP1) location and place it on my current FTP(Lets say FTP2) location. The issue is, these are daily files (in a pattern Sales_YYYYMMDD_report.csv) and are placed every day on FTP1 and my process usually runs on Monday(eg. 09-Sept-2013) which has to use the file of the previous week starting from Sunday(eg. 01-Sept-2013) to Saturday(eg. 07-Sept-2013) place them on FTP2 location and then run the Informatica process. For an instance, if I run the process on Monday,09-Sept-2013, I have to pull all the files from FTP1 which have file names such as

Code:
Sunday file --> Sales_20130901_report.csv  
Monday file --> Sales_20130902_report.csv  
Tuesday file --> Sales_20130903_report.csv  
Wednesday file --> Sales_20130904_report.csv  
Thursday file --> Sales_20130905_report.csv  
Friday file --> Sales_20130906_report.csv  
Saturday file --> Sales_20130907_report.csv

How can I achieve this in a shell script? I know the part to get the files from another FTP, but I am not sure how to get the 7 files.

P.S: I cannot use the file creation/last modified timestamps to get the files. Irrespective of the created timestamp and the day I run my Informatica process, I have to get the files which have last week's dates in the file names and put in my FTP2 location and then continue with them.

Please help...
# 2  
Old 09-09-2013
This is an example script script that creates a 'ftp' script that 'puts' the files from the last sunday thru saturday to remote host:
Code:
echo 'ftp -vn FTP2 <<ENDSCRIPT' >>ftp_script.sh
echo 'user userid pass' >>ftp_script.sh
echo 'cd /path/to/dir' >>ftp_script.sh
echo 'lcd /path/to/dir' >>ftp_script.sh
echo put Sales_`date -d '-8day' +%Y%m%d`_report.csv >>ftp_script.sh
echo put Sales_`date -d '-7day' +%Y%m%d`_report.csv >>ftp_script.sh
echo put Sales_`date -d '-6day' +%Y%m%d`_report.csv >>ftp_script.sh
echo put Sales_`date -d '-5day' +%Y%m%d`_report.csv >>ftp_script.sh
echo put Sales_`date -d '-4day' +%Y%m%d`_report.csv >>ftp_script.sh
echo put Sales_`date -d '-3day' +%Y%m%d`_report.csv >>ftp_script.sh
echo put Sales_`date -d '-2day' +%Y%m%d`_report.csv >>ftp_script.sh
echo quit >>ftp_script.sh
echo 'ENDSCRIPT' >>ftp_script.sh
echo 'rc=$?' >>ftp_script.sh
echo 'if [[ $rc != 0 ]];    then' >>ftp_script.sh
echo '  echo "Error occured...$rc" `date "+%Y-%m-%d-%H.%M.%S"`' >>ftp_script.sh
echo 'else' >>ftp_script.sh
echo '  echo "Successful transfer...$rc" `date "+%Y-%m-%d-%H.%M.%S"`' >>ftp_script.sh
echo 'fi' >>ftp_script.sh

Code:
$ cat ftp_script.sh
ftp -vn FTP2 <<ENDSCRIPT
user userid pass
cd /path/to/dir
lcd /path/to/dir
put Sales_20130901_report.csv
put Sales_20130902_report.csv
put Sales_20130903_report.csv
put Sales_20130904_report.csv
put Sales_20130905_report.csv
put Sales_20130906_report.csv
put Sales_20130907_report.csv
quit
ENDSCRIPT
rc=$?
if [[ $rc != 0 ]];      then
  echo "Error occured...$rc" `date "+%Y-%m-%d-%H.%M.%S"`
else
  echo "Successful transfer...$rc" `date "+%Y-%m-%d-%H.%M.%S"`
fi

# 3  
Old 09-11-2013
Hi Spacebar,

Your code is very helpful. However, I see in your code that, you are subtracting the dates as if running the process on Mondays only, but there are few scenarios where I run the job on Tuesdays or any day of the week too (eg. due to long weekends or holidays or sometimes the files are not ready on the FTP1). Can you please suggest a different code(or modification to the current one) to get the files for the previous week no matter which day or when I run the job in the current week.

Thanks

---------- Post updated at 02:04 PM ---------- Previous update was at 11:26 AM ----------

Hi Spacebar,

I got the answer to my question Smilie. I used the following:

Code:
date -d "last saturday -1day"  +%Y%m%d

Thanks
Dhruuv.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove dates from file names

Hi, I'm using a shell script. I have extracted current date files to a directory1 and the date should be removed on both sides of a CSV file. FYI... I'm looking to remove the date from the file name and not inside the CSV file. Directory1 2017-07-12_gmr_tag_log_20170711.csv... (0 Replies)
Discussion started by: shivamayam
0 Replies

2. Shell Programming and Scripting

Need Help:Shell Script for Solaris to change the dates in a file by one week

I have to increase the date by one week in an input when script is executed in solaris. I was able to acheive this using ksh script that is working in Linux enivironment, when i execute the same script in Solaris i am getting below error: /var/tmp\n\r-> ./script.ksh date: illegal option -- d... (3 Replies)
Discussion started by: sriramanaramoju
3 Replies

3. Shell Programming and Scripting

How to get the dates from a list of file names?

Hi All, I have a .txt file with the list of filenames as given below. /dev_data/dev3/ctl/20120622_Employee.txt /dev_data/dev3/ctl/20120623_Employee.txt /dev_data/dev3/ctl/20120624_Employee.txt I want to read this file & write the dates alone from the filenames into a .done file. ... (6 Replies)
Discussion started by: dsfreddie
6 Replies

4. Shell Programming and Scripting

Download previous day files through FTP

Hi All, I have a scenario where I need to download the previous day modified files from other server to my server through FTP . Could any one please send me the shell script for the same. I used the following but I dont know how to proceed after this. ftp -n -i -v $IP <<ENDOFinPUT >>... (3 Replies)
Discussion started by: sarathchandrach
3 Replies

5. UNIX for Dummies Questions & Answers

Device Files names and location

HI there. I am studying System administration right now and am stuck on a question that I have failed to find the answer for in my book. The question is: Do device files need to be in the /dev directory and to they need to follow a naming convention? My answer (so far) is that since the... (7 Replies)
Discussion started by: Carl1976
7 Replies

6. UNIX for Dummies Questions & Answers

Checking files in ftp location

I am having a requirement to check whether files were there in the ftp location and if the files were not there then need to send a mail to someone.Suppose there were files like A,B,C,D,etc.. if the file A was not there then mail to someone similarly for all the files. Thanks for your help. ... (2 Replies)
Discussion started by: nimu1979
2 Replies

7. Shell Programming and Scripting

how to put file from one ftp to another ftp location

hi, I have one file located in ftp.I want to place that file in different ftp location. Up to now i am doing it manually using get command and copying that file in to my local system and then putting that file in different ftp using put command. now i want to automate that using shell... (1 Reply)
Discussion started by: prasee
1 Replies

8. Shell Programming and Scripting

Get Files from ftp which are uploaded recent week

Hi All, Here is a brief scenario for my requirement .. There is a directory in FTP Server, where would files be uploaded on weekly basic. I need to get those files which are uploaded during this week and not the files which are uploaded the previous week and download them to locale... (1 Reply)
Discussion started by: narramadan
1 Replies

9. Shell Programming and Scripting

grep a log file to filter previous dates

Hi, I have problem of filtering a log file from my perl script. #cat /data/pinpe.csv_20070731 | nawk -v FS=, '{print $1','$18','$22','$26}' | grep -w 100 | grep -w 1 | nawk '{print $4}' Below is the output: 2009-06-16 2009-01-29 2009-06-02 2008-03-05 2007-08-05 2007-09-24... (5 Replies)
Discussion started by: pinpe
5 Replies

10. Shell Programming and Scripting

transferring files using ftp but mantaining the dates

Hi guys, Is there a way I can ftp some files without changing the last-modified dates from them ? I haven't seen any ftp option to do so. Or, alternatively, how do I change the last-modified date for a file, if this is possible... ? Thanks in advance, Abrahao. (4 Replies)
Discussion started by: 435 Gavea
4 Replies
Login or Register to Ask a Question