UNIX for Beginners Questions & Answers

We are receiving data feed files in SFTP location daily. so the folder structure in SFTP location is abc/def/studyname_1/outbound/zipped files
So we will be getting different studies and for each study a folder is created abc/def/studyname_2/outbound/zipped files , abc/def/studyname_3/outbound/zipped files

Script needs to go to each study name and then fetch the zipped files to another Windows location.

Here I would want to have a reference table created in SQL where I have listed all the incoming zipped files and its SFTP location. So the script also needs to cross verify with the sql table and then pick-up the files.

Or

we can fetch zip filenames and then insert into sql table and mark it as processed.

I understand we can use MGET to fetch all the files in SFTP location. I am confused on how I can achieve the tracking of the zip files and how to go through each study and fetch files.

Code:

if [ $ftpmethod = SFTP_MGET_NOEXT ]
then
/usr/bin/expect -f - <<EOFMGET 
spawn sftp $remoteusername@$remotehost
expect "password:"
send "$remotepwd\n" 
expect "sftp> "
send "cd $remotedir\n"
expect "sftp> "
send "lcd $localdir\n"
expect "sftp> "
send "mget $filename*\n"
expect "sftp> "
send "sleep 120\n"
expect "sftp> "
send "exit\n"
interact
EOFMGET
fi

Moderator's Comments:

Please use CODE tags as required by forum rules!