Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Automated File Transfer Script Post 303008810 by Sidhant on Wednesday 6th of December 2017 03:01:10 PM
Old 12-06-2017
Automated File Transfer Script

We are receiving data feed files in SFTP location daily. so the folder structure in SFTP location is abc/def/studyname_1/outbound/zipped files
So we will be getting different studies and for each study a folder is created abc/def/studyname_2/outbound/zipped files , abc/def/studyname_3/outbound/zipped files

Script needs to go to each study name and then fetch the zipped files to another Windows location.

Here I would want to have a reference table created in SQL where I have listed all the incoming zipped files and its SFTP location. So the script also needs to cross verify with the sql table and then pick-up the files.

Or

we can fetch zip filenames and then insert into sql table and mark it as processed.

I understand we can use MGET to fetch all the files in SFTP location. I am confused on how I can achieve the tracking of the zip files and how to go through each study and fetch files.

Code:
if [ $ftpmethod = SFTP_MGET_NOEXT ]
then
/usr/bin/expect -f - <<EOFMGET 
spawn sftp $remoteusername@$remotehost
expect "password:"
send "$remotepwd\n" 
expect "sftp> "
send "cd $remotedir\n"
expect "sftp> "
send "lcd $localdir\n"
expect "sftp> "
send "mget $filename*\n"
expect "sftp> "
send "sleep 120\n"
expect "sftp> "
send "exit\n"
interact
EOFMGET
fi

Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!

Last edited by RudiC; 12-06-2017 at 04:08 PM.. Reason: Added CODE tags.
 

10 More Discussions You Might Find Interesting

1. UNIX and Linux Applications

Automated file transfer

I want to automate the file transfer from a remote site to my pc over a WAN. Can anyone please suggest an industry standard secure file transfer tool for a windows environment? I am looking for an industry accepted, more secure than traditional FTP. Thanks! (1 Reply)
Discussion started by: damienjine
1 Replies

2. Shell Programming and Scripting

Perl automated file transfer

Hi, Firstly, I have no experience (at all) with shell scripting. So please, go easy on me :) I did a search for what I was looking for although there were a few things available here and on the net I couldn't find anything tailored to what I am looking for. Simply put, I have two servers.... (2 Replies)
Discussion started by: lastrider
2 Replies

3. UNIX for Advanced & Expert Users

Automated SCP script passing password to preserve source file timestamp

Hi My requirement is i want to copy files from remote server to the local server and also i need to preserve the timestamp of the remote file. By using scp -p , it is working fine in the interactive call but it is not preserving he file timestamp when i use it in the non interactive scp call... (1 Reply)
Discussion started by: skumar75
1 Replies

4. Shell Programming and Scripting

Expect script file transfer failure

I use expect to spawn an sftp file transfer. On occasion the transfer fails or doesn't complete correctly, resulting in: "sftp> cd /returns sftp> rename /returns/TESTFILE.TXT /returns/archive/TESTFILE.TXT Couldn't rename file "/returns/TESTFILE.TXT" to "/returns/archive/TESTFILE.TXT": Failure... (0 Replies)
Discussion started by: tjb1959
0 Replies

5. Shell Programming and Scripting

Problem in Weekly file Transfer script

I have made a script which transfers some files of the entire week , but the script fails when the next month is started. For e.g; if i run the script on 5th may , but i need to transfer files of its previous week which is from 24th April to 30th april ,the script fails, i have this loop in the... (2 Replies)
Discussion started by: vee_789
2 Replies

6. Shell Programming and Scripting

Automated script to take 1000 records from the file every week.

I have a file having n number of records .i want first 1000 records from the file and store in temporary file to process on sunday. I want script should should automatically take the next 1000 records for processing on next sunday. can we do it using head and tail head -1000 | tail... (2 Replies)
Discussion started by: sonam273
2 Replies

7. Shell Programming and Scripting

File transfer script

Hi, I need a shell script to transfer a file from one server(unix box) to another server(windows box). I have the details of the source and destination Ip's. source path : /home/UNIX/server filename:abc.txt Destination folder: D:/UNIX/test I am using AIX server. Type of shell :... (1 Reply)
Discussion started by: NareshN
1 Replies

8. Red Hat

UNIX script for daily file transfer ???

I need to create a script which needs to transfer the "<filename>+yyyymmdd".dat file from a unix machine to a linux machine.. daily. we can use an active batch to schedule the script. however, the script needs to detect if the new file is present, if present, then transfer. any help ?? (1 Reply)
Discussion started by: ravikodi
1 Replies

9. Shell Programming and Scripting

Deleting local server file from automated FTP script

Hi, I want to delete a file on the local server, while connected to remote server through FTP. I am using the below code for this $FTP_CMD -v -n $HOST <<*! >> $LOGFILE 2>&1 user $USER $PASSWORD cd $DIR ... (11 Replies)
Discussion started by: jhilmil
11 Replies

10. Shell Programming and Scripting

How to make script for file transfer?

Hi All, Please suggest me how to make script for file transfer from server X to another server Y. I have some directory path in server X as below: /home/directory_1/ . . /home/directory_n/ where some text files are available in each directory where some files records zero and some files... (6 Replies)
Discussion started by: aaditya321
6 Replies
Char(3) 						User Contributed Perl Documentation						   Char(3)

NAME
PDL::Char -- PDL subclass which allows reading and writing of fixed-length character strings as byte PDLs SYNOPSIS
use PDL; use PDL::Char; my $pchar = PDL::Char->new( [['abc', 'def', 'ghi'],['jkl', 'mno', 'pqr']] ); $pchar->setstr(1,0,'foo'); print $pchar; # 'string' bound to "", perl stringify function # Prints: # [ # ['abc' 'foo' 'ghi'] # ['jkl' 'mno' 'pqr'] # ] print $pchar->atstr(2,0); # Prints: # ghi DESCRIPTION
This subclass of PDL allows one to manipulate PDLs of 'byte' type as if they were made of fixed length strings, not just numbers. This type of behavior is useful when you want to work with charactar grids. The indexing is done on a string level and not a character level for the 'setstr' and 'atstr' commands. This module is in particular useful for writing NetCDF files that include character data using the PDL::NetCDF module. FUNCTIONS
new Function to create a byte PDL from a string, list of strings, list of list of strings, etc. # create a new PDL::Char from a perl array of strings $strpdl = PDL::Char->new( ['abc', 'def', 'ghij'] ); # Convert a PDL of type 'byte' to a PDL::Char $strpdl1 = PDL::Char->new (sequence (byte, 4, 5)+99); $pdlchar3d = PDL::Char->new([['abc','def','ghi'],['jkl', 'mno', 'pqr']]); string Function to print a character PDL (created by 'char') in a pretty format. $char = PDL::Char->new( [['abc', 'def', 'ghi'], ['jkl', 'mno', 'pqr']] ); print $char; # 'string' bound to "", perl stringify function # Prints: # [ # ['abc' 'def' 'ghi'] # ['jkl' 'mno' 'pqr'] # ] # 'string' is overloaded to the "" operator, so: # print $char; # should have the same effect. setstr Function to set one string value in a character PDL. The input position is the position of the string, not a character in the string. The first dimension is assumed to be the length of the string. The input string will be null-padded if the string is shorter than the first dimension of the PDL. It will be truncated if it is longer. $char = PDL::Char->new( [['abc', 'def', 'ghi'], ['jkl', 'mno', 'pqr']] ); $char->setstr(0,1, 'foobar'); print $char; # 'string' bound to "", perl stringify function # Prints: # [ # ['abc' 'def' 'ghi'] # ['foo' 'mno' 'pqr'] # ] $char->setstr(2,1, 'f'); print $char; # 'string' bound to "", perl stringify function # Prints: # [ # ['abc' 'def' 'ghi'] # ['foo' 'mno' 'f'] -> note that this 'f' is stored "f" # ] atstr Function to fetch one string value from a PDL::Char type PDL, given a position within the PDL. The input position of the string, not a character in the string. The length of the input string is the implied first dimension. $char = PDL::Char->new( [['abc', 'def', 'ghi'], ['jkl', 'mno', 'pqr']] ); print $char->atstr(0,1); # Prints: # jkl perl v5.12.1 2009-10-17 Char(3)
All times are GMT -4. The time now is 12:38 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy