Sending a file to 24 Server locations parallely


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sending a file to 24 Server locations parallely
# 1  
Old 02-14-2010
Sending a file to 24 Server locations parallely

Hi All,

I have to send a processed file to 24 different server locations. I feel, if this job can be done parallel in the background - Time will come down.

I found the script to FTP the file for a single server location through past Unix posts as below:
Code:
#!/usr/bin/ksh
ftp -v -n "ipaddress" 
user "Username" "Username"

cd /distant/directory
put filename
If error >> ${LOG}
mail -x john@gmail.com -s FTP failed to the server ${ipaddress}
else
echo "FTP Successful to the server ${ipaddress} >> ${LOG}
quit

How to automate the script to send the Source File to Different FTP Locations.

For eg: If I create a file with all the 24 different FTP Target Locations in "Filelist.txt" as below:
Code:
Filelist.txt:
------------
ipaddress,Username,password
172.90.232.1,john,john
172.90.232.2,john,john
172.90.232.3,john,john
172.90.232.4,john,john
.
.
.
172.90.232.24,john,john

And pass the above file values as parameters to the original FTP Script script.

Can some one help me how to implement this solution in Unix scripting will be appreciated.

Thanks,
vsmeruga.

Last edited by Scott; 02-14-2010 at 12:18 PM.. Reason: Code tags, please...
# 2  
Old 02-14-2010
Code:
#!/usr/bin/ksh
while read ip user password
do
ftpscript $ip $user $password
done <filelist.txt

# 3  
Old 02-14-2010
Hi Jgt,

Thank you very much. I will try it tomorrow at office.

As per my understanding, this script reads a file: filelist.txt
and pass the 3 parameters(24 lines, 24 times) to the calling function: ftpscript right.

Is it possible to do this FTP parallely to reduce time.

Thanks,
vsmeruga
# 4  
Old 02-14-2010
You can add an ampersand (&) to the end of the ftpscript line, and that will release each copy to run independently. Depending upon where these files are going on your network, and the network speed, there may or may not be much difference in total throughput, and having each connection running more slowly might cause timeout errors.
One thing that I noticed, you have commas separating the fields in the input data file. Either use a text editor to replace these commas with spaces, or add
Code:
IFS=","

prior to the read statement in the script.

Last edited by jgt; 02-14-2010 at 04:12 PM.. Reason: Added comma/space delimiter
# 5  
Old 02-17-2010
Hi Jgt,

i tried the below script as you suggested. But i am not able to work it out.

When i run the script, It stops at the Message: "FTPing the file from the server". not going further. Please tell me what i am doing wrong.

etluser : 24ftplocations_i.sh
FTPing the file from the server


Script(24ftplocations_i.sh) code is as below:
Code:
#!/usr/bin/ksh
export filename=/grid/PowerCenter/stage/velocity_r3_dev/inbound/ATRPU/filelist.txt
export SOURCE_DIR=/grid/PowerCenter/stage/velocity_r3_dev/inbound/ATRPU
export ICOMS_FTP_TGT_DIR1=/dw/cdrsp/stage/velocity/vmedia/input/ATRPU/ICOM_SERV1
export ICOMS_FTP_TGT_DIR2=/dw/cdrsp/stage/velocity/vmedia/input/ATRPU/ICOM_SERV2
export FILE_MASK="ATRPU_RP_ATU"

ftp_data_file()
{
    
   # local parameters
   local FTP_FUNCTION=$1
   local FILE_NAME=$2
   local FTP_SERVER=$3        			# remote delivery server
   local FTP_USER=$4					# ftp user name
   local FTP_PASS=$5 					# ftp user pwd
   local FTP_TARGET_DIR=$6				# local target file directory
   
	# local variables
   local SOURCE_DIR=${SOURCE_DIR}   
         
	echo "FTPing the file from the server"

   # ftp file from remote server to local informatica server
   ftp -nv ${FTP_SERVER} <<-!!
	user ${FTP_USER} ${FTP_PASS}
	cd ${FTP_TARGET_DIR}
	 ascii
        prompt
	${FTP_FUNCTION} ${FILE_NAME}
	bye
	!!

   co_error_check $? "Failed to ${FTP_FUNCTION} file ${SOURCE_DIR}/${FILE_NAME} to ${FTP_TARGET_DIR}" 		
}

# main function

while IFS="," read ipaddress username password
do

# ftp file from remote server to local server
	cd ${SOURCE_DIR}

ftp_data_file mput ${FILE_MASK}*.csv ${ipaddress} ${username} ${password}   ${ICOMS_FTP_TGT_DIR1}
ftp_data_file mput ${FILE_MASK}*.csv ${ipaddress} ${username} ${password}   ${ICOMS_FTP_TGT_DIR2}
done <$filename[COLOR="#738fbf"]


Last edited by Scott; 02-17-2010 at 10:11 AM.. Reason: Code tags, PLEASE!
# 6  
Old 02-17-2010
!!, your end of input marker has to start in column 1. You appear to have a tab in front of it.
# 7  
Old 02-17-2010
Hi Jit,

I am very sorry. I am really not so good in understanding unix part. I could not understand what you are suggesting.

Please can you be more clear in your comments.

Thanks,
Vijaya Meruga.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Web Development

Sending file to WebDav Server

Hi All, I am using a webdav server host name : abc.xyz.com.ak username : user123 password : password123 port : 80 I need to send files stored in my windows system to the server, any idea how to do it. I dont know how to create a url in linux for webdav server using details. (2 Replies)
Discussion started by: prakhar_dubey
2 Replies

2. UNIX for Dummies Questions & Answers

Edit locations in a file

Hi , I have a file which looks like this source1 LEN predictive 392879 394347 0.955489 + . Name=sa000003.1;ID=sa000003;Alias=sa121751.1; source1 LEN descriptive_1 391082 392878 . . . Parent=sa000003.1;supp_id=.1805.1; ... (3 Replies)
Discussion started by: siya@
3 Replies

3. Shell Programming and Scripting

Inserting commas into file at set locations...

Hey everyone. What would be the best method to insert commas (or comma-quotes) into a file at set locations. Every line in the file looks like this: 1234567890123456 123456BIGAUDIODYNAMITE33 123.12 123456 12345678901234MARK E WILLIAMS 123456 The comma's should... (6 Replies)
Discussion started by: Astrocloud
6 Replies

4. Shell Programming and Scripting

Perl: Sending file from UNIX server to Windows server

I'm trying to write a Perl script where a file from a UNIX server box connects to a Windows server box and copies that file into the Window box. The main problem I have right now is that whenever I try to connect to the Windows box, the connection is refused. The error message that always pops... (2 Replies)
Discussion started by: kooshi
2 Replies

5. UNIX for Advanced & Expert Users

Insert Delimiter at fixed locations in a flat file

Hi Can somebody help me with solution for this PLEASE? I have a flat file and need to insert delimiters at fixed positions in all the lines so that I can easily convert into EXCEL with columns defined as per their width. For Example Here is the file { kkjhdhal sdfewss sdtereetyw... (7 Replies)
Discussion started by: jd_mca
7 Replies

6. Shell Programming and Scripting

Perl and sending file to server

Hi, I am trying to write some test code for bigger project where my perl script will send file over to server. This is the current SERVER and CLIENT code I have so far. When I type "hi" from client I get hello back from server and like wise I send send command I get respond back the problem... (1 Reply)
Discussion started by: Dabheeruz
1 Replies

7. Solaris

Sending a file to a server

Dear Experts, please help i am very much confused here and not getting the right path, actually i want to send a file from client to a server, initially i used mmap() which is failing most of the time so i want to use general IO streams to do that, i can open a file in "rb" mode and read... (1 Reply)
Discussion started by: vin_pll
1 Replies

8. Shell Programming and Scripting

executing 2 or more files inside another file parallely

hi All, i have 3 files file1 file2 file3 these 3 files are shell (KSH) scripts. i want to place all these 3 shell scripts in another file file99 and execute these files simultaneoulsy inside the file99 these 3 files should be present and executed simultaneously when i excute file99 (nohup... (1 Reply)
Discussion started by: dareman123
1 Replies

9. Solaris

file locations...

Hi Guys, There was a post that I saw here a while ago regarding file system layout and what to put where, which I am unable to find now.. A user here posted a man page that list what each partition should have in it and what it is used for and were you should install custom packages. ie:... (1 Reply)
Discussion started by: Tornado
1 Replies

10. Programming

sending file from server to client

hi dear i m very new to socket programing . i need the source code which sends file from server to client . i mean both server n client programe which sends file . can u do this for me please my email id is email id removed regards bilal (1 Reply)
Discussion started by: bilal
1 Replies
Login or Register to Ask a Question