Visit Our UNIX and Linux User Community


Script to transfer files without xcom


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script to transfer files without xcom
# 1  
Old 05-08-2012
Script to transfer files without xcom

Hi All,

I want to make a script in which I want to transfer files from current server to another.
The problem is that I don't have xcom installed on this so I was thinking of using scp command.
The code I've written till now is -
Code:
# Transferring latest inactive log file to another server
echo "Checking if there are inactive log files on the home directory...."
HOME_DIR=/o2/logs/weblogic/o2prl/
if [ `ls ${HOME_DIR}o2prl-05.log0* |wc -l` ];
then
echo "Inactive files are present"
echo "Continuing to further check the latest inactive file"
# Counting the number of inactive files
filename=`ls ${HOME_DIR}o2prl-05.log0* | tail -1`
file=`basename $filename`
# Start file transfer
scp ${HOME_DIR}${file} n412426@blx07ap09:/o2/home/
else
echo "There are currently no inactive log files on the home directory...."
fi

This is not running, basically I want to transfer file without xcom.
Please suggest.
Thanks
# 2  
Old 05-08-2012
You have lots of options including ftp, remsh/rsh, sftp, NFS, samba and probably more. It depends what you are trying to acheive overall, if your servers are local, is the data sensitive (e.g. client personal details or company secret values)

Can you elaborate a little to stop us shooting off down an unacceptable path?

What are your OS versions for the two sides?




Robin
Liverpool/Blackburn
UK
This User Gave Thanks to rbatte1 For This Post:
# 3  
Old 05-08-2012
Quote:
Originally Posted by rbatte1
You have lots of options including ftp, remsh/rsh, sftp, NFS, samba and probably more. It depends what you are trying to acheive overall, if your servers are local, is the data sensitive (e.g. client personal details or company secret values)

Can you elaborate a little to stop us shooting off down an unacceptable path?

What are your OS versions for the two sides?




Robin
Liverpool/Blackburn
UK
Thanks for the reply Robin.
The OS versions of the two sides are -
GNU/Linux 2.6.18-194.3.1.el5
The data that I need to transfer is sensitive and the servers are not local and they have super user accounts too.
# 4  
Old 05-08-2012
and what you want to achieve at the end? between those two boxes?
# 5  
Old 05-08-2012
Probably better to avoid sending in files with a superuser account if you can avoid it. Can you create a standard account that has write permission to the required area on the target server? It would be a little safer.

After that you are probably better with SFTP. Would a plain FTP (of a similar sized file) transfer in a suitable time? You say that it is sensitive, but this is just to get an idea of throughput and what is acceptable. If we can avoid compression, that saves a bit of work.

Other options depend on the tools you have available and the network access, for instance, do you have an encryption tool such as PGP? We could use this with rsh/remsh to copy the data.

It's all a bit of trying to fully understand the requirements at first along with options, limits and potential problems. I have a few ways in my head, so it's trying to pick the best avenue.



What more can you tell us?



Robin
Liverpool/Blackburn
UK
# 6  
Old 05-08-2012
Quote:
Originally Posted by rbatte1
Probably better to avoid sending in files with a superuser account if you can avoid it. Can you create a standard account that has write permission to the required area on the target server? It would be a little safer.

After that you are probably better with SFTP. Would a plain FTP (of a similar sized file) transfer in a suitable time? You say that it is sensitive, but this is just to get an idea of throughput and what is acceptable. If we can avoid compression, that saves a bit of work.

Other options depend on the tools you have available and the network access, for instance, do you have an encryption tool such as PGP? We could use this with rsh/remsh to copy the data.

It's all a bit of trying to fully understand the requirements at first along with options, limits and potential problems. I have a few ways in my head, so it's trying to pick the best avenue.



What more can you tell us?



Robin
Liverpool/Blackburn
UK
As per the requirement, compression not required as such.
I tried using ftp as -
Code:
# Transferring latest inactive log file to another server
echo "Checking if there are inactive log files on the home directory...."
HOME_DIR=/o2/logs/weblogic/o2prl/
HOST=blx07ap09
USER=o2prlods
PASSWD=xxxxxxx
if [ `ls ${HOME_DIR}o2prl-05.log0* |wc -l` ];
then
echo "Inactive files are present"
echo "Continuing to further check the latest inactive file"
# Counting the number of inactive files
filename=`ls ${HOME_DIR}o2prl-05.log0* | tail -1`
file=`basename $filename`
# Start file transfer
ftp -i -n $HOST
user ${USER} ${PASSWD}
cd ${HOME_DIR}
put ${HOME_DIR}$file
else
echo "There are currently no inactive log files on the home directory...."
fi

However as I am new to this area, hence, this is not working.

---------- Post updated at 04:57 PM ---------- Previous update was at 04:49 PM ----------

Quote:
Originally Posted by busyboy
and what you want to achieve at the end? between those two boxes?
I want to transfer file from the server I am making script in to the server blx07ap09.
# 7  
Old 05-08-2012
Be aware that a plain FTP will pass your data accross the network in readable form if someone can read in the network packets. Not a good idea to test with live data in any case, but also your credentials are passed in clear text. If you are using a superuser account, then you can easily be comprimised Smilie

The problem with your script is that when FTP starts, it has no input defined, so it will default to the screen. You would need to do something like:-
Code:
# Start file transfer
ftp -i -n $HOST  <<-EOFTP
user ${USER} ${PASSWD}
cd ${HOME_DIR}
put ${HOME_DIR}$file
EOFTP
else
......

... to get it to run.

We should be using SFTP for this, but it is set up and called in a different way. Hopefully the steps are as follows (please correct me anyone else that sees this)
  1. Create standard user account on target server with write permission to the required area
  2. Login in as the new account and generate a pair of encryption keys
  3. FTP the public key to the source server
  4. Add the new account to /etc/ftpusers to prevent normal FTP connections
  5. Test the SFTP connection

So, tackling these:-
  1. I'm assuming that this is okay.
  2. Generating keys is accomplished with the command ssh-keygen if you have installed the SSL software. Just run it on the command line and press ENTER to the prompts of where to store it and use a blank passphrase.
  3. I assume you can acheive this. The file to copy is .ssh/id_rsa.pub in the home directory of the new user. Put it on the source server as .ssh/authorized_keys ensuring the directory is rwx --- ---
  4. Add the account with the editor of your choice.
  5. Run sftp $targethost. It will ask you to verify the target firsdt time. You should see either an encryption key fingerprint string or picture. Accept this and it will create a record in .ssh/known_hosts allowing access next time, but warning you if it changes. Close the SFTP with the usual bye quit or close for a normal FTP.
If you get all that running, we can carry on with automating it, but best to get this far first. Then others can chip in too. I do not proclaim to be an expert in this (or any other) area.



I hope that this helps to start off with. Do ask for help if you want it and I haven't explained properly.



Robin
Liverpool/Blackburn
UK
This User Gave Thanks to rbatte1 For This Post:

Previous Thread | Next Thread
Test Your Knowledge in Computers #746
Difficulty: Medium
The Z80 is an 8-bit microprocessor introduced by Zilog.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with a script to transfer files from Solaris to windows

Hi Please can you tell me what could be wrong with the following scriptto transfer files from solaris 10 to windows machine: #!/bin/sh HOST=<IP> USER=administrator PASSWD=xyz123zyx /usr/bin/ftp -inv <<EOF connect $HOST user $USER $PASSWD cd Documents binary mput *.sh bye EOF (23 Replies)
Discussion started by: fretagi
23 Replies

2. Shell Programming and Scripting

Script to Exclude Files That Still On Transfer..

Hi. I want to schedule a job at some directory will several files in it. But there maybe a situation whereby some of the files at the point of the schedule are still transferring during that time. So I want to skip those files from being processed. Two method that come to my mind: 1.... (5 Replies)
Discussion started by: aimy
5 Replies

3. Shell Programming and Scripting

Transfer files from one server with bash script

Hello to all, I want to copy from one server to another files of last 24 hours with size between 500MB and 2GB. The code below searches last files in 24 hours. find . -mtime -1 In order to copy faster I'd like to compress the files before copying them. How to automate the process of... (8 Replies)
Discussion started by: Ophiuchus
8 Replies

4. Shell Programming and Scripting

Help with script to transfer files from one server to another

Hi I have the following script: #!/bin/sh set -x touch -mt 201210040000 /tmp/ref1 touch -mt 201210042359 /tmp/ref2 find /fs1/bscsrtx/BSCS_ix/WORK/LOG -type f \( -newer /tmp/ref1 -a ! -newer /tmp/ref2 \) > file_lst scp $(< file_lst) root@10.100.48.76:/ano2005/fs1_2015/LOG/ but somehow its... (7 Replies)
Discussion started by: fretagi
7 Replies

5. Shell Programming and Scripting

Script to transfer files from Solaris to windows

Hi All Please can you help, Ive wrote the following script on a solaris 10 server to transfer files to a windows machine. #!/usr/bin/sh cd /moneta_polled01/download HOST=10.100.4.72 USER=user1 PASSWD=V7stop /usr/bin/ftp -v $HOST <<EOF user $USER $PASSWD cd tmp binary put... (7 Replies)
Discussion started by: fretagi
7 Replies

6. Shell Programming and Scripting

Need script using cp to transfer files to another folder

Hi guys , how are you doing? I am tryng to make a script using cp that i would like to do this: echo "Enter the name files to tranfer" then user has to enter via keyboard this kind of output file1 file 2 file 3 file x (i mean the number of the files could vary and their name too)... (1 Reply)
Discussion started by: REX:)
1 Replies

7. Shell Programming and Scripting

Script to transfer files

Hi, I am new to scripting, I need to write a script to transfer .TXT files from Server A (ftp) to Server B. Once the files in server B i need to transfer it to server C (sftp). (not transfer of files directly from server A to Server C ) Thanks! Regards Sendhil ---------- Post updated at... (3 Replies)
Discussion started by: Sendhil.Kumaran
3 Replies

8. Shell Programming and Scripting

Script for FTP (transfer only new files)

Hi everybody, I just want to transfer files with FTP (mget and mput). The problem is that I dont want to overwrite any existing files and don't want to transfer them again (e.g. using the rename-function). So I only want to transfer new files with mget and mput. My first idea was to create... (3 Replies)
Discussion started by: inoxx
3 Replies

9. Shell Programming and Scripting

shell script to transfer files from Unix to Windows

I need to write a shell script to transfer files from Unix server to windows machine. This script will be scheduled on scheduler to run at specified intervals #!/bin/ksh ftp -n alaska <<End-Of-Session user sss01 sample cd /home/sss01 lcd D:/sample mget *.txt bye when I executed the... (15 Replies)
Discussion started by: knag
15 Replies

10. Shell Programming and Scripting

How to transfer files (By using generic script) when using sftp

I have written generic script to transfer files from one machine to other machine(By passing the command line arguments like server name, user name ,location of the files, file names etc) but when i am using sftp, what are the things I have to specify in the code Is it necessary to specify... (0 Replies)
Discussion started by: gsri
0 Replies

Featured Tech Videos