Sponsored Content
Top Forums Shell Programming and Scripting Automated scp using shell & expect Post 302984565 by veeresh_15 on Thursday 27th of October 2016 09:58:36 AM
Old 10-27-2016
Oracle Automated scp using shell & expect

Hi All,

I have written a script to automate scp of files. Most of the times it works fine except few cases. I want your help and suggestions to fix these failures.

I have used expect & shell to do the automated scp. Below is code


Code:
$ cat scp.ksh
#!/bin/ksh
inputfile=$1
fdest_sid=$2
echo -n "Enter Fully Qualified Source Server Name:"
read -r rserver
echo -n "Enter userid of source server:"
read -r ruser
echo -n "Enter password for source server:"
stty -echo
read -r rpassword
stty echo
echo  ""
export rserver
export ruser
export rpassword

export logfile=file_transfer_$(date +%Y%M%d_%H%M%S).log
> $logfile

cnt=$(cat $inputfile|wc -l)
var=$(cat $inputfile|tr '\n' ' '|sed -e 's/.$//g')
scp_cmd=$(echo "scp $ruser@$rserver:'${var}' $fdest_sid")
./expect_scp "$scp_cmd" "$cnt" >> $logfile 2>&1




$ cat expect_scp
#!/usr/bin/expect
set scp_cmd [lindex $argv 0]
set cnt [lindex $argv 1]
set rpass "$env(rpassword)\r"
set prompt "(%|#|\$) $"
set j 0
spawn ksh
send "$scp_cmd\r"
expect "Are you sure you want to continue connecting (yes/no)?" {send "yes\r"}
expect "*assword:"  {send "$rpass\r"}
while {$j < $cnt} {
        expect {
                "ETA" {
                        puts "still transferring..."
                        exp_continue;        
                        }
                "100%" {
                        puts "done"
                        }
                }
        incr j
}
expect "*$"
send {"exit\r"}


In file transfer log, i can see the shell command that i pass to expect script are send to the spawned shell with some control characters.

eg:

command passed to expect script:
Code:
scp oracle@server.domain.com:'/FS1/DB1/ETL_STAGING_A01.dbf /FS1/DB1/KALIDO_IN_D03.dbf /FS2/DB1/ETL_STAGING_A02.dbf /FS2/DB1/KALIDO_IN_D04.dbf /FS1/DB1/ETL_STAGING_A03.dbf /FS1/DB1/KALIDO_IN_D05.dbf /FS2/DB1/ETL_STAGING_A04.dbf /FS2/DB1/KALIDO_IN_D06.dbf /FS1/DB1/ETL_STAGING_A05.dbf' /FS3/DB2


Expect Script sending below command to shell:
Code:
$ scp oracle@server.domain.com:'/FS1/DB1/ETL ^H_STAGING ^H_A01.dbf /FS1/DB1/KALIDO ^H_IN ^H_D03.dbf /dbbcnprod/or^M$ oradata02/DB1/ETL ^H_STAGING ^H_A02.dbf /FS2/DB1/KALIDO ^H_IN ^H_D04.d                                                                                 <^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^Hbf /FS1/DB1/ETL ^H_STAGING ^H_A03.dbf /FS1/DB1/                                                                                ^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^M$ WH/KALIDO ^H_IN ^H_D05.dbf /FS2/DB1/ETL ^H_STAGING ^H_A04.dbf /dbbcnprod/o                                                                                ^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^Hradata02/DB1/KALIDO ^H_IN ^H_D06.dbf /FS1/DB1/ETL ^H_STAGING ^H_A05.^M$  ^H_D06.dbf /FS1/DB1/ETL ^H_STAGING ^H_A05.dbf' /u02/oradata03/DB2                                                                                ^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^M


These control charcters are causing scp failures some times.
How do i get rid of these control characters?


Regards,
Veeresham
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

scp automated script

Hi Unix gurus, I am trying to create a script to automate the copying of files daily from one server to another using the scp command. --> #!/bin/ksh KEY="$HOME/.ssh/SSHKEY" if ;then echo "Private key not found at $KEY" >> $LOGFILE echo "* Please create it with \"ssh-keygen -t dsa\" *"... (5 Replies)
Discussion started by: gholdbhurg
5 Replies

2. Solaris

SCP & SSH errors

Hi I am trying to scp a file between to servers (both on same subnet and can see each other). However, whenever I try I get the following error: ld.so.1: ssh: fatal: relocation error: file /usr/local/bin/ssh: symbol EVP_CIPHER_CTX_key_length: referenced symbol not found lost connection I... (4 Replies)
Discussion started by: skewbie
4 Replies

3. Shell Programming and Scripting

How to put scp in background inside expect

Gents, I have a wrapper script passing couple of information to an expect script mainly responsible for spawning scp and providing the password (which is transmitted down from the main script). the main script prepare the transfer to couple of servers, idea being to transfer the files in... (3 Replies)
Discussion started by: luc3004
3 Replies

4. UNIX for Advanced & Expert Users

Automated SCP script passing password to preserve source file timestamp

Hi My requirement is i want to copy files from remote server to the local server and also i need to preserve the timestamp of the remote file. By using scp -p , it is working fine in the interactive call but it is not preserving he file timestamp when i use it in the non interactive scp call... (1 Reply)
Discussion started by: skumar75
1 Replies

5. Shell Programming and Scripting

Automated SCP from a job?

Hi all - I have a script which runs on the OS level, but refuses to run as a cron or as an Oracle job. The Script is pretty straight forward: #!/bin/bash username="MyUsername" host="Remote.server" path="Remote.directory/files/*.*" password="MyPassword" expect -c " spawn /usr/bin/scp... (3 Replies)
Discussion started by: danimaltex
3 Replies

6. Shell Programming and Scripting

scp not working in expect script

Hi All, I run the scp command in shell prompt without issue, but when on expect script as below: #!/usr/bin/expect spawn scp /var/spool/sms/failed.tar.gz abc@10.10.12.2:/home/abc expect "abc@10.10.12.2's password: " send "abcfef\r" exit 0 It looks not working at all and the... (3 Replies)
Discussion started by: elingtey
3 Replies

7. AIX

How to use 'expect' to pass UID & Password to a "for loop" in shell script?

Friends, Need someone's help in helping me with the below requirement for a script: > For a list of servers(over 100+), I need to login into each of them(cannot configure password-less ssh) & grab few configuration details < I know, this is possible through expect programming in a simple... (2 Replies)
Discussion started by: thisissouvik
2 Replies

8. Shell Programming and Scripting

How to use 'expect' to pass UID & Password to a "for loop" in shell script?

Friends, Need someone's help in helping me with the below requirement for a script: > For a list of servers(over 100+), I need to login into each of them(cannot configure password-less ssh) & grab few configuration details < I know, this is possible through expect programming in a simple... (14 Replies)
Discussion started by: thisissouvik
14 Replies

9. Shell Programming and Scripting

scp script getting timed out with expect

Hi, I have an expect script where in i am trying to scp a folder but it is getting timed out. Any help will be appreciated. (I don't have the option for sharing keys) expect -c 2> /dev/null " spawn scp -r -o NumberOfPasswordPrompts=1 -o StrictHostKeyChecking=no root@10.10.10.10:test_dir... (2 Replies)
Discussion started by: temp_user
2 Replies

10. Shell Programming and Scripting

Using Expect and scp. Not moving directories

An Expect script using scp to copy directories is not moving the subdirectories as wanted. The directory structure is: computerA (user 'test') /home/test/parentDir/subdir1 /home/test/parentDir/subdir2 and I want them copied to computerB (user 'archive') /home/archive/subdir1... (1 Reply)
Discussion started by: nohj ordpl
1 Replies
All times are GMT -4. The time now is 09:50 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy