Shell Programming and Scripting

Hi All,

I am trying to take Bulk Log files from around 24 Remote servers using my shell script. I am using SCP for this. I need to take log files every 15 min, so I am thinking about the parallel processing mechanism in UNIX.The 24 remote machines username,servername reading from properties file. My code will look like this :

while read list
do
----
nohup scp $FRONTEND_USER@$FRONTEND_SERVER:/REMOTEDIR/LOG_FILENAME.* . &
----
done < listdatafile.properties
wait // wait command is using for all the 24 background process to finish before doing the next part of the script

The problem now i am facing is it is doing the first SCP and prompting for enter key to come out from SCP command..I can't press enter key every time because this process is a cronjob. Can anybody help me from this problem ? Any suggestions in the existing part also most welcome.

Thanks and Regards,
Scorpio

Strange it's waiting for "Enter". You can ssh/scp without entering a passphrase right? You also created the keys without entering a passphrase, right?
Maybe try it out with "-B" or one of the ssh options "-o ....".

If nothing of that works, maybe setup up a central rsync server as alternative which can handle parallel transfers. You can define in the rsyncd.conf how many parallel connections are allowed.

Hi ,

" You also created the keys without entering a passphrase, right? "
sorry
I didn't understand your question ?

While doing the SCP ommand without & , it will not prompt for enter key to press. But that time the process will not run in background also..

Is it possible to give the enter key as a input parameter ? if yes how i can do that ?

Thanks and Regards,
Scorpio

When you create ssh keys for ssh or scp connections with ssh-keygen, you will be asked to enter a passphrase. For batch jobs/scripts you will not enter anything there but just press enter so you will not be asked everytime doing ssh/scp for a passphrase to enter.

Usually you don't have to press enter, even when using scp in a script via cron. So I recommended trying out inserting -B or -o with an approriate ssh option. Did you try out -B for example?

YEs , I tried nohup scp -B $FRONTEND_USER@$FRONTEND_SERVER:/REMOTEDIR/LOG_FILENAME.* . &

it is still prompting for enter key to press

Sorry, I can't reproduce the problem - it's working in my environment. If no one else gives a hint, maybe move to rsync.

You also should look into PDSH. There's a module to enable a command (pdcp) to work in parallel, where you specify the number of parallel operations. It will use rcp or scp, as you wish.