Solaris

I have a simple makefile which I use to sync libraries files from /source/sybase directory on the source machine where the Makefile resides to /destination/folder on the various destination machines like machine1, machine2, machine3 using solaris utility 'rsync'. So when I run "make -f makefilename synclibraries", the sync goes through fine on all the machines. However if one of the machines (eg: machine2) is down, then makefile aborts midway because rsync fails and skips the sync on all the subsequent machines. Is there a way to capture the rsync error inside the Makefile and skip those machines which are down and proceed with syncing the subsequent machines ? If so how can I modify the below Makefile code. This would be really helpful because in future I would be expanding this makefile to handle 100s' of machines and it would be hard to track and rerun, if Make aborts midway due to an issue with any of those machines

MACHINELIST = loginname@machine1 \
loginname@machine2 \
loginname@machine3 \
loginname@machine4

synclibraries :
for target in $(MACHINELIST); do /usr/local/bin/rsync -avb --suffix .bk -e 'ssh -x' -n /source/sybase $$target:/destination/folder; done


Your help/suggestions would be greatly appreciated

thanks
Waavman

1. I know you have made the investment in using make but is this the correct tool for the job?

2. Split up the makefile into separate make file tasks, so that each makefile is a task that should complete together.

3. have a script that runs the make files tasks and logs the errors.

I know the error the Makefile is reporting. For example if machine1 has some connection issues then rsync cannot connect to machine1 and returns error. As a result Makefile aborts with fatal error instead of trying to run rsync on the next host which is machine2. See the Error output below. So how could I have the Makefile continue syncing machine2, machine3, machine4 etc. instead of just aborting. Its easy to remove the offending hostname from the makefile and rerun when the number of hosts is small like here. But when we have over 100 machines, the Makefile runs for several hours and to keep waiting for it to fail on some machine somewhere in the middle and then manually fix the makefile and rerun gets tedious. So and self correcting mechanism within the makefile to catch these errors and skip these erroneous hosts and proceed to the next machine in the list would be automated and convenient. Is there any way of achieving this ?
See Error output below when machine1 is the offending UNIX box.

for target in loginname@machine1 loginname@machine2 loginname@machine3; do /usr/local/bin/rsync -avb --suffix .bk -e 'ssh -x' -n /source/sybase $target:/destination/folder; done
warning: Connecting to machine1 failed: No address associated to the name
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(359)
*** Error code 12
make: Fatal error: Command failed for target `scsmerllistsync'


thanks
waavman

By having them as separate make targets (rules sets) and using a script that calls make on the outside and does not stop depending on the return code.

Code:

for machine in mach1 mach2 mach3
do
     if make -f make.mak $machine
     then
             echo $machine synced
     else
             echo 1>$2 $machine failed
     fi
done

The idea with make is it's supposed to stop when a target cannot be made.

thanks Porter. let me try tinkering the makefile like that and see how it goes.

The script mechanism works fine when one of the machines is completely down and does not have an IP address as rsync/ssh to that machine returns an error and so make terminates and make -f make.mak $machine returns an error code. However if the target machine has a corrupt drive or some similar issue, rsync/ ssh to that machine does not return error. Instead it hangs trying to connect to that machine
So when I run
if make -f make.mak $machine as you suggested below,
if a particular $machine has a corrupt drive, then rsync within the makefile just hangs trying to connect to that machine just as ssh to such a machine would hang without returning and error code. As a result the script calling the makefile hangs at the if clause ( if make -f make.mak $machine ) without proceeding to the next statement in the script. Is there a way I can signal an error when make -f make.mak $machine does not return after a while for any machine since it is not able to rsync / ssh to $machine due to issues such as corrupt drive etc. ?

thanks
waavman

The question of course is how would you know it's hanging due to a severe failure rather than just a busy network or busy server?

One possible solution is to run sleep in parallel but it starts getting messy very quickly trying to synchronise the various completion options. A solution would be to develop a generic timeout script that can run any command with a timeout. But you do need to solve the question regarding how do you know its failing?