IP Networking

I'm working on a queue manager type application. I'd like to be able to delegate a large job constituted of many smaller task to a handful (or potentially dozens) of commodity linux machines on a LAN. It is quite likely this LAN will be ethernet but it might end up being Fibre Channel or Infiniband. I expect I'll be using TCP for all communication between nodes because it's been implemented on all of these network technologies and results in reliable, point-to-point data transmission.

HOWEVER -- and this is the real question here -- I'd like to be able to attach a new machine to the network and have it notify the other machines that it is available to work on these tasks. At the very least it must notify the queue manager. I know how this would be done if the new node was aware which ip was the queue manager or if the queue manager knows the network address of the new node. Is there a way to do it without specifically apprising the computers of each other's presence? I.e., can they just 'find each other' ?

The most simple way I can think of has exactly one manager on any given LAN segment. Upon power-up, the client then sends a gratuitous ARP packet, announcing it's presence.

On a related note, you might be interested in some of the software and protocols used to build a Beowulf cluster

Pludi, you are awesome. Thanks for that.

Glad to be of help.

What I forgot to mention is that the Linux arping utility can send such an packet, eg

Code:

# arping -U 192.168.1.5

will tell the other devices on the segment that your IP is 192.168.1.5. It's open source, so you can take a few pointers off it, or use it directly from some script.