You can't connect 10baseT to a serial port! It won't work. 10baseT is not a serial port. They've got nothing to do with each other unless you've set up something crazy like SLIP.
I can't be certain from the description you've given me, but it sounds like the server has a nonstandard serial port connector that
looks like 10baseT -- not unheard of -- and you've found or made an adaptor cable for it to connect to your windows machine's serial port. If that's correct, you need to communicate with the serial port, not the network -- a serial port has nothing to do with networking unless you've set up SLIP or something. Think of a serial port as a single, devoted telnet connection that always exists and never closes.
I reccomend
tutty for communicating with serial consoles in Windows. Select the 'serial' protocol when you open it, leave the phone number blank since you're not talking to a modem, and pick the port options the console needs under the
connection -> serial options.