Solaris

Hi all,

I am writing this post in order to share my experience with the SUN nxge ethernet interfaces and try to see if some of you managed to solve similar issues than the ones we experienced here with T5520 running Solaris 10.

First, we noticed that it was somehow difficult to have those links working at 1Gbps.
After lots of tests and a call to SUN support, we configured the interfaces , by modifying the nxge.conf file so that the autonegotiation is switched on (mandatory if you want to reach 1Gpps) but the only speed advertized is 1Gbps/full-duplex.
Configuring e1000g interfaces is much easier: only activate autonegotiation and it works immediately at 1Gbps/full-duplex

Second issue, and for that one we are still looking for a solution, is the instability of those links when using them as heartbeat in a Veritas cluster environment.
We first blamed the Huawei switches that were connected at the other end and replaced those switches by simple hubs purchased at the supermaket. This was better as we did not loose completely both heartbeat links at the same time but still we sometimes see hb lost messages reported by Veritas.
We have configured the second heartbeat on a e1000g interface and this one never give any message.

Has anyone of ou experienced similar issue and if so, how did ou overcome it (solaris patches, modification of driver parameters, re-configuration at veritas level, ...) ?

Thank you in advance !
Fape

I didn't think there even was such a thing as an ethernet "hub" for gigabit. Never seen anything but switches.

What exactly did you install?

You're right, I think what shops sell as hubs are in fact L2 switches.
Ours is from D-link.
As the links we wanted to interconnect are only used for heartbeats, we finally configured them as 100Mbps/full-duplex, fixed speed.

We used to sell d-link products until one month we had nearly 50% of them come back faulty. One nearly caught fire... Our experience since then hasn't convinced me they've improved much at all.

We're stuck with a few we stocked, unwilling to sell them to anyone, and unable to find much use for them internally; the "gigabit" switches we have negotiate at gigabit speeds, but only get 100baseT transfer rates, slowing the network down to their speed by spamming pause packets.

Hopefully they'll function well enough at 100baseT for you, but watch them.

OK, thank you that's good to know.
One thing I did not mention is that on another system, with similar setup we used professionnal switches from Extreme (Summit X250e-24t) and when connecting the nxge inerface to those we see similar hb lost messages from Veritas.
Exact messages are: "...kern.notice] LLT-INFO V14-1-1-10023 lost <x> hb seq ..."

While connecting the Extreme switches to the e1000g interface, there is no single message.
This on top of the the negotiation issues mentionned earlier is why we thing the nxge are the source of the problem.

i work for years with nxge interfaces in all kinds of sun hardware (m3000-m9000 and t-series). also they are in cluster environments (sun cluster) and i've never seen any of the above problems... i remember there was a batch of nxge cards with wrong mac adresses on them, which caused some problems. but thats all i remember about problems with nxge interfaces...