UNIX for Advanced & Expert Users

Hello all,

I have two twins Proliant DL580 G3 with RHEL4U4 clustered with Oracle RAC with two nics working in each one.

All nics have the same firmware, driver and type of physical cable connected (Cat 5E). All ports of the switch have the same configuration but only one of the nics of only one server can negociate to 1Gbps.

The main difference between nic working 1Gbps and the rest is the others nics Support 1000baseT/Full but this mode is NOT Advertised.

I have readed similar posts but in this case, I have two twin boxes which only work one interface of one box... It's very rare.

Configuration
------------------------------
Options: autoneg on
driver: tg3 -> version 3.92n 04B8459653BDAA6D1C7B798
version: 3.92n
firmware-version: 5704-v3.27b, ASFIPMIc v2.36
Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet (rev 10)
Kernel: 2.6.9-78.0.17.ELsmp


Thanks and regards.

Last night we change the cable in server is working Gigabit and use it for the server can't negotiate Gbit and we used the same switch port with no results....

¿Any suggestion why this nic can't negotiate Gigabit?

Hi Rubick,

for testing switch the cable of one of the ports where your adapter is not negotiating with the cable in the port that is negotiating ... if the other adapter works, it is a problem with the switch / switchport, if not it's an adapter problem. Did you try different switchport settings ? Do you experience any difference when setting the switchport to 1000/full rather than auto?

Rgds
zxmaus

We did a lot of test about 3 hours with service lost because is heartbeat interface, we tried many many differents configuracions in switch and adapter side (auto,full, etc...) with no results... We use the same cable and switch port with adapter working properly and adapter never shows advertisment 1000.

In this case, we have more servers and only one adapter of one server supports and advertisment one card in 1000/full others, only supports without advertisment.

Support tell us upgrade of all components... but I don't want to make this upgrade because one server is working with the same level of firmwares/drivers and I wan't to know the root problem not only "customer please, upgrade and maybe works"

By the other hand, maybe I will ask for mother board change as these interfaces are integrated.

I will upgrade this "X-file" when I come back of my holidays

Thanks.

Have you used ethtool to see statistics, settings, etc? See http://lxr.linux.no/#linux+v2.6.30.5/drivers/net/tg3.c for lots of information about this driver internals.

Yes, and ethool shows I commented before:

Code:

# ethtool eth1
Settings for eth1:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full ----> Suppports Gbit
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Full
                                100baseT/Half 100baseT/Full
                                --- Where is Advertised Gbit mode?

Looking at the source code, if a NIC is set in low power mode, it will not autonegotiate 1000baseT.

Suggest you dump the NVRAM of this NIC and a "good" NIC and see if they are the same.