I am posting this as an FYI. I burned a lot of time on digging for this solution. The problem was performance between newer AIX p series boxes and a sun 4500 with a fiber nic. I tried many misc. solutions but this turned out to fix the issue. My bandwidth benchmark went from around 20MB/sec to 60MB/sec. after the change. (the other symptom I had was on the my cisco catalyst 6500 series switch the RxPause grew to 272448910)

I made the change from the console without rebooting.

Hope this helps someone! -Chuck

IBM guys comments:

There have been some reported problems with FC 5700, 5701, 5706 and 5707 which are the newer Gigabit Ethernet adapters we support in the pSeries systems. The devices have two TCP offload features, large_send (TCP Resegmentation) and chksum_offload (Checksum Offload), that are intended
to be performance features. In some instances the devices work as designed, but there have been issues with some software combinations where the offload features have caused problems. It is not difficult to disable the two features, but you want to make sure you have them either on or off for all devices that used the feature. To disable the large_send and chksum_offload you will detach the IP interfaces, put the device into a defined state, change the device configuration then reinitialized the
device and its interfaces. Sometimes it is easier to reboot the server as their may be application issues when IP interfaces are temporarily removed. The example shows how to change one device, ent1, and if you have other devices of the same type you will want to modify them as well.

1. If you can reboot:

chdev -l ent1 -a large_send=no -a chksum_offload=no -P
shutdown -Fr

2. If you can't reboot, but can take a very short communications outage
Note: if you get a device busy error when running the rmdev command then
try the ifconfig sequences again. If
you continue to get device busy error then use proceedure 1.

#!/bin/ksh
ifconfig en1 down detach
ifconfig et1 down detach
rmdev -l ent1
chdev -l ent1 -a large_send=no -a chksum_offload=no
# Checks to make sure device driver is unloaded
genkex | grep /usr/lib/drivers/pci/goentdd
# If above command returned no output then the driver is unloaded. If not,
use proceedure 1.
mkdev -l ent1
mkdev -l en1
mkdev -l inet0
exit

very interesting stuff.

thanks for the post.

altho we dont have fiber cards, we use etherchannel here on our aix machines.