UNIX for Advanced & Expert Users

I don't know it that's already solved, but if not, I wold see if your NICs have "ierrs". Sometimes data corruptions ocurrs in package transmitions. We had some problems here. Here folows an example:
# netstat -ni
Name Mtu Net/Dest Address Ipkts Ierrs Opkts Oerrs # netstat -ni
Name Mtu Net/Dest Address Ipkts Ierrs Opkts Oerrs Collis Queue
lo0 8232 127.0.0.0 127.0.0.1 124727 0 124727 0 0 0
ge0 1500 172.19.148.0 172.19.151.85 410543681 95593 221423296 0 0 0
ge1 1500 172.19.148.0 172.19.151.84 259411750 0 161355918 0 0 0
ge2 1500 10.152.231.0 10.152.231.150 258896122 0 496691048 0 0 0
hme0 1500 20.10.1.0 20.10.1.40 781 0 3 0 0 0


In ge0 (gigabit interface) we had some "ierrs" while "ftp'ing". That corrupted data. The FTP did not return any error, and completed successfull. But we had corruption problem. As we have two Nics in the same Network (Sun's IPMP thing) I forced my FTP connection to the other NIC (ge1) and then the data corruption problem was gone. Maybe this would help.

Htsubamoto

A few trouble shooting points:

1) have you tried to Gzip or tar the file before transfer?

2) have you tried to copy it to another host to check if it is a platform issue?

3) Can you establish an rlogini/rcp relationship and do a direct copy from one to the other?

4) If possible can you perform a reset on the NIC card to see if that may clear any errors? Of course this may interrupt normal dataflow while it is resetting.

Hi,

so first the problem is still alive But we have the right direction I think.

We were able to identify the problem, it looks like a linux - ia64 - NFS problem. We tried other platforms, and didn't get this error.

Only with our linux, ia64, fileserver is it possible to reporduce this problem.

At this moment our development is investigating the NFS code.

If someone is interested, I can public our solution for the problem, when our develoment send me the solution.

Regards
Alex

Hi again,

thank you to all of you very very much, with your hints, I was able to go in the right direction.

Next time, we will drink a beer together !!

Alex

Yes please post your answer.

I am sure that someone else may have this problem and need a starting point...

Hi all,

today I received the answer from our development related to the data corruption problem.
The solution is XFS, our used XFS has a bug, but see below the detailed answer :

--- snip ---

We have now found the cause of the data corruption.
This is original bug from XFS.

The special timing of the delete operation of a block and
flush operation from the memory leads to the data corruption.

To be prepared for next I/O operation, XFS pre-allocates the data
on the memory larger than what is actually needed.
Sometimes also unnecessary block are removed from pre-allocated data
on the memory to efficiently use the memory resources.
This happens when memory resource are fully utilized.

When this "pre-allocation" and "removal of unnecessary data"
occurs with certain timing, next data to be processed is written
in the area where just removed from the pre-allocation which shouldn't
happen. Since the data is written into the wrong place, the block
in the memory which should contain the data does not contain the
actual data which causes the data corruption when it is flushed to

Following conditions are needed to have the bug to aprear:

- Memory is fully loaded
- I/O is done via NFS
- Large file is being written continuously and concurrently
(magnitude of GB)

--- snap ---

may be someone has a similar problem, so thanks a lot to all and have a nice day.

Regards
Alex