UNIX for Advanced & Expert Users

I will try to provide as many details as possible but this one has me scratching my head...

A few months ago I was asked to set up a test syslog server for internal use only. I chose to go the following route for simplicity:

1. I used Gentoo Linux so that I could have the most stripped down and highest performing box possible for the task
2. I also decided to use the Xen virtualization system so that I could easily migrate this server from this box to any other Xen box with minimal or no downtime
3. I chose syslog-ng as the syslog server since it was part of Gentoo's portage archives
4. I used the ReiserFS for all filesystems including the dedicated syslog filesystem

Xen details:

1. After installing the base Gentoo distro, I installed Xen from source and compiled a new kernel with support for everything I needed including the ReiserFS
2. I took the primary disk device and used LVM (again for simplicity's sake sinc e it's much more flexible than partitions) to slice it up for Domain0 (the "host" OS for lack of a better term) and the other domains, one of which was my syslog server
3. I then built my syslog VM (Domain1) based off the stripped down base OS for Domain0 on an LVM and unmounted it in Domain0
4. The new vg was then mounted as ReiserFS within Domain1 and I booted
5. I then emerged syslog-ng and got it configured to work as needed

Everything seemed fine. I was able to capture syslog... no problem. For one day's worth of Pix data I had a 48 Gigabyte syslog file. This is where the problem comes in. It appears the ReiserFS has no problem handling the file size. But if I attempt to transfer it off to any other system using FTP I get an error. I thought it might have something to do with the target system's file system. But one of the boxes I was testing with was an HP-UX box with vxfs and the largefiles option enabled. It SHOULD be able to handle a 48 gigabyte file. So to test further I decided to create a file that was slightly less than 4 gigs and see if it would transfer. Still didn't work. IN fact the only way a file will transfer is if I'm below 2 gigs. This is where I'm a bit confused. If the filesystem that currently contains the file can handle super large files, then why won't it transfer those files using FTP? See below:

FTP gives me the following:

Code:

ftp> get pix.log
local: pix.log remote: pix.log
200 PORT command successful.
550 pix.log: not a plain file.

I've tried pushing the files from the syslog server (ftp client) to the target hosts (ftp server). And I've tried pulling from the syslog server (ftp server) to the target hosts (ftp client). The target hosts have all been either Unix (Linux, Solaris, HP-UX) or VMS boxes. The VMS box is the final target actually and the other Unix boxes were just used to try and troubleshoot this problem. The VMS box is using a version of the RMS filesystem that can also support files larger than 2 gigs. I was only able to successfully copy the large file to another Linux box using scp. Multinet's SCP on the OpenVMS box fails with the following error:

Code:

VMS_I> scp2 "root@10.0.1.1::/logs/pix.log" test.fil
root@10.0.1.1's password: 
SCP2: FATAL: DISK$MULTINET_V44_A:[MULTINET_V44A.MULTINET.SSH2.APPS.SSH]SSHFC_TRA
NSFER.C;1:3004 SshFCTransfer (function name unavailable) Assertion failed: tdata
->read_bytes <= tdata->source_file->attributes->size

Finally I also tried running a Samba server on the syslog server and using DEC's SMB client to copy the file but it failed after copying 2 gigs. Note; the underlying file system (RMS) does not have a 2 gig file limitation. I suspect that the SMB implementation from DEC is the limitation here.

Anyone have any ideas?

Aside from split, no.

A lot of programs use 32 bit file pointers, limiting the offset to 2+GB. Is there a reason why the logfile has to be whole during the transfer? You could certainly
try on the Gentoo end:

Code:

#!/bin/ksh
split -b 2147483647 /var/log/hugefile  ./huge
echo "
verbose
open vmsbox
user username pwd
mput ./huge*
bye
" | ftp -n

It's been so long since played VMS sysmgr I've forgotten the DCL to reconstruct file file. If you can execute a UNIX shell -

Code:

find . -name 'huge*' -exec cat {} \; >> anotherhugefile

Well that's the solution I came up with but my supervisor really wants the file whole. However, in my experimenting I'm starting to wonder if it's a limitation of the ftp client and/or server I'm working with. The VMS guys here routinely do transfers of large files with no problem between the VMS boxes, so I'm getting laughed at. :\ My next experiment is to try and move from ReiserFS to something else since I've found some hits with Google that point to problems with ReiserFS and FTP at times. I'm going to try ext3 next...

I tried with ext3 instead of ReiserFS and still had the same problem. So I switched FTP servers on the syslog server. I went from "Net FTPD" to "PureFTPD". When I did that, I was able to PULL the file to the VMS box (ftp client) from the syslog server (ftp server) with no problem. And now, when I try to push the file from the syslog server (ftp client) to the VMS box (ftp server), I get a different error:

Code:

ftp> put /logs/pix.log test.fil
local: /logs/pix.log remote: test.fil
local: /logs/pix.log: File too large

I did some Googling on that and tried changing the ulimit settings on the syslog server from "unlimited" to 4 gigs just to see if it made a difference. Unfortunately, it didn't change things. I'm now going to try the 'ncftp' client to see if that gives me anything different.

I wound up installing 'ncftp' as the ftp client on my syslog server and used ncftpput to transfer the file to the VMS box. That worked. It would appear that the ftp client that Gentoo installs by default has some kind of limitation. It's "Net FTP" which I supposed would make some sense as the FTPD was from the same project.

deckard-

Did you try to cp this file somewhere by any chance and run into the same problem? I have a similar problem here with one disk but the file can't even be cp'd. I also get "File too large".

-haidood