UNIX for Dummies Questions & Answers

I had received a complaint recently that 'wc' is not returning a correct byte count for large files (in 10s of GB). Does 'wc' have any such limitation? If yes, would 'ls' be a better option to fetch file size.

I am planning on replacing wc with ls for calculating file sizes in the existing code. Please let me know if you see any issues with this change.

One interesting thing I noted while trying to simulate the problem was, 'wc' takes way longer time than 'ls'. Can anyone provide a reasoning for this? (See the code and its result below)

Thanks!
Sreenivas

============== Script ========================
timestamp=`date`
echo $timestamp
lscount=`ls -l two.dat|awk '{print $5}'`
echo $lscount
timestamp=`date`
echo $timestamp
wccount=`wc -c two.dat|awk '{print $1}'`
echo $wccount
timestamp=`date`
echo $timestamp
=============== Result =========================
Mon Apr 24 05:18:29 PDT 2006
2303923104
Mon Apr 24 05:18:29 PDT 2006
2303923104
Mon Apr 24 05:33:16 PDT 2006
========================================

1. did you verify the complaint? ie., see that wc returned an incorrect value compared with ls

2. Are large files enabled/installed correctly on your system?

1. I tried simulating it with a 2 GB file but couldn't. I havent yet tried a 10/20 Gb file.
2. I dont understand this question. ls works fine, so am assuming the installation (?) was proper.

You can't really compare ls and wc. In simple terms:

The ls command reads UNIX's directory information - which contains the file's size and location (as well as many other things)
The wc command reads the _file_ to extract various bit of information (number of lines, characters, words)

So, ls reads information about the file whereas wc reads the file itself and, as you would expect, wc will take longer the larger the file whereas ls will not.

cheers

It is possible when wc is proprietary and/or it wasn't upgraded along with other components. Some "wc" try to use /tmp and if it exceeds it, fails. I believe you can circumvent it by streaming to the stdout simply as a byte count:

cat filename | wc -c

You also may wish to verify /tmp file system as well as a system restrictions for max memory (per system) / max file size (per fs)/ and max buffer sizes; to see if there are any limits were hit. As I said it may be “wc” needs to be updated as well (it may happen as your wc has been compiled for a previous version you had and has some issues with a current version). It is hard to help more without OS specifics as all of them have different facilities to go over. Get “UNIX Essentials and UNIX Core” DVD it helps in such situations.

UUoC

wc -c < filename

Cheers
ZB

The same egg but a profile view.