The only compromise in "wc -l" is the probable FILE* i/o, which could be rewritten to do raw read() or, even faster and more dangerous to system throughput, mmap64() using a 64 bit compiler (flushes all RAM to backing store). It is still brute force.
That is why I mentioned the 'storage of earlier byte counts' method. You could have it run all day, periodically updating the byte total report.
If the app logged as it wrote, every N lines or N seconds, which ever came first, then you could just tail the log.
If it is not our code, you could even write a pass-through logger and use a (possibly named) pipe to get access to the output stream, if the app can be configured for output to a (possibly named) pipe.
---------- Post updated at 05:02 PM ---------- Previous update was at 03:42 PM ----------
Not the weepy face! Try this fast wc -l stdin C bit using read() and quarter meg buffer:
Code:
#include <stdio.h>
#include <errno.h>
int main(){
char buf[262144];
long long ct = 0L ;
char *cp ;
int ret ;
while ( 0 < ( ret = read( 0, buf, sizeof(buf)))
|| ( ret == -1
&& ( errno == EAGAIN
|| errno == EINTR ))){
if ( ret < 0 )
continue ;
for ( cp = buf + ret - 1 ; cp >= buf ; cp-- ){
if ( *cp == '\n' ){
ct++ ;
}
}
}
if ( ret ){
perror( "fwcl: stdin" );
exit( 1 );
}
printf( "%lld\n", ct );
exit( 0 );
}
"mysrc/fwcl.c" line 37 of 37 --100%--
$ wc -l <.profile
70
$ fwcl <.profile
70
$
Location: Saint Paul, MN USA / BSD, CentOS, Debian, OS X, Solaris
Posts: 2,288
Thanks Given: 430
Thanked 480 Times in 395 Posts
Hi.
If one can be satisfied with an estimate, then a code that samples the file can be very fast.
As DGPickett said, seeks are fast. This demo code, esmele, reads the first 100 lines of the file (almost a GB), and skips to 6000 characters before the EOF, reading again (88 lines in this situation). The mean lengths are calculated and then the estimate is made based on another quickly-accessible characteristic, the length of the file via stat. The accuracy compared to wc is within 2%. The time is (essentially) constant, although if one were to choose to read percentages of the file, say 3% at the beginning, middle, and end, one could be more accurate, at the expense of taking more time.
Code:
% ./compare-esmele-wc
-----
File characteristics:
-rw-r--r-- 1 955M Oct 16 05:54 /tmp/test-one-gb
-----
Time and result of esmele on /tmp/test-one-gb:
real 0m0.011s
user 0m0.008s
sys 0m0.004s
14958698
-----
Time and result of wc on /tmp/test-one-gb:
real 0m2.739s
user 0m1.212s
sys 0m0.480s
14754910
-----
Ratio of wc / es counts:
0.986377
I've created quite a collection of tcl scripts which have buttons, radio buttons, check boxes, text fields, etc. These tcl scripts in turn call and execute several hundred sh, csh, bash, perl scripts and pass in the args based on the gui selections on the same and other redhat machines. We're... (4 Replies)
Is there any other editor, installed by 'default' in Sparc Solaris10, besides vi?
I'd like to avoid installing anything new.
If not, how to make vi more user-friendly?
thanks. (8 Replies)
I have an automated testing script that relies on the dev box being able to see production's (NFS) share. It uses rsync and ssh to handle transfers and command execution; however, it also needs the production share mounted in order to run Perl code against it when Unix commands via ssh will not do.... (2 Replies)
Hi Folks...
Is there an alternative for ikecert(SunOS) - man info - "manipulates the machine's on-filesystem public-key certificate databases" in linux?
Can we use pkcs7, pkcs8 or something like that?...
I also came across ssh-keygen and ssh-keygen2...
My best guess is to use ssh-certtool... (0 Replies)
Hello to all board members!!
I have a problem on a HP-UX system. I should write a script. Therefore I need to search after IP addresses in the output of a command.
On Debian this works: ifconfig | egrep -o "{1,3}\.{1,3}\.{1,3}\.{1,3}"
The script where i need this is not ifconfig, but... (2 Replies)
I have a perl script that just does a `du -sk -x` and formats it to look groovy ( the argument can be a directory but usually is like /usr/local/* )
#!/usr/bin/perl
use strict;
use warnings;
my $sizes = `du -x -sk @ARGV | sort -n`;
my $total = 0;
print "MegaBytes Name\n";
for(split... (1 Reply)
i=1
while
do
mm=02
dd=03
yy=2008
echo "$mm$dd$yy"
i=$(( i+1))
echo "$i"
done
whenever i execute the script above i will get the error below:
syntax error at line 30: `i=$' unexpected (3 Replies)
I have to implement switches (options) like this in my script.
./myscript -help
./myscript -dir /home/krish -all
./myscript -all
getopts allows switches to have one character (like a, b, etc.). How can I customize it for handling the above situation? Or, is there any alternative to... (3 Replies)
10-15-2010
