Alternative for wc -l Post: 302463214

Sponsored Content

Top Forums Shell Programming and Scripting Alternative for wc -l Post 302463214 by drl on Saturday 16th of October 2010 11:08:51 AM

10-16-2010

Registered User

Hi.

If one can be satisfied with an estimate, then a code that samples the file can be very fast.

As DGPickett said, seeks are fast. This demo code, esmele, reads the first 100 lines of the file (almost a GB), and skips to 6000 characters before the EOF, reading again (88 lines in this situation). The mean lengths are calculated and then the estimate is made based on another quickly-accessible characteristic, the length of the file via stat. The accuracy compared to wc is within 2%. The time is (essentially) constant, although if one were to choose to read percentages of the file, say 3% at the beginning, middle, and end, one could be more accurate, at the expense of taking more time.

Code:

% ./compare-esmele-wc 

-----
 File characteristics:
-rw-r--r-- 1 955M Oct 16 05:54 /tmp/test-one-gb

-----
 Time and result of esmele on /tmp/test-one-gb:

real	0m0.011s
user	0m0.008s
sys	0m0.004s
14958698

-----
 Time and result of wc on /tmp/test-one-gb:

real	0m2.739s
user	0m1.212s
sys	0m0.480s
14754910

-----
 Ratio of wc / es counts:
0.986377

Best wishes ... cheers, drl

drl

View Public Profile for drl

Find all posts by drl

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

getopts alternative?

I have to implement switches (options) like this in my script. ./myscript -help ./myscript -dir /home/krish -all ./myscript -all getopts allows switches to have one character (like a, b, etc.). How can I customize it for handling the above situation? Or, is there any alternative to...

2. Shell Programming and Scripting

help with while loop or any other alternative?

i=1 while do mm=02 dd=03 yy=2008 echo "$mm$dd$yy" i=$(( i+1)) echo "$i" done whenever i execute the script above i will get the error below: syntax error at line 30: `i=$' unexpected

3. Shell Programming and Scripting

du alternative in perl

I have a perl script that just does a `du -sk -x` and formats it to look groovy ( the argument can be a directory but usually is like /usr/local/* ) #!/usr/bin/perl use strict; use warnings; my $sizes = `du -x -sk @ARGV | sort -n`; my $total = 0; print "MegaBytes Name\n"; for(split...

4. HP-UX

alternative for egrep -o on HP-UX

Hello to all board members!! I have a problem on a HP-UX system. I should write a script. Therefore I need to search after IP addresses in the output of a command. On Debian this works: ifconfig | egrep -o "{1,3}\.{1,3}\.{1,3}\.{1,3}" The script where i need this is not ifconfig, but...

5. Shell Programming and Scripting

Alternative for ikecert

Hi Folks... Is there an alternative for ikecert(SunOS) - man info - "manipulates the machine's on-filesystem public-key certificate databases" in linux? Can we use pkcs7, pkcs8 or something like that?... I also came across ssh-keygen and ssh-keygen2... My best guess is to use ssh-certtool...

6. Solaris

Alternative to sshfs?

I have an automated testing script that relies on the dev box being able to see production's (NFS) share. It uses rsync and ssh to handle transfers and command execution; however, it also needs the production share mounted in order to run Perl code against it when Unix commands via ssh will not do....

7. Solaris

vi alternative

Is there any other editor, installed by 'default' in Sparc Solaris10, besides vi? I'd like to avoid installing anything new. If not, how to make vi more user-friendly? thanks.

8. Shell Programming and Scripting

Looking for an alternative to Tcl

I've created quite a collection of tcl scripts which have buttons, radio buttons, check boxes, text fields, etc. These tcl scripts in turn call and execute several hundred sh, csh, bash, perl scripts and pass in the args based on the gui selections on the same and other redhat machines. We're...

LEARN ABOUT LINUX

times

TIMES(P)						     POSIX Programmer's Manual							  TIMES(P)

NAME

       times - write process times

SYNOPSIS

       times

DESCRIPTION

       The  times  utility  shall  write  the accumulated user and system times for the shell and for all of its child processes, in the following
       POSIX locale format:

	      "%dm%fs %dm%fs
%dm%fs %dm%fs
", <shell user minutes>,
		  <shell user seconds>, <shell system minutes>,
		  <shell system seconds>, <children user minutes>,
		  <children user seconds>, <children system minutes>,
		  <children system seconds>

       The four pairs of times shall correspond to the members of the <sys/times.h> tms structure (defined  in	the  Base  Definitions	volume	of
       IEEE Std 1003.1-2001, Chapter 13, Headers) as returned by times(): tms_utime, tms_stime, tms_cutime, and tms_cstime, respectively.

OPTIONS

       None.

OPERANDS

       None.

STDIN

       Not used.

INPUT FILES

       None.

ENVIRONMENT VARIABLES

       None.

ASYNCHRONOUS EVENTS

       Default.

STDOUT

       See the DESCRIPTION.

STDERR

       The standard error shall be used only for diagnostic messages.

OUTPUT FILES

       None.

EXTENDED DESCRIPTION

       None.

EXIT STATUS

       Zero.

CONSEQUENCES OF ERRORS

       Default.

       The following sections are informative.

APPLICATION USAGE

       None.

EXAMPLES

	      $ times
	      0m0.43s 0m1.11s
	      8m44.18s 1m43.23s

RATIONALE

       The times special built-in from the Single UNIX Specification is now required for all conforming shells.

FUTURE DIRECTIONS

       None.

SEE ALSO

       Special Built-In Utilities

COPYRIGHT

       Portions of this text are reprinted and reproduced in electronic form from IEEE Std 1003.1, 2003 Edition, Standard for Information Technol-
       ogy -- Portable Operating System Interface (POSIX), The Open Group Base Specifications Issue 6, Copyright (C) 2001-2003 by the Institute of
       Electrical  and	Electronics  Engineers, Inc and The Open Group. In the event of any discrepancy between this version and the original IEEE
       and The Open Group Standard, the original IEEE and The Open Group Standard is the referee document. The original Standard can  be  obtained
       online at http://www.opengroup.org/unix/online.html .

IEEE
/The Open Group						       2003								  TIMES(P)