Alternative for wc -l Post: 302463075

Sponsored Content

Top Forums Shell Programming and Scripting Alternative for wc -l Post 302463075 by DGPickett on Friday 15th of October 2010 05:02:29 PM

10-15-2010

Registered User

The only compromise in "wc -l" is the probable FILE* i/o, which could be rewritten to do raw read() or, even faster and more dangerous to system throughput, mmap64() using a 64 bit compiler (flushes all RAM to backing store). It is still brute force.

That is why I mentioned the 'storage of earlier byte counts' method. You could have it run all day, periodically updating the byte total report.

If the app logged as it wrote, every N lines or N seconds, which ever came first, then you could just tail the log.

If it is not our code, you could even write a pass-through logger and use a (possibly named) pipe to get access to the output stream, if the app can be configured for output to a (possibly named) pipe.

---------- Post updated at 05:02 PM ---------- Previous update was at 03:42 PM ----------

Not the weepy face! Try this fast wc -l stdin C bit using read() and quarter meg buffer:

Code:

#include <stdio.h>
#include <errno.h>

int main(){

        char buf[262144];
        long long ct = 0L ;
        char *cp ;
        int ret ;


        while ( 0 < ( ret = read( 0, buf, sizeof(buf)))
         || ( ret == -1
           && ( errno == EAGAIN
             || errno == EINTR ))){

                if ( ret < 0 )
                        continue ;

                for ( cp = buf + ret - 1 ; cp >= buf ; cp-- ){
                        if ( *cp == '\n' ){
                                ct++ ;
                         }
                 }
         }

        if ( ret ){
                perror( "fwcl: stdin" );
                exit( 1 );
         }

        printf( "%lld\n", ct );

        exit( 0 );
 }

"mysrc/fwcl.c" line 37 of 37 --100%-- 

$ wc -l <.profile
70
$ fwcl <.profile
70
$

DGPickett

View Public Profile for DGPickett

Find all posts by DGPickett

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

getopts alternative?

I have to implement switches (options) like this in my script. ./myscript -help ./myscript -dir /home/krish -all ./myscript -all getopts allows switches to have one character (like a, b, etc.). How can I customize it for handling the above situation? Or, is there any alternative to...

2. Shell Programming and Scripting

help with while loop or any other alternative?

i=1 while do mm=02 dd=03 yy=2008 echo "$mm$dd$yy" i=$(( i+1)) echo "$i" done whenever i execute the script above i will get the error below: syntax error at line 30: `i=$' unexpected

3. Shell Programming and Scripting

du alternative in perl

I have a perl script that just does a `du -sk -x` and formats it to look groovy ( the argument can be a directory but usually is like /usr/local/* ) #!/usr/bin/perl use strict; use warnings; my $sizes = `du -x -sk @ARGV | sort -n`; my $total = 0; print "MegaBytes Name\n"; for(split...

4. HP-UX

alternative for egrep -o on HP-UX

Hello to all board members!! I have a problem on a HP-UX system. I should write a script. Therefore I need to search after IP addresses in the output of a command. On Debian this works: ifconfig | egrep -o "{1,3}\.{1,3}\.{1,3}\.{1,3}" The script where i need this is not ifconfig, but...

5. Shell Programming and Scripting

Alternative for ikecert

Hi Folks... Is there an alternative for ikecert(SunOS) - man info - "manipulates the machine's on-filesystem public-key certificate databases" in linux? Can we use pkcs7, pkcs8 or something like that?... I also came across ssh-keygen and ssh-keygen2... My best guess is to use ssh-certtool...

6. Solaris

Alternative to sshfs?

I have an automated testing script that relies on the dev box being able to see production's (NFS) share. It uses rsync and ssh to handle transfers and command execution; however, it also needs the production share mounted in order to run Perl code against it when Unix commands via ssh will not do....

7. Solaris

vi alternative

Is there any other editor, installed by 'default' in Sparc Solaris10, besides vi? I'd like to avoid installing anything new. If not, how to make vi more user-friendly? thanks.

8. Shell Programming and Scripting

Looking for an alternative to Tcl

I've created quite a collection of tcl scripts which have buttons, radio buttons, check boxes, text fields, etc. These tcl scripts in turn call and execute several hundred sh, csh, bash, perl scripts and pass in the args based on the gui selections on the same and other redhat machines. We're...

LEARN ABOUT CENTOS

papi_get_thr_specific

PAPI_get_thr_specific(3)					       PAPI						  PAPI_get_thr_specific(3)

NAME

       PAPI_get_thr_specific -

       Retrieve a pointer to a thread specific data structure.

SYNOPSIS

Detailed Description
       @par Prototype:
	       int PAPI_get_thr_specific( int tag, void **ptr );

       @param tag
	   An identifier, the value of which is either PAPI_USR1_TLS or
	   PAPI_USR2_TLS. This identifier indicates which of several data
	   structures associated with this thread is to be accessed.
       @param ptr
	   A pointer to the memory containing the data structure.

       @retval PAPI_OK
       @retval PAPI_EINVAL
	   The @em tag argument is out of range.

       In C, PAPI_get_thr_specific PAPI_get_thr_specific will retrieve the pointer from the array with index @em tag.
       There are 2 user available locations and @em tag can be either
       PAPI_USR1_TLS or PAPI_USR2_TLS.
       The array mentioned above is managed by PAPI and allocated to each
       thread which has called PAPI_thread_init.
       There is no Fortran equivalent function.

       @par Example:

	int ret;
	HighLevelInfo *state = NULL;
	ret = PAPI_thread_init(pthread_self);
	if (ret != PAPI_OK) handle_error(ret);

	// Do we have the thread specific data setup yet?

       ret = PAPI_get_thr_specific(PAPI_USR1_TLS, (void *) &state);
       if (ret != PAPI_OK || state == NULL) {
	   state = (HighLevelInfo *) malloc(sizeof(HighLevelInfo));
	   if (state == NULL) return (PAPI_ESYS);
	   memset(state, 0, sizeof(HighLevelInfo));
	   state->EventSet = PAPI_NULL;
	   ret = PAPI_create_eventset(&state->EventSet);
	   if (ret != PAPI_OK) return (PAPI_ESYS);
	   ret = PAPI_set_thr_specific(PAPI_USR1_TLS, state);
	   if (ret != PAPI_OK) return (ret);
       }
       *

       See Also:
	   PAPI_register_thread PAPI_thread_init PAPI_thread_id PAPI_set_thr_specific

Author
       Generated automatically by Doxygen for PAPI from the source code.

Version 5.2.0.0 						  Tue Jun 17 2014					  PAPI_get_thr_specific(3)