Sponsored Content
Full Discussion: Counting entries in a file
Top Forums Shell Programming and Scripting Counting entries in a file Post 302545270 by agama on Sunday 7th of August 2011 11:32:47 PM
Old 08-08-2011
Small tweeks to the original awk to show number of new IPs in the current bin compared to the previous.

Code:
#!/usr/bin/env ksh

awk -v bin_size=${1:-5} '
    function dump( )
    {
        if( NR == 1 )
            return;

        new_count = 0;
        for( u in unique )              # compute total in this bin that were not in last bin
            if( last_bin[u] == 0 )
                new_count++;

        printf( "%3d %3d %3d\n", bin+1, total, new_count );
        bin++;
    }

    {
        if( $1+0 >= next_bin )
        {
            dump( );
            next_bin = $1 + bin_size;

            delete last_bin;
            for( u in unique )              # copy hits from this bin
                last_bin[u] = 1;
            delete unique;
            total = 0;
        }

        unique[$2]++
        total++;
    }
    END {
        if( total )
            dump( );
    }
'

exit

Have fun!
This User Gave Thanks to agama For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Counting words in a file

I'm trying to figure out a way to count the number of words in the follwing file: cal 2002 > file1 Is there anyway to do this without using wc but instead using the cut command? (1 Reply)
Discussion started by: r0mulus
1 Replies

2. Shell Programming and Scripting

Help me in counting records from file

Hi, Please help me in counting the below records(1st field) from samplefile: Expected output: Count Descr ------------------------------------------- 7 Mean manager 14 ... (7 Replies)
Discussion started by: prashant43
7 Replies

3. Programming

Counting the words in a file

Please find the below program. It contains the purpose of the program itself. /* Program : Write a program to count the number of words in a given text file */ /* Date : 12-June-2010 */ # include <stdio.h> # include <stdlib.h> # include <string.h> int main( int argc, char *argv ) {... (6 Replies)
Discussion started by: ramkrix
6 Replies

4. Shell Programming and Scripting

Counting multiple entries in a file using awk

Hi, I have a big file (~960MB) having epoch time values (~50 million entries) which looks like 897393601 897393601 897393601 897393601 897393602 897393602 897393602 897393602 897393602 897393603 897393603 897393603 897393603 and so on....each time stamp has more than one... (6 Replies)
Discussion started by: sajal.bhatia
6 Replies

5. Shell Programming and Scripting

Counting duplicate entries in a file using awk

Hi, I have a very big (with around 1 million entries) txt file with IPv4 addresses in the standard format, i.e. a.b.c.d The file looks like 10.1.1.1 10.1.1.1 10.1.1.1 10.1.2.4 10.1.2.4 12.1.5.6 . . . . and so on.... There are duplicate/multiple entries for some IP... (3 Replies)
Discussion started by: sajal.bhatia
3 Replies

6. Shell Programming and Scripting

Counting characters within a file

Ok say I wanted to count every Y in a data file. Then set Y as my delimiter so that I can separate my file by taking all the contents that occur BEFORE the first Y and store them in a variable so that I may use this content later on in my program. Then I could do the same thing with the next Y's... (5 Replies)
Discussion started by: puttster
5 Replies

7. UNIX for Dummies Questions & Answers

Counting feilds entries with Perl

Hi All, I have a small problem of counting the number of times a particular entry that exists in a horizontal string of elements and a vertical feild (column of entries). For example AATGGTCCTGExpected outputA=2 C=2 G=3 T=3 I have an idea to do this but I dont know how to do that if these entries... (1 Reply)
Discussion started by: pawannoel
1 Replies

8. Shell Programming and Scripting

Counting lines in a file using awk

I want to count lines of a file using AWK (only) and not in the END part like this awk 'END{print FNR}' because I want to use it. Does anyone know of a way? Thanks a lot. (7 Replies)
Discussion started by: guitarist684
7 Replies

9. Shell Programming and Scripting

Need help of counting no of column of a file

Hi All , I got stuck on the below scenario.If anyone can help me ,that will be really helpful. I have a target hdfs file layout.I need to know the no of column in that file. Target_RECRD_layout { ABC_ID EN NOTNULLABLE, ABC_COUNTRY CHARACTER ENCODING ASCII NOTNULLABLE, ... (5 Replies)
Discussion started by: STCET22
5 Replies

10. Shell Programming and Scripting

Counting and print from file

Dear community, I have an already filtered log on my machine, something like: WARN 2016.03.10 10:59:01.136 logging.LogAlarmListener raise ALARMWARNINGRAISED Alarm NODE-NetworkAccessGroup.Client.41283 SERVICEDOWN-41283.WC severity WARNING raised: Service 41283.WC protocoltype client is down... (13 Replies)
Discussion started by: Lord Spectre
13 Replies
tabs(1) 						      General Commands Manual							   tabs(1)

NAME
tabs - set tabs on a terminal SYNOPSIS
tabs [-v[n]] [-ahuUV] file... DESCRIPTION
The tabs program clears and sets tab-stops on the terminal. This uses the terminfo clear_all_tabs and set_tab capabilities. If either is absent, tabs is unable to clear/set tab-stops. The terminal should be configured to use hard tabs, e.g., stty tab0 OPTIONS
General Options -Tname Tell tabs which terminal type to use. If this option is not given, tabs will use the $TERM environment variable. If that is not set, it will use the ansi+tabs entry. -d The debugging option shows a ruler line, followed by two data lines. The first data line shows the expected tab-stops marked with asterisks. The second data line shows the actual tab-stops, marked with asterisks. -n This option tells tabs to check the options and run any debugging option, but not to modify the terminal settings. The tabs program processes a single list of tab stops. The last option to be processed which defines a list is the one that determines the list to be processed. Implicit Lists Use a single number as an option, e.g., "-5" to set tabs at the given interval (in this case 1, 6, 11, 16, 21, etc.). Tabs are repeated up to the right margin of the screen. Use "-0" to clear all tabs. Use "-8" to set tabs to the standard interval. Explicit Lists An explicit list can be defined after the options (this does not use a "-"). The values in the list must be in increasing numeric order, and greater than zero. They are separated by a comma or a blank, for example, tabs 1,6,11,16,21 tabs 1 6 11 16 21 Use a '+' to treat a number as an increment relative to the previous value, e.g., tabs 1,+5,+5,+5,+5 which is equivalent to the 1,6,11,16,21 example. Predefined Tab-Stops X/Open defines several predefined lists of tab stops. -a Assembler, IBM S/370, first format -a2 Assembler, IBM S/370, second format -c COBOL, normal format -c2 COBOL compact format -c3 COBOL compact format extended -f FORTRAN -p PL/I -s SNOBOL -u UNIVAC 1100 Assembler PORTABILITY
X/Open describes a +m option, to set a terminal's left-margin. Very few of the entries in the terminal database provide this capability. The -d (debug) and -n (no-op) options are extensions not provided by other implementations. Documentation for other implementations states that there is a limit on the number of tab stops. While some terminals may not accept an arbitrary number of tab stops, this implementation will attempt to set tab stops up to the right margin of the screen, if the given list happens to be that long. SEE ALSO
tset(1), infocmp(1), ncurses(3NCURSES), terminfo(5). This describes ncurses version 5.9 (patch 20110404). tabs(1)
All times are GMT -4. The time now is 08:52 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy