Summing columns over group of lines Post: 302798367

Sponsored Content

Top Forums Shell Programming and Scripting Summing columns over group of lines Post 302798367 by Don Cragun on Wednesday 24th of April 2013 10:26:36 AM

04-24-2013

Registered User

The following awk script is a little more complex than you requested. It allows processing of multiple input files, prints an end of file separator if more than one input file is given, and prints a warning if there are lines left at the end of a file that don't make up a complete 4 line set.

Since you said all values in a 4 line set are constant in fields 1 and 3, I used the values in the last line of the set instead of in the 1st line of the set (it saved me from needing to create two more variables). If you really need the 1st line's values instead of the last line's values, it won't be hard to change this script to do that.

As always, if you're using a Solaris/SunOS system, use /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk instead of awk.

Code:

awk '
FNR == 1 {
        # Check for incomplete set at end of previous file.
        if(l) {
                printf("%d line(s) skipped at end of %s.\n", l, file)
                for(i = 5; i <= n; i++) s[i] = 0
        }
        # Print file trailer if more than 1 file has been seen.
        if(nf++) printf("================== End of data from file %s\n", file)
        # Process headers: Print output headers, determine field count.
        printf("%s %s ", $1, $3)
        for(i = 5; i <= NF; i++) printf("%s%s", $i, i == NF ? "\n" : " ")
        n = NF          # set number of fields for this file
        l = 0           # set number of lines in current set
        file = FILENAME # save filename for diagnostics
        next
}
{       for(i = 5; i <= n; i++) s[i] += $i}
++l == 4 {
        l = 0
        printf("%d %d ", $1, $3)
        for(i = 5; i <= NF; i++) {
                printf("%6.4f%s", s[i], i == NF ? "\n" : " ")
                s[i] = 0
        }
}
END {   if(l) printf("%d line(s) skipped at end of %s.\n", l, file)
        if(nf > 1) printf("================== End of data from file %s\n", file)
}' input

This User Gave Thanks to Don Cragun For This Post:

Don Cragun

View Public Profile for Don Cragun

Find all posts by Don Cragun

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Summing the columns of a file

Hi All, I have a file like - num.txt 12, 34, 65, line1 34, 65, 89, line2 43, 65, 77, line3 I want to do two things - 1. Add first three columns of each line and print the line with largest value. i.e. (12+34+65) for 1st line and so on. 2. Add middle column of each line i.e....

2. Shell Programming and Scripting

Awk: Summing values with group criteria

Hi Guys, I have a text file with ";" like separator F1;F2;F3;F4;F5 444;100041;IT;GLOB;1800000000 444;100041;TM;GLOB;1000000000 444;10300264;IT;GLOB;2000000000 444;10300264;IT;GLOB;2500000000 I have to sum the cullums F5 for same F2 and F3 collums The result must be: ...

3. Shell Programming and Scripting

Summing values in columns

Basically I have to process a text file which has been sorted this way: John 12 John 13 John 10 John 900 Peter 20 Peter 30 Peter 32 The first column is a name, and the second an arbitrary value, both delimited by a space. How can I sum them up such that it would become: John 935...

4. Shell Programming and Scripting

Summing over specific lines and replacing the lines with the sum using sed, awk

Hi friends, This is sed & awk type question. I have a text file which has numbers spread all over the file. I want to sum the series of numbers whenever i find it and produce an output file with the sum. For example ###start of input text file #### abc def ghi 1 2 3 4 kjld random...

5. Shell Programming and Scripting

Summing over specific lines and replacing the lines with the sum

Hi friends, This is sed & awk type question. It is slightly different from my previous question. I have a text file which has numbers spread all over the file. I want to sum the series of numbers (but no more than 10 numbers in series) whenever i find it and produce an output file with the...

6. Homework & Coursework Questions

HELP with Unix scripts in summing columns in a file.

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: Hi guys, i'm a new guy here, and it's my first time creating a unix script. can you guys help me out here? i'd...

7. Homework & Coursework Questions

HELP with Unix scripts in summing columns in a file

1. The problem statement, all variables and given/known data: Hi guys, i'm a new guy here, and it's my first time creating a unix script. can you guys help me out here? i'd really appreciate it. :( Here's my problem: This is the file i'm using, it has 6 columns, the first three columns are...

8. Shell Programming and Scripting

Please Help!!!! Awk for summing columns based on selected column value

a,b,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z,aa,bb,cc,dd,ee,ff,gg,hh,ii a thru ii are digits and strings.... The awk needed....if coloumn 9 == i (coloumn 9 is string ), output the sum of x's(coloumn 22 ) in all records and sum of y's (coloumn 23 ) in all records in a file (records.txt)....

9. Shell Programming and Scripting

Summing columns in line

I have a file with the following format AAAAA 1.34B 0.76B 0.00B 0.00B 0.00B 0.00B 0.00B 0.00B 0.00B 0.00B 0.00B 0.00B 0.00B 0.00B 0.90B 0.00B 0.00B 0.46B 0.00B 0.03B 0.00B ...

10. Shell Programming and Scripting

Summing per group in a loop

I want to sum and average all other columns by first column GR1 1 4 7 GR1 2 5 8 GR1 3 6 9 GR2 11 14 17 GR2 13 16 19 GR3 1 3 5 GR3 2 4 6 For a limited number of columns I can do...

LEARN ABOUT FREEBSD

tabs

TABS(1) 						    BSD General Commands Manual 						   TABS(1)

NAME

     tabs -- set terminal tabs

SYNOPSIS

     tabs [-n | -a | -a2 | -c | -c2 | -c3 | -f | -p | -s | -u] [+m[n]] [-T type]
     tabs [-T type] [+[n]] n1[,n2,...]

DESCRIPTION

     The tabs utility displays a series of characters that clear the hardware terminal tab settings then initialises tab stops at specified posi-
     tions, and optionally adjusts the margin.

     In the first synopsis form, the tab stops set depend on the command line options used, and may be one of the predefined formats or at regular
     intervals.

     In the second synopsis form, tab stops are set at positions n1, n2, etc.  If a position is preceded by a '+', it is relative to the previous
     position set.  No more than 20 positions may be specified.

     If no tab stops are specified, the ``standard'' UNIX tab width of 8 is used.

     The options are as follows:

     -n      Set a tab stop every n columns.  If n is 0, the tab stops are cleared but no new ones are set.

     -a      Assembler format (columns 1, 10, 16, 36, 72).

     -a2     Assembler format (columns 1, 10, 16, 40, 72).

     -c      COBOL normal format (columns 1, 8, 12, 16, 20, 55)

     -c2     COBOL compact format (columns 1, 6, 10, 14, 49)

     -c3     COBOL compact format (columns 1, 6, 10, 14, 18, 22, 26, 30, 34, 38, 42, 46, 50, 54, 58, 62, 67).

     -f      FORTRAN format (columns 1, 7, 11, 15, 19, 23).

     -p      PL/1 format (columns 1, 5, 9, 13, 17, 21, 25, 29, 33, 37, 41, 45, 49, 53, 57, 61).

     -s      SNOBOL format (columns 1, 10, 55).

     -u      Assembler format (columns 1, 12, 20, 44).

     +m[n], +[n]
	     Set an n character left margin, or 10 if n is omitted.

     -T type
	     Output escape sequence for the terminal type type.

ENVIRONMENT

     The LANG, LC_ALL, LC_CTYPE and TERM environment variables affect the execution of tabs as described in environ(7).

     The -T option overrides the setting of the TERM environment variable.  If neither TERM nor the -T option are present, tabs will fail.

EXIT STATUS

     The tabs utility exits 0 on success, and >0 if an error occurs.

SEE ALSO

     expand(1), stty(1), tput(1), unexpand(1), termcap(5)

STANDARDS

     The tabs utility conforms to IEEE Std 1003.1-2001 (``POSIX.1'').

HISTORY

     A tabs utility appeared in PWB UNIX.  This implementation was introduced in FreeBSD 5.0.

BUGS

     The current termcap(5) database does not define the 'ML' (set left soft margin) capability for any terminals.

BSD
								   May 20, 2002 							       BSD