Sum of two columns in a file Post: 302685885

Sponsored Content

Top Forums Shell Programming and Scripting Sum of two columns in a file Post 302685885 by Don Cragun on Monday 13th of August 2012 01:32:50 PM

08-13-2012

Registered User

According to POSIX, -F "|^" or any other mechanism that sets FS to the two characters <vertical-line><circumflex> produces undefined results because <vertical-line> is special in an extended regular expression (ERE) except when it appears in a bracket expression. Also in an ERE ^ is an anchor (even when it is not the first character in the ERE) except when it appears in a bracket expression, in a collating symbol ([.^.]), or equivalence class ([=^=]). So, any of the solutions here that set FS to a string starting with "|" is not portable. And any of the solution here that set FS to a string ending with "^" is not portable. Setting FS to "[|^]" is valid, but makes both <vertical-line> and <circumflex> field separators, not the two character string "|^". (This is why the solutions that set FS to "[|^]" had to sum fields 9 and 15 instead of fields 5 and 8. Note that this solution won't work if there are any <vertical-line> or <circumflex> characters in the input that aren't adjacent pairs and it that order).

Since "^" is an anchor in an ERE, setting FS to "^" produces undefined results. (It is anchoring the empty expression following the <circumflex> to the beginning of the string. But since this isn't a replacement [where an empty expression is replaced by the previously matched expression], the standards don't specify what this means and the results are not portable.)

I'm sure the solutions given all work on one or more implementations. Several of them fail with syntax errors when using Mac OS X's awk.

To portably search for "|^" as a field delimiter and sum the numeric values in the 5th and 8th columns, I think you need something like:

Code:

awk -F "[|][[.^.]]" ' {sum+=$5+$8}
END {print "Sum of 2 columns 5 & 8 which is",sum}' input

which should work on any system that meets the POSIX awk utility requirements.

This User Gave Thanks to Don Cragun For This Post:

Don Cragun

View Public Profile for Don Cragun

Find all posts by Don Cragun

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Sum of three columns - in 4N columns file

Hi All, happy new year. I have a file with 4xN columns like 0.0000e+00 0.0000e+00 7.199E+07 7.123E+07 6.976E+07 6.482E+07 5.256E+07 2.523E+07 0.0000e+00 0.0000e+00 8.641E+07 8.550E+07 8.373E+07 7.780E+07 6.309E+07 3.028E+07...

2. UNIX for Dummies Questions & Answers

Sum of all columns in all files in one output file

If I have say 4 files like this: File1: 1 3 4 7 7 0 5 7 5 9 1 2 7 4 8 File2: 1 4 6 2 5 7 1 2 3 6 0 3 0 3 8 File3:

3. Shell Programming and Scripting

Sum columns

Hi All, I'm new to this forum. So please be patience with me! :) I have a file that looks like this (all rows have the same number of columns): 19 20 30 15 17 38 51 60 74 85 96 07 .... 10 20 44 59 39 88 13 77 30 10 11 12 .... . . . I want to sum the value of first field to all the...

4. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns match on two rows

Hi all, I know this sounds suspiciously like a homework course; but, it is not. My goal is to take a file, and match my "ID" column to the "Date" column, if those conditions are true, add the total number of minutes worked and place it in this file, while not printing the original rows that I...

5. Shell Programming and Scripting

Sum numeric columns contained in a plain text file

Hi everyone, Here are the contents of a plain text file created by a SQL query: SUM(T.TRNQTY) COUNT(D.TRNSEQ) ---------------- ---------------- 1380 46 1393 59 2680 134 740 37 ...

6. Shell Programming and Scripting

Get the SUM of TWO columns SEPARATELY by doing GROUP BY on other columns

My File looks like: "|" -> Field separator A|B|C|100|1000 D|E|F|1|2 G|H|I|0|7 D|E|F|1|2 A|B|C|10|10000 G|H|I|0|7 A|B|C|1|100 D|E|F|1|2 I need to do a SUM on Col. 5 and Col.6 by grouping on Col 1,2 & 3 My expected output is: A|B|C|111|11100

7. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns satisfy the condition

HI All, I'm embedding SQL query in Script which gives following output: Assignee Group Total ABC Group1 17 PQR Group2 5 PQR Group3 6 XYZ Group1 10 XYZ Group3 5 I have saved the above output in a file. How do i sum up the contents of this output so as to get following output: ...

8. UNIX for Beginners Questions & Answers

Group by columns and add sum in new columns

Dear Experts, I have input file which is comma separated, has 4 columns like below, BRAND,COUNTRY,MODEL,COUNT NIKE,USA,DUMMY,5 NIKE,USA,ORIGINAL,10 PUMA,FRANCE,DUMMY,20 PUMA,FRANCE,ORIGINAL,15 ADIDAS,ITALY,DUMMY,50 ADIDAS,ITALY,ORIGINAL,50 SPIKE,CHINA,DUMMY,1O And expected output add...

9. UNIX for Beginners Questions & Answers

Copy columns from one file into another and get sum of column values and row count

I have a file abc.csv, from which I need column 24(PurchaseOrder_TotalCost) to get the sum_of_amounts with date and row count into another file say output.csv abc.csv- UTF-8,,,,,,,,,,,,,,,,,,,,,,,,, ...

10. UNIX for Advanced & Expert Users

Need Optimization shell/awk script to aggreagte (sum) for all the columns of Huge data file

Optimization shell/awk script to aggregate (sum) for all the columns of Huge data file File delimiter "|" Need to have Sum of all columns, with column number : aggregation (summation) for each column File not having the header Like below - Column 1 "Total Column 2 : "Total ... ......

LEARN ABOUT ULTRIX

ppmtosixel

ppmtosixel(1)						      General Commands Manual						     ppmtosixel(1)

NAME

       ppmtosixel - convert a portable pixmap into DEC sixel format

SYNOPSIS

       ppmtosixel [-raw] [-margin] [ppmfile]

DESCRIPTION

       Reads  a  portable  pixmap  as input.  Produces sixel commands (SIX) as output.	The output is formatted for color printing, e.g. for a DEC
       LJ250 color inkjet printer.

       If RGB values from the PPM file do not have maxval=100, the RGB values are rescaled.  A printer control header and a color assignment table
       begin the SIX file.  Image data is written in a compressed format by default.  A printer control footer ends the image file.

OPTIONS

       -raw   If  specified,  each  pixel  will  be explicitly described in the image file.  If -raw is not specified, output will default to com-
	      pressed format in which identical adjacent pixels are replaced by "repeat pixel" commands.  A raw file is often an order	of  magni-
	      tude larger than a compressed file and prints much slower.

       -margin
	      If  -margin  is not specified, the image will be start at the left margin (of the window, paper, or whatever).  If -margin is speci-
	      fied, a 1.5 inch left margin will offset the image.

PRINTING

       Generally, sixel files must reach the printer unfiltered.  Use the lpr -x option or cat filename > /dev/tty0?.

BUGS

       Upon rescaling, truncation of the least significant bits of RGB values may result in poor color conversion.  If the original PPM maxval was
       greater	than  100,  rescaling  also reduces the image depth.  While the actual RGB values from the ppm file are more or less retained, the
       color palette of the LJ250 may not match the colors on your screen.  This seems to be a printer limitation.

SEE ALSO

       ppm(5)

AUTHOR

       Copyright (C) 1991 by Rick Vinci.

								   26 April 1991						     ppmtosixel(1)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Sum of three columns - in 4N columns file

Discussion started by: f_o_555

2. UNIX for Dummies Questions & Answers

Sum of all columns in all files in one output file

Discussion started by: cosmologist

3. Shell Programming and Scripting

Sum columns

Discussion started by: Aderson Nascime

4. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns match on two rows

Discussion started by: mtucker6784