Sponsored Content
Top Forums Shell Programming and Scripting Average across multiple columns group by Post 302932980 by Don Cragun on Monday 26th of January 2015 07:18:56 PM
Old 01-26-2015
Quote:
Originally Posted by ritakadm
sorry about not providing with a sample input, I`m using cygwin. The data range is 0 to 100,000 should be outputted upto 2 decimal places.
Data is 83000 lines, not very big.

Yes, the code calculates the average correctly for only the 7th column, although I should
populate arr as arr[$1" "$2" "$3" "$4" "$5] to get all the variables delimited.

Sample input has 4 data columns, I have many in the original data starting col7 until NF ($22).

Code:
  a1 b1 c1 d1 e1 12 13 14 15
  a1 b1 c1 d1 e1 14 15 16 17
  a1 b1 c1 d1 e1 13 14 15 16
  a2 b1 c1 d1 e1 112 113 114 115
  a2 b1 c1 d1 e1 114 115 116 117
  a2 b1 c1 d1 e1 113 114 115 116

Output should be

Code:
  a1 b1 c1 d1 e1 13 14 15 16
  a2 b1 c1 d1 e1 113 114 115 116

The code you showed us in your 1st post in this thread skips data in the 1st line of your file (which I assumed was intended to skip over a header line). But, I don't see any headers in this sample. Is there a header, or not? If there s a header, should it be copied to the output?

Is the number of fields constant in an input file, or can it vary from line to line?

It looks like there is a leading space in your sample input and output. Is a leading space required in your output?

Do you want 2 decimal places in all computed output fields, or do you want values to be printed without decimal places (as in your sample output) in cases where the computed result is an integral value?

You say you want to calculate averages for fields 7 through NF, but your sample data also calculates the average for field 6? Is field 6 supposed to be ignored in calculations and removed from the output, or is field 6 to be averaged as well as fields 7 through NF?
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Taking the average of two columns and printing it on a new column

Hi, I have a space delimited text file that looks like the following: Aa 100 200 Bb 300 100 Cc X 500 Dd 600 X Basically, I want to take the average of columns 2 and 3 and print it in column 4. However if there is an X in either column 2 or 3, I want to print the non-X value. Therefore... (11 Replies)
Discussion started by: evelibertine
11 Replies

2. Shell Programming and Scripting

Average of columns with values of other column with same name

I have a lot of input files that have the following form: Sample Cq Sample Cq Sample Cq Sample Cq Sample Cq 1WBIN 23.45 1WBIN 23.45 1CVSIN 23.96 1CVSIN 23.14 S1 31.37 1WBIN 23.53 1WBIN 23.53 1CVSIN 23.81 1CVSIN 23.24 S1 31.49 1WBIN 24.55 1WBIN 24.55 1CVSIN 23.86 1CVSIN 23.24 S1 31.74 ... (3 Replies)
Discussion started by: isildur1234
3 Replies

3. Shell Programming and Scripting

How to calculate average of two columns and copy into another file?

Hi, I need help with the awk command. I have a folder with aprox 500 files each one with two columns and I want to print in a new file, the average of column 1 and average of column 2 and the name of each file. Input files are: File-1: 100 99 20 99 50 99 50 99 File-2: 200 85... (3 Replies)
Discussion started by: Lokaps
3 Replies

4. Shell Programming and Scripting

Get the SUM of TWO columns SEPARATELY by doing GROUP BY on other columns

My File looks like: "|" -> Field separator A|B|C|100|1000 D|E|F|1|2 G|H|I|0|7 D|E|F|1|2 A|B|C|10|10000 G|H|I|0|7 A|B|C|1|100 D|E|F|1|2 I need to do a SUM on Col. 5 and Col.6 by grouping on Col 1,2 & 3 My expected output is: A|B|C|111|11100 (2 Replies)
Discussion started by: machomaddy
2 Replies

5. Shell Programming and Scripting

Match first two columns and average third from multiple files

I have the following format of input from multiple files File 1 24.01 -81.01 1.0 24.02 -81.02 5.0 24.03 -81.03 0.0 File 2 24.01 -81.01 2.0 24.02 -81.02 -5.0 24.03 -81.03 10.0 I need to scan through the files and when the first 2 columns match I... (18 Replies)
Discussion started by: ncwxpanther
18 Replies

6. Emergency UNIX and Linux Support

Average columns based on header name

Hi Friends, I have files with columns like this. This sample input below is partial. Please check below for main file link. Each file will have only two rows. ... (8 Replies)
Discussion started by: jacobs.smith
8 Replies

7. Shell Programming and Scripting

Average across multiple columns - awk

Hi forum members, I'm trying to get an average of multiple columns in a csv file using awk. A small example of my input data is as follows: cu,u3o8,au,ag -9,20,-9,3.6 0.005,30,-9,-9 0.005,50,10,3.44 0.021,-9,8,3.35 The following code seems to do most of what I want gawk -F","... (6 Replies)
Discussion started by: theflamingmoe
6 Replies

8. UNIX for Beginners Questions & Answers

Group by columns and add sum in new columns

Dear Experts, I have input file which is comma separated, has 4 columns like below, BRAND,COUNTRY,MODEL,COUNT NIKE,USA,DUMMY,5 NIKE,USA,ORIGINAL,10 PUMA,FRANCE,DUMMY,20 PUMA,FRANCE,ORIGINAL,15 ADIDAS,ITALY,DUMMY,50 ADIDAS,ITALY,ORIGINAL,50 SPIKE,CHINA,DUMMY,1O And expected output add... (2 Replies)
Discussion started by: ricky1991
2 Replies

9. Shell Programming and Scripting

Average of a columns from three files

hello, I have three files in the following order ==> File1 <== 1 20977000 20977000 A C 1.00 0,15 15 45 1 115829313 115829313 G A 0.500 6,7 13 99 ==> File2 <== 1 20977000 20977000 A C 1.00 0,13 13 39 1 115829313 ... (5 Replies)
Discussion started by: nans
5 Replies

10. UNIX for Beginners Questions & Answers

Average of columns

I have files that have the following columns chr pos ref alt sample 1 sample 2 sample 3 chr2 179644035 G A 1,107 0,1 58,67 chr7 151945167 G T 142,101 100,200 500,700 chr13 31789169 CTT CT,C 6,37,8 0,0,0 15,46,89 chr22 ... (3 Replies)
Discussion started by: nans
3 Replies
ASCII(7)						     Linux Programmer's Manual							  ASCII(7)

NAME
ascii - ASCII character set encoded in octal, decimal, and hexadecimal DESCRIPTION
ASCII is the American Standard Code for Information Interchange. It is a 7-bit code. Many 8-bit codes (such as ISO 8859-1, the Linux default character set) contain ASCII as their lower half. The international counterpart of ASCII is known as ISO 646. The following table contains the 128 ASCII characters. C program 'X' escapes are noted. Oct Dec Hex Char Oct Dec Hex Char ------------------------------------------------------------------------ 000 0 00 NUL '' 100 64 40 @ 001 1 01 SOH (start of heading) 101 65 41 A 002 2 02 STX (start of text) 102 66 42 B 003 3 03 ETX (end of text) 103 67 43 C 004 4 04 EOT (end of transmission) 104 68 44 D 005 5 05 ENQ (enquiry) 105 69 45 E 006 6 06 ACK (acknowledge) 106 70 46 F 007 7 07 BEL 'a' (bell) 107 71 47 G 010 8 08 BS '' (backspace) 110 72 48 H 011 9 09 HT ' ' (horizontal tab) 111 73 49 I 012 10 0A LF ' ' (new line) 112 74 4A J 013 11 0B VT 'v' (vertical tab) 113 75 4B K 014 12 0C FF 'f' (form feed) 114 76 4C L 015 13 0D CR ' ' (carriage ret) 115 77 4D M 016 14 0E SO (shift out) 116 78 4E N 017 15 0F SI (shift in) 117 79 4F O 020 16 10 DLE (data link escape) 120 80 50 P 021 17 11 DC1 (device control 1) 121 81 51 Q 022 18 12 DC2 (device control 2) 122 82 52 R 023 19 13 DC3 (device control 3) 123 83 53 S 024 20 14 DC4 (device control 4) 124 84 54 T 025 21 15 NAK (negative ack.) 125 85 55 U 026 22 16 SYN (synchronous idle) 126 86 56 V 027 23 17 ETB (end of trans. blk) 127 87 57 W 030 24 18 CAN (cancel) 130 88 58 X 031 25 19 EM (end of medium) 131 89 59 Y 032 26 1A SUB (substitute) 132 90 5A Z 033 27 1B ESC (escape) 133 91 5B [ 034 28 1C FS (file separator) 134 92 5C '\' 035 29 1D GS (group separator) 135 93 5D ] 036 30 1E RS (record separator) 136 94 5E ^ 037 31 1F US (unit separator) 137 95 5F _ 040 32 20 SPACE 140 96 60 ` 041 33 21 ! 141 97 61 a 042 34 22 " 142 98 62 b 043 35 23 # 143 99 63 c 044 36 24 $ 144 100 64 d 045 37 25 % 145 101 65 e 046 38 26 & 146 102 66 f 047 39 27 ' 147 103 67 g 050 40 28 ( 150 104 68 h 051 41 29 ) 151 105 69 i 052 42 2A * 152 106 6A j 053 43 2B + 153 107 6B k 054 44 2C , 154 108 6C l 055 45 2D - 155 109 6D m 056 46 2E . 156 110 6E n 057 47 2F / 157 111 6F o 060 48 30 0 160 112 70 p 061 49 31 1 161 113 71 q 062 50 32 2 162 114 72 r 063 51 33 3 163 115 73 s 064 52 34 4 164 116 74 t 065 53 35 5 165 117 75 u 066 54 36 6 166 118 76 v 067 55 37 7 167 119 77 w 070 56 38 8 170 120 78 x 071 57 39 9 171 121 79 y 072 58 3A : 172 122 7A z 073 59 3B ; 173 123 7B { 074 60 3C < 174 124 7C | 075 61 3D = 175 125 7D } 076 62 3E > 176 126 7E ~ 077 63 3F ? 177 127 7F DEL Tables For convenience, let us give more compact tables in hex and decimal. 2 3 4 5 6 7 30 40 50 60 70 80 90 100 110 120 ------------- --------------------------------- 0: 0 @ P ` p 0: ( 2 < F P Z d n x 1: ! 1 A Q a q 1: ) 3 = G Q [ e o y 2: " 2 B R b r 2: * 4 > H R f p z 3: # 3 C S c s 3: ! + 5 ? I S ] g q { 4: $ 4 D T d t 4: " , 6 @ J T ^ h r | 5: % 5 E U e u 5: # - 7 A K U _ i s } 6: & 6 F V f v 6: $ . 8 B L V ` j t ~ 7: ' 7 G W g w 7: % / 9 C M W a k u DEL 8: ( 8 H X h x 8: & 0 : D N X b l v 9: ) 9 I Y i y 9: ' 1 ; E O Y c m w A: * : J Z j z B: + ; K [ k { C: , < L l | D: - = M ] m } E: . > N ^ n ~ F: / ? O _ o DEL NOTES
History An ascii manual page appeared in Version 7 of AT&T UNIX. On older terminals, the underscore code is displayed as a left arrow, called backarrow, the caret is displayed as an up-arrow and the ver- tical bar has a hole in the middle. Uppercase and lowercase characters differ by just one bit and the ASCII character 2 differs from the double quote by just one bit, too. That made it much easier to encode characters mechanically or with a non-microcontroller-based electronic keyboard and that pairing was found on old teletypes. The ASCII standard was published by the United States of America Standards Institute (USASI) in 1968. SEE ALSO
iso_8859-1(7), iso_8859-10(7), iso_8859-13(7), iso_8859-14(7), iso_8859-15(7), iso_8859-16(7), iso_8859-2(7), iso_8859-3(7), iso_8859-4(7), iso_8859-5(7), iso_8859-6(7), iso_8859-7(7), iso_8859-8(7), iso_8859-9(7) COLOPHON
This page is part of release 3.44 of the Linux man-pages project. A description of the project, and information about reporting bugs, can be found at http://www.kernel.org/doc/man-pages/. Linux 2009-02-12 ASCII(7)
All times are GMT -4. The time now is 11:03 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy