Repeating awk command


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Repeating awk command
# 1  
Old 05-11-2009
Repeating awk command

Hi all,

I have an awk command that needs to be ran multiple times in a script on one file containing lots of fields of data.

The file look like this (the numbers are made up):
Code:
1234      2222     2223     2222
123        2223     3333    2323
3333      3321     3344    4444

The file contains more fields than that but that's how it looks.

The command works out the average number from each field but to execute in the script could mean repeating it up to 72 times! (there could be up to 72 fields of data in the file).

The command
Code:
awk '{ s += $1 } END { print s/NR }' datafile > avefile

This works out to average of field 1 ($1) and puts it into avefile - I want to avoid having to do that for 72 lines in the script, obviously redirecting the subsequent output to the avefile. Is there any way of making the command run in the script with out the need of printing it some many times.

Thanks for any responses.

Last edited by nistleloy; 05-11-2009 at 06:25 PM.. Reason: Fixed code tags
# 2  
Old 05-11-2009
Quote:
Originally Posted by nistleloy
Hi all,

I have an awk command that needs to be ran multiple times in a script on one file containing lots of fields of data.

The file look like this (the numbers are made up):
Code:
1234      2222     2223     2222
123        2223     3333    2323
3333      3321     3344    4444

The file contains more fields than that but that's how it looks.

The command works out the average number from each field but to execute in the script could mean repeating it up to 72 times! (there could be up to 72 fields of data in the file).

The command
Code:
awk '{ s += $1 } END { print s/NR }' datafile > avefile

This works out to average of field 1 ($1) and puts it into avefile - I want to avoid having to do that for 72 lines in the script, obviously redirecting the subsequent output to the avefile. Is there any way of making the command run in the script with out the need of printing it some many times.

To run a piece of code multiple times, use a while or for loop, but you don't need to do that; it can all be done with a single awk script:

Code:
awk '{
for ( n = 1; n <= NF; ++n ) tot[n] += $n
if ( NF > max ) max = NF
}
END {
for ( n = 1; n <= max; ++n ) printf "Field %d: %d\n", n, tot[n] / NR 
}
' datafile > avfile

# 3  
Old 05-12-2009
Thank for your response, much appreciated.

/nistleloy
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Indexing each repeating pattern of rows in a column using awk/sed

Hello All, I have data like this in a column. 0 1 2 3 0 3 4 5 6 0 1 2 3 etc. where 0 identifies the start of a pattern in my data. So I need the output like below using either awk/sed. 0 1 (2 Replies)
Discussion started by: ks_reddy
2 Replies

2. Shell Programming and Scripting

awk and or sed command to sum the value in repeating tags in a XML

I have a XML in which <Amt Ccy="EUR">3.1</Amt> tag repeats. This is under another tag <Main>. I need to sum all the values of <Amt Ccy=""> (Ccy may vary) coming under <Main> using awk and or sed command. can some help? Sample looks like below <root> <Main> ... (6 Replies)
Discussion started by: bk_12345
6 Replies

3. UNIX for Dummies Questions & Answers

Awk: print all URL addresses between iframe tags without repeating an already printed URL

Here is what I have so far: find . -name "*php*" -or -name "*htm*" | xargs grep -i iframe | awk -F'"' '/<iframe*/{gsub(/.\*iframe>/,"\"");print $2}' Here is an example content of a PHP or HTM(HTML) file: <iframe src="http://ADDRESS_1/?click=5BBB08\" width=1 height=1... (18 Replies)
Discussion started by: striker4o
18 Replies

4. UNIX for Dummies Questions & Answers

Using sed command to remove multiple instances of repeating headers in one file?

Hi, I have catenated multiple output files (from a monte carlo run) into one big output file. Each individual file has it's own two line header. So when I catenate, there are multiple two line headers (of the same wording) within the big file. How do I use the sed command to search for the... (1 Reply)
Discussion started by: rebazon
1 Replies

5. Shell Programming and Scripting

grep or awk looking for repeating text

I am looking for a way to find the below pattern in text. 777777,111,08-20-2011 111111,222,08-20-2011 777777,111,07-24-2011 777777,222,07-24-2011 111111,222,07-22-2011 I would like to find a way to print every line in a file where the first 6 numbers match and there is different... (6 Replies)
Discussion started by: ffdstanley
6 Replies

6. Shell Programming and Scripting

Removing repeating lines from a data frame (AWK)

Hey Guys! I have written a code which combines lots of files into one big file(.csv). However, each of the original files had headers on the first line, and now that I've combined the files the headers are interspersed throughout the new combined data frame. For example, throughout the data... (21 Replies)
Discussion started by: gd9629
21 Replies

7. Shell Programming and Scripting

Repeating Substitution Command on VI

Hello Folks, how to write a command on vi that allow to repeat last substitution command? Here what I want to do : 1 2 3 1 2 3 1 2 3 :.,+2s/\n/ /And I obtain : 1 2 3 1 2 3 1 (5 Replies)
Discussion started by: gogol_bordello
5 Replies

8. UNIX for Advanced & Expert Users

Repeat output of last command w/o repeating last command

Is there a way to repeat the output of the last command for filtering without running the command again? All I could think of was to copy all the data to a text file and process it that way, is there another way? Like say I want to grep server.server.lan from a dtrace that was pages long after I... (5 Replies)
Discussion started by: glev2005
5 Replies

9. UNIX for Advanced & Expert Users

unix script for repeating a command with a variable

Hi need urgent help , for creating unix script . To collect system name,This is command i want to execute n (integer) no. of times for for a differnt IP addresses .IP is variable in every execution. Other string & collecter name is constant . snmpGet %IP% sysName.0 -c <string> -S <datacollecter... (2 Replies)
Discussion started by: langdatyagi
2 Replies

10. UNIX for Dummies Questions & Answers

repeating previous argument on command line?

Hi, is there a way in bash--or any other shell--to repeat the preceding argument on the command line? E.g., let's say I want to rename the file "/var/www/conf/httpd.conf" to "/var/www/conf/httpd.conf.bak". I want to be able to type mv /var/www/conf/httpd.conf, and then press a command key that... (6 Replies)
Discussion started by: hadarot
6 Replies
Login or Register to Ask a Question