Extract numbers from text file work out average


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract numbers from text file work out average
# 1  
Old 04-22-2010
Extract numbers from text file work out average

Just wondering if someone could assist me with shell script I'm trying to write. I need to read the final column of a text file (shown below) and workout what the average number is. The text file will have a variable number of lines, I just want the script to pull out the values in the final field (which I've done using cut), add them all together then divide by the total number of rows. Really struggling with this! Help much appreciated Smilie

abcd e544f222e0jk55 1503
abcd e544f222e0jk56 1504
abcd e544f222e0jk57 1505
abcd e544f222e0jk58 1506
abcd e544f222e0jk59 1507
abcd e544f222e0jk60 1508
abcd e544f222e0jk61 1509
abcd e544f222e0jk62 1510
abcd e544f222e0jk63 1511
abcd e544f222e0jk64 1512
abcd e544f222e0jk65 1513
abcd e544f222e0jk66 1514
abcd e544f222e0jk67 1515
abcd e544f222e0jk68 1516
abcd e544f222e0jk69 1517
abcd e544f222e0jk70 1518
abcd e544f222e0jk71 1519
abcd e544f222e0jk72 1520
abcd e544f222e0jk73 1521
abcd e544f222e0jk74 1522
abcd e544f222e0jk75 1523
abcd e544f222e0jk76 1524
abcd e544f222e0jk77 1525
abcd e544f222e0jk78 1526
abcd e544f222e0jk79 1527
abcd e544f222e0jk80 1528
abcd e544f222e0jk81 1529
abcd e544f222e0jk82 1530
abcd e544f222e0jk83 1531
abcd e544f222e0jk84 1532
# 2  
Old 04-22-2010
Hi.

One with awk...
Code:
$ cat file1
abcd e544f222e0jk55 1503
abcd e544f222e0jk56 1504
abcd e544f222e0jk57 1505
abcd e544f222e0jk58 1506
abcd e544f222e0jk59 1507
abcd e544f222e0jk60 1508
abcd e544f222e0jk61 1509
abcd e544f222e0jk62 1510
abcd e544f222e0jk63 1511
abcd e544f222e0jk64 1512
abcd e544f222e0jk65 1513
abcd e544f222e0jk66 1514
abcd e544f222e0jk67 1515
abcd e544f222e0jk68 1516
abcd e544f222e0jk69 1517
abcd e544f222e0jk70 1518
abcd e544f222e0jk71 1519
abcd e544f222e0jk72 1520
abcd e544f222e0jk73 1521
abcd e544f222e0jk74 1522
abcd e544f222e0jk75 1523
abcd e544f222e0jk76 1524
abcd e544f222e0jk77 1525
abcd e544f222e0jk78 1526
abcd e544f222e0jk79 1527
abcd e544f222e0jk80 1528
abcd e544f222e0jk81 1529
abcd e544f222e0jk82 1530
abcd e544f222e0jk83 1531
abcd e544f222e0jk84 1532

$ awk '{T+= $NF} END { print T/NR }' file1
1517.5

# 3  
Old 04-22-2010
many thanks for the reply scottn, I get the following:

awk: line 1: syntax error at or near END

script is:

Code:
#!/bin/sh
ls -l logs/db
#echo "please enter the name of the file you wish to analyse:"
#read filetoint
cat stats.txt
awk '!(T+= $NF) END { print T/NR }' stats.txt


Last edited by Scott; 04-22-2010 at 05:01 PM.. Reason: Please use code tags
# 4  
Old 04-22-2010
Hi.

I changed my awk slightly. Try that one.

It doesn't look like the error from Solaris, but if it is, use /usr/xpg4/bin/awk or nawk.
# 5  
Old 04-22-2010
change
Code:
awk '!(T+= $NF) END { print T/NR }' stats.txt

to
Code:
awk '!{T+= $NF} END { print T/NR }' stats.txt



---------- Post updated at 22:06 ---------- Previous update was at 22:02 ----------

@scottn

I've found a 6 yr. old posting of Ygor here.
Now I'm almost able to "read" your code.
So "T" means Total and "$NF" means 3rd column, right?
I really can't understand how "$NF" stands for 3rd column....
# 6  
Old 04-22-2010
many thanks for this, I'm actually scripting it on ubuntu 9.10? now returns the following error:

awk: line 1: syntax error at or near {
# 7  
Old 04-22-2010
Quote:
Originally Posted by pseudocoder
change
Code:
awk '!(T+= $NF) END { print T/NR }' stats.txt

to
Code:
awk '!{T+= $NF} END { print T/NR }' stats.txt



---------- Post updated at 22:06 ---------- Previous update was at 22:02 ----------

@scottn

I've found a 6 yr. old posting of Ygor here.
Now I'm almost able to "read" your code.
So "T" means Total and "$NF" means 3rd column, right?
I really can't understand how "$NF" stands for 3rd column....
Hi pseudocoder.

Yes, I'm sure this question has been asked and answered many many times.

The first awk you quote is my original one, which I changed, but which rich@ardz quoted.

It works fine (at least for me Smilie)
Code:
$ awk '!(T+= $NF) END { print T/NR }' file1
1517.5

T is a running total, $NF is the value of the last field and NR is the number of rows.

---------- Post updated at 10:13 PM ---------- Previous update was at 10:10 PM ----------

Quote:
Originally Posted by rich@ardz
many thanks for this, I'm actually scripting it on ubuntu 9.10? now returns the following error:

awk: line 1: syntax error at or near {
Can you paste (i.e. not type) exactly the command you are running? This awk is not by any means complicated.

Thanks.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Average result of text file

Hi all I have created a script to allow me to gain a number of file times and how long they take to execute. I have been asked to get the average time for the log. The log will update every 5 minutes and never be empty. 141 152 157 161 168 169 171 179 202 207 229 651 666 714... (3 Replies)
Discussion started by: simpsa27
3 Replies

2. Shell Programming and Scripting

Extract numbers from file name-how to ?

Hello friends,I am new to Unix programming. how do I achieve the following in Unix shell script (I am running ksh on AIX) extract the number from name of file? My file format is like "LongFileName-1234.020614-221030.txt" now I want to extract value which is between (-) hyphen and (.) dot... (4 Replies)
Discussion started by: f150
4 Replies

3. UNIX for Dummies Questions & Answers

Extract Numbers from a log file

Hi, I am trying to grep/extract the number list from this log file, can I get some help on this. I can grep the word 'href' to see the numbers, but it is resulting with the complete line. Content of my file: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"> <html> <head>... (4 Replies)
Discussion started by: Sajjadmehdi
4 Replies

4. Shell Programming and Scripting

Color prompt with file numbers does not work anymore

I have used this color prompt on my servers for long time, in file ~\.bashrc Black="\" Dark="\" Blue="\" LBlue="\" Green="\" LGreen="\" Cyan="\" LCyan="\" Red="\" LRed="\" Purple="\" LPurple="\" Brown="\" Yellow="\" LGray="\" White="\" Reset="\" PS1="$Yellow\u@\h $LBlue\w... (4 Replies)
Discussion started by: Jotne
4 Replies

5. Shell Programming and Scripting

Need to extract only decimal numbers for a glob of text

If you have a look at this thread, you'll see that users have been posting the output a script which are numbers that range from 2 to 5 decimal places. If I dump this entire thread to txt file, how can I: 1) Delete everything except for numbers of the following formats (where 'x' is a digit and... (5 Replies)
Discussion started by: graysky
5 Replies

6. UNIX for Dummies Questions & Answers

Extracting lines from a text file based on another text file with line numbers

Hi, I am trying to extract lines from a text file given a text file containing line numbers to be extracted from the first file. How do I go about doing this? Thanks! (1 Reply)
Discussion started by: evelibertine
1 Replies

7. UNIX Desktop Questions & Answers

Calculate average for rows in a text file

Dear Gurus, I have tab-delimited text files with matrix containing values. The first column is a identifier and other columns have the corresponding values. I would like to calculate the average value (total number/number of entries) for all entries from 2nd column to the last column in row... (3 Replies)
Discussion started by: Unilearn
3 Replies

8. Shell Programming and Scripting

Need help please with Grep/Sed command to extract text and numbers from a file

Hello All, I need to extract lines from a file that contains ALPHANUMERIC and the length of Alphanumeric is set to 16. I have pasted the sample of the lines from the text file that I have created. My problem is that sometimes 16 appears in other part of the line. I'm only interested to... (14 Replies)
Discussion started by: mnassiri
14 Replies

9. UNIX for Dummies Questions & Answers

Extract numbers from .txt file

I need to extract all the p-value numbers and the rho numbers from a .txt file and write them as coma separated values in a new file. Ideally I would get two files in the end, one for p- values and one for rho. Any suggestions? I appreciate your help!!! The .txt file looks essentially like this... (5 Replies)
Discussion started by: eggali
5 Replies

10. Shell Programming and Scripting

BASH. Need to extract some numbers and take the average

Hey all, I ran some simulations, of which the output is 100s of files. I've used grep to extract the vital information needed from the files. This has made my task somewhat easier. But I still need to perform some mathematical calculations (average and geometrical average) on the results of the... (9 Replies)
Discussion started by: slackjack
9 Replies
Login or Register to Ask a Question