Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

BASH. Need to extract some numbers and take the average

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting BASH. Need to extract some numbers and take the average
# 1  
Old 11-01-2009
BASH. Need to extract some numbers and take the average
Hey all,

I ran some simulations, of which the output is 100s of files. I've used grep to extract the vital information needed from the files. This has made my task somewhat easier. But I still need to perform some mathematical calculations (average and geometrical average) on the results of the grep command (which is saved to a txt file).

The general format of the output of grep is (just a crude example):
Code:
a-4.txt number of eggs used is ## in #  cartons
b-4.txt number of eggs used is ## in #  cartons
b-6.txt number of eggs used is ## in #  cartons
a-6.txt number of eggs used is ## in #  cartons
a-8.txt number of eggs used is ## in #  cartons
b-8.txt number of eggs used is ## in #  cartons
a-10.txt number of eggs used is ## in #  cartons
b-10.txt number of eggs used is ## in #  cartons

My aim here is to get the average number of cartons (#) needed to pack ## number of eggs. Here I want to average all the a-4, a-6, a-8, and a-10 and then all the b-4, b-6, b-8, and b-10 cartons. SO my end result is to have an average number of cartons for both a and b. As you can imagine doing this by hand with results that can go up to 100 is very time consuming. Can someone please assist me with a BASH script that can make my job simpler?

Thank you.
# 2  
Old 11-01-2009
Code:
 cat abc.txt | perl -e 'while ( <>){ push @{$hash{$1}},$2 if (/^([a-z]+)\-(\d+)\..*/);} foreach $k(keys %hash){ my $total = 0; ($total+=$_) for @{$hash{$k}}; printf ("%s\t%d\t%.2f\n",$k,$total,$total/($#{$hash{$k}}+1));}'

Replace abc.txt with your file name

HTH,
PL
# 3  
Old 11-01-2009
I dont understand perl Smilie So I cant make modifications to the script. Could you provide some comments or a translation to BASH? I would ideally like to adapt this to many situations.
# 4  
Old 11-02-2009
Something like this? (bash code):
Code:
grep cartons [ab]-* |
{ while read x x x x x eggs x cartons x; do
    (( total_eggs+=eggs ))
    (( total_cartons+=cartons ))
  done
  echo "A total number of $total_eggs eggs in $total_cartons cartons makes an average of $(( total_eggs/total_cartons )) eggs per carton"
}

Code:
$> grep cartons [ab]-*
a-10.txt:number of eggs used is 55 in 4 cartons
a-4.txt:number of eggs used is 97 in 3 cartons
a-6.txt:number of eggs used is 22 in 1 cartons
a-8.txt:number of eggs used is 44 in 9 cartons
b-10.txt:number of eggs used is 87 in 5 cartons
b-4.txt:number of eggs used is 85 in 5 cartons
b-6.txt:number of eggs used is 36 in 6 cartons
b-8.txt:number of eggs used is 88 in 2 cartons

Result:
Code:
A total number of 514 eggs in 35 cartons makes an average of 14 eggs per carton

# 5  
Old 11-02-2009
Quote:
Originally Posted by daptal
Code:
 cat abc.txt | perl -e 'while ( <>){ ..... }

Replace abc.txt with your file name

HTH,
PL
if you want to use Perl, use Perl. There's no need to cat the file. Pass it into perl as an argument. Also, don't cram everything into one line like that. Its hard to read and troubleshoot. Indent your code where necessary.
# 6  
Old 11-02-2009
Quote:
Originally Posted by Scrutinizer
Something like this? (bash code):
Code:
grep cartons [ab]-* |
{ while read x x x x x eggs x cartons x; do
    (( total_eggs+=eggs ))
    (( total_cartons+=cartons ))
  done
  echo "A total number of $total_eggs eggs in $total_cartons cartons makes an average of $(( total_eggs/total_cartons )) eggs per carton"
}

Code:
$> grep cartons [ab]-*
a-10.txt:number of eggs used is 55 in 4 cartons
a-4.txt:number of eggs used is 97 in 3 cartons
a-6.txt:number of eggs used is 22 in 1 cartons
a-8.txt:number of eggs used is 44 in 9 cartons
b-10.txt:number of eggs used is 87 in 5 cartons
b-4.txt:number of eggs used is 85 in 5 cartons
b-6.txt:number of eggs used is 36 in 6 cartons
b-8.txt:number of eggs used is 88 in 2 cartons

Result:
Code:
A total number of 514 eggs in 35 cartons makes an average of 14 eggs per carton

Hi,

What exactly does the highlighted code do? Also, what if the structure of the string changes? Like if it was:
b-4.txt:85 eggs in 5 cartons

I would be grateful if you could explain how you extracted the numbers.
# 7  
Old 11-02-2009
Hi slackjack, the grep statement extracts those lines that contain the word 'cartons' from every file that is starts with 'a-' or with 'b-' The output is fed into the curly brace block (which is needed because in bash the fields will loose their values outside the while loop). The output is then fed into the read statement of which the first 5 fields and field 7 are discarded into a dummy variable called 'x'. Any further fields are all assigned to the last x.

If the line to look for in those files were "85 eggs in 5 cartons", the read statement would be:

Code:
read eggs x x cartons x
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Substr with % - extract numbers only

# cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.5 (Maipo) I have this script that will monitor filesystems and send me e-amil alerts. #! /bin/ksh DIST_LIST=monitor@...com WORKDIR=/home/monitor WARNLEVEL=90 MAIL_SUBJ="filesystems monitor on "$(hostname) ... (3 Replies)
Discussion started by: danielshell
3 Replies

2. Solaris

How is Load Average computed and what are better numbers?

Hello All, What is load average and how is it computed in Solaris 10? What are the different ranges for normal, warning and danger signs? Kindly clarify. Thank you, Sunil Kumar (3 Replies)
Discussion started by: msgforsunil
3 Replies

3. Shell Programming and Scripting

Extract numbers from file name-how to ?

Hello friends,I am new to Unix programming. how do I achieve the following in Unix shell script (I am running ksh on AIX) extract the number from name of file? My file format is like "LongFileName-1234.020614-221030.txt" now I want to extract value which is between (-) hyphen and (.) dot... (4 Replies)
Discussion started by: f150
4 Replies

4. Shell Programming and Scripting

awk or Bash: Cumulative average

For the data I would like to parse down and for each parsing I want a cumulative averaging, stored in an array that can be output. I.e. 546/NR = 546 (546+344)/NR=(546+344)/2 = etc. For N record input I want N values of the average (a block averaging effectively) Any... (3 Replies)
Discussion started by: chrisjorg
3 Replies

5. Shell Programming and Scripting

How to take a Average of numbers from different files?

Hi, I have 3 to 4 different files, from that i need to take a Average of numbers from a particular column. here i have to take 4th column, that should present in diff. file. File 1: Col1 col2 col3 col4 1 11 sa 12.00 2 22 sb 134.59 3 33 sc 11.99 4 44 sd 12.44 Col1 col2 col3... (8 Replies)
Discussion started by: Shenbaga.d
8 Replies

6. Shell Programming and Scripting

Bash script affect load average

Hello I have created next scritpt to do the next: chekp if host is alive. When the host down, launch telnet other equip to do checks. When execute the script the load average of the machines increase. For example: Before launch script top - 11:14:56 up 14 days, 18:06, 3 users, load... (3 Replies)
Discussion started by: capilla
3 Replies

7. Shell Programming and Scripting

Number of elements, average value, min & max from a list of numbers using awk

Hi all, I have a list of numbers. I need an awk command to find out the numbers of elements (number of numbers, sort to speak), the average value the min and max value. Reading the list only once, with awk. Any ideas? Thanks! (5 Replies)
Discussion started by: black_fender
5 Replies

8. UNIX for Dummies Questions & Answers

Taking a average of a column of numbers

Hey all, I am relatively poor at programming and unfortunately don't have time to read about programming at this current moment. I wanted to be able to run a simple command to read a column of numbers in a file and give me the average of those numbers. In addition if I could specify the... (2 Replies)
Discussion started by: Leonidsg
2 Replies

9. Shell Programming and Scripting

Extract numbers from text file work out average

Just wondering if someone could assist me with shell script I'm trying to write. I need to read the final column of a text file (shown below) and workout what the average number is. The text file will have a variable number of lines, I just want the script to pull out the values in the final field... (14 Replies)
Discussion started by: rich@ardz
14 Replies

10. Shell Programming and Scripting

extract numbers from a word

Hi ppl, I am a bit lost on this...can some one assist. I know this can be down with awk or sed, but i cant get the exact syntax right. I need to only extract the numbers from a signle word ( eg abcd.123.xyz ) How can i extract 123 only ? Thanks (14 Replies)
Discussion started by: systemali
14 Replies
Login or Register to Ask a Question