Help in extracting multiple files and taking average at same time | Unix Linux Forums | Shell Programming and Scripting

  Go Back    


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

Help in extracting multiple files and taking average at same time

Shell Programming and Scripting


Closed Thread    
 
Thread Tools Search this Thread Display Modes
    #1  
Old 08-23-2008
ahjiefreak ahjiefreak is offline
Registered User
 
Join Date: Nov 2007
Last Activity: 11 December 2008, 9:39 AM EST
Posts: 132
Thanks: 0
Thanked 0 Times in 0 Posts
Help in extracting multiple files and taking average at same time

Hi,

I have 20 files which have respective 50 lines with different values.

I would like to process each line of the 50 lines in these 20 files one at a time and do an average of 3rd field ($3) of these 20 files. This will be output to an output file.

Instead of using join to generate whole bunch of redundant files and then compute the average, im looking any other possible better way to do the above right away.

E.g

apple.txt
tool1 2.00 4 30.20
tool2 3.00 5 40.22
tool3 2.00 6 45.32
....
tool50 ...........


orange.txt
tool1 1.00 2 30.20
tool2 4.00 3 40.22
tool3 6.00 4 45.32
...
tool50 ...

bar.txt
tool1 2.10 1 30.20
tool2 3.04 4 40.22
tool3 2.02 5 45.32
...
tool50 .....

and the remaining 17 files of different names.

The output would be:-
tool1 (4+2+1+....)/20
tool2 (5+3+4+...)/20
tool3 (6+4+5+...)/20
....
tool50....

Please advise. THanks.
Sponsored Links
    #2  
Old 08-23-2008
vgersh99's Avatar
vgersh99 vgersh99 is offline Forum Advisor  
Forum Advisor
 
Join Date: Feb 2005
Last Activity: 29 November 2014, 12:11 AM EST
Location: Foxborough, MA
Posts: 7,672
Thanks: 153
Thanked 586 Times in 551 Posts
Pls show the proof of the effort.

Last edited by vgersh99; 08-23-2008 at 01:05 PM.. Reason: jumping the gun - need to see the proof of the effort.
Sponsored Links
    #3  
Old 08-23-2008
ghostdog74 ghostdog74 is offline
Registered User
 
Join Date: Sep 2006
Last Activity: 9 November 2014, 12:24 AM EST
Posts: 2,669
Thanks: 0
Thanked 18 Times in 18 Posts
I see you have been here for a while. How about showing some code on what you did that doesn't work?
    #4  
Old 08-23-2008
ahjiefreak ahjiefreak is offline
Registered User
 
Join Date: Nov 2007
Last Activity: 11 December 2008, 9:39 AM EST
Posts: 132
Thanks: 0
Thanked 0 Times in 0 Posts
Hi,

My initial thought/idea was:-

!/bin/awk

filename={apple.txt,orange.txt,bar.txt................}

for file in filename;do

while(getline<"file"); do

val+=$3;
count++;

done

done
tada=val/count;

print $1"," tada>output.txt

That's my first intuition to the above case problem using awk.
Sponsored Links
    #5  
Old 08-23-2008
ghostdog74 ghostdog74 is offline
Registered User
 
Join Date: Sep 2006
Last Activity: 9 November 2014, 12:24 AM EST
Posts: 2,669
Thanks: 0
Thanked 18 Times in 18 Posts
Is that just your pseudocode or you are definitely writing an awk script? you seem to be mixing shell and awk syntaxes all over.
Sponsored Links
    #6  
Old 08-23-2008
era era is offline Forum Advisor  
Herder of Useless Cats (On Sabbatical)
 
Join Date: Mar 2008
Last Activity: 28 March 2011, 6:41 AM EDT
Location: /there/is/only/bin/sh
Posts: 3,653
Thanks: 0
Thanked 10 Times in 8 Posts
awk offers arrays for precisely this type of problem. Make each item with a count a key of the array, then at END print the results. If all keys occur in all files you can simply divide by the number of input files, otherwise you will need to collect both the sum and the count (divisor) for each key.
Sponsored Links
    #7  
Old 08-25-2008
summer_cherry summer_cherry is offline Forum Advisor  
Registered User
 
Join Date: Jun 2007
Last Activity: 27 November 2014, 1:23 AM EST
Location: Beijing China
Posts: 1,294
Thanks: 0
Thanked 24 Times in 24 Posts
below code should be ok, it assume all file with same naming convention .txt,
nomatter how many files, it can process them


Code:
sum=`ls -l *.txt | wc -l`
paste -d" " *.txt | nawk -v s="$sum" '{
	for(i=0;i<=s-1;i++)
	{
		t=3+(i*4)
		temp=temp+$t
	}
	print $1"---->"temp/s
	temp=0
}'


Last edited by summer_cherry; 08-25-2008 at 02:56 AM..
Sponsored Links
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Taking a average of a column of numbers Leonidsg UNIX for Dummies Questions & Answers 2 01-15-2012 04:02 PM
Computing average values from multiple text files rbredereck Shell Programming and Scripting 3 08-01-2011 07:25 AM
Extracting/condensing text from multiple files to multiples files born2phase Shell Programming and Scripting 2 05-06-2011 05:25 AM
Extracting columns from multiple files with awk orcaja Shell Programming and Scripting 10 09-23-2010 03:56 AM
Average of elements throught multiple files chillmaster Shell Programming and Scripting 10 08-08-2008 12:28 PM



All times are GMT -4. The time now is 02:10 AM.