Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

reading and calculating no. of occurrence in awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting reading and calculating no. of occurrence in awk
# 1  
Old 11-14-2010
PHP reading and calculating no. of occurrence in awk
SmilieHi,

I have got results from my simulation and I did extract the time field from the results file into new file. Then I was trying to calculate no. of occurrence for each set of time. However, I couldn't.

For example :
0.2 --
0.2 --
0.21016 --
0.21016 --
0.21016 --
.
.
.
0.299387
0.3 --

the time increase by 0.1 and so on. I want to calculate no of occurrence for each 0.1 increase (e.g from 0.2 --to --0.3 how many lines or occurrence?)

I wrote script in awk but not working: (any suggestion)

Code:
BEGIN{ 
    no_occurrence=0;
    nextTime=0.1;
    incrMent=0.1;
    count=0;
    RS= null;
}{    
    while (RS!=null){
    time= $0;
    no_occurrence++;
    count++;
        if (time>nextTime){
            print "time :" time ;
            print "count" count;
            nextTime= nextTime+incrMent;
            
        }
    }
}END{}

# 2  
Old 11-14-2010
No need to initialise variables to zero in begin.
Reset counter for next range.
Only count after you test value is in range.
Output counts of last range in END block:

Code:
BEGIN{ 
    nextTime=incrMent=0.1;
}
{    
    time= $0;
    while (time>nextTime){
        printf "%3.1f - %3.1f  count %d\n", nextTime - incrMent, nextTime, count;
        nextTime+=incrMent;
        count=0;
    }
    no_occurrence++;
    count++;
}END{
    printf "%3.1f - %3.1f  count %d\n", nextTime - incrMent, nextTime, count;
}

Last edited by Chubler_XL; 11-14-2010 at 12:59 PM..
# 3  
Old 11-14-2010
MySQL Thanks
Thank you very much for your time and help. it works SmilieSmilieSmilieSmilieSmilieSmilie
as I'm beginner to awk scripting.
Could you explain to me what's the point of END and I've noticed you print the results in side end block.
# 4  
Old 11-14-2010
The END block executes after all the input has been read, in this case we print the results we have been accumulating for the last time range.

For example if the last few entries in your file were:
Code:
0.87674
0.87898
0.88123

The count variable would be counting lines in the 0.8 - 0.9 range. No record greater than 0.9 exists in the file so the count for this range hasn't been output by the main block. The END block prints this final result.
Last edited by Chubler_XL; 11-14-2010 at 03:25 PM.. Reason: Tidy up formatting
# 5  
Old 11-14-2010
Depending on the expected result
Code:
cut -c1-3 inputfile | uniq -c

or
Code:
uniq inputfile | cut -c1-3 | uniq -c

# 6  
Old 11-14-2010
Your the maaaan,
I really appreciate your help

Thanks alot>>>>>>>>

again I have small problem I have a file with many fields. I want to extract field 2, 8 where field 8 could be either 1 or 2

I tried to extract it, it works but in a strange way. it gives a new file contain field 2 ($2) and $8 without any respect to the value of $8....

For example
1.222 1
1.230 1
1.3 2
I want each value of field 8 $8 to be printed with its own set.
Like :
file called for example ones:
1.222 1
1.230 1

another file called two contains;
1.33 2

Note I tried to do it in command line direct got the same !!!


My code is :
Code:
BEGIN{ 
if ($8 == "2") ;
}
{ print $2, $8;
}
END {}

# 7  
Old 11-14-2010
How about this:
Code:
BEGIN { system("rm -f ones"); system("rm -f two"); }
($8 == 1) { print $2, $8 >> "ones" }
($8 == 2) { print $2, $8 >> "two" }

BEGIN block removes files "ones" and "two" (just incase they are already there from previous run of awk program).
You could also do this in the shell before calling awk, and have the program always append to ones and two if they are already there.

Note any line that contains a value other than 1 or 2 in field 8 is just ignored.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Calculating correlations across columns in awk

Hello, I came across a previous thread "awk-calculating-simple-correlation-rows" which calculated correlations across rows in awk. Code: awk '{ a = 0; for (i = 2; i <= NF; ++i) a += $i; a /= NF-1 b = 0; for (i = 2; i <= NF; ++i) b += ($i - a) ^ 2; b = sqrt(b) if... (7 Replies)
Discussion started by: Ross
7 Replies

2. UNIX for Dummies Questions & Answers

Calculating cumulative frequency using awk

Hi, I wanted to calculate cumulative frequency distribution of my data that involves several arithmetic calls. I did things in excel but its taking me forever. this is what I want to do: var1.txt contains n observations which I have to compute for frequency which is given by 1/n and subsequently... (7 Replies)
Discussion started by: ida1215
7 Replies

3. Shell Programming and Scripting

Calculating Running Variance Using Awk

Hi all, I am attempting to calculate a running variance for a file containing a column of numbers. I am using the formula variance=sum((x-mean(x))^2)/(n-1), where x is the value on the current row, and mean(x) is the average of all of the values up until that row. n represents the total number... (1 Reply)
Discussion started by: Jahn
1 Replies

4. Shell Programming and Scripting

Calculating average with awk

I need to find the average from a file like: data => BW:123 M:30 RTD:0 1 0 1 0 0 1 1 1 1 0 0 1 1 0' data => BW:123 N:30 RTD:0 1 0 1 0 0 1 1 1 1 0 0 1 1 0' data => BW:123 N:30 RTD:0 1 0 1 0 0 1 1 1 1 0 0 1 1 0' data => BW:123 N:30 RTD:0 1 0 1 0 0 1 1 1 1 0 0 1 1 0' data => BW:123 N:30 RTD:0 1... (4 Replies)
Discussion started by: Slagle
4 Replies

5. Shell Programming and Scripting

Calculating the epoch time from standard time using awk and calculating the duration

Hi All, I have the following time stamp data in 2 columns Date TimeStamp(also with milliseconds) 05/23/2012 08:30:11.250 05/23/2012 08:30:15.500 05/23/2012 08:31.15.500 . . etc From this data I need the following output. 0.00( row1-row1 in seconds) 04.25( row2-row1 in... (5 Replies)
Discussion started by: ks_reddy
5 Replies

6. Shell Programming and Scripting

AWK way of calculating growth

Hi All, IS there any 'awk' way to manipulate following data? Fruit Date Count Apple 20/08/2011 5 Apple 27/08/2011 7 Apple 05/09/2011 11 Apple 12/09/2011 3 Apple 19/09/2011 25 . . . . Orange 20/08/2011 9 Orange 27/08/2011 20 Orange 27/08/2011 7 Orange 05/09/2011 15 Orange... (3 Replies)
Discussion started by: aniketdixit
3 Replies

7. Shell Programming and Scripting

Calculating an integer with awk

I would like to extract a number from $0 and calculate if it can be devided by 25. Though the number can also be less then 25 or bigger than 100. How do i extract the number and how can the integer be calculated? String: "all_results">39</span>I am looking for the number between "all_results"> ... (5 Replies)
Discussion started by: sdf
5 Replies

8. Shell Programming and Scripting

Awk program for calculating dates.

Hi All, I have a txt file which has hundreds of lines and 41 fields. I have a requirement to pick up field 14 from the text file which is a date fiels in the following format. Field 14 : Data Type : NUMERIC DATE (YYYYMMDD) Field Length : 8 Example of Data :20090415 Field 42 : Data Type... (2 Replies)
Discussion started by: nua7
2 Replies

9. Shell Programming and Scripting

Calculating totals in AWK

Hello, With the following small script I list the size of documents belonging to a certain user by each time selecting the bytes-field of that file ($7). Now it fills the array with every file it finds so in the end the output of some users contains up to 200.000 numbers. So how can I calculate... (7 Replies)
Discussion started by: Hille
7 Replies

10. UNIX for Dummies Questions & Answers

awk + last occurrence

Hi, I'm attempting to search, using awk, a pattern range in a file. Something like: >awk '/first bit of text.../,/...last bit of text/' file Is it possible to print only the last (or first) occurrence of the pattern range this way? Thanks for any suggestions. Al (2 Replies)
Discussion started by: agibbs
2 Replies
Login or Register to Ask a Question