Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk - find average interarrival times for each unique page

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk - find average interarrival times for each unique page
# 8  
Old 05-02-2011
Code:
function sd(N,average,array,line2)
 {for(i in array) {split(i,x,":") 
  if(x[2]==line2) sum+=(x[1]-average)^2
  }
 s=N==1?"0":sqrt(sum/(N-1))
 return s
 sum=0}
{v1=$2==v2?v1:$1
a[$2]=$1-v1
v2=$2
b[$2]++
c[$1":"$2]
d[$2]+=$1}
END{
 for(i in b) {
  print i,a[i]/(b[i]==1?1:b[i]-1),sd(b[i],d[i]/b[i],c,i)|"sort"}
  }

# 9  
Old 05-02-2011
yinyuemi,

How can I test this? Do I just run this on the same file I have? It looks I have to pass it 4 parameters?

Quote:
Originally Posted by yinyuemi
Code:
function sd(N,average,array,line2)
 {for(i in array) {split(i,x,":") 
  if(x[2]==line2) sum+=(x[1]-average)^2
  }
 s=N==1?"0":sqrt(sum/(N-1))
 return s
 sum=0}
{v1=$2==v2?v1:$1
a[$2]=$1-v1
v2=$2
b[$2]++
c[$1":"$2]
d[$2]+=$1}
END{
 for(i in b) {
  print i,a[i]/(b[i]==1?1:b[i]-1),sd(b[i],d[i]/b[i],c,i)|"sort"}
  }

# 10  
Old 05-02-2011
save the code as awk.script
Code:
cat file
0.000 55588
0.000 55592
3.2320 55592
117.964 55596
530.841 55596
928.232 55596
117.964 55600
530.841 55600
630.789 55600
700.232 55600

awk -f awk.script file
55588 0 0
55592 3.232 2.28537
55596 405.134 515.903
55600 194.089 260.771

# 11  
Old 05-03-2011
yinyuemi,

Maybe I didn't explain it clearly...

For this file:
Code:
cat file
0.000 55588
0.000 55592
3.2320 55592
117.964 55596
530.841 55596
928.232 55596
117.964 55600
530.841 55600
630.789 55600
700.232 55600

The results should be like this:
Code:
awk -f awk.script file
55588 0 0
55592 3.232 0
55596 405.134 7.743
55600 194.089 155.207

For page 55588, there is no intArriTime time since there is only 1 request for that page. So both the aveIntArrTime and the stdDev are both 0.

For page 55592, there is only 1 intArriTime (3.232 - 0 = 3.232). So the aveIntArriTime is also 3.232. The stdDev is 0 since there is only 1 aveIntArriTime.

For page 55596, there are 3 ArriTimes, which means there are 2 intArriTimes, which are 412.877 and 397.391, respectively.

So the aveIntArrTime is (412.877 + 397.391)/2 = 405.134.

And the stdDev for page 55596 is:
stdDev = square_root { [ (412.877-405.134)^2 + (397.391-405.134)^2 ] / 2 }
stdDev = 7.743

Similar logic follows for page 55600.

Hopefully this clears things up. Thanks for your time.
# 12  
Old 05-03-2011
Code:
function sd(N,average,array,line2)
 {for(i in array) {split(i,x,":") 
  if(x[2]==line2) sum[line2]+=(x[1]-average)^2
  }
 s=N<=2?"0":sqrt(sum[line2]/(N-1))
 return s
 }
{v1=$2==v2?v1:$1
a[$2]=$1-v1
b[$2]++
intr=$1-v3;
$2==v2?c[intr":"$2":"++p]:intr=0
v2=$2
v3=$1
}
END{for(i in b) {
  avr=a[i]/(b[i]<=1?1:b[i]-1);print i,avr,sd(b[i],avr,c,i)|"sort"}
  }
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Awk: Page number with total number of pages, EG Page 1 of 5

So I've worked how to add page numbers based on regex. It's using the footer text. How do we get the total amount added so we have page number with the total number of pages? Desired output: Page No:1 of 5 Thanks in advance. (15 Replies)
Discussion started by: tugar
15 Replies

2. Shell Programming and Scripting

Using awk to find unique, how to save results?

I am very very new to this (as in, I didn't even know awk existed till today) I have a huuuuge csv file. In column 1, there is a ton of emails. I need to find which emails are unique, and save those rows to a separate file. I also need to find which emails are duplicates, and save a record of... (10 Replies)
Discussion started by: shonna
10 Replies

3. Shell Programming and Scripting

Find the average and the different

Hi , Every day I'll get a file, in that I have to match today's file(20130619) third column to previous files (20130618,20130617), that is 124 present in previous files or not. If it matches then I have take the average values of 5th column of 124 from yesterdays and day before yesterdays file,... (5 Replies)
Discussion started by: Shenbaga.d
5 Replies

4. Shell Programming and Scripting

awk to find lines containing word that occur multiple times

i have a script that scans a log file every 10 minutes. this script remembers the last line of the log and then uses it to continue monitoring the log when it runs again 10 minutes later. the script searches the log for a string called MaxClients. now, how can i make it so that when the... (7 Replies)
Discussion started by: SkySmart
7 Replies

5. Shell Programming and Scripting

AWK script to split data and find average

Input: 2.58359023380340e+02 1.43758864405595e+02 -7.65700666212508e+00 1.06460208083228e+02 1.26185441783936e+02 -3.41389169427027e+01 -1.40393299309592e+02 -3.07758776849508e+01 1.45067703495838e+02 1.79405834959073e+02 5.06666234594205e+01 OUT 2.0105894389e+02 (average of... (8 Replies)
Discussion started by: chrisjorg
8 Replies

6. Shell Programming and Scripting

awk based script to find the average of all the columns in a data file

Hi All, I need the modification for the below mentioned code (found in one more post https://www.unix.com/shell-programming-scripting/27161-script-generate-average-values.html) to find the average values for all the columns(but for a specific rows) and print the averages side by side. I have... (4 Replies)
Discussion started by: ks_reddy
4 Replies

7. Homework & Coursework Questions

Find the Maximum value and average of a column

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: I am trying to complete a script which will allow me to find: a) reads a value from the keyboard. (ask the... (4 Replies)
Discussion started by: dstewie
4 Replies

8. UNIX for Dummies Questions & Answers

Compare unique ID's to Create and Delete Times

I have thousands of lines a day of data I would like to sort out. Every sessions has the 3 lines below. I want to figure out each sessions length from Creation to Deletion. Every one has a unique session ID logevent3:<190>Nov 20 08:41:06 000423df255c: 6|4096|RC|CAC: Created CAC session ID... (2 Replies)
Discussion started by: bpfoster7
2 Replies

9. SCO

Find access times

I am working on a SCO Unixware 7.1.4 server and I have been asked to determine over the last year when a file was accessed, not just the last time it was accessed. Is there anyway to figure this out? Thanks in advance, Kevin Harnden (1 Reply)
Discussion started by: chefsride
1 Replies
Login or Register to Ask a Question