Shell Programming and Scripting

Hi,
I have two files A and B and would like to use A as a filter. Like this:

File A.txt: Contains a list of IP addresses, each one occurring only once and in order:

Code:

10.0.0.1
10.0.0.2
10.0.0.4
10.0.0.8

File B.txt: Contains the same IP addresses with a corresponding ping time each (in fact, file A.txt is generated with a simple sort -u command from file B.txt). B.txt could consist of thousands of rows but I have sorted all addresses and ping times in ascending order like this with another filter:

Code:

10.0.0.1     2
10.0.0.1    18
10.0.0.2     6
10.0.0.4     8
10.0.0.4    14
10.0.0.8    18

For each IP address in file A, I want to calculate the min/med/max ping time and the packet delay variation (difference in time between min/max) and store that in a separate file. So the result should look like this:

Code:

   IP      Min Ave Max PDV
10.0.0.1    2  10  18   16
10.0.0.2    6   6   6    0
10.0.0.4    8  11  14    6
10.0.0.8   18  18  18    0

Have tried quite a few different ways to solve this. One obvious way would probably be to loop through file A, and for each IP address I do a grep in file B and direct the result into a new file which is analysed separately. Something like this:

Code:

while read -r LINE ; do cat B.txt | grep $LINE > file$line.txt ; done < A.txt

I know the syntax won't do the trick (not any other similar syntax either, have tried them all), but if I could get one file per IP address I could figure out how to analyze the data.

After monitoring forums, I found out that a better way to do this on such big text files would be to use awk. Have found many examples in this forum, but none that takes entries from one file as input and uses that to define the scope for another file.

I have tried to combine 'awk' and 'read' but as the rookie I am I only manage to get some grey hair. Even this simple one (only trying to read A.txt and see if I can get the IP address out and match it with the left column in B.txt) fails:

Code:

while read -r LINE ; do awk '{if ($LINE[1]==$1) print $1}' B.txt ; done < A.txt

Just wonder if someone could give some advise here? Have really scanned after solutions in other threads on many forums but just can't pull it together.

As side note: Would be very interesting to understand if all of this could be done in a single command in some way also (including my first steps where A.txt is created and B.txt is sorted). Generating new files all the time means writing to the flash over and over and also add code to clean up those files as I get new files all the time. Would like to avoid that if possible.

Thanks!
Z

The sample data you have shown has averages that happen to work out to whole numbers. Do you always want the averages displayed as values rounded to whole numbers? If not, how many decimal places do you want in the output?

What formula are you using to calculate the packet delay variation? How many decimal places do you want to display in the output in the PDV column?

Does the output need to be sorted by IP address? (If not, the sort step can probably be eliminated.)

Here is a solution that works with the unsorted original file:

Code:

awk '
{ T[$1]+=$2
  if(!($1 in M)) M[$1]=X[$1]=$2
  else if($2<M[$1]) M[$1]=$2
  else X[$1]=$2
  C[$1]++
}
END {
  printf "%-17s %4s %4s %4s %4s\n", "IP", "Min", "Ave", "Max", "PDV"
  for (I in T)
     printf "%-17s %4d %4d %4d %4d\n", I, M[I], T[I]/C[I],X[I],X[I]-M[I]
}' infile

output:

Code:

IP                 Min  Ave  Max  PDV
10.0.0.1             2   10   18   16
10.0.0.2             6    6    6    0
10.0.0.4             8   11   14    6
10.0.0.8            18   18   18    0

Hi Don,
Thanks for reaching out. Made the sample data like that for a simple printout, but I realize it wasn't smart . One decimal would be sufficient for all entries in the result file.

The packet delay variation (jitter) is obtained by first finding the max and min ping times for an IP address and then substract the min from the max. Ex: max=10, min=2, PDV=10-2=8.

The output doesn't have to be sorted. If I need that later I can fix it. If more than one row is required that's also fine. What I mainly want is to avoid writing/deleting files if possible.

Thanks!
Z

For 1 decimal value change the printf format to %6.1f instead of %4d, or course the headings should also be changed to %6s

Works!! Thanks a million Chubler_XL!

If I'm reading Chubbier_XL's code correctly, if the input file is not sorted, the max value (X[$1]) may not be saved correctly. The following slightly modified code (adjusted for 1 decimal point in the average output and to save the correct maximum value for each IP address) should work:

Code:

awk '
{	if(!($1 in c)) m[$1] = M[$1] = $2
	else	if($2 > M[$1]) M[$1] = $2
	else	if($2 < m[$1]) m[$1] = $2
	s[$1] += $2
	c[$1]++
}
END {	printf("%-17s %3s %-5s %3s %3s\n", "   IP Address", "Min", " Ave",
		"Max", "PDV")
	printf("%17s %3s %5s %3s %3s\n", "=================", "===", "=====",
		"===", "===")
	for(i in c)
		printf("%-17s %3d %5.1f %3d %3d\n", i, m[i], s[i] / c[i], M[i],
			M[i] - m[i])
}' B.txt

which, with your sample input, produces the output:

Code:

   IP Address     Min  Ave  Max PDV
================= === ===== === ===
10.0.0.1            2  10.0  18  16
10.0.0.2            6   6.0   6   0
10.0.0.4            8  11.0  14   6
10.0.0.8           18  18.0  18   0

If someone else wants to try this on a Solaris system, change awk to /usr/xpg4/bin/awk.