UNIX for Dummies Questions & Answers

Hi, here's some ideas applied to Your data... assuming (including any number of space):

Code:

    6643 total packets received
    6718 total packets received
    7293 total packets received
    7785 total packets received

The first example is a method I use a lot in different versions, basically presenting the diff in a value from it's previous value. Good for analysing data from log files for example. And since there is always a beginning for everything, I chose in this case, 0:

Code:

#!/bin/bash
cnt=0
oldval=0
while read newval x y z; do
	echo diff=$((newval-oldval))
	oldval=$newval
	((cnt++))
done < tmpIP

And if You want only diff from every other line (if this is what You meant):

Code:

#!/bin/bash
cnt=0
while read newval x y z; do
	[ $((cnt % 2)) -eq 1 ] && echo diff=$((newval-oldval))
	oldval=$newval
	((cnt++))
done < tmpIP

Try changing [ $((cnt % 2)) -eq 1 ] for a different result. It's a construct to return true if You are on an even line. Well, er... yes, it's uneven with regard to cnt, but we usually we start counting from 0, so, oh, whatever... fiddle about with it! Change -eq to 0 for a different, "shifted", result.
The x y z is just to consume values from tmpIP file, otherwise the whole line would be considered one variable.
The read is good for ignoring whitespace, as many other clu's are.

Well, just my 2 öre!

/Lakris

PS Oh, almost forgot... maybe You could consider piping Your data collecting directly through the while loop, and not using intermediate files. Just a thought.

Thanks Lakris.

Currently I have my data collecting running through a function so every time the script is ran another piece of data is collected. The idea is that using chmod the script is ran x amount times between set intervals so I can determine network usage on a machine. Hence the working out of the differences between the values as demonstrated in your 2nd code.

To pipe into the while loop I would have to run the netstat -s command, followed by the grep to pick out the data I wanted (this case IP total packets received). Would I then need to put that data into a file to be stored, so I can then use it in the while loop - bearing in mind that this data could be collected over a 24hr period.

So I've been building the script to collect the data first then perform the calculations.
For instance:

Code:

collect_function  #Collect new data and put it in tempIP file 
cnt=0
while read newval x y z; do
        [ $((cnt % 2)) -eq 1 ] && echo diff=$((newval-oldval))
        oldval=$newval
        ((cnt++))
done < tempIP > tempdiff

I'm not sure how piping into the loop would work with collecting the data

Thanks for your input

Ok, I noticed that You are appending output, and if this is going on for some time, maybe You want some kind of average? With distinct time steps? For each time the data collection has taken place? It's hard to guess what the relevant analysis of the data would be.

Find max? min? Average? Time series? Related to time of day? There's lots of tools doing this, such as ntop and xosview. Mostly real time. They rely on data found in files in /proc/net, a part of the file system that is continuously updated with network related data (snapshots).

For example, I have a "system icon" in my IceWM that monitors network usage contiuously. It graphs usage in its icon. When I hover the mouse pointer over it it displays statistics about in/out, current, average, etc. I can't find it now but I'm pretty certain that that data is in one of the files in /proc/net.

Google for it. You may have better luck going for the source (netstat is also using /proc/net) and accumulating data from that, rather than creating logic around miscellaneous shell programming.

I am sorry if I can't give any better directions, I think I will explore this tomorrow...

/Lakris

The analysis I'm trying to come out with is to determine over a given amount of time when network usage is low. So theoretically over say 10, 30 minute time-slots I can see which time-slot is on average the best time-slot to start a data backup.

I envisage a file that can be interrogated about all the collected time-slots network usage and work out each time-slots average and say "time slot" x is on average the best time to perform a backup of a system. The time-slots time, that is on average the lowest network usage, is then put into a backup system (AMANDA) to perform the backup. The time-slots are related to time of day.

Just explored the /proc/net/dev file which produces this result
Edited for space and format reasons.

Code:

 
 nistleloy@****:/proc/net> cat dev 
 Inter- face   |      Receive                         |           Transmit 
                  bytes        packets                  bytes        packets    
     lo:            2260          38                      2260         38
   eth1:           6288256      4939                  351334       3370

Looking at that information this would be more clinical and create a more manageable analysis of network usage compared to working individual protocols out. Now all I have need to do is run the data collection based on the total transmit / receive data. I'd probably use the packet data.

Great spot Lakris.

I also looked at the two tools you pointed out and they will interrogate this information to. Ironically the netstat -s command does not provide the succinct information that /pro/net/dev does. Furthermore looking at XP netstat command this info does come back! Just looking around that dir and I think netstat basis it's info on proc/net/protocols amongst over things hence why when I was trying to code with the command I was gathering small bits of data and building them up logically.

Just had a look around my system apps and found "Network tool" that shows the interface stats gathered from the above file. Although it doesn't display any graphs just neatly presented numbers for each interface.

Once again thank you for your input.

Np!

After running this script

Code:

cnt=0
while read newval x y z; do
        echo $((newval-oldval))
        oldval=$newval
        ((cnt++))
done < dump2 > final

on this file with these numbers:

Code:

nistleloy@****:~/Documents> cat dump2
84657
93406
112554
112607
123780
254687

i get this answer back:

Code:

nistleloy@****:~/Documents> cat final
84657
8749
19148
53
11173
130907

Is there anyway of stopping the the first reading in dump2 file being subtracted from 0 and ending up in the final file as it's really messing up the results! I far as I can see it shouldn't perform any maths on that reading at all but it is.

Thanks for any tips/advice.

Hi, You could use something like
[ $cnt -gt 0 ] && echo ...
as the first statement in loop.

/Lakris