How to calculate the entropy of a single directory that contains many files
Hello,
I'm new member of shell scripting and i face some difficulties. To begin, i try to write an algorithm that calculate from one directory containing nfdump files (288) the entropy of one day 24hours. Each of the file is 5 min interval
(nfdump -r nfcapd.200908250000 -s srcip) 1st
(nfdump -r nfcapd.200908250005 -s srcip) 2nd
(nfdump -r nfcapd.200908250010 -s srcip) 3rd
...
(nfdump -r nfcapd.200908252350 -s srcip) last
taking the flows (first 10) to calculate the entropy of the first 5 min interval i did it (i hope so in the right way) and i took the results of the 1st 5 min interval entropy. the code that i have written is:
Code:
#!/bin/sh
#x1 to x10 are variables which include values. Top 10 srcip Addr ordered by flows
x1=7576
x2=5691
x3=5130
x4=4239
x5=4136
x6=3387
x7=3070
x8=2780
x9=2125
x10=2011
#sum is the total flows
sum=551892
#calculate the values for each srcip Addr (10)
calx1=`echo "scale=6;$x1 / $sum" | bc`
calx2=`echo "scale=6;$x2 / $sum" | bc`
calx3=`echo "scale=6;$x3 / $sum" | bc`
calx4=`echo "scale=6;$x4 / $sum" | bc`
calx5=`echo "scale=6;$x5 / $sum" | bc`
calx6=`echo "scale=6;$x6 / $sum" | bc`
calx7=`echo "scale=6;$x7 / $sum" | bc`
calx8=`echo "scale=6;$x8 / $sum" | bc`
calx9=`echo "scale=6;$x9 / $sum" | bc`
calx10=`echo "scale=6;$x10 / $sum" | bc`
#calculate the logarithmic values from x1 to x10
l1=`echo "scale=6;l (${calx1})" | bc -l`
l2=`echo "scale=6;l (${calx2})" | bc -l`
l3=`echo "scale=6;l (${calx3})" | bc -l`
l4=`echo "scale=6;l (${calx4})" | bc -l`
l5=`echo "scale=6;l (${calx5})" | bc -l`
l6=`echo "scale=6;l (${calx6})" | bc -l`
l7=`echo "scale=6;l (${calx7})" | bc -l`
l8=`echo "scale=6;l (${calx8})" | bc -l`
l9=`echo "scale=6;l (${calx9})" | bc -l`
l10=`echo "scale=6;l (${calx10})" | bc -l`
#calculate the total entropy for 5 min interval
RESULT=`echo "scale=6;${calx1} * ${l1} + ${calx2} * ${l2} + ${calx3} * ${l3} + ${calx4} * ${l4} + ${calx5} * ${l5} + ${calx6} * ${l6} + ${calx7} * ${l7} + ${calx8} * ${l8} + ${calx9} * ${l9} + ${calx10} * ${l10}" | bc -l`
echo "Entropy for 5 min interval is ${RESULT}"
I used the shannon entropy formula for this.
My main problem now is how to make a matrix -array of all 288 files of 5 min interval and how is possible to combine the above algorithm with the matrix? Please if someone knows how to do this just to give me some help - advise
Thanks
I have multiple files test1, test2, test3 etc. I want to move to a different directory with ABC_ prefixed to every file and and current dat time as postfix using a single command. (I will be using this is sftp with ! (command for local server).
I have tried the following but it gives error
... (5 Replies)
I need to copy a complete directory structure into a new location. But I want to have all files copied into one directory and leave out the directory structure. So all files must be placed in one directory. (4 Replies)
I am trying to use awk to calculate the average of all lines in $2 for every file in a directory. The below bash seems to do that, but I cannot figure out how to capture the string before the _ as the output file name and have it be tab-delimeted. Thank you :).
Filenames in... (3 Replies)
Hi,
I have 2 csv/txt files with single columns. I am trying to merge them using paste, but its not working..
output3.csv:
flowerbomb
everlon-jewelry
sofft
steve-madden
dolce-gabbana-watchoutput2.csv:
http://www1.abc.com/cms/slp/2/Flowerbomb
http://www1.abc.com/cms/slp/2/Everlon-Jewelry... (5 Replies)
I am trying to do the following task :
export ENV=aaa
export ENV_PATH=$(cd /apps | ls | grep $ENV)
However, it's not working. What's the way to change to directory and search some file in that directory in single command
Please help. (2 Replies)
i have below list of files in a directory.
/root/admin/files/file1.txt
/root/admin/files/file2.txt
/root/admin/files/file3.txt
/root/admin/files/pattern.txt
/root/admin/files/server.txt
i need combine the above text files in the below sequence,
file1.txt, pattern.txt,server.txt =>... (8 Replies)
Hi,
Is there a way to find all the files from a specific location and then zip them into a single file, even if they are in multiple directories? (3 Replies)
hi
sorry for grammar mistakes but i am writting these fro tablet and am not realy used to ot so well yet....
i am in the middle of doing work here and hope some1 ca answear my question asap please :-)
How to calculate the amount of internal fragmentation using script.
cd directory ...... (6 Replies)
Hi,
I have written one script to calculate total space of all file in one directory, ignoring subdirectory, it works fine.
Now, I've been trying to calculate all files which includes files in any subdirectories.
I use recursive function to do this, but it can work only if there is only one... (4 Replies)
10-03-2009
How to calculate the entropy of a single directory that contains many files