Sponsored Content
Top Forums Shell Programming and Scripting How to calculate the entropy of a single directory that contains many files Post 302358619 by draxmas on Saturday 3rd of October 2009 01:27:14 PM
Old 10-03-2009
Data How to calculate the entropy of a single directory that contains many files

Hello,
I'm new member of shell scripting and i face some difficulties. To begin, i try to write an algorithm that calculate from one directory containing nfdump files (288) the entropy of one day 24hours. Each of the file is 5 min interval
(nfdump -r nfcapd.200908250000 -s srcip) 1st
(nfdump -r nfcapd.200908250005 -s srcip) 2nd
(nfdump -r nfcapd.200908250010 -s srcip) 3rd
...
(nfdump -r nfcapd.200908252350 -s srcip) last

taking the flows (first 10) to calculate the entropy of the first 5 min interval i did it (i hope so in the right way) and i took the results of the 1st 5 min interval entropy. the code that i have written is:

Code:
#!/bin/sh
#x1 to x10 are variables which include values. Top 10 srcip Addr ordered by flows
x1=7576
x2=5691
x3=5130
x4=4239
x5=4136
x6=3387
x7=3070
x8=2780
x9=2125
x10=2011
#sum is the total flows
sum=551892
#calculate the values for each srcip Addr (10)
calx1=`echo "scale=6;$x1 / $sum" | bc`
calx2=`echo "scale=6;$x2 / $sum" | bc`
calx3=`echo "scale=6;$x3 / $sum" | bc`
calx4=`echo "scale=6;$x4 / $sum" | bc`
calx5=`echo "scale=6;$x5 / $sum" | bc`
calx6=`echo "scale=6;$x6 / $sum" | bc`
calx7=`echo "scale=6;$x7 / $sum" | bc`
calx8=`echo "scale=6;$x8 / $sum" | bc`
calx9=`echo "scale=6;$x9 / $sum" | bc`
calx10=`echo "scale=6;$x10 / $sum" | bc`
#calculate the logarithmic values from x1 to x10
l1=`echo "scale=6;l (${calx1})" | bc -l`
l2=`echo "scale=6;l (${calx2})" | bc -l`
l3=`echo "scale=6;l (${calx3})" | bc -l`
l4=`echo "scale=6;l (${calx4})" | bc -l`
l5=`echo "scale=6;l (${calx5})" | bc -l`
l6=`echo "scale=6;l (${calx6})" | bc -l`
l7=`echo "scale=6;l (${calx7})" | bc -l`
l8=`echo "scale=6;l (${calx8})" | bc -l`
l9=`echo "scale=6;l (${calx9})" | bc -l`
l10=`echo "scale=6;l (${calx10})" | bc -l`
#calculate the total entropy for 5 min interval
RESULT=`echo "scale=6;${calx1} * ${l1} + ${calx2} * ${l2} + ${calx3} * ${l3} + ${calx4} * ${l4} + ${calx5} * ${l5} + ${calx6} * ${l6} + ${calx7} * ${l7} + ${calx8} * ${l8} + ${calx9} * ${l9} + ${calx10} * ${l10}" | bc -l`
echo "Entropy for 5 min interval is ${RESULT}"

I used the shannon entropy formula for this.

My main problem now is how to make a matrix -array of all 288 files of 5 min interval and how is possible to combine the above algorithm with the matrix? Please if someone knows how to do this just to give me some help - advise
Thanks
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to calculate file's size in directory and subdirectory

Hi, I have written one script to calculate total space of all file in one directory, ignoring subdirectory, it works fine. Now, I've been trying to calculate all files which includes files in any subdirectories. I use recursive function to do this, but it can work only if there is only one... (4 Replies)
Discussion started by: KLL
4 Replies

2. Emergency UNIX and Linux Support

calculate internal fragmentation in directory

hi sorry for grammar mistakes but i am writting these fro tablet and am not realy used to ot so well yet.... i am in the middle of doing work here and hope some1 ca answear my question asap please :-) How to calculate the amount of internal fragmentation using script. cd directory ...... (6 Replies)
Discussion started by: me.
6 Replies

3. Shell Programming and Scripting

Find all the files under a specific directory and zip them into a single file.

Hi, Is there a way to find all the files from a specific location and then zip them into a single file, even if they are in multiple directories? (3 Replies)
Discussion started by: rudoraj
3 Replies

4. Shell Programming and Scripting

how to find first files in a directory and combine them as a single file?

i have below list of files in a directory. /root/admin/files/file1.txt /root/admin/files/file2.txt /root/admin/files/file3.txt /root/admin/files/pattern.txt /root/admin/files/server.txt i need combine the above text files in the below sequence, file1.txt, pattern.txt,server.txt =>... (8 Replies)
Discussion started by: vel4ever
8 Replies

5. Shell Programming and Scripting

Change to directory and search some file in that directory in single command

I am trying to do the following task : export ENV=aaa export ENV_PATH=$(cd /apps | ls | grep $ENV) However, it's not working. What's the way to change to directory and search some file in that directory in single command Please help. (2 Replies)
Discussion started by: saurau
2 Replies

6. Shell Programming and Scripting

Paste 2 single column files to a single file

Hi, I have 2 csv/txt files with single columns. I am trying to merge them using paste, but its not working.. output3.csv: flowerbomb everlon-jewelry sofft steve-madden dolce-gabbana-watchoutput2.csv: http://www1.abc.com/cms/slp/2/Flowerbomb http://www1.abc.com/cms/slp/2/Everlon-Jewelry... (5 Replies)
Discussion started by: ajayakunuri
5 Replies

7. Shell Programming and Scripting

Bash to calculate average of all files in directory and output by part of filename

I am trying to use awk to calculate the average of all lines in $2 for every file in a directory. The below bash seems to do that, but I cannot figure out how to capture the string before the _ as the output file name and have it be tab-delimeted. Thank you :). Filenames in... (3 Replies)
Discussion started by: cmccabe
3 Replies

8. Shell Programming and Scripting

Copy files recursively to one single directory

I need to copy a complete directory structure into a new location. But I want to have all files copied into one directory and leave out the directory structure. So all files must be placed in one directory. (4 Replies)
Discussion started by: ReneVL
4 Replies

9. Shell Programming and Scripting

Move multiple files to different directory using a single command

I have multiple files test1, test2, test3 etc. I want to move to a different directory with ABC_ prefixed to every file and and current dat time as postfix using a single command. (I will be using this is sftp with ! (command for local server). I have tried the following but it gives error ... (5 Replies)
Discussion started by: Soham
5 Replies
RANDOM_HARVEST(9)					   BSD Kernel Developer's Manual					 RANDOM_HARVEST(9)

NAME
random_harvest -- gather entropy from the kernel for the entropy device SYNOPSIS
#include <sys/types.h> #include <sys/random.h> void random_harvest(void *entropy, u_int size, u_int bits, u_int frac, enum esource source); DESCRIPTION
The random_harvest() function is used by device drivers and other kernel processes to pass data that is considered (at least partially) sto- chastic to the entropy device. The caller should pass a pointer (to no more than 16 bytes) of the ``random'' data in entropy. The argument size contains the number of bytes pointed to. The caller should very conservatively estimate the number of random bits in the sample, and pass this in bits or frac. If the estimated number of bits per sample is an integer, then bits is used, and frac is 0. Otherwise, for low-entropy samples, ``fractional'' entropy can be supplied in frac. (This is considered to be frac / 1024 bits of entropy.) The source is chosen from RANDOM_WRITE, RANDOM_KEYBOARD, RANDOM_MOUSE, RANDOM_NET and RANDOM_INTERRUPT, and is used to indicate the source of the entropy. Interrupt harvesting has been simplified for the kernel programmer. If a device driver registers an interrupt handler with BUS_SETUP_INTR(9) or bus_setup_intr(9), then it is only necessary to include the INTR_ENTROPY bit in the flags argument to have that interrupt source be used for entropy harvesting. SEE ALSO
random(4), BUS_SETUP_INTR(9) AUTHORS
The FreeBSD random(4) entropy device and supporting documentation was written by Mark R V Murray. BSD
February 6, 2002 BSD
All times are GMT -4. The time now is 05:22 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy