Sponsored Content
Top Forums Shell Programming and Scripting How to calculate the entropy of a single directory that contains many files Post 302358619 by draxmas on Saturday 3rd of October 2009 01:27:14 PM
Old 10-03-2009
Data How to calculate the entropy of a single directory that contains many files

Hello,
I'm new member of shell scripting and i face some difficulties. To begin, i try to write an algorithm that calculate from one directory containing nfdump files (288) the entropy of one day 24hours. Each of the file is 5 min interval
(nfdump -r nfcapd.200908250000 -s srcip) 1st
(nfdump -r nfcapd.200908250005 -s srcip) 2nd
(nfdump -r nfcapd.200908250010 -s srcip) 3rd
...
(nfdump -r nfcapd.200908252350 -s srcip) last

taking the flows (first 10) to calculate the entropy of the first 5 min interval i did it (i hope so in the right way) and i took the results of the 1st 5 min interval entropy. the code that i have written is:

Code:
#!/bin/sh
#x1 to x10 are variables which include values. Top 10 srcip Addr ordered by flows
x1=7576
x2=5691
x3=5130
x4=4239
x5=4136
x6=3387
x7=3070
x8=2780
x9=2125
x10=2011
#sum is the total flows
sum=551892
#calculate the values for each srcip Addr (10)
calx1=`echo "scale=6;$x1 / $sum" | bc`
calx2=`echo "scale=6;$x2 / $sum" | bc`
calx3=`echo "scale=6;$x3 / $sum" | bc`
calx4=`echo "scale=6;$x4 / $sum" | bc`
calx5=`echo "scale=6;$x5 / $sum" | bc`
calx6=`echo "scale=6;$x6 / $sum" | bc`
calx7=`echo "scale=6;$x7 / $sum" | bc`
calx8=`echo "scale=6;$x8 / $sum" | bc`
calx9=`echo "scale=6;$x9 / $sum" | bc`
calx10=`echo "scale=6;$x10 / $sum" | bc`
#calculate the logarithmic values from x1 to x10
l1=`echo "scale=6;l (${calx1})" | bc -l`
l2=`echo "scale=6;l (${calx2})" | bc -l`
l3=`echo "scale=6;l (${calx3})" | bc -l`
l4=`echo "scale=6;l (${calx4})" | bc -l`
l5=`echo "scale=6;l (${calx5})" | bc -l`
l6=`echo "scale=6;l (${calx6})" | bc -l`
l7=`echo "scale=6;l (${calx7})" | bc -l`
l8=`echo "scale=6;l (${calx8})" | bc -l`
l9=`echo "scale=6;l (${calx9})" | bc -l`
l10=`echo "scale=6;l (${calx10})" | bc -l`
#calculate the total entropy for 5 min interval
RESULT=`echo "scale=6;${calx1} * ${l1} + ${calx2} * ${l2} + ${calx3} * ${l3} + ${calx4} * ${l4} + ${calx5} * ${l5} + ${calx6} * ${l6} + ${calx7} * ${l7} + ${calx8} * ${l8} + ${calx9} * ${l9} + ${calx10} * ${l10}" | bc -l`
echo "Entropy for 5 min interval is ${RESULT}"

I used the shannon entropy formula for this.

My main problem now is how to make a matrix -array of all 288 files of 5 min interval and how is possible to combine the above algorithm with the matrix? Please if someone knows how to do this just to give me some help - advise
Thanks
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to calculate file's size in directory and subdirectory

Hi, I have written one script to calculate total space of all file in one directory, ignoring subdirectory, it works fine. Now, I've been trying to calculate all files which includes files in any subdirectories. I use recursive function to do this, but it can work only if there is only one... (4 Replies)
Discussion started by: KLL
4 Replies

2. Emergency UNIX and Linux Support

calculate internal fragmentation in directory

hi sorry for grammar mistakes but i am writting these fro tablet and am not realy used to ot so well yet.... i am in the middle of doing work here and hope some1 ca answear my question asap please :-) How to calculate the amount of internal fragmentation using script. cd directory ...... (6 Replies)
Discussion started by: me.
6 Replies

3. Shell Programming and Scripting

Find all the files under a specific directory and zip them into a single file.

Hi, Is there a way to find all the files from a specific location and then zip them into a single file, even if they are in multiple directories? (3 Replies)
Discussion started by: rudoraj
3 Replies

4. Shell Programming and Scripting

how to find first files in a directory and combine them as a single file?

i have below list of files in a directory. /root/admin/files/file1.txt /root/admin/files/file2.txt /root/admin/files/file3.txt /root/admin/files/pattern.txt /root/admin/files/server.txt i need combine the above text files in the below sequence, file1.txt, pattern.txt,server.txt =>... (8 Replies)
Discussion started by: vel4ever
8 Replies

5. Shell Programming and Scripting

Change to directory and search some file in that directory in single command

I am trying to do the following task : export ENV=aaa export ENV_PATH=$(cd /apps | ls | grep $ENV) However, it's not working. What's the way to change to directory and search some file in that directory in single command Please help. (2 Replies)
Discussion started by: saurau
2 Replies

6. Shell Programming and Scripting

Paste 2 single column files to a single file

Hi, I have 2 csv/txt files with single columns. I am trying to merge them using paste, but its not working.. output3.csv: flowerbomb everlon-jewelry sofft steve-madden dolce-gabbana-watchoutput2.csv: http://www1.abc.com/cms/slp/2/Flowerbomb http://www1.abc.com/cms/slp/2/Everlon-Jewelry... (5 Replies)
Discussion started by: ajayakunuri
5 Replies

7. Shell Programming and Scripting

Bash to calculate average of all files in directory and output by part of filename

I am trying to use awk to calculate the average of all lines in $2 for every file in a directory. The below bash seems to do that, but I cannot figure out how to capture the string before the _ as the output file name and have it be tab-delimeted. Thank you :). Filenames in... (3 Replies)
Discussion started by: cmccabe
3 Replies

8. Shell Programming and Scripting

Copy files recursively to one single directory

I need to copy a complete directory structure into a new location. But I want to have all files copied into one directory and leave out the directory structure. So all files must be placed in one directory. (4 Replies)
Discussion started by: ReneVL
4 Replies

9. Shell Programming and Scripting

Move multiple files to different directory using a single command

I have multiple files test1, test2, test3 etc. I want to move to a different directory with ABC_ prefixed to every file and and current dat time as postfix using a single command. (I will be using this is sftp with ! (command for local server). I have tried the following but it gives error ... (5 Replies)
Discussion started by: Soham
5 Replies
g_sham(1)					 GROMACS suite, VERSION 4.5.4-dev-20110404-bc5695c					 g_sham(1)

NAME
g_sham - read/write xmgr and xvgr data sets VERSION 4.5.4-dev-20110404-bc5695c SYNOPSIS
g_sham -f graph.xvg -ge gibbs.xvg -ene esham.xvg -dist ener.xvg -histo edist.xvg -bin bindex.ndx -lp prob.xpm -ls gibbs.xpm -lsh enthalpy.xpm -lss entropy.xpm -map map.xpm -ls3 gibbs3.pdb -mdata mapdata.xvg -g shamlog.log -[no]h -[no]version -nice int -[no]w -xvg enum -[no]time -b real -e real -ttol real -n int -[no]d -bw real -[no]sham -tsham real -pmin real -dim vector -ngrid vector -xmin vector -xmax vector -pmax real -gmax real -emin real -emax real -nlevels int -mname string DESCRIPTION
g_sham makes multi-dimensional free-energy, enthalpy and entropy plots. g_sham reads one or more .xvg files and analyzes data sets. The basic purpose of g_sham is to plot Gibbs free energy landscapes (option -ls) by Bolzmann inverting multi-dimensional histograms (option -lp), but it can also make enthalpy (option -lsh) and entropy (option -lss) plots. The histograms can be made for any quantities the user supplies. A line in the input file may start with a time (see option -time) and any number of y-values may follow. Multiple sets can also be read when they are separated by & (option -n), in this case only one y-value is read from each line. All lines starting with and @ are skipped. Option -ge can be used to supply a file with free energies when the ensemble is not a Boltzmann ensemble, but needs to be biased by this free energy. One free energy value is required for each (multi-dimensional) data point in the -f input. Option -ene can be used to supply a file with energies. These energies are used as a weighting function in the single histogram analysis method by Kumar et al. When temperatures are supplied (as a second column in the file), an experimental weighting scheme is applied. In addition the vales are used for making enthalpy and entropy plots. With option -dim, dimensions can be gives for distances. When a distance is 2- or 3-dimensional, the circumference or surface sampled by two particles increases with increasing distance. Depending on what one would like to show, one can choose to correct the histogram and free-energy for this volume effect. The probability is normalized by r and r2 for dimensions of 2 and 3, respectively. A value of -1 is used to indicate an angle in degrees between two vectors: a sin(angle) normalization will be applied. Note that for angles between vec- tors the inner-product or cosine is the natural quantity to use, as it will produce bins of the same volume. FILES
-f graph.xvg Input xvgr/xmgr file -ge gibbs.xvg Input, Opt. xvgr/xmgr file -ene esham.xvg Input, Opt. xvgr/xmgr file -dist ener.xvg Output, Opt. xvgr/xmgr file -histo edist.xvg Output, Opt. xvgr/xmgr file -bin bindex.ndx Output, Opt. Index file -lp prob.xpm Output, Opt. X PixMap compatible matrix file -ls gibbs.xpm Output, Opt. X PixMap compatible matrix file -lsh enthalpy.xpm Output, Opt. X PixMap compatible matrix file -lss entropy.xpm Output, Opt. X PixMap compatible matrix file -map map.xpm Output, Opt. X PixMap compatible matrix file -ls3 gibbs3.pdb Output, Opt. Protein data bank file -mdata mapdata.xvg Output, Opt. xvgr/xmgr file -g shamlog.log Output, Opt. Log file OTHER OPTIONS
-[no]hno Print help info and quit -[no]versionno Print version info and quit -nice int 19 Set the nicelevel -[no]wno View output .xvg, .xpm, .eps and .pdb files -xvg enum xmgrace xvg plot formatting: xmgrace, xmgr or none -[no]timeyes Expect a time in the input -b real -1 First time to read from set -e real -1 Last time to read from set -ttol real 0 Tolerance on time in appropriate units (usually ps) -n int 1 Read sets separated by & -[no]dno Use the derivative -bw real 0.1 Binwidth for the distribution -[no]shamyes Turn off energy weighting even if energies are given -tsham real 298.15 Temperature for single histogram analysis -pmin real 0 Minimum probability. Anything lower than this will be set to zero -dim vector 1 1 1 Dimensions for distances, used for volume correction (max 3 values, dimensions 3 will get the same value as the last) -ngrid vector 32 32 32 Number of bins for energy landscapes (max 3 values, dimensions 3 will get the same value as the last) -xmin vector 0 0 0 Minimum for the axes in energy landscape (see above for 3 dimensions) -xmax vector 1 1 1 Maximum for the axes in energy landscape (see above for 3 dimensions) -pmax real 0 Maximum probability in output, default is calculate -gmax real 0 Maximum free energy in output, default is calculate -emin real 0 Minimum enthalpy in output, default is calculate -emax real 0 Maximum enthalpy in output, default is calculate -nlevels int 25 Number of levels for energy landscape -mname string Legend label for the custom landscape SEE ALSO
gromacs(7) More information about GROMACS is available at <http://www.gromacs.org/>. Mon 4 Apr 2011 g_sham(1)
All times are GMT -4. The time now is 12:02 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy