Sponsored Content
Top Forums Shell Programming and Scripting How to calculate the entropy of a single directory that contains many files Post 302358619 by draxmas on Saturday 3rd of October 2009 01:27:14 PM
Old 10-03-2009
Data How to calculate the entropy of a single directory that contains many files

Hello,
I'm new member of shell scripting and i face some difficulties. To begin, i try to write an algorithm that calculate from one directory containing nfdump files (288) the entropy of one day 24hours. Each of the file is 5 min interval
(nfdump -r nfcapd.200908250000 -s srcip) 1st
(nfdump -r nfcapd.200908250005 -s srcip) 2nd
(nfdump -r nfcapd.200908250010 -s srcip) 3rd
...
(nfdump -r nfcapd.200908252350 -s srcip) last

taking the flows (first 10) to calculate the entropy of the first 5 min interval i did it (i hope so in the right way) and i took the results of the 1st 5 min interval entropy. the code that i have written is:

Code:
#!/bin/sh
#x1 to x10 are variables which include values. Top 10 srcip Addr ordered by flows
x1=7576
x2=5691
x3=5130
x4=4239
x5=4136
x6=3387
x7=3070
x8=2780
x9=2125
x10=2011
#sum is the total flows
sum=551892
#calculate the values for each srcip Addr (10)
calx1=`echo "scale=6;$x1 / $sum" | bc`
calx2=`echo "scale=6;$x2 / $sum" | bc`
calx3=`echo "scale=6;$x3 / $sum" | bc`
calx4=`echo "scale=6;$x4 / $sum" | bc`
calx5=`echo "scale=6;$x5 / $sum" | bc`
calx6=`echo "scale=6;$x6 / $sum" | bc`
calx7=`echo "scale=6;$x7 / $sum" | bc`
calx8=`echo "scale=6;$x8 / $sum" | bc`
calx9=`echo "scale=6;$x9 / $sum" | bc`
calx10=`echo "scale=6;$x10 / $sum" | bc`
#calculate the logarithmic values from x1 to x10
l1=`echo "scale=6;l (${calx1})" | bc -l`
l2=`echo "scale=6;l (${calx2})" | bc -l`
l3=`echo "scale=6;l (${calx3})" | bc -l`
l4=`echo "scale=6;l (${calx4})" | bc -l`
l5=`echo "scale=6;l (${calx5})" | bc -l`
l6=`echo "scale=6;l (${calx6})" | bc -l`
l7=`echo "scale=6;l (${calx7})" | bc -l`
l8=`echo "scale=6;l (${calx8})" | bc -l`
l9=`echo "scale=6;l (${calx9})" | bc -l`
l10=`echo "scale=6;l (${calx10})" | bc -l`
#calculate the total entropy for 5 min interval
RESULT=`echo "scale=6;${calx1} * ${l1} + ${calx2} * ${l2} + ${calx3} * ${l3} + ${calx4} * ${l4} + ${calx5} * ${l5} + ${calx6} * ${l6} + ${calx7} * ${l7} + ${calx8} * ${l8} + ${calx9} * ${l9} + ${calx10} * ${l10}" | bc -l`
echo "Entropy for 5 min interval is ${RESULT}"

I used the shannon entropy formula for this.

My main problem now is how to make a matrix -array of all 288 files of 5 min interval and how is possible to combine the above algorithm with the matrix? Please if someone knows how to do this just to give me some help - advise
Thanks
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to calculate file's size in directory and subdirectory

Hi, I have written one script to calculate total space of all file in one directory, ignoring subdirectory, it works fine. Now, I've been trying to calculate all files which includes files in any subdirectories. I use recursive function to do this, but it can work only if there is only one... (4 Replies)
Discussion started by: KLL
4 Replies

2. Emergency UNIX and Linux Support

calculate internal fragmentation in directory

hi sorry for grammar mistakes but i am writting these fro tablet and am not realy used to ot so well yet.... i am in the middle of doing work here and hope some1 ca answear my question asap please :-) How to calculate the amount of internal fragmentation using script. cd directory ...... (6 Replies)
Discussion started by: me.
6 Replies

3. Shell Programming and Scripting

Find all the files under a specific directory and zip them into a single file.

Hi, Is there a way to find all the files from a specific location and then zip them into a single file, even if they are in multiple directories? (3 Replies)
Discussion started by: rudoraj
3 Replies

4. Shell Programming and Scripting

how to find first files in a directory and combine them as a single file?

i have below list of files in a directory. /root/admin/files/file1.txt /root/admin/files/file2.txt /root/admin/files/file3.txt /root/admin/files/pattern.txt /root/admin/files/server.txt i need combine the above text files in the below sequence, file1.txt, pattern.txt,server.txt =>... (8 Replies)
Discussion started by: vel4ever
8 Replies

5. Shell Programming and Scripting

Change to directory and search some file in that directory in single command

I am trying to do the following task : export ENV=aaa export ENV_PATH=$(cd /apps | ls | grep $ENV) However, it's not working. What's the way to change to directory and search some file in that directory in single command Please help. (2 Replies)
Discussion started by: saurau
2 Replies

6. Shell Programming and Scripting

Paste 2 single column files to a single file

Hi, I have 2 csv/txt files with single columns. I am trying to merge them using paste, but its not working.. output3.csv: flowerbomb everlon-jewelry sofft steve-madden dolce-gabbana-watchoutput2.csv: http://www1.abc.com/cms/slp/2/Flowerbomb http://www1.abc.com/cms/slp/2/Everlon-Jewelry... (5 Replies)
Discussion started by: ajayakunuri
5 Replies

7. Shell Programming and Scripting

Bash to calculate average of all files in directory and output by part of filename

I am trying to use awk to calculate the average of all lines in $2 for every file in a directory. The below bash seems to do that, but I cannot figure out how to capture the string before the _ as the output file name and have it be tab-delimeted. Thank you :). Filenames in... (3 Replies)
Discussion started by: cmccabe
3 Replies

8. Shell Programming and Scripting

Copy files recursively to one single directory

I need to copy a complete directory structure into a new location. But I want to have all files copied into one directory and leave out the directory structure. So all files must be placed in one directory. (4 Replies)
Discussion started by: ReneVL
4 Replies

9. Shell Programming and Scripting

Move multiple files to different directory using a single command

I have multiple files test1, test2, test3 etc. I want to move to a different directory with ABC_ prefixed to every file and and current dat time as postfix using a single command. (I will be using this is sftp with ! (command for local server). I have tried the following but it gives error ... (5 Replies)
Discussion started by: Soham
5 Replies
g_anaeig(1)					 GROMACS suite, VERSION 4.5.4-dev-20110404-bc5695c				       g_anaeig(1)

NAME
g_anaeig - analyzes the eigenvectors VERSION 4.5.4-dev-20110404-bc5695c SYNOPSIS
g_anaeig -v eigenvec.trr -v2 eigenvec2.trr -f traj.xtc -s topol.tpr -n index.ndx -eig eigenval.xvg -eig2 eigenval2.xvg -comp eigcomp.xvg -rmsf eigrmsf.xvg -proj proj.xvg -2d 2dproj.xvg -3d 3dproj.pdb -filt filtered.xtc -extr extreme.pdb -over overlap.xvg -inpr inprod.xpm -[no]h -[no]version -nice int -b time -e time -dt time -tu enum -[no]w -xvg enum -first int -last int -skip int -max real -nframes int -[no]split -[no]entropy -temp real -nevskip int DESCRIPTION
g_anaeig analyzes eigenvectors. The eigenvectors can be of a covariance matrix ( g_covar) or of a Normal Modes analysis ( g_nmeig). When a trajectory is projected on eigenvectors, all structures are fitted to the structure in the eigenvector file, if present, otherwise to the structure in the structure file. When no run input file is supplied, periodicity will not be taken into account. Most analyses are performed on eigenvectors -first to -last, but when -first is set to -1 you will be prompted for a selection. -comp: plot the vector components per atom of eigenvectors -first to -last. -rmsf: plot the RMS fluctuation per atom of eigenvectors -first to -last (requires -eig). -proj: calculate projections of a trajectory on eigenvectors -first to -last. The projections of a trajectory on the eigenvectors of its covariance matrix are called principal components (pc's). It is often useful to check the cosine content of the pc's, since the pc's of random diffusion are cosines with the number of periods equal to half the pc index. The cosine content of the pc's can be calculated with the program g_analyze. -2d: calculate a 2d projection of a trajectory on eigenvectors -first and -last. -3d: calculate a 3d projection of a trajectory on the first three selected eigenvectors. -filt: filter the trajectory to show only the motion along eigenvectors -first to -last. -extr: calculate the two extreme projections along a trajectory on the average structure and interpolate -nframes frames between them, or set your own extremes with -max. The eigenvector -first will be written unless -first and -last have been set explicitly, in which case all eigenvectors will be written to separate files. Chain identifiers will be added when writing a .pdb file with two or three structures (you can use rasmol -nmrpdb to view such a .pdb file). Overlap calculations between covariance analysis: Note: the analysis should use the same fitting structure -over: calculate the subspace overlap of the eigenvectors in file -v2 with eigenvectors -first to -last in file -v. -inpr: calculate a matrix of inner-products between eigenvectors in files -v and -v2. All eigenvectors of both files will be used unless -first and -last have been set explicitly. When -v, -eig, -v2 and -eig2 are given, a single number for the overlap between the covariance matrices is generated. The formulas are: difference = sqrt(tr((sqrt(M1) - sqrt(M2))2)) normalized overlap = 1 - difference/sqrt(tr(M1) + tr(M2)) shape overlap = 1 - sqrt(tr((sqrt(M1/tr(M1)) - sqrt(M2/tr(M2)))2)) where M1 and M2 are the two covariance matrices and tr is the trace of a matrix. The numbers are proportional to the overlap of the square root of the fluctuations. The normalized overlap is the most useful number, it is 1 for identical matrices and 0 when the sampled subspaces are orthogonal. When the -entropy flag is given an entropy estimate will be computed based on the Quasiharmonic approach and based on Schlitter's formula. FILES
-v eigenvec.trr Input Full precision trajectory: trr trj cpt -v2 eigenvec2.trr Input, Opt. Full precision trajectory: trr trj cpt -f traj.xtc Input, Opt. Trajectory: xtc trr trj gro g96 pdb cpt -s topol.tpr Input, Opt. Structure+mass(db): tpr tpb tpa gro g96 pdb -n index.ndx Input, Opt. Index file -eig eigenval.xvg Input, Opt. xvgr/xmgr file -eig2 eigenval2.xvg Input, Opt. xvgr/xmgr file -comp eigcomp.xvg Output, Opt. xvgr/xmgr file -rmsf eigrmsf.xvg Output, Opt. xvgr/xmgr file -proj proj.xvg Output, Opt. xvgr/xmgr file -2d 2dproj.xvg Output, Opt. xvgr/xmgr file -3d 3dproj.pdb Output, Opt. Structure file: gro g96 pdb etc. -filt filtered.xtc Output, Opt. Trajectory: xtc trr trj gro g96 pdb cpt -extr extreme.pdb Output, Opt. Trajectory: xtc trr trj gro g96 pdb cpt -over overlap.xvg Output, Opt. xvgr/xmgr file -inpr inprod.xpm Output, Opt. X PixMap compatible matrix file OTHER OPTIONS
-[no]hno Print help info and quit -[no]versionno Print version info and quit -nice int 19 Set the nicelevel -b time 0 First frame (ps) to read from trajectory -e time 0 Last frame (ps) to read from trajectory -dt time 0 Only use frame when t MOD dt = first time (ps) -tu enum ps Time unit: fs, ps, ns, us, ms or s -[no]wno View output .xvg, .xpm, .eps and .pdb files -xvg enum xmgrace xvg plot formatting: xmgrace, xmgr or none -first int 1 First eigenvector for analysis (-1 is select) -last int 8 Last eigenvector for analysis (-1 is till the last) -skip int 1 Only analyse every nr-th frame -max real 0 Maximum for projection of the eigenvector on the average structure, max=0 gives the extremes -nframes int 2 Number of frames for the extremes output -[no]splitno Split eigenvector projections where time is zero -[no]entropyno Compute entropy according to the Quasiharmonic formula or Schlitter's method. -temp real 298.15 Temperature for entropy calculations -nevskip int 6 Number of eigenvalues to skip when computing the entropy due to the quasi harmonic approximation. When you do a rotational and/or transla- tional fit prior to the covariance analysis, you get 3 or 6 eigenvalues that are very close to zero, and which should not be taken into account when computing the entropy. SEE ALSO
gromacs(7) More information about GROMACS is available at <http://www.gromacs.org/>. Mon 4 Apr 2011 g_anaeig(1)
All times are GMT -4. The time now is 10:21 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy