Directory sizes loop optimization


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Directory sizes loop optimization
# 1  
Old 05-16-2008
Directory sizes loop optimization

I have the following script:

#!/usr/bin/ksh

export MDIR=/datafiles

NAME=$1
SERVER=$2
DIRECTORY=$3
DATABASE=$4
ID=$5

export dirlist=`/usr/bin/ssh -q $ID@$SERVER find $DIRECTORY -type d -print`
for dir in $dirlist
do
SIZE=`</dev/null /usr/bin/ssh -q $ID@$SERVER du -ks $dir`
echo $NAME $DATABASE $SIZE $DIRECTORY>> $MDIR/bldtuout.txt
done


It is running forever, but does return the correct results. Is there a faster way? My goal is to have the directory sizes of all directories under a given path.
# 2  
Old 05-16-2008
Why use the for-loop when it can all be done by find.

Code:
/usr/bin/ssh -q $ID@$SERVER find $DIRECTORY -type d -exec du -sk {} \;

# 3  
Old 05-16-2008
So, it should look like this:

#!/usr/bin/ksh

export MDIR=/datafiles

NAME=$1
SERVER=$2
DIRECTORY=$3
DATABASE=$4
ID=$5

SIZE=`/usr/bin/ssh -q $ID@$SERVER find $DIRECTORY -type d -exec du -sk {} \;`
echo $NAME $DATABASE $SIZE $DIRECTORY>> $MDIR/bldtuout.txt
done


When I try to run it, I get find: incomplete statement What am I doing wrong?
# 4  
Old 05-16-2008
You probably need to double (or triple or quadruple) the backslash in order for the remote ssh to receive it correctly.
# 5  
Old 05-16-2008
Put it in double-quotes and use $(cmd) instead of `cmd` which makes it much easier to follow.

Code:
SIZE=$(/usr/bin/ssh -q $ID@$SERVER "find $DIRECTORY -type d -exec du -sk {} \;")

# 6  
Old 05-16-2008
Thank you! That did it.

Two more questions:
Now the output is all on one line, is there a quick way to parse it to multiple lines?

When the find creates an error, it is written to the screen, is there a way to get it writen to an output file?

Last edited by la_womn; 05-16-2008 at 06:43 PM..
# 7  
Old 05-17-2008
I figured it out. My script now looks like:

Quote:
#!/usr/bin/ksh

export MDIR=$PS_HOME/datafiles

NAME=$1
SERVER=$2
DIRECTORY=$3
DATABASE=$4
ID=$5

echo "*" $NAME $DATABASE $DIRECTORY >> $MDIR/bldtuout.txt
/usr/bin/ssh -q $ID@$SERVER "find $DIRECTORY -type d -exec du -ks {} \;" >> $MDIR/bldtuout.txt 2> $MDIR/bldterr.txt
Thank yiou to everyone for your help.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to list files names and sizes in a directory and output result to the file?

Hi , I'm trying to list the files and output is written to a file. But when I execute the command , the output file is being listed. How to exclude it ? /tmp file1.txt file2.txt ls -ltr |grep -v '-' | awk print {$9, $5} > output.txt cat output.txt file1.txt file2.txt output.txt (8 Replies)
Discussion started by: etldeveloper
8 Replies

2. Shell Programming and Scripting

Loop through files in a directory

Hello How do I loop through files in a specific directory ? This script is not working! #! /bin/bash FILES=/usr/desktop/input/* For f in $FILES; do awk '-v A="$a" -v B="$b" {$6=($1-64)/2 ;$7=((10^($6/10))/A)^(1/B) ; print}' OFS="\t" $f > /root/Desktop/output/$f.txt; done ... (7 Replies)
Discussion started by: ali.seifaddini
7 Replies

3. Shell Programming and Scripting

One file in to one directory using a for loop

So I have several files: 1.txt 2.txt 3.txt 4.txt and several directories: one two three four I want to put one file in one directory, thusly: 1.txt to /one 2.txt to /two 3.txt to /three 4.txt to /four but I want to use a for loop to do something like this: !#/usr/bin/bash... (5 Replies)
Discussion started by: os2mac
5 Replies

4. UNIX for Dummies Questions & Answers

Loop through directory and extract sub directory names

I am trying to loop through folders and extract the name of the lowest level subfolder I was running the script below, it returns /bb/bin/prd/newyork /bb/bin/prd/london /bb/bin/prd/tokyo I really want newyork london tokyo I couldn't find a standard variable for the lowest level... (1 Reply)
Discussion started by: personalt
1 Replies

5. UNIX for Advanced & Expert Users

df vs du for directory sizes

Is it better to use df or du to calculate directory sizes? I tried both and got different numbers with both. $ du -h /home 1.7G /home/bob1 1.7G /home $ df -h /home Filesystem Size Used Avail Use% Mounted on /dev/mapper/VG-lv_home 25G 1.9G 22G ... (2 Replies)
Discussion started by: cokedude
2 Replies

6. Shell Programming and Scripting

Script Optimization - large delimited file, for loop with many greps

Since there are approximately 75K gsfiles and hundreds of stfiles per gsfile, this script can take hours. How can I rewrite this script, so that it's much faster? I'm not as familiar with perl but I'm open to all suggestions. ls file.list>$split for gsfile in `cat $split`; do csplit... (17 Replies)
Discussion started by: verge
17 Replies

7. Shell Programming and Scripting

to compare total directory structure and get sizes of all f on two different servers

Hello every one, Iam newbie to this forum and shell programming &scripting. i needed to compare each and every folder of two separate servers. Actually I have copied some directory structure from one server to second server, to build on second server the files all should be copied... (3 Replies)
Discussion started by: mannam srinivas
3 Replies

8. Shell Programming and Scripting

Script for checking and reporting file sizes in a directory.

Hi, Need help for a Script for checking and reporting database file sizes in a directory. Request you to please give your valuable inputs. Thanks a lot in advance. Best Regards, Marconi (1 Reply)
Discussion started by: marconi
1 Replies

9. Shell Programming and Scripting

Loop through files in a directory

Hi, I want to write bash script that will keep on looking for files in a directory and if any file exists, it processes them. I want it to be a background process, which keeps looking for files in a directory. Is there any way to do that in bash script? I can loop through all the files like... (4 Replies)
Discussion started by: rladda
4 Replies

10. UNIX for Dummies Questions & Answers

Directory sizes

Can someone tell me how to read these damn sizes. i mean, i prefer to see sizes in MB but that is not the case when you do an ls -l on directories. i have a had time converting these to MB just for verification purposes, what would a directory size like this = 3499990308 represent in MB or... (3 Replies)
Discussion started by: TRUEST
3 Replies
Login or Register to Ask a Question