Quote:
Originally Posted by cfajohnson
As soon as a new file is created, you can gzip the previous one.
|
Not the most enlightening statement
but I understand what you mean.
I did some playing and found that the export program will create a file of 4k initially, then stop using it while it builds up a list of objects to export. It then returns a lock on the file and fills the file up to the 1gb size.
I modified my code to use a du and fuser test to check that the file was bigger than 4k and was not being used by any user. I find that if i use parallelism on the export the program will create
n number of files and then populate them, it may start with files 1,2,3, and 4, but 1,3, and 4 suddenly reach 1Gb so it creates 5,6,7 to continue the parallel tasks. file 2 is still not full (for whatever reason).
Files 1,3, and 4 are now unused but the gzip_func does not seem to want to gzip the files until file 2 is also unused - which often might not be until the end of the export. Can you please have a look at the code below and see if you can spot an obvious error? I want the code to really start gzipping when the 2 tests are passed, whether it can only gzip 1 file or up to
n threads. Any ideas?
Code:
gzip_func() {
started=0
threads=4
for filename in ${EXP_DIR}/*.dmp
do
# Check if file is bigger than 8K and is not being used
if [ `du -sk "${filename}"|awk '{print $1}'` -gt "8" ] && [ `fuser "${filename}" 2>/dev/null | wc -m` -eq "0" ]; then
# Loop through files until 4 are started (to match threads)
if [ ${started} -lt ${threads} ]; then
started=$(( $started + 1 ))
echo "gzip ${filename}"
$GZIPCMD "${filename}" &
list_of_pids="${list_of_pids} $!"
else
print "wait ${list_of_pids}"
wait ${list_of_pids}
list_of_pids=""
started=0
fi
else
echo "${filename} is still being written to, trying next file..."
fi
done
}
# Export creation - note done in background to allow gzip loop to run
expdp '"/ as sysdba"' directory=DPUMP_DIR_ADHOC dumpfile=ram6_full%U.dmp logfile=ram6_full.log filesize=1024m full=y parallel=4 &
while [ `find ${EXP_DIR} -name \*.dmp|wc -l` -gt "0" ]; do
gzip_func
print "out of loop wait ${list_of_pids}"
wait ${list_of_pids}
list_of_pids=""
sleep 5
done
