Removing tmp file too quickly?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Removing tmp file too quickly?
# 1  
Old 03-05-2010
Removing tmp file too quickly?

Still trying to get the basics down and I would like a different solution to what I'm currently doing and a better understanding of why it's happening. I've written a simple backup script that tars individual directories and then dumps them to a NFS drive. STDERR is being dumped into a process which echos a date stamp, and then the error to a tmp file (/tmp/backerrors.log). At the end of the script you will see I am mailing the tmp file if it has any data in it, after which I call for it to remove the file. Now here is where I get confused. Before the mail command is finished, the file is getting removed and cutting off bits of the message body.
If you run something like this, you will see what I mean (for me, the last line is being removed before the mail command is finished):

Code:
#!/bin/bash
#

#Want to enable error log emails?
EMAILLOGS="true"
EMAILTO="me@domain.com"
SUBJECT="Test Script"

#Where to dump errors if EMAILLOGS is false
ERRORLOG="/var/log/backuplog"

##Where are we sending stderr..
if [ $EMAILLOGS != "false" ]; then
        exec 2> >(while read line; do echo -e "$(date):${line}\n"; done >>/tmp/errorlog)
else
        exec 2>>$ERRORLOG
fi

echo -e "**error**\n**error**\n**error**" >&2
sleep 1
echo "piping to the error log for timestamp testing, one second" >&2
sleep 2
echo "two seconds" >&2


#Mail the errorlog if /tmp/backerrors.log exists and is > 0
if [ -s /tmp/errorlog ]; then
        /usr/bin/mail -s "$SUBJECT" "$EMAILTO" < /tmp/errorlog
else
        rm -f /tmp/errorlog
fi

rm -f /tmp/errorlog


Here is my script, at the end of which I've given it a 5 second sleep to complete the mail command. This can't be the correct way of doing, I assume..


Code:
#Want to enable error log emails?
EMAILLOGS="true"
EMAILTO="me@domain.com"
SUBJECT="Backup Error Log"

#Where to dump errors if EMAILLOGS is false
ERRORLOG="/var/log/backup/error.log"

#Set some main variables
set $(date)
MOUNTLOC="/mnt/bak"

##What is the root we are reading directories from?
DATA="/mail"

##Which directories do you want to exclude?
EXCLUDE="oldusers"

##What is the root we are writing to?
BACK=$MOUNTLOC

##Mount the NFS I'm backing to
mount $MOUNTLOC

##Variables to check if files exist
WEEKLYFILES=$(ls $BACK/weeklybak/*.tgz 2>/dev/null | wc -l)
INCREMFILES=$(ls $BACK/data_diff/*.tgz 2>/dev/null | wc -l)

##Where are we sending stderr..
if [ $EMAILLOGS == "true" ]; then
        ##This is process substitution: Read in sterr lines,
        ##echo back timstamp: error, then dump into a tmp file for mailing
        exec 2> >(while read line; do echo -e "$(date):${line}\n"; done >/tmp/backerrors.log)
else
        exec 2>>$ERRORLOG
fi

##Script start
TIMESTART=$(date +%Y\/%m\/%d-%H:%M)
printf "*--------------------$TIMESTART--------------------*\n"
if [ "$1" = "Sun" ] || [ "$WEEKLYFILES" = "0" ];  then
        #If it's Sunday, or a full backup doesn't exist, remove incremented files and logs
        if [ "$INCREMFILES" != "0" ];  then
                echo "Purging old incremental files..."
                rm -fv $BACK/data_diff/*.tgz
                rm -fv $BACK/snars/*.snar
        fi
                #Just in case something is left over
                echo "Purging old full backups..."
                rm -fv $BACK/weeklybak/*.tgz

        printf "Running full backup...\n"

        cd $DATA
        for DIRECTORY in $(ls -d *); do
                tar                                                                     \
                        --create --gzip                                                 \
                        --file=$BACK/weeklybak/${DIRECTORY}_$6-$2-$3.tgz                \
                        --listed-incremental=$BACK/snars/${DIRECTORY}_backup.snar       \
                        --exclude="$EXCLUDE"                                            \
                        $DIRECTORY
        done
        cd ~

else
printf "Running incremental backup of emails...\n"
        cd $DATA
        for DIRECTORY in $(ls $DATA); do
                tar                                                                     \
                        --create --gzip                                                 \
                        --file=$BACK/data_diff/${DIRECTORY}_$6-$2-$3.tgz                \
                        --listed-incremental=$BACK/snars/${DIRECTORY}_backup.snar       \
                        --exclude="$EXCLUDE"                                            \
                        $DIRECTORY
        done
        cd ~
fi

        #Grab my scripts and cronjobs
        printf "Grabbing cronjobs and scripts...\n"

        tar                                     \
                --create --gzip                 \
                --file=$BACK/rootscripts.tgz    \
                /root/scripts

        tar                                     \
                --create --gzip                 \
                --file=$BACK/crontabs.tgz       \
                /var/spool/cron/crontabs

#Mail the errorlog if /tmp/backerrors.log exists and is > 0
if [ -s /tmp/backerrors.log ]; then
        /usr/bin/mail -s "$SUBJECT" "$EMAILTO" < /tmp/backerrors.log
else
        rm -f /tmp/backerrors.log
fi

##Define time, now that we've finished
TIMEFINISHED=$(date +%Y\/%m\/%d-%H:%M)

##Log finished time
printf "Finished...\n"
printf "*--------------------$TIMEFINISHED--------------------*\n\n\n"

##Umount the NFS
umount $MOUNTLOC

#Give our mail command time to process, then remove the back up errors
sleep 5
rm -f /tmp/backerrors.log
exit

# 2  
Old 03-05-2010
You can run the command in the background and wait for it:

Code:
/usr/bin/mail -s "$SUBJECT" "$EMAILTO" < /tmp/backerrors.log &
wait

# 3  
Old 03-05-2010
In the UNIX mail program try doing a 'set sendwait' and see if it changes anything.

I think you might be able to set this 'sendwait' thing inside the mail.rc file. I have not tested this - just an idea for you to consider.

Here is what I found about the 'sendwait' thing on the web:

Code:
sendwait
Wait for the background mailer to finish before returning. The default is nosendwait.

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to quickly substitute pattern within certain range of a huge file?

I have big files (some are >300GB!) that need substitution for some patterns, for example, change Multiple Spaces into Tab. I used this oneliner:sed '1,18s/ \{1,\}/\t/g' infile_big.sam > outfile_big.sambut it seems very slow as the job is still running after 24 hours! In this example, only the... (8 Replies)
Discussion started by: yifangt
8 Replies

2. HP-UX

Core file under /tmp

Hi Guys Recently we have a issue where the core file under /tmp grows and there by /tmp becomes 100% full in our HP unix server. Any ideas or suggestions about how we can resolve this. (3 Replies)
Discussion started by: newtoaixos
3 Replies

3. Solaris

Permission of tmp file

The script written by csh, when it running it make some tmp file, the process need to read the tmp file to complete but tmp unable to open, don't have permission. Anyone know the way to automatic chmod the tmp file when run process. (2 Replies)
Discussion started by: Diabolist9
2 Replies

4. AIX

/tmp file system full

Hi, I would like to know if /tmp file system is full, wheather it will affect the peformance of application installed on AIX. if Memory and CPU are not heavily utilized. Regards, Manoj. (1 Reply)
Discussion started by: manoj.solaris
1 Replies

5. Shell Programming and Scripting

Matching a file with extension tmp

Hi all, I am writing a shell script for processing files in a directory. I need to read files in the directory and process them and write it to another file. For example, if the directory contains the following files, file1,file2,file3 I want to process these files and create... (3 Replies)
Discussion started by: ananddr
3 Replies

6. UNIX for Dummies Questions & Answers

need solution for this quickly. please quickly.

Write a nawk script that will produce the following report: ***FIRST QUARTERLY REPORT*** ***CAMPAIGN 2004 CONTRIBUTIONS*** ------------------------------------------------------------------------- NAME PHONE Jan | ... (5 Replies)
Discussion started by: p.palakj.shah
5 Replies

7. Shell Programming and Scripting

How to quickly show Nth line from the file

Hi all, How can I quickly show Nth line from the huge file(at least more than 15GB)? I used the following script but seems slower. See 2717298 th line. head -2717298 data0802.dat | tail -1 Thank you very much (4 Replies)
Discussion started by: mr_bold
4 Replies

8. Solaris

Increase size of /tmp swap File

Hi Guys I need to increase the size of my /tmp swap file. What is the easiest way to do this. Thanks Carson (2 Replies)
Discussion started by: cmackin
2 Replies

9. Solaris

after init all /tmp file has been removed

I'm new in Solaris server After the system support reboot the Solaris server, all the files in /tmp has been removed, is that normal under Solaris or under different init level will get different result? which init level will do that? (5 Replies)
Discussion started by: yesthomas
5 Replies

10. UNIX for Dummies Questions & Answers

monitoring /tmp and /var/tmp for suspicous activity

Hello, does anyone have a script that can check the contents of the /tmp directory and for example e-mail the directory content if anything other than session files are present? Maybe there are better ways to monitor suspicous /tmp and /var/tmp activity, if so I'm listening :) (1 Reply)
Discussion started by: jamesbond
1 Replies
Login or Register to Ask a Question