Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Compare File & Copy Replace if Successful

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare File & Copy Replace if Successful
# 1  
Old 01-10-2013
Linux Compare File & Copy Replace if Successful
Hi All,

I have written a shell script that creates a backup of my MySQL database. The script performs the following functions:
  1. Creates a Backup of the MySQL database
  2. Compresses the Backup
  3. Copies the Backup to a Remote Server
  4. Send an E-Mail displaying the size of the Backup
  5. Removes any left over files on the source server not needed.
What the script doesn't do, but what I need it to do:
  1. Check the newly created Backup against the existing backup on the remote server
  2. If the new backup is smaller than the old backup, send a WARNING Notification via e-mail / sms.
  3. If the new backup is larger or equal in size to the old backup, replace the old backup on the remote server with the new backup and then send the successful notification stated in point 4.
Thanks, any help here is really appreciated

Operating systems being used:
Source Server: Ubuntu 12.04.1 LTS
Destination Server: Fedora release 13 (Goddard)
Last edited by SalientAnimal; 01-10-2013 at 10:17 AM.. Reason: Adding OS versions
# 2  
Old 01-10-2013
Do you just want to check the size? Run an ls on the remote server, and save the result in a variable, then use that for your logics. Try like (assuming ssh is the remote connection tool)
Code:
SIZE=$(ssh user@node.remote.domain  "ls -l /path/to/backup"|awk '{print $5}')

# 3  
Old 01-14-2013
I want to check the size, as well as if the backup completed successfully before replacing the old files on the remote server with the newly created files.

This is what my scripts looks like:

Code:
#!/bin/bash
# ===================================================================== #
#  OTRS BACKUP SCRIPT WITH SYSTEM INFORMATION  #
#    SCRIPT CREATED 15/11/2012   #
#               SCRIPT LAST MODIFIED 10/01/2013    #
# ===================================================================== #

# ===================================================================== #
#    VARIABLES THAT CAN BE USED IN THE SCRIPT   #
# ===================================================================== #
# DIRECTORY WHERE FILES ARE STORED
BACKUPDIR="/app/temp"
# CREATES A TITLE VARIABLE
TITLE="OTRS System Status"
# SPECIFY THE CONTENT OF THE E-MAIL TO BE SENT
MAILCONTENT="files"
# SETS THE MAXIMUM SIZE OF THE E-MAIL 4000 = +/- 5MB
MAXATTSIZE="4000"
# E-MAIL ADDRESS WHERE THE E-MAIL WILL BE SENT
# MAILADDR="myemail@mydomain.com myemail2@mydomain.com"
MAILADDR="myemail@mydomain.com myemail3@mydomain.com"

# DEFINE THE E-MAIL BODY
SUBJECTFILE=$BACKUPDIR/sqldump.txt
# CREATE THE FILE TO ATTACH
ATTACHMENT=status.html

# ===================================================================== #
#     FUNCTIONS    #
#      STUBBING IS USED AS A TEMPORY PLACE HOLDER FOR FUTURE CODE       #
# ===================================================================== #

MYSQL_DUMP ()
{
 mysqldump systembackup > systembackup.sql --user=root --password=mypassword && mysqldump systembackup2 > systembackup2.sql --user=root --password=mypassword
}
# ---- END OF MYSQL DUMP ---- #

COMPRESSION ()
{
        tar -jcvf systembackup.tar.bz2 systembackup.sql && tar -jcvf systembackup2.tar.bz2 systembackup2.sql
}
# ---- END OF COMPRESSION ---- #

REMOVE_FILES ()
{
        rm systembackup.sql && rm systembackup2.sql && rm systembackup.tar.bz2 && rm systembackup2.tar.bz2
}
# ---- END OF COMPRESSION ---- #

DRIVE_SPACE ()
{
    echo "<h2>Filesystem Storage</h2>"
    echo "<pre>"
    df -h
    echo "</pre>"
}
# ---- END OF DRIVE SPACE ---- #

BACKUP_SIZE ()
{
    echo "<h2>Backup Size</h2>"
    echo "<pre>"
    du -h systembackup.tar.bz2 && du systembackup.tar.bz2 
    du -h systembackup2.tar.bz2 && du systembackup2.tar.bz2
    echo "</pre>"
}
# ---- END OF BACKUP SIZE ---- #

FILE_COMPARE ()
{
  echo "<pre>"
 if [ systembackup.sql -nt systembackup.sql ] && [ systembackup2.sql -nt systembackup2.sql ]; then
  echo "New file created"
 else 
  echo "No change identified"
 fi
  echo "</pre>"
}
# ---- END OF FILE COMPARE ---- #
MOVE_OFF_SERVER ()
{
  scp systembackup.tar.bz2 root@255.255.255.10:/systembackupbackup
  echo $?
  scp systembackup2.tar.bz2 root@255.255.255.10:/systembackupbackup
}
# ---- END OF FILE MOVE ---- #
 
# ===================================================================== #
#            GATHERS AND DISPLAYS THE STATUS OF THE SYSTEM              #
# ===================================================================== #
ATTACHMENT ()
{
cat <<- _status_ > $ATTACHMENT

                <html>
                <head>
                <title>$TITLE</title>
                </head>
                <body>
                <h1>$TITLE</h1>
                <p></p>
                $(DRIVE_SPACE)
                $(BACKUP_SIZE)
                $(FILE_COMPARE)
                <body>
                </html>
_status_
}
# ---- CALL THE FUNCTION TO CREATE AND ATTACH THE FILE ---- #
 
 
# ===================================================================== #
#        EXECUTING FUNCTIONS     #
# ===================================================================== #
# ---- CALL INDIVIDUAL FUNCTIONS IN CORRECT SEQUENCE ---- #
cd $BACKUPDIR
MYSQL_DUMP
COMPRESSION
MOVE_OFF_SERVER
ATTACHMENT
REMOVE_FILES

# ===================================================================== #
#      GATHERS INFORMATION AND SEND MESSAGE ATTACHMENT  #
# ===================================================================== #

 
if [ "$MAILCONTENT" = "files" ]
 then
 uuencode $ATTACHMENT $ATTACHMENT | mailx -s "System Status $DATE" -- $MAILADDR
fi

exit 0

Last edited by SalientAnimal; 01-14-2013 at 02:42 AM.. Reason: Adding Code
# 4  
Old 01-14-2013
Quote:
Originally Posted by SalientAnimal
I want to check the size, as well as if the backup completed successfully before replacing the old files on the remote server with the newly created files. . . .
Then, collect the backups' exit status' into variables, use my proposal to get the remote old backup size (or, keep it in a file locally for later easy reference) into another var, and build some logics around your function calls based on those vars.
# 5  
Old 01-14-2013
I'm slowly progressing on this. I have managed to get my script to now return the size of the original files. I now need to get the script to compare which is large and then to send the correct notification.

Either: Backup Successful OR Backup Failed.

This is the code I have added:
Code:
CHECK_EXISTING ()
{
echo "<h2>Old Backup Size</h2>"
echo "<pre>"
ssh root@host "ls -l -h /directory/of/backup.tar.bz2"|awk '{print $5}'
ssh root@host "ls -l -h /directory/of/backup2.tar.bz2"|awk '{print $5}'
echo "</pre>"
}

# 6  
Old 01-14-2013
When programming, you should always strive for the most efficient (ressources, time, ...) code if the correct functionality allows. In the case above, logging in twice to the remote system is quite wasteful, so try to use sth. like ls -l /.../backup?... to get the two (or more) sizes in one go. Assigning them to variables can be done later on the local host.
BTW - if you want to base your decision to copy or not on the file size, the -h option to ls is the worst you can do - the differences will be lumped together to Megs or Gigs but you need to be aware of every single byte change!
# 7  
Old 01-14-2013
Cool, thanks for the feedback. I'm still busy working on the script and being new to scripting I'm learning as I go along.

Will remove the -h command from my script.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare file dates before copy

Sometimes when I boot, my system goes into emergency mode. I then use Clonezilla to restore an image. Usually the image is older than the current date. This is part of a backup script that runs as a startup program. cd /home/andy/bin/ zip -u -q Ubuntu_Scripts.zip *.sh *.rb *.c *.py... (22 Replies)
Discussion started by: drew77
22 Replies

2. Shell Programming and Scripting

Compare file size and then copy/overwrite

// Redhat I have this code working, but need to add one more qualification so that I don't overwrite the files. #!/bin/sh cd /P2/log/cerner_prod/millennium/archive/ for f in * do || continue #If this isn't a regular file, skip it. && continue #If a backup already... (2 Replies)
Discussion started by: Daniel Gate
2 Replies

3. UNIX for Dummies Questions & Answers

Confirm the execution of copy command is successful or not

All, I have to copy huge file from one location to another using python . I want to make sure the execution of Copy command is successful and all the files are copied properly (there has not been any abrupt interruption to the copy process or error like no space available is encountered during... (5 Replies)
Discussion started by: IshuGupta
5 Replies

4. Shell Programming and Scripting

Compare & Copy Directories : Bash Script Help

Beginner/Intermediate shell; comfortable in the command line. I have been looking for a solution to a backup problem. I need to compare Directory 1 to Directory 2 and copy all modified or new files/directories from Directory 1 to Directory 3. I need the directory and file structure to be... (4 Replies)
Discussion started by: Rod
4 Replies

5. Red Hat

copy & replace text

how can i copy a certain word from a text file then use this word to replace in another text file?? i tried to use something like: awk '{print "Hit the",$1,"with your",$2}' /aaa/qqqq.txt > uu.txt but i can't add an argument to point to the second file which i will replace in. please... (8 Replies)
Discussion started by: mos33
8 Replies

6. Shell Programming and Scripting

replace & with &amp; xml file

Hello All I have a xml file with many sets of records like this <mytag>mydata</mytag> <tag2>data&</tag2> also same file can be like this <mytag>mydata</mytag> <tag2>data&</tag2> <tag3>data2&amp;data3</tag3> Now i can grep & and replace with &amp; for whole file but it will replace all... (4 Replies)
Discussion started by: lokaish23
4 Replies

7. UNIX for Dummies Questions & Answers

How to compare 2 files & get specific value & replace it in other file.

Hiiii Friends I have 2 files with huge data. I want to compare this 2 files & if they hav same set of vales in specific rows & columns i need to get that value from one file & replace it in other. For example: I have few set data of both files here: a.dat: PDE-W 2009 12 16 5 29 11.11 ... (10 Replies)
Discussion started by: reva
10 Replies

8. Shell Programming and Scripting

How do I search first&second string & copy all content between them to other file?

Hi All, How do I search first string & second string and copy all content between them from one file to another file? Please help me.. Thanks In Advance. Regards, Pankaj (12 Replies)
Discussion started by: pankajp
12 Replies

9. Shell Programming and Scripting

XML Copy & replace problem

I probably could have done this at one time, but, the years and no need has left my scripting skills lacking and I'm unable to work this problem out. https://www.unix.com/images/smilies/frown.gif :( Using Linux, have a great many xml files in which there may be multiple occurrence of a line of... (13 Replies)
Discussion started by: xenixuser
13 Replies

10. Shell Programming and Scripting

Compare & replace contents within a file

I have 2 files file1 1 TMQUEUE QUE1 STMW633A 100 DMADM DOMGRPSTMW633A STMW633A 100 GWADM GWTGRPSTMW633A STMW633A 100 GWADM GWTGRPSTMW633AA STMW633A 100 GWADM GWTGRPSTMW638A STMW638A 100 TMSYSEVT EVTGRPSTMW633A STMW633A 100 TMSYSEVT ... (2 Replies)
Discussion started by: kaustubh137
2 Replies
Login or Register to Ask a Question