This file is saved in my root directory replacing the old/exisiting backups. To improve on this I would like to have a part of the script create the backup, then once the backup is created, check if the backup was completed successfully, and then check if the new backups are larger than the old backups. If both these conditions are true, it must/can replace the old backup files, else it needs to send a mail to say "WARNING - BACKUP FAILED" with possible causes fort he fail.
I usually gzip these files, as they are text-like, using a pipe so the data never hits disk uncompressed. Then you can keep several backups in the same space, namded for the db and date-time of the backup. It might even run faster, not so much disk i/o.
You can get the file length from ls -l or in blocks from ls -s, easily read and compared in shell. However, it might be simpler to send yourself a listing of the files via email.
You can back up both databases at the same time, too, independently in case one hangs.
I have the follwoing script running at the moment and now like mentioned in the first part of my post I want to compare compare the new backup to the backup of the previous day. If the new backup is smaller than that of the previous day it menas that something has gone wrong therefore it needs to either re-run the backup or send an e-mail to advise that there was an error running the backup.
If the new backup is larger than the old back up, it needs to replace the old backup. Here is my current script:
Code:
#!/bin/bash
# ===================================================================== #
# BACKUP SCRIPT WITH SYSTEM INFORMATION #
# SCRIPT CREATED 15/11/2012 #
# SCRIPT LAST MODIFIED 17/11/2012 #
# ===================================================================== #
# ===================================================================== #
# VARIABLES THAT CAN BE USED IN THE SCRIPT #
# ===================================================================== #
# DIRECTORY WHERE FILES ARE STORED
BACKUPDIR="/root/temp"
# CREATES A TITLE VARIABLE
TITLE=" System Status"
# SPECIFY THE CONTENT OF THE E-MAIL TO BE SENT
MAILCONTENT="files"
# SETS THE MAXIMUM SIZE OF THE E-MAIL 4000 = +/- 5MB
MAXATTSIZE="4000"
# E-MAIL ADDRESS WHERE THE E-MAIL WILL BE SENT
MAILADDR="recipient@mydomain.com"
# DEFINE THE E-MAIL BODY
SUBJECTFILE=$BACKUPDIR/sqldump.txt
# CREATE THE FILE TO ATTACH
ATTACHMENT=$BACKUPDIR/Status.html
# ===================================================================== #
# FUNCTIONS #
# STUBBING IS USED AS A TEMPORY PLACE HOLDER FOR FUTURE CODE #
# ===================================================================== #
MYSQL_DUMP ()
{
mysqldump > .sql --user=myusername --password=mypassword && mysqldump 2 > 2.sql --user=myusername --password=mypassword
}
# ---- END OF MYSQL DUMP ---- #
COMPRESSION ()
{
tar cvf .tar .sql && tar cvf 2.tar 2.sql
}
# ---- END OF COMPRESSION ---- #
REMOVE_FILES ()
{
rm .sql && rm 2.sql
}
# ---- END OF COMPRESSION ---- #
DRIVE_SPACE ()
{
echo "<h2>Filesystem Storage</h2>"
echo "<pre>"
df -h
echo "</pre>"
}
# ---- END OF DRIVE SPACE ---- #
BACKUP_SIZE ()
{
echo "<h2>Backup Size</h2>"
echo "<pre>"
du -h .tar && du .tar
du -h 2.tar && du 2.tar
echo "</pre>"
}
# ---- END OF BACKUP SIZE ---- #
FILE_COMPARE ()
{
echo "<pre>"
if [ .sql -nt .sql ] && [ 2.sql -nt 2.sql ]; then
echo "New file created"
else
echo "No change identified"
fi
echo "</pre>"
}
# ---- END OF FILE COMPARE ---- #
# ===================================================================== #
# EXECUTING FUNCTIONS #
# ===================================================================== #
# ---- CALL INDIVIDUAL FUNCTIONS IN CORRECT SEQUENCE ---- #
MYSQL_DUMP
COMPRESSION
REMOVE_FILES
# ===================================================================== #
# GATHERS AND DISPLAYS THE STATUS OF THE SYSTEM #
# ===================================================================== #
ATTACHMENT ()
{
cat <<- _Status_ > $ATTACHMENT
<html>
<head>
<title>$TITLE</title>
</head>
<body>
<h1>$TITLE</h1>
<p></p>
$(DRIVE_SPACE)
$(BACKUP_SIZE)
$(FILE_COMPARE)
<body>
</html>
_Status_
}
# ---- CALL THE FUNCTION TO CREATE AND ATTACH THE FILE ---- #
ATTACHMENT
# ===================================================================== #
# GATHERS INFORMATION AND SEND MESSAGE ATTACHMENT #
# ===================================================================== #
if [ "$MAILCONTENT" = "files" ]
then
uuencode $ATTACHMENT $ATTACHMENT | mailx -s " System Status $DATE" -- $MAILADDR
fi
exit 0
You can capture size, field 5 of "ls -l", and compare it with "if (( $sz1 > $sz2 ))" to decide to move = rename = mv the file or send an email.
I'd use gzip on each dump file during the dump, and use zip, not tar, to archive them, as zip can retrieve random files to stdout efficiently.
Don't be afraid of the pipe. Shell programming with no pipes tends to have lots of storage space usage, high i/o resource cost, higher run time and latency, less use of multiprocessing, tends to leave junk intermediate temp files around, does not allow parallel running or use by other users because of file names and permissions.
Thanks for the advice DGPickett. Where would one use the pipe? Sorry I know its probably a very basic question, but I'm still new to shell scripting, infact this is actually my first script.
I'm going to have to see if I understand what you mean with the
Quote:
You can capture size, field 5 of "ls -l", and compare it with "if (( $sz1 > $sz2 ))" to decide to move = rename = mv the file or send an email.
but I am sure I will figure it out... I hope, lol.
Well, "ls -l flat-file-name" prints out the permissions, link count, owner user name, group name, size, mod date-time, and file name. You can pick up fields with "read" but be careful in bash off a pipe, where it is in a subshell.
Code:
export sz1=$(
ls -l file1 |(
read perm lks unm gnm sz x
echo $sz
) )
In ksh the last process on a pipeline is the invoking shell, so you can say:
Code:
ls -l file1 | read perm lks unm gnm sz1 x
I am not sure if read creates an exported variable, if you need it exported, but I think so. Not to be lazy, don't read, try it and know, yes:
Code:
$ echo zzz | read qq ; ( echo $qq )
zzz
$
Between "try it and see" and Google, you just cannot say "I do not know" so much any more!
The x variable absorbs the rest of the line, which is nice as file date-time has a varible format.
By calling awk, you can get field 5 more directly, but why buy a knife when you have a drawer full? You could pass the ls -l output to a shell function that does "echo $5". That's direct and still in the same shell exec(). An exec() is about 10 times more expensive than a fork().
Last edited by DGPickett; 11-21-2012 at 04:58 PM..
Thanks... Sorry I haven't had time to try these. But will try work on it as soon as I can and let you know how it all works out. Really appreciate all the help.
Hello,
I have two text tab delimited files
File 1 has 30 columns. I am pasting only first 9
Chr Position Ref Alt Score Gene HGVS_C HGVS_P Coding_Consequence dbSNP
1 17312743 C T 1 Gene1 - ... (2 Replies)
I need a way to to check if a value in a file that has this XML format is less than or equal to current system date/time. if it is I need to override it with a future date/time:
Here is the data sample:
<?xml version="1.0"... (1 Reply)
Hi All,
I have 2 files in the following format :
File 1
S00999999|BHANU|TEST|007 JOHN DOE APT 999||VENGA HIGHWAY|MA|09566|SCO DUAL|20140201|20140331|20140401|20140630|20140327|
S00888888|BU|TES|009 JOHN DOE APT 909||SENGA HIGHWAY|MA|08566|SCO... (1 Reply)
Hi All,
I have written a shell script that creates a backup of my MySQL database. The script performs the following functions:
Creates a Backup of the MySQL database
Compresses the Backup
Copies the Backup to a Remote Server
Send an E-Mail displaying the size of the Backup
Removes any... (6 Replies)
hi Friends
need to compare columns in one file where the data looks like below
laptop,IBM
phone,samsung
car,rental
user1,laptop
user2,laptop
user3,phone
want to get output as
laptop,IBM
phone,samsung
car,rental
user1,IBM
user2,IBM
user3,samsung
need to seach $2 in array of $1 and... (4 Replies)
ok, so here is the issue, I have 2 arrays.
I need to be able to create a loop that will find ${ARRAY1 in the text doc, and replace it with ${ARRAY2 then write the results. I already have that working. The problem is, I need it to do that same result across however many items are in the 2... (2 Replies)
Hi All,
Need help on this
I have 2 files
one file file1 which has several entries as :
define service{
hostgroup_name !host1,!host5,!host6,.*
service_description check_nrpe
}
define service{
hostgroup_name !host2,!host4,!host6,.*
service_description check_opt
}
another... (2 Replies)
Hiiii Friends
I have 2 files with huge data. I want to compare this 2 files & if they hav same set of vales in specific rows & columns i need to get that value from one file & replace it in other.
For example: I have few set data of both files here:
a.dat:
PDE-W 2009 12 16 5 29 11.11 ... (10 Replies)
Hi,
I have a file which contains names with counts for eg:
0622 0031 JOHN MAX 20080622003104. STAT 1.
0622 0031 BILL MAX 20080622003104. STAT 7.
and I have an exception file containing.
BILL
Can anyone help to write a script... (7 Replies)
11-16-2012
Compare and Replace