Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Backup Rationalisation Script - Help Required Post 302551418 by jeked on Tuesday 30th of August 2011 10:18:36 AM
Old 08-30-2011
Backup Rationalisation Script - Help Required

Ok so once again im back with what is probably a beginner question although somewhat more complicated (for me) than the last.

Background:
A client has a daily backup which is carried out via rsync.
Due to this, when they move a file around that file is then coppied a second time.
On top of this, when a file is deleted off of the client server it isnt deleted off of the backup server. (Rsync options i believe)

Im am working on a script that does the following:

Entities:
1. Client File Server (A)
2. Offsite Backup Server (B)

Current:
1. Backup sync runs daily.
2. Changed / new files on A in comparison to B are replaced / copied onto B.
3. Deleted files on A are ignored in this process

Proposed:
Additional script created to do the following:
1. Created a file of all files currently on A.
2. Compare with a file of all files currently on B.
3. If:
a) A has a file B doesn't < Warning report on email
b) B has a file A doesn't < Mark for deletion
c) A has a modified file to B < Warning report on email
4. Do the following:
-i < Identify / Create a report with what would have been deleted
5. If no restriction is placed then delete.

Thoughts:
1. Rsync with option delete files on B that are no longer on A

2. Personal script.
Methods within script:
Main
< Call method

Creation
< Create A file
< Create B file

Comparison
< Check for discrepancies between files
< Create file with discrepancies

Output
< Send error messages to unison email account
< Display files to be deleted on screen then delete

Outputed Email:
1. No. files processed
2. List of deleted files + size for deletion
3. Total deleted size
4. Errors/Warnings

< Email sent regardless of parameter selected or not.

So far this is what i have:
Code:
#!/bin/sh

#Initalise Variables
clientDIR=0
backupDIR=0
clientFile=0
backupFile=0
dateStamp=0
saveLocation=0

#1. Create client file:
clientDIR=/home/[USR]/
dateStamp=`date +%d-%m-%Y`
saveLocation=/home/root/scripts
clientFile=$saveLocation/clientFile@$dateStamp.txt

cd $clientDIR
for i in "$clientDIR"
do
       ls -lstR > $clientFile
done

#2. Create backup file:
#ssh to offsite backup location using key rather than pw
#backupFile=insert ssh login without pw using key to ip + /backupFile@$dateStamp.txt

#TEMP TEST FILE
backupFile=$saveLocation/backupFile@$dateStamp.txt

cd $clientDIR
for i in "$clientDIR"
do
        ls -lstR > $backupFile
done

#3: To compare files:
#cmp <- Compare two files
#Not working

#comm <- Compare two sorted files line by line
#Not working

#diff <- Display the differences between two files
diff -y $clientFile $backupFile > $saveLocation/diffOutput@$dateStamp.txt
#diff command doesnt recognise files due to order....

#4: If:
#       a) A has a file B doesn't < Warning report
#       b) B has a file A doesn't < Mark for deletion
#       c) A has a modified file to B < Warning report

#5. Do the following:
#       -i < Identify / Create a report with what would have been deleted

#6. If no restriction is placed then delete.

echo "Done <- Debugging"

So what i need from you guys will be a fair bit of help (i think) but if we can start with how i can compare the files generated. ATM with the diff command due to files being in a different order with the backup server having more files than the client, naturally the file lines wont match up.

Is there an easier way to do this? Any help is appreciated. Feel free to tell me im going the wrong way about this and to try something else.

Many regards and sorry to any that read this super long post,
Regards,
John Crombie
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script help required!

Hi there, i am trying to create a script that checks for the existence of users on the system, if they exist then their details should print on the screen with a message that the id is in use. I am having a bit of trouble with it. Any ideas? Cheers Kev! (8 Replies)
Discussion started by: kev112
8 Replies

2. Shell Programming and Scripting

script required

The line is like this +abc+def+mgh+ddsdsd+sa i.e. words seperated by +. There is a plus in the beginning. i want to conver this line to abc, def, mgh, ddsdsd, sa please provide the logic in the form of a shell script Thanks in advance (13 Replies)
Discussion started by: skyineyes
13 Replies

3. Shell Programming and Scripting

Getting required fields from a test file in required fromat in unix

My data is something like shown below. date1 date2 aaa bbbb ccccc date3 date4 dddd eeeeeee ffffffffff ggggg hh I want the output like this date1date2 aaa eeeeee I serached in the forum but didn't find the exact matching solution. Please help. (7 Replies)
Discussion started by: rdhanek
7 Replies

4. Shell Programming and Scripting

Help required to get a script

Hi Experts, I am very beginner in Bash Shell Scripting. Can anyone please guide me to create a script which should show the most busy file systems in sort basis as there are a lot of file systems on the server. I was told this task to be done by my IT lead and I must have to do this in... (3 Replies)
Discussion started by: naw_deepak
3 Replies

5. Shell Programming and Scripting

rsync backup mode(--backup) Are there any options to remove backup folders on successful deployment?

Hi Everyone, we are running rsync with --backup mode, Are there any rsync options to remove backup folders on successful deployment? Thanks in adv. (0 Replies)
Discussion started by: MVEERA
0 Replies

6. Shell Programming and Scripting

Help with Backup Shell Script for Network Device Configuration backup

HI all, im new to shell scripting. need your guidence for my script. i wrote one script and is attached here Im explaining the requirement of script. AIM: Shell script to run automatically as per scheduled and backup few network devices configurations. Script will contain a set of commands... (4 Replies)
Discussion started by: saichand1985
4 Replies

7. UNIX for Dummies Questions & Answers

Perl Script:how to find how many parameters are required to run the script

How to find how many parameters are required to run a Perl script? (1 Reply)
Discussion started by: Lakshman_Gupta
1 Replies

8. AIX

GTAR - new ways for faster backup - help required

We are taking backup of our application data(cobol file system, AIX/unix) before and after EOD job runs. The data size is approximately 260 GB in biggest branch. To reduce the backup time, 5 parallel execution is scheduled through control-m which backups up the files in 5 different *.gz. The job... (2 Replies)
Discussion started by: Bharath_79
2 Replies

9. AIX

GTAR - new ways to faster backup - help required

We are taking backup of our application data(cobol file system, AIX/unix) before and after EOD job runs. The data size is approximately 260 GB in biggest branch. To reduce the backup time, 5 parallel execution is scheduled through control-m which backups up the files in 5 different *.gz. The job... (8 Replies)
Discussion started by: Bharath_79
8 Replies

10. Shell Programming and Scripting

Help required to get a backup script working

Hi all I have a unix based firewall, which creates a daily backup file on the device. I need a script to scp this file over to a remote server. I can get this working daily using a basic script and a cron job. However, I only want it to send the latest config back up file and currently... (4 Replies)
Discussion started by: jimmyzoom
4 Replies
All times are GMT -4. The time now is 05:37 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy