Ok so once again im back with what is probably a beginner question although somewhat more complicated (for me) than the last.
Background:
A client has a daily backup which is carried out via rsync.
Due to this, when they move a file around that file is then coppied a second time.
On top of this, when a file is deleted off of the client server it isnt deleted off of the backup server. (Rsync options i believe)
Im am working on a script that does the following:
Entities:
1. Client File Server (A)
2. Offsite Backup Server (B)
Current:
1. Backup sync runs daily.
2. Changed / new files on A in comparison to B are replaced / copied onto B.
3. Deleted files on A are ignored in this process
Proposed:
Additional script created to do the following:
1. Created a file of all files currently on A.
2. Compare with a file of all files currently on B.
3. If:
a) A has a file B doesn't < Warning report on email
b) B has a file A doesn't < Mark for deletion
c) A has a modified file to B < Warning report on email
4. Do the following:
-i < Identify / Create a report with what would have been deleted
5. If no restriction is placed then delete.
Thoughts:
1. Rsync with option delete files on B that are no longer on A
2. Personal script.
Methods within script:
Main
< Call method
Creation
< Create A file
< Create B file
Comparison
< Check for discrepancies between files
< Create file with discrepancies
Output
< Send error messages to unison email account
< Display files to be deleted on screen then delete
Outputed Email:
1. No. files processed
2. List of deleted files + size for deletion
3. Total deleted size
4. Errors/Warnings
< Email sent regardless of parameter selected or not.
So far this is what i have:
So what i need from you guys will be a fair bit of help (i think) but if we can start with how i can compare the files generated. ATM with the diff command due to files being in a different order with the backup server having more files than the client, naturally the file lines wont match up.
Is there an easier way to do this? Any help is appreciated. Feel free to tell me im going the wrong way about this and to try something else.
Many regards and sorry to any that read this super long post,
Regards,
John Crombie
I think all the tasks you want to achieve can be done using rsync options only.
No need to write down a separate script for that.
rsync has option for dry run, i.e. instead of syncing data just compare files and provide list of files (building file list... o/p) which are going to be sync in process.
As you mentioned, rsync has option for deleting files which are on A and not in B.
The only thing is if you want to check which files are going to be deleted from B before getting deleted. For that can you check running dry run with --delete option and check what o/p comes?
One better approach is to setup backup policy as
Delete/Archive everything on B on Friday night, depending upon disk space.
Take full backup on Sat/sun.
Run incremental backup everyday till next Sat/Sun.
I know, you would have more queries, most welcome.
Ok so i took into account what you siad about rsync and managed something but now i want to finish what i started...after redefining what i need to do to myself i came to the following:
This is what i need processFolder() to do :P
And this is what i currently have....
Any furthur input on this? ASAP but i know you all have lives
Hi all
I have a unix based firewall, which creates a daily backup file on the device.
I need a script to scp this file over to a remote server.
I can get this working daily using a basic script and a cron job.
However, I only want it to send the latest config back up file and currently... (4 Replies)
We are taking backup of our application data(cobol file system, AIX/unix) before and after EOD job runs. The data size is approximately 260 GB in biggest branch. To reduce the backup time, 5 parallel execution is scheduled through control-m which backups up the files in 5 different *.gz. The job... (8 Replies)
We are taking backup of our application data(cobol file system, AIX/unix) before and after EOD job runs. The data size is approximately 260 GB in biggest branch. To reduce the backup time, 5 parallel execution is scheduled through control-m which backups up the files in 5 different *.gz. The job... (2 Replies)
HI all, im new to shell scripting. need your guidence for my script. i wrote one script and is attached here
Im explaining the requirement of script.
AIM: Shell script to run automatically as per scheduled and backup few network devices configurations. Script will contain a set of commands... (4 Replies)
Hi Everyone,
we are running rsync with --backup mode, Are there any rsync options to remove backup folders on successful deployment?
Thanks in adv. (0 Replies)
Hi Experts,
I am very beginner in Bash Shell Scripting. Can anyone please guide me to create a script which should show the most busy file systems in sort basis as there are a lot of file systems on the server.
I was told this task to be done by my IT lead and I must have to do this in... (3 Replies)
My data is something like shown below.
date1 date2 aaa bbbb ccccc
date3 date4 dddd eeeeeee ffffffffff ggggg hh
I want the output like this
date1date2 aaa eeeeee
I serached in the forum but didn't find the exact matching solution. Please help. (7 Replies)
The line is like this
+abc+def+mgh+ddsdsd+sa
i.e. words seperated by +. There is a plus in the beginning.
i want to conver this line to
abc, def, mgh, ddsdsd, sa
please provide the logic in the form of a shell script
Thanks in advance (13 Replies)
Hi there,
i am trying to create a script that checks for the existence of users on the system, if they exist then their details should print on the screen with a message that the id is in use. I am having a bit of trouble with it.
Any ideas?
Cheers
Kev! (8 Replies)