Do incremental backup without tar files up


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Do incremental backup without tar files up
# 1  
Old 07-02-2010
Tools Do incremental backup without tar files up

I'm trying to do a incremental backup for a big NFS. Since space is not an issue, I don't want to compress them or end up with a big tarball for full backup( and a series of small tarballs for incremental backup). Basically I want the TAR backup/restore functionality but not TAR files up.
Requirements:
1. files/directories are transferred directly so that I can browse through the backup easily.
2. Have a snapshot file that can be used to do incremental backup. (like the one Tar generates, which used to keep track of changes)
Any suggestions? Thanks!

Note: Rsync is not an option, because target filesystem is not unix-like. It only supports GET/LIST/PUT.
# 2  
Old 07-05-2010
Just me... I'd use Linux as the holder of the backup data. rsync, as mentioned is the best solution, but in lieu of that, sounds like you'll have to write a screen scraping (?) ftp like manager to loop through and grab everything. Could mean multiple passes. Might help to know the target system? You'd be surprised at what people have experience with.

I'd use Linux, because assuming you can roll through and pull the whole tree across ala ftp style (maybe you're system supports scripting on xfer??).... then you can use the GNU cp -l option to create a link shadow of the backed up area... this is a cheap mans snapshotting. So as long as the backup and shapshots on the receiving system are within the SAME filesystem, you can use this cheap snapshot mechanism to keep track of multiple days... AND you can browse through it all.
# 3  
Old 07-05-2010
What are the two Operating Systems involved? Please be very very specific.

How far apart (in network terms) are the computers? Are there any stoppers such as firewalls or restricted bandwidth?

What network file transfer software do you have available? Obviously GET/LIST/PUT implies ftp but with better knowledge of your systems there may be more?

If we are confined to ftp, do you have enough disc space to copy back an archive when you want individual files or do you expect the target system to deal with such requests?

How big in measurable units is "big" ?
# 4  
Old 07-06-2010
Quote:
Originally Posted by methyl
What are the two Operating Systems involved? Please be very very specific.

How far apart (in network terms) are the computers? Are there any stoppers such as firewalls or restricted bandwidth?

What network file transfer software do you have available? Obviously GET/LIST/PUT implies ftp but with better knowledge of your systems there may be more?

If we are confined to ftp, do you have enough disc space to copy back an archive when you want individual files or do you expect the target system to deal with such requests?

How big in measurable units is "big" ?
I'm backup things from NFS on RHEL5 to S3, but I cant use all the features that S3 provides(specifically versioning), because I need to go through a mandatory encryption.

bandwidth is not an issue.

I dont have space to copy back a full archive, but I am considering splitting them into small parts.

data is about 5TB

Last edited by overmindxp; 07-06-2010 at 01:43 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Incremental Backup using tar for more than one path

Hi everyone; I need to create a shell script for full and incremental backup purposes. for incremental backup only for one folder (test1) located in /home/test1 syntax below works fine and have no problem: tar -cvf /home/backup/backup-0.tar.gz -g /home/test1/test.snar /home/test Problem is here... (5 Replies)
Discussion started by: momed131
5 Replies

2. Shell Programming and Scripting

Append files (backup .tar)

Hi all, I have to write a script which will keep for a particular user (username is first argument) backups of the account area to another. The script should accept as the second argument is a directory (or file), create one by storing a copy of the argument (using tar) and copies it to the list... (0 Replies)
Discussion started by: peter20
0 Replies

3. UNIX for Dummies Questions & Answers

Create an incremental tar

Hello, I need to create a tar file with a list of files from a directory. The directory has about 1000+ files of which I only need to create a tar ball of 150 files. The 150 files I need in the tar ball do not have common names or a common start letter. No two file names match, example: 1st... (3 Replies)
Discussion started by: babyPen1985
3 Replies

4. Shell Programming and Scripting

Backup script to split and tar files

Hi Guys, I'm very new to bash scripting. Please help me on this. I'm in need of a backup script which does the ff. 1. If a file is larger than 5GB. split it and tar the file. 2. Weekly backup file to amazon s3 using s3rsync 3. If a file is unchanged it doesn't need to copy to amazon s3 ... (4 Replies)
Discussion started by: ganitolngyundre
4 Replies

5. Shell Programming and Scripting

Backup files in tar format

Hi, I need backup all the files(including sub directories files ) which we modified today and create the tar file with the filename_<current_date>.tar Thanks (1 Reply)
Discussion started by: gavemani
1 Replies

6. Windows & DOS: Issues & Discussions

Incremental Backup

I have a folder /root/test in a centos 5.3 system. I want to take an incremental backup of the contents of the folder in the C:\Downloads folder of a windows system present in the same lan as the linux system. What are the ways of executing this plan? Kindly help (0 Replies)
Discussion started by: proactiveaditya
0 Replies

7. Shell Programming and Scripting

Incremental backup

Hi, I would like to create a daily incremental backup of a directory with all of the files within and add a timestamp (year-month-day) to the tar.gz file. I have the following, but it doesn't backup the inside files of the directory. #!/bin/bash tar -czf... (1 Reply)
Discussion started by: agasamapetilon
1 Replies

8. UNIX for Dummies Questions & Answers

incremental backup

Hi All.. i am trying to write a script which will give the incremental tar backup of all files with latest timestam. i tried with find -mmin -2 but if it takes half on hour or something to creat the tar itself, then no meaning in using the above command. so please help me to find the... (2 Replies)
Discussion started by: Usha Shastri
2 Replies

9. SCO

Incremental Backup using TAR in sco Openserver 5.0

Incremental Backup using TAR in sco Openserver 5.0 Dear all I am using sco openserver 5.0. I wanted to take backup of two folder (each 600 MB size) with lot of files. I used to take backup using tar command daily using a script. But the same takes more time. Is there any way to take backup... (0 Replies)
Discussion started by: jamcalicut
0 Replies

10. Shell Programming and Scripting

tar - incremental backup

Hello everyone! I'm trying to make incremental tar archives of a folder for an example. On the box I use is UNIX AIX installed. I tried some sample codes I found on several web pages but with no success. Don't know what I'm doing wrong. Please write some sample code to make incremental tar... (0 Replies)
Discussion started by: Funky_ass
0 Replies
Login or Register to Ask a Question