Compare & Copy Directories : Bash Script Help Post: 302744797

Sponsored Content

Top Forums Shell Programming and Scripting Compare & Copy Directories : Bash Script Help Post 302744797 by Rod on Friday 14th of December 2012 08:05:14 PM

12-14-2012

Registered User

Compare & Copy Directories : Bash Script Help

Beginner/Intermediate shell; comfortable in the command line.

I have been looking for a solution to a backup problem. I need to compare Directory 1 to Directory 2 and copy all modified or new files/directories from Directory 1 to Directory 3. I need the directory and file structure to be mirrored on Directory 3. Another way of thinking about the logic is: Dir1 - Dir2 = Dir3. I want Dir3 to be portable between users/machines so I don't think an incremental backup with hardlinks will work. Seems like it should be easy right?

I found a scripts that seems like it could do the trick. However, I'm having some trouble getting it to work properly. It compares Dir1 to Dir2 and copies all new or modified files into Dir3 but it does NOT recreate the directory structure on the target. Instead all files end up in a flat folder.

From Statckoverflow . com

Code:

#!/bin/bash
#
# setup folders for our different stages
#
Dir1=/Users/username/source1
Dir2=/Users/username/source2
Dir3=/Users/username/target
#
cd $Dir1
for f in *f
do
# Diff the files - ignore the output...
    diff $f $Dir2 > /dev/null 2>&1
# ...but get the status
    status=$?
    if [ $status -eq 0 ] ; then
# Files are identical - don't copy the file
        echo $f unchanged
    elif [ $status -eq 1 ] ; then
# Files differ - copy new file
        echo $f changed
        cp $f $diff
    elif [ $status -eq 2 ] ; then
# Old file doesn't exist - copy new file
        echo old $f does not exist
        cp $f $diff
    fi
done

I started my reading with rsync. I've used it for back up - locally and over ssh between NAS servers. It should be able to do the job, right? I just couldn't find examples to guide me and I don't know enough and / or have enough time to do it unaided. Researching it I got side tracked by diff which I didn't know much about but it looked promising.

My preference is to do this with a bash script. Anyone have any ideas?

Thanks!

Last edited by Rod; 12-14-2012 at 09:56 PM..

Rod

View Public Profile for Rod

Find all posts by Rod

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How do i collect Date & Time from Different Directories in a script

How do i collect Date & Time from Different Directories in a script The script iam using for a single directory is : ls -l | grep awk '{print $8}'

2. Shell Programming and Scripting

Bash Script to Read & Write on different directories

Hi, root@server] df -h 121G 14G 101G 12% /home 147G 126G 14G 91% /backup We having our site files and images are storing in /backup/home/user/files/ through symbolic link created in /home directory pointing in /backup directory as following. root@server] cd /home...

3. Shell Programming and Scripting

Bash scripting to compare N number of files located in two directories

I want to compare "N" (around 2000+) number of huge files located in a directory A against "N" files located in a different directory using Bash scripting. Please help me with any scripts available. Thanks.

4. UNIX for Dummies Questions & Answers

Compare two directories and copy differing files

Hello there, I'm a total noob to shell scripting. :) What I want to do is compare the contents of Folder A and Folder B, and copy any files in Folder A that do not exist in Folder B over to Folder B. I have gotten so far as: diff -rq folderA folderB which returns the names of the files,...

5. Shell Programming and Scripting

Need a shell script to compare two directories and produces the output

Hi, I am using solaris OS 10 and Bash shell.I need a script which will compare the two directories and produces the output. Step 1: In detail say suppoose I have machine one and have a directory dir1. Script should iterate through the directories and subdirectories inside and produce the output...

6. Shell Programming and Scripting

Script to copy certain info from several directories

Hi, I am writing a script to copy certain file name in txt file . It is working fine if I provide a single directory name (for example "/eos/uscms/store/user/pooja04//analysis2012/525/data/doubleele/2012/datav1/" ) where those specific files are present ending with *root . But I want to modify...

7. Shell Programming and Scripting

Compare File & Copy Replace if Successful

Hi All, I have written a shell script that creates a backup of my MySQL database. The script performs the following functions: Creates a Backup of the MySQL database Compresses the Backup Copies the Backup to a Remote Server Send an E-Mail displaying the size of the Backup Removes any...

8. Shell Programming and Scripting

Shell script to copy particular file from directories recursively

I have directory path in which there are several sub directories. In all these sub dir there will be one env.cnf file. I want to copy this env.cnf file from each sub dir's and place them in destination path by creating same filename as sub dir_env.cnf. After copying env.cnf files from source...

9. Shell Programming and Scripting

Need shell script to compare directories and delete files on target server

Hello, I need help in writing the shell script for below mentioned case. There are 2 servers(server A, server B). A cronjob syncs files between these 2 servers. Existing script is copying files from A to B. This is done using the command rsync. However, the files are not deleted...

10. Shell Programming and Scripting

Compare directories and copy differences (files) in a another directory

Hey im working on script that can compare 2 directory and check difference, then copy difference files in third diretory. here is the story: in folder one we have 12 subfolder and in each of them near 500 images hosted. 01 02 03 04 05 06 07 08 09 10 11 12 in folder 2 we have same subfolder...

LEARN ABOUT V7

gendiff

GENDIFF(1)						      General Commands Manual							GENDIFF(1)

NAME

       gendiff - utility to aid in error-free diff file generation

SYNOPSIS

       gendiff <directory> <diff-extension>

DESCRIPTION

       gendiff	is  a  rather  simple script which aids in generating a diff file from a single directory.  It takes a directory name and a "diff-
       extension" as its only arguments.  The diff extension should be a unique sequence of characters added to the end of all	original,  unmodi-
       fied files.  The output of the program is a diff file which may be applied with the patch program to recreate the changes.

       The  usual  sequence  of events for creating a diff is to create two identical directories, make changes in one directory, and then use the
       diff utility to create a list of differences between the two.  Using gendiff eliminates the need for the  extra,  original  and	unmodified
       directory copy.	Instead, only the individual files that are modified need to be saved.

       Before editing a file, copy the file, appending the extension you have chosen to the filename.  I.e. if you were going to edit somefile.cpp
       and have chosen the extension "fix", copy it to somefile.cpp.fix before editing it.  Then edit the first copy (somefile.cpp).

       After editing all the files you need to edit in this fashion, enter the directory one level above where your source code resides, and  then
       type

	   $ gendiff somedirectory .fix > mydiff-fix.patch

       You should redirect the output to a file (as illustrated) unless you want to see the results on stdout.

SEE ALSO

       diff(1), patch(1)

AUTHOR

       Marc Ewing <marc@redhat.com>

4th Berkeley Distribution					  Mon Jan 10 2000							GENDIFF(1)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How do i collect Date & Time from Different Directories in a script

Discussion started by: laknar

2. Shell Programming and Scripting

Bash Script to Read & Write on different directories

Discussion started by: mirfan

3. Shell Programming and Scripting

Bash scripting to compare N number of files located in two directories

Discussion started by: Sangtha

4. UNIX for Dummies Questions & Answers

Compare two directories and copy differing files

Discussion started by: raaaaaa