Visit Our UNIX and Linux User Community


compare files in two directories and output changed files to third directory


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting compare files in two directories and output changed files to third directory
# 1  
Old 05-26-2009
compare files in two directories and output changed files to third directory

I have searched about 30 threads, a load of Google pages and cannot find what I am looking for. I have some of the parts but not the whole. I cannot seem to get the puzzle fit together.

I have three folders, two of which contain different versions of multiple files, dist/file1.php dist/file2.php etc and dist_old/file1.php dist_old/file2.php etc.

The third folder contains only the changed files between the two. If /dist/file1.php differs from /dist_old/file1.php then a file named dist_upgrade/file1.php exists with the same contents as dist/file1.php.

Currently I am manually coping the different files using winmerge and I would like to automate some of this if possible.

What I would like to do is compare the dist folder with the dist_old folder then have files that are not the same copied to dist_upgrade/file.ext retaining the directory structure and full contents of the new file in the dist folder.

Thanks for any advice,
Best Regards,
Brandon
# 2  
Old 05-26-2009
This could be useful
Code:
#!/bin/sh
cd dist
for a in `ls`; do
   
   if [ ! -f "../dist_old/$a" ]; then
       continue
   fi
   diff $a ../dist_old/$a > /dev/null
   if [[ "$?" == "1" ]]; then
      cp $a ../dist_upgrade
   fi
done


Last edited by edgarvm; 05-26-2009 at 08:35 PM..
# 3  
Old 05-26-2009
Similar to edgarvm's but coping with new files as well as different ones...

Code:
new=/dist
old=/dist_old
diff=/dist_upgrade

cd $new
for f in *f
do
# Diff the files - ignore the output...
    diff $f $old > /dev/null 2>&1
# ...but get the status
    status=$?
    if [ $status -eq 0 ] ; then
# Files are identical - don't copy the file
        echo $f unchanged
    elif [ $status -eq 1 ] ; then
# Files differ - copy new file
        echo $f changed
        cp $f $diff
    elif [ $status -eq 2 ] ; then
# Old file doesn't exist - copy new file
        echo old $f does not exist
        cp $f $diff
    fi
done

You'll actually get a status 2 if a file exists but is unreadable as well, but I've ignored that as you can make everything readable before you start!
This User Gave Thanks to JerryHone For This Post:
# 4  
Old 05-26-2009
Thanks for the responses you guys are top notch. I have made a few modifications to get this working recursively using edgarvm's example. I also incorporated JerryHone's idea about dealing with non-existent files

Code:
#!/bin/bash

# setup folders for our different stages
DIST=/var/www/localhost/htdocs/dist/
DIST_OLD=/var/www/localhost/htdocs/dist_old/
DIST_UPGRADE=/var/www/localhost/htdocs/dist_upgrade/

cd $DIST

list=`find . -type f`

for a in $list; do
   if [ ! -f "$DIST_OLD$a" ]; then
        cp --parents $a $DIST_UPGRADE
      continue
   fi
   diff $a $DIST_OLD$a > /dev/null
   if [[ "$?" == "1" ]]; then
        # File exists but is different so copy changed file
        cp --parents $a $DIST_UPGRADE
   fi
done

Thanks for the advice.
Regards,
Brandon

Last edited by bkeep; 05-26-2009 at 11:04 PM.. Reason: Got it working with above code
# 5  
Old 08-26-2009
thanks you guys, this script helped me a lot!

but today I found out that it can't process spaces in file names properly so I modified it a bit using the advise from here:

Unixjunkie Blog: Handling Filenames With Spaces


this script seems to process such files properly (although I didn't test it too much):

Code:
#!/bin/bash

# setup folders for our different stages
DIST=/var/www/localhost/htdocs/dist/
DIST_OLD=/var/www/localhost/htdocs/dist_old/
DIST_UPGRADE=/var/www/localhost/htdocs/dist_upgrade/

cd $DIST

find . -type f | while read filename
do
   if [ ! -f "$DIST_OLD$filename" ]; then
        cp --parents "$filename" $DIST_UPGRADE
      continue
   fi
   diff "$filename" "$DIST_OLD$filename" > /dev/null
   if [[ "$?" == "1" ]]; then
        # File exists but is different so copy changed file
        cp --parents $filename $DIST_UPGRADE
   fi
done

cheers!

Previous Thread | Next Thread
Test Your Knowledge in Computers #682
Difficulty: Easy
Steve Job's was the primary founder and majority shareholder of Pixar
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare directories and copy differences (files) in a another directory

Hey im working on script that can compare 2 directory and check difference, then copy difference files in third diretory. here is the story: in folder one we have 12 subfolder and in each of them near 500 images hosted. 01 02 03 04 05 06 07 08 09 10 11 12 in folder 2 we have same subfolder... (2 Replies)
Discussion started by: nimafire
2 Replies

2. Shell Programming and Scripting

Compare files to pull changed records only

Hi, I am using Sun Solaris - SunOS. I have two fixed width files shown below. I am trying to find the changes in the records in the Newfile.txt for the records where the key column matches. The first column is a key column (example: A123). If there are any new or deletion of records in the... (4 Replies)
Discussion started by: Saanvi1
4 Replies

3. AIX

How to backup a directory (sub-directories/files) files from one server on to other ?

Hello, Server A: /directory1/ Server B: /Backups/ i wanted to backup contents of /directory1 from "server A" on to "Server B" every 1 hour. If there is any change in (only new/differences) contents on serverA (directory1/) supposed to be backeup on next run. I did used rsync command to... (5 Replies)
Discussion started by: System Admin 77
5 Replies

4. Shell Programming and Scripting

Compare files of 2 directories

Hi all, I have 2 directories dir1 and dir2 which contains many xml files. I need to compare files of dir1 with that of dir2 and if they match, I need to cut it from dir1 and paste it in dir2. I need to do this thru scripts. I'm currently investigating on the diff command. Please help me write... (6 Replies)
Discussion started by: frum
6 Replies

5. UNIX for Advanced & Expert Users

Find all files in the current directory excluding hidden files and directories

Find all files in the current directory only excluding hidden directories and files. For the below command, though it's not deleting hidden files.. it is traversing through the hidden directories and listing normal which should be avoided. `find . \( ! -name ".*" -prune \) -mtime +${n_days}... (7 Replies)
Discussion started by: ksailesh1
7 Replies

6. UNIX for Dummies Questions & Answers

Compare files in two directories

Hi All, I have two directories that has some files, some of the files are common to both of them like : ls -l dir1 file1 file2 file3 ls -l dir2 file1 file2 file3 file4 file5 Now i want to get the files from dir2 that are not present in dir1 (means i want to get... (2 Replies)
Discussion started by: mukulverma2408
2 Replies

7. Shell Programming and Scripting

How to list all the files, directories and sub-directories in the current path except one directory?

Can anyone come up with a unix command that lists all the files, directories and sub-directories in the current directory except a folder called log.? Thank you in advance. (7 Replies)
Discussion started by: Manjunath B
7 Replies

8. UNIX for Dummies Questions & Answers

compare all files under directories

Hello I am very new to Unix. I am actually using the C shell to write a program that will compare all the files in the directory and subdirectores and print out the ones that are identical, I am assuming identical by name or text Thank you (2 Replies)
Discussion started by: ga.miami56
2 Replies

9. Shell Programming and Scripting

mget * (obtein files from current directory but not the files form sub-directories)

Hello, Using the instruction mget (within ftp) and with "Interactive mode off", I want to get all files from directory (DirAA), but not the files in sub-directories. The files names don't follow any defined rule, so they can be just letters without (.) period Directory structure example: ... (0 Replies)
Discussion started by: Peter321
0 Replies

10. Shell Programming and Scripting

How to compare two flat files and get changed data

Hi, I need to compare two flat files (yesterday & today's data) and get only the changed data from flat files. In flat file i dont have data column or anything its just a string data in flat file.Can any one please let me know the script With Regds Shashi (3 Replies)
Discussion started by: jtshashidhar
3 Replies

Featured Tech Videos