Copying updated files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Copying updated files
# 1  
Old 03-09-2012
Copying updated files

I have a directory with source code and want to figure out the files whose content has changed.

Suppose I have two version of development work called hstmy.1.0 and hstmy.1.7. Directory structure are basically the same. However there may be additional files and directories, whose contents have changed (new source codes, bug fixes, etc).

Code:
── hstmy.1.0
   ├── baseLib
   ├── bin
   │   ├── awk
   │   ├── ksh
   │   ├── prog
   │   └── tcsh
   ├── fortran
   ├── others
   ├── scripts
   └── source


Code:
── hstmy.1.7
   ├── baseLib
   ├── bin
   │   ├── awk
   │   ├── ksh
   │   ├── prog
   │   └── tcsh
   ├── fortran
   ├── others
   ├── scripts
   └── source

I want to know which files differ and copy them to a temporary directory

As example, hstmy.1.0 is as shown below

Code:
── hstmy.1.0
    ├── baseLib
   │   ├── common.hh
   │   ├── Complex.hh
   │   ├── CSpInterp1D.hh
   │   ├── CSpInterp2D.hh
   │   ├── Darwin.hh
   │   ├── DFT1.hh
   │   ├── DFT2.hh
   │   ├── dftUtils.hh
   │   ├── DynBaseObj.hh
   │   ├── FPnInterp1D.hh
   │   ├── FPnInterp2D.hh
   │   ├── GenFunc.hh
   │   ├── Interp1D.hh
   │   ├── Interp2D.hh
   │   ├── invers.hh
   │   ├── LinAlg.hh
   │   ├── LinInterp1D.hh
   │   ├── LinInterp2D.hh
   │   ├── List.hh
   │   ├── Matrix.hh
   │   ├── ObjFunc.hh
   │   ├── Optimize.hh
   │   ├── ParseEl.hh
   │   ├── Parsing.hh
   │   ├── Poly1.hh
   │   ├── Poly2.hh
   │   ├── Random.hh
   │   ├── Simplex.hh
   │   ├── Stack.hh
   │   ├── String.hh
   │   ├── Tensor2.hh
   │   ├── Tree.hh
   │   ├── Vect2.hh
   │   ├── Vector.hh
   │   └── Verbose.hh
   ├── bin
   │   ├── awk
   │   │   ├── addNoise-dat.awk
   │   │   ├── addNoise-dat.awk.pdf
   │   │   ├── addNoise-dat.awk.ps
   │   │   ├── addNoise-xt.awk
   │   │   ├── awkNotes.txt
   │   │   ├── change-log.awk
   │   │   ├── change-text.awk
   │   │   ├── checkSRDist.awk
   │   │   ├── checkSRDist-dat.awk
   │   │   ├── checkSRDist-ry.awk
   │   │   ├── checkSRDist-xt.awk
   │   │   ├── createAnmly-zc.awk
   │   │   ├── example.awk
   │   │   ├── jcd2zc.awk
   │   │   ├── log2cmd.awk
   │   │   ├── modeli2cmod.awk
   │   │   ├── n02-z30.ry
   │   │   ├── n02-z30.ry.bck
   │   │   ├── n02-z30.xt.bck
   │   │   ├── removeSndSp-zc.awk
   │   │   ├── removeSRDist-dat.awk
   │   │   ├── removeSRDist-ry.awk
   │   │   ├── removeSRDist-xt.awk
   │   │   ├── result
   │   │   ├── test.ry
   │   │   ├── test.ry.bck
   │   │   ├── test.txt
   │   │   ├── test.txt.bck
   │   │   ├── test.xt
   │   │   ├── test.xt.bck
   │   │   ├── timeDistAbs.txt
   │   │   ├── timeDistAbs.txt.bck
   │   │   ├── timeDistAbs.txt.bck.bck
   │   │   ├── xt2dat.awk
   │   │   ├── xt2dat-v2.awk
   │   │   ├── zc2cmd.awk
   │   │   ├── zc2zfc.awk
   │   │   └── zp2pmd.awk
   │   ├── ksh
   │   │   ├── check-srdist.ksh
   │   │   ├── comp-xt.ksh
   │   │   ├── convert-zfc-fcmod.ksh
   │   │   ├── cr-data-tinv.ksh
   │   │   ├── crmisfit.ksh
   │   │   ├── crmodel.ksh
   │   │   ├── hseis-processing.ksh
   │   │   ├── pl-jcd-zc.ksh
   │   │   ├── plmisfit.ksh
   │   │   ├── plmodel.ksh
   │   │   ├── plry.ksh
   │   │   ├── plryxt.ksh
   │   │   ├── plxy.ksh
   │   │   ├── raytrac.ksh
   │   │   ├── rmsrdist.ksh
   │   │   ├── tdarwin.ksh
   │   │   └── tsimplex.ksh
   │   ├── prog
   │   │   ├── getmisf
   │   │   ├── getpdf
   │   │   ├── getvel
   │   │   ├── linint
   │   │   ├── raytrac
   │   │   ├── tdarwin
   │   │   ├── tracepdf
   │   │   └── tsimplex
   │   └── tcsh
   │       ├── browseDir.tcsh
   │       ├── checkSRDist.tcsh
   │       ├── create-data-tinv.tcsh
   │       ├── createDocs.tcsh
   │       ├── createMisfit.tcsh
   │       ├── createModel.tcsh
   │       ├── createTrvt.tcsh
   │       ├── file0p12-22344.txt
   │       ├── file0p12-23234.txt
   │       ├── file0p12-77364.txt
   │       ├── file0p12.txt
   │       ├── file0p12.txt.bck
   │       ├── hseis-processing.tcsh
   │       ├── plotMisfit.tcsh
   │       ├── plotModel.tcsh
   │       ├── plotryxt.tcsh
   │       ├── plotxy.tcsh
   │       ├── printTable.tcsh
   │       ├── raytrac.tcsh
   │       ├── removeFiles.tcsh
   │       ├── removeRays.tcsh
   │       ├── removeSRDist.tcsh
   │       ├── removeText.tcsh
   │       ├── removeTrvt.tcsh
   │       ├── renameFiles.tcsh
   │       ├── renameFiles-tcsh-to-csh.run
   │       ├── replaceText.tcsh
   │       ├── sortList.tcsh
   │       ├── tcshrc.csh
   │       ├── tdarwin.tcsh
   │       ├── test.tcsh
   │       ├── tsimplex.tcsh
   │       └── zfc2fcmod.tcsh
   ├── fortran
   │   ├── generate_rgaussian
   │   ├── generate_rgaussian.f90
   │   ├── generate_rgaussian.o
   │   ├── hseistools.mod
   │   ├── linint
   │   ├── linint.f90
   │   ├── linint.o
   │   ├── rgaussian.f90
   │   ├── rgaussian.o
   │   ├── split_files_tt.f
   │   └── tt_resid.f
   ├── Makefile
   ├── Makefile.bck
   ├── Makefilediag
   ├── Makefile.old
   ├── others
   │   ├── asac2xy.cc
   │   ├── asacdump.cc
   │   ├── convtv.cc
   │   ├── cwt.cc
   │   ├── fourier.cc
   │   ├── pp2sac.cc
   │   ├── sac.h
   │   ├── segy.h
   │   ├── sucrpi.cc
   │   └── sufppk.cc
   └── source
       ├── CSpLay.hh
       ├── ElMap.hh
       ├── FPnLay.hh
       ├── GetMisf.cc
       ├── GetPDF.cc
       ├── GetVel.cc
       ├── Layer.hh
       ├── LayInt.hh
       ├── LayMod.hh
       ├── LinLay.hh
       ├── Map.hh
       ├── ModMisfit.hh
       ├── RayField.hh
       ├── Ray.hh
       ├── RayTrac.cc
       ├── Rdt.cc
       ├── Source.hh
       ├── Surface.cc
       ├── Tdarwin.cc
       ├── Tomog.hh
       ├── TraceMis.cc
       ├── TracePDF.cc
       ├── Tsimplex.cc
       └── VelMod.hh

# 2  
Old 03-09-2012
Hi kristinu,

Some questions:
  1. Omit files that exist in one branch but not in the other one?
  2. Compare both text and binary files?
  3. When found files that differ: copy both or one of them? and if that case, from which branch?
  4. Temporary dir will be plain? I mean, if found a file that differ inside two subdirectories, the script will have to create those subdirectories in the temporary dir or could be enough to copy the file without the tree structure?
  5. Why don't you use a source control system for the job?
# 3  
Old 03-09-2012
There might be a better way to use rsync for this, but this ought to work unless you've been putting spaces in filenames:

Code:
rsync -r --list-only a b | # Find changes and list them
        awk '/^[^d]/ { print $5 }' | # Get only filenames
        xargs tar -cf - | # Bundle them up
        tar -C /path/to/destination -xf - # Extract elsewhere

# 4  
Old 03-09-2012
Quote:
Originally Posted by birei
Hi kristinu,

Some questions:
  1. Omit files that exist in one branch but not in the other one?
  2. Compare both text and binary files?
  3. When found files that differ: copy both or one of them? and if that case, from which branch?
  4. Temporary dir will be plain? I mean, if found a file that differ inside two subdirectories, the script will have to create those subdirectories in the temporary dir or could be enough to copy the file without the tree structure?
  5. Why don't you use a source control system for the job?

I only care about source code changes (.hh, .cc, .awk, .tcsh, .ksh, .f, .F)

I want to just transfer the files without the tree structure to a folder, let's call it updatedcode. Will transfer only from
hstmy.1.7 the files that differ from hstmy.1.0

Yes, my intention is to use svn, but I am only starting on it. In the meantime I have to continue working.

---------- Post updated at 07:31 PM ---------- Previous update was at 07:23 PM ----------

Quote:
Originally Posted by Corona688
There might be a better way to use rsync for this, but this ought to work unless you've been putting spaces in filenames:

Code:
rsync -r --list-only a b | # Find changes and list them
        awk '/^[^d]/ { print $5 }' | # Get only filenames
        xargs tar -cf - | # Bundle them up
        tar -C /path/to/destination -xf - # Extract elsewhere

There are no spaces in file names. I am working from home and work. I just want to transfer the updated files so I can have a look at the work done on the latest version.

I created source1 and source2. They are identical to directory source. I made changes to source2/TracePDF.cc.

However when I did
Code:
rsync -r --list-only source source2

I got

Code:
drwxr-xr-x        4096 2012/03/09 22:38:03 source
-rw-r--r--       12869 2012/02/08 13:58:30 source/CSpLay.hh
-rw-r--r--        4737 2012/02/08 11:28:26 source/ElMap.hh
-rw-r--r--       18955 2012/02/08 13:58:30 source/FPnLay.hh
-rw-r--r--        4034 2012/02/08 11:29:13 source/GetMisf.cc
-rw-r--r--        4144 2012/02/08 11:29:48 source/GetPDF.cc
-rw-r--r--        7442 2012/02/08 11:30:24 source/GetVel.cc
-rw-r--r--        6172 2012/02/08 13:58:27 source/LayInt.hh
-rw-r--r--        8250 2012/02/08 13:58:21 source/LayMod.hh
-rw-r--r--       24893 2012/02/08 13:58:26 source/Layer.hh
-rw-r--r--       12864 2012/02/08 13:58:22 source/LinLay.hh
-rw-r--r--       18886 2012/02/08 13:58:22 source/Map.hh
-rw-r--r--       23943 2012/02/08 13:58:23 source/ModMisfit.hh
-rw-r--r--       24137 2012/02/08 13:58:23 source/Ray.hh
-rw-r--r--       25086 2012/02/08 13:58:23 source/RayField.hh
-rw-r--r--       23521 2012/03/07 23:03:04 source/RayTrac.cc
-rw-r--r--        8052 2012/02/08 11:40:36 source/Rdt.cc
-rw-r--r--       10668 2012/02/08 13:58:18 source/Source.hh
-rw-r--r--        2582 2012/02/08 11:41:42 source/Surface.cc
-rw-r--r--       14682 2012/02/07 18:51:24 source/Tdarwin.cc
-rw-r--r--       19959 2012/02/08 13:58:18 source/Tomog.hh
-rw-r--r--        6434 2012/02/08 11:42:53 source/TraceMis.cc
-rw-r--r--        6617 2012/02/08 11:43:23 source/TracePDF.cc
-rw-r--r--       11190 2012/02/08 11:04:55 source/Tsimplex.cc
-rw-r--r--       24784 2012/02/08 13:58:14 source/VelMod.hh

rather than just
Code:
source2/TracePDF.cc


Last edited by kristinu; 03-09-2012 at 08:47 PM..
# 5  
Old 03-09-2012
Yes, that's what all the awk nonsense is for, to get the proper filename out of it.

It may also be expecting source/* in the destination, as well as the source, in which case a symbolic link would fix it.
# 6  
Old 03-10-2012
I tried it but it saved all the files in the directory test rather than just the one file
source2/TracePDF.cc
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Comparing two files to get only records to be inserted and updated

Hello all, Please help me for a script that compares two files and reads only those records that are to be inserted and updated. File1: c_id name place contact_no 1 abc xyz 34567 10 efg uvw 82725 6 hjk wth 01823 2 iuy ... (4 Replies)
Discussion started by: T@ni@
4 Replies

2. Shell Programming and Scripting

Updated files using awk

If third column contains null value then it should delete the row and i want to know the files which contains null value in 3rd column.Below code is not showing the updated files correctly..not sure what the issue is.plz help me #!/usr/bin/sh Scripts=/ushhquest/data001/Scripts cd... (5 Replies)
Discussion started by: katakamvivek
5 Replies

3. Shell Programming and Scripting

Updated files information

I am using the below script to remove the rows which contains null values in the 3rd column.My requirement here is want to get the filenames which row is removed .please help me. #!/usr/bin/sh Scripts=/ushhquest/data001/Scripts cd /ushhquest/data011/TgtFiles/MonthlyData ls CUSTADDR*.txt >... (4 Replies)
Discussion started by: katakamvivek
4 Replies

4. UNIX for Dummies Questions & Answers

How do crontab files get updated with changes?

I have an ant script that puts my crontabtemplate changes out on a solaris server. But I am scratching my head trying to figure out what to do next on getting the actual crontab file updated with changes. Somone have suggestions on reads or links to figure this out? thx in advance (2 Replies)
Discussion started by: vsekvsek
2 Replies

5. Shell Programming and Scripting

Old time stamp being updated for new files

Hello Friends I am facing a weird problem :confused:, we receive thousands of files in my system on a daily basis, access time stamp on some of the files are being updated as old time stamp like 1968-01-19, Could some one help me what could be causing this? so that i can narrow down the problem... (4 Replies)
Discussion started by: Prateek007
4 Replies

6. Shell Programming and Scripting

Find last updated files and rename

HI I have a requirement to find the last updated files from a directory whcih has subdirectories and inside them we have files with .txt,.doc,.xls .. extensions. i have to find those files which were updated in the last 1hr and rename the files with respective <sub-directory>_<filename> and copy... (3 Replies)
Discussion started by: ramse8pc
3 Replies

7. Shell Programming and Scripting

to find the last updated file from different groups of files.

Hi i have many sets of files as shown below(here i have shown 2 sets) basel_aa_20091030.txt basel_aa_20091130.txt basel_aa_20091230.txt basel_bb_20091030.txt basel_bb_20091130.txt basel_bb_20091230.txt from each set of files i need to select the latest updated file(there are... (3 Replies)
Discussion started by: jagadeeshn04
3 Replies

8. UNIX for Advanced & Expert Users

copying of files by userB, dir & files owned by userA

I am userB and have a dir /temp1 This dir is owned by me. How do I recursively copy files from another users's dir userA? I need to preserve the original user who created files, original group information, original create date, mod date etc. I tried cp -pr /home/userA/* . ... (2 Replies)
Discussion started by: Hangman2
2 Replies

9. Solaris

files updated in last 10 hours should be moved

Hi, I would like to move all files that are updated in last 10 hrs. to some temporary folder. Please help. (3 Replies)
Discussion started by: sanjay1979
3 Replies

10. UNIX for Dummies Questions & Answers

recently updated files

How do I find files those have been updated in the last 24 hours, sort them by size descending and then display the top of the long list? (6 Replies)
Discussion started by: shantanuo
6 Replies
Login or Register to Ask a Question