Copy only files whiich do not exist on Target Directory


 
Thread Tools Search this Thread
Operating Systems HP-UX Copy only files whiich do not exist on Target Directory
# 1  
Old 12-02-2009
Copy only files whiich do not exist on Target Directory

Hi,

I am using HP-UX B 11.23

I want to copy files from one directory into another directory in such a way that, only the nonexistent files in target directory are copied from source directory

e.g.
dir1 has
file1
file2
file3
dir2 has
file1
file3
now I want that a command (non-interactive) to copy only file2 from dir1 to dir2

Thanks and Regards,
Chetana
# 2  
Old 12-03-2009
Code:
cd dir1
 for i in `ls`; do  if [ -e $i ]; then echo "$i exists"; else echo "$i doesnot exist"; cp $i ../dir2/;  fi; done

HTH,
PL
# 3  
Old 12-03-2009
Hi Daptal

Many Thanks for the reply

The option you suggested works.
However I am not sure if it will ipmact the performance, since the (exists) test will be carried for each file individually

This is like -exec
Do we have any option like xargs which will execute only once for this?

Initially I thought of 'diff' for comparing the files in directories but it compare contents of the files as well which is not feasible in my case since I have binary files of quite big size

These are Oracle archivelog files, each 150-200 Mb and large number of quantity

Also I tried following which too is slow

ls -ltr dir1>dir1FileList.txt
ls -ltr dir2>dir2FileList.txt
diff dir1FileList.txt dir2FileList.txt|awk '{print $9}'>reqdFiles.txt

Thanks and Regards,
Chetana
# 4  
Old 12-03-2009
have you tried using diff on the directories directly, instead of sending listings to text files and processing that?

Code:
diff dir1 dir2

gives output like the following:

Code:
Only in dir1: file1
Only in dir1: file2
Only in dir2: file3

So you could do the following (assuming dir1 is the source and dir2 is the target):

Code:
for FILE in $(diff dir1 dir2|grep dir1 | cut -f4 -d' ')
do
  cp -p dir1/${FILE} dir2/${FILE}
done

# 5  
Old 12-03-2009
I dont think it would be impacting the performance much .I am not quite confident abt this as i have not done benchmarking on this.

Also if the performance is really bad you can try this

ls -l dir1 | awk ' {print $5,$8}' | sort -k 2,2 > file1
ls -l dir2 | awk ' {print $5,$8}' | sort -k 2,2 > file2

If the file 1 in dir 1 is same as file 1 in dir2 dont bother checking the sizes just take the file name only and write a simple one liner to get the file names that donotexist

HTH,
PL
# 6  
Old 12-04-2009
rsync can help you ?!

i am not sure, but i will suggest you to check the feasibility.
# 7  
Old 12-04-2009
Hi.

You could also do something like:

Code:
false | cp -i $DIR1/* $DIR2

or

yes no | cp -i $DIR1/* $DIR2

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell scripting for moving folder specific files into target directory of that country folder.

I need help to write shell script to copy files from one server to another server. Source Directory UAE(inside i have another folder Misc with files inside UAE folder).I have to copy this to another server UAE folder( Files should be copied to UAE folder and Misc files should be copied in target... (3 Replies)
Discussion started by: naresh2389
3 Replies

2. Post Here to Contact Site Administrators and Moderators

How to count successfully copy files source to target location with check directory in Linux?

Hi guys...please any one help me .... how to copy files from source to target location if 5 files copied successfully out of 10 files then implement success=10 and if remaining 5 files not copied successfully then count error=5 how to implement this condition with in loop i need code linux... (0 Replies)
Discussion started by: sravanreddy
0 Replies

3. Shell Programming and Scripting

Copy the files in directory and sub folders as it is to another directory.

How to copy files from one directory to another directory with the subfolders copied. If i have folder1/sub1/sub2/* it needs to copy files to folder2/sub1/sub2/*. I do not want to create sub folders in folder2. Can copy command create them automatically? I tried cp -a and cp -R but did... (4 Replies)
Discussion started by: santosh2626
4 Replies

4. UNIX for Dummies Questions & Answers

Rsync copy files if dont exist

I have a setup where I have two drives. TV TVbackup For what ever reason, I have a lot of content on my TVbackup drive which isn't on my TV drive. I want to copy all the files across which are on TVbackup but are not currently on TV. If there is a file with the same name but a... (2 Replies)
Discussion started by: Spadez
2 Replies

5. Red Hat

Unable to copy files due to many files in directory

I have directory that has some billion file inside , i tried copy some files for specific date but it's always did not respond for long time and did not give any result.. i tried everything with find command and also with xargs.. even this command find . -mtime -2 -print | xargs ls -d did not... (2 Replies)
Discussion started by: before4
2 Replies

6. UNIX for Dummies Questions & Answers

copy files as space exist in file name..

Hi, i am having a directory in which files are having space in the name . $ls -1 aa b.txt my file.pdf lost file.csv foo_file.txti want to copy those file to some where with date +%F as extension . But it failed for the file having space. #!/bin/sh ls -1 >tt for var in `cat tt` do b=$var... (2 Replies)
Discussion started by: posix
2 Replies

7. Shell Programming and Scripting

copy files from remote server (B) to target server (A)?

Hi All, what is the comand to log off the remote server? I have 2 servers A, B. I need to find all files older than 7 days on server B and copy over to server A. My logic is: login the remote server: ================= ssh hostB cd /data/test find . -mtime -7 -ls | awk '{print... (4 Replies)
Discussion started by: Beginer0705
4 Replies

8. Shell Programming and Scripting

Check if certain files exist in a directory, if not add name to a textfile

We recieve some logs on our windows box via FTP on a daily basis, in the same directory. I would like to check for missing logs files and add their name to a text file. Something like... Check if C:\logs\file1_currentdate exists (if not, add file1_currentdate to... (1 Reply)
Discussion started by: SunnyK
1 Replies

9. Shell Programming and Scripting

How to check a file exist and do a copy of other files

Hi, I would like to perform bash which would check the file A.txt to be size 0 or not. If the size is 0, I would copy file B.txt to replace A.txt. Please help. Thanks. -Jason (6 Replies)
Discussion started by: ahjiefreak
6 Replies

10. Shell Programming and Scripting

Copy files from one directory to another

I need to copy about 13 Tb of data from one directory and subdirectories to the other (another mount point). If I run this as a cron, say between 10 pm and 7 am, not all of the files will be copied over. Is there a way of 'resuming' the copy the following evenings until all files are copied over? (0 Replies)
Discussion started by: hd2006
0 Replies
Login or Register to Ask a Question