Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: HELP! Need to compare 2 folders on 2 different systems, and copy unmatched filenames to other folder
Top Forums Shell Programming and Scripting HELP! Need to compare 2 folders on 2 different systems, and copy unmatched filenames to other folder Post 302591564 by jim mcnamara on Thursday 19th of January 2012 11:49:36 PM
Old 01-20-2012
This assumes you set up ssh keys for the user in question
Code:
ssh server1 'cd /data/archive && ls | cksum | grep -v -i directory' > server1.lis
ssh server2 'cd /data/archive && ls | cksum | grep -v -i directory' > server2.lis
awk '{arr[$0]++} END{ for (i in arr){if(arr[i]==1){print arr[i]} } }' server1.lis server2.lis >uniq.txt

This gives you a list of files that are different, either in content and/or in name.
You can then scp the files to server3. I'm tired right now so I will opt for not worrying about failures
too much.
Code:
awk '{print $(NF)}' uniq.txt |
while read fname 
do
    scp server1:/data/archive/${fname} server3:/data/archive/${fname}  || 
    scp server2:/data/archive/${fname} server3:/data/archive/${fname}
done < uniq.txt

TEST this first.
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Auto copy for files from folder to folder upon instant writing

Hello all, I'm trying to accomplish that if a file gets written to folder /path/to/a/ it gets automatically copied into /path/to/b/ the moment its get written. I thought of writing a shell script and cron it that every X amount of minutes it copies these files over but this will not help me... (2 Replies)
Discussion started by: Bashar
2 Replies

2. Shell Programming and Scripting

copy some files from users home folders to my folder

i have users home directories in /home all the users have some files starting with character e and i want to copy all these files in a folder in my (root) home using a script i tried the script for i in m5 do cd m5 cp e1* /home/pc/exam cd .. done but get these... (3 Replies)
Discussion started by: pcrana
3 Replies

3. Shell Programming and Scripting

copy all files with the same filenames as those in another folder

Hi, all: I've got two folders, folder A contains some image files (say, 100 files) in .jpg format; folder B contains all description files (say, 500 files) in .txt format. All image files in folder A are able to find their corresponding description files in folder B. That is to say,... (3 Replies)
Discussion started by: jiapei100
3 Replies

4. Shell Programming and Scripting

Find files of type within folder copy multiple results to different folders

Ok question number two: I'd like to search a directory for multiple file types (rar, txt, deb) and depending on what's found, copy those files to folders named Rar, TextFiles, and Debs. I'm looking for speed here so the faster the script the better. I want it to be a function that I pass 1 argument... (4 Replies)
Discussion started by: DC Slick
4 Replies

5. Shell Programming and Scripting

Change all filenames under different folders...

Hi, all: I'd love to use shell script to change all filenames under different folders once for all: I've got over 100 folders, in each of them, there is a file named "a.ppm". I wanna change all these "a.ppm" to "b.ppm", and still . Visually, the directory structure looks like: and hope... (1 Reply)
Discussion started by: jiapei100
1 Replies

6. Shell Programming and Scripting

How best to remove certain characters from filenames and folders recursively

hello, I'm trying to figure out which tool is best for recursively renaming and files or folders using the characters \/*?”<>| in their name. I've tried many examples that use Bash, Python and Perl, but I'm not much of a programmer I seem to have hit a roadblock. Does anyone have any... (15 Replies)
Discussion started by: prometheon123
15 Replies

7. Shell Programming and Scripting

Linux Script to compare two folders and copy missing files

Hi, I need help in shell scripting. If someone can help me, that would be great! Problem. I want Linux Script to compare two folders and copy missing files. Description. I have two directories /dir1 /dir2 I need to copy all distinct/new/unique/missing files from /dir1 and that... (1 Reply)
Discussion started by: S.Praveen Kumar
1 Replies

8. Shell Programming and Scripting

How to copy files with the same filenames as those in another folder to that same folder?

Hello All A similar question like this was asked before but I need to change part of the question. I've two folders, Folder A contains some image files in 150 subfolders; Folder B contains text files in 350 subfolders. All image files in Folder A have the same filename as the text... (5 Replies)
Discussion started by: chlade
5 Replies

9. Shell Programming and Scripting

How do you compare two folders and copy the difference to a third folder in a remote server?

How do you compare one local folder and a remote folder and copy the difference to a third folder in a remote folder.e.g. Folder A -- Is in a remote server and it has the following files TEST1.OUT TEST2.OUT TEST3.OUT Folder B --Is in a local server and it has the following files ... (5 Replies)
Discussion started by: cumeh1624
5 Replies

10. Shell Programming and Scripting

Script to compare partial filenames in two folders and delete duplicates

Background: I use a TV tuner card to capture OTA video files (.mpeg) and then my Plex Media Server automatically optimizes the files (transcodes for better playback) and places them in a new directory. I have another Plex Library pointing to the new location for the optimized .mp4 files. This... (2 Replies)
Discussion started by: shaky
2 Replies

LEARN ABOUT DEBIAN

zip_fopen

ZIP_FOPEN(3)						     Library Functions Manual						      ZIP_FOPEN(3)

NAME

       zip_fopen , - .Nm zip_fopen_index open file in zip archive for reading

LIBRARY

       libzip (-lzip)

SYNOPSIS

       #include <zip.h>

       struct zip_file * zip_fopen(struct zip *archive, const char *fname, int flags);

       struct zip_file * zip_fopen_index(struct zip *archive, zip_uint64_t index, int flags);

DESCRIPTION

       The zip_fopen function opens the file name fname in archive.  The flags argument specifies how the name lookup should be done, according to
       the values are described in zip_name_locate(3).	Also, the following values may be or'ed to it.

	      ZIP_FL_COMPRESSED  Read the compressed data.  Otherwise the data is uncompressed by zip_fread.

	      ZIP_FL_UNCHANGED	 Read the original data from the zip archive, ignoring any changes made to the file.

       The zip_fopen_index function opens the file at position index.

       If encrypted data is encountered, the functions call zip_fopen_encrypted(3) or zip_fopen_index_encrypted(3) respectively, using the default
       password set with zip_set_default_password(3).

RETURN VALUES

       Upon successful completion, a

       struct zip_file pointer is returned.  Otherwise, NULL is returned and the error code in archive is set to indicate the error.

ERRORS

	      [ZIP_ER_CHANGED]
		  The file data has been changed.

	      [ZIP_ER_COMPNOTSUPP]
		  The compression method used is not supported.

	      [ZIP_ER_ENCRNOTSUPP]
		  The encryption method used is not supported.

	      [ZIP_ER_MEMORY]
		  Required memory could not be allocated.

	      [ZIP_ER_READ]
		  A file read error occurred.

	      [ZIP_ER_SEEK]
		  A file seek error occurred.

	      [ZIP_ER_ZLIB]
		  Initializing the zlib stream failed.

       The function zip_fopen may also fail and set for any of the errors specified for the routine zip_name_locate(3).

       The function zip_fopen_index may also fail with ZIP_ER_INVAL if index is invalid.

SEE ALSO

       libzip(3), zip_fclose(3), zip_fread(3), zip_get_num_entries(3), zip_name_locate(3), zip_set_default_password(3)

AUTHORS

       Dieter Baron <dillo@giga.or.at> and Thomas Klausner <tk@giga.or.at>

NiH								 February 14, 2011						      ZIP_FOPEN(3)
All times are GMT -4. The time now is 06:32 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy