Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

A directory of common files from two others

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers A directory of common files from two others
# 1  
Old 12-19-2011
A directory of common files from two others
Hi all,
that' my problem

Code:
dir1:
fileA
fileB
fileC

dir2:
fileA
fileC

I need a command in shell to obtain a dir with common files:
dir_common:
Code:
fileA
fileC

thank u all
Last edited by zxmaus; 12-19-2011 at 09:44 PM..
# 2  
Old 12-19-2011
Simple script that lists both directories and prints anything that there are two of.

Code:
#!/usr/bin/env ksh
ls $1 $2 | awk '{ c[$1]++; } END { for( x in c ) if( c[x] == 2 ) print x; }'

A bit more complex script that lists the overlap and marks files have the same name, but don't have the same content, with an asterisk (*):

Code:
#!/usr/bin/env ksh

md5sum $1/* $2/* 2>/dev/null | awk '
    {
        basename = $NF;
        gsub( ".*/", "", basename );
        md5[basename,++idx[basename]] = $1;
        count[basename]++;
    }

    END {
        for( x in count )
            if( count[x] == 2 )
                printf( "%s %s\n", x, md5[x,1] == md5[x,2] ? "" : "*" );
    }
'

# 3  
Old 12-19-2011
Or you can use rsync to do the trick:

Code:
rsync -nr --existing dir1/ dir2/

Last edited by verdepollo; 12-19-2011 at 02:43 PM.. Reason: typo
This User Gave Thanks to verdepollo For This Post:
sdebasis
# 4  
Old 12-19-2011
Thank u Agama.

And now if I would like to copy instead print the files?

Code:
ls $1 $2 | awk '{ c[$1]++; } END { for( x in c ) if( c[x] == 2 ) print x; }'

I would like to copy the files x in a new directory
Last edited by zxmaus; 12-19-2011 at 09:43 PM..
# 5  
Old 12-19-2011
This is ambiguous (and therefore impossible to program). When there are files with the same name but in different directories, you need to specify which file to copy to the new directory. If we assume that "dir1" is the master directory, then the decision is made.
The fact that two files have the same name does not imply that they have the same contents.
# 6  
Old 12-20-2011
Methyl,
yes it' ambiguos but I know the files have same contents cause another program check their md5sum.
However it's correct to consider dir1 as a master.

Solution with awk is charming but I dont know it, and i dont how to put a copy command inside that awk code, a copy command instead of print command
# 7  
Old 12-20-2011
One way of doing it would be this:

Code:
if [[ -z $3 ]]
then
    echo "usage: $0 source-dir check-dir target-dir"
    exit 1
fi
# probably should also check to see if all three are directories and fail if not.

ls $1 $2 | awk -v sd="$1" -v td=$3 '{ c[$1]++; } END { for( x in c ) if( c[x] == 2 ) printf( "cp %s/%s %s/\n", sd, x, td ); }'

Supplies the target directory to the script as the third parameter and copies the files from the first (source) directory there if they are found in the check (second) directory. The output of the script is the copy commands which when verified can be piped to your shell of choice.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Comparing two files and list the difference with common first line content of both files

I have two file as given below which shows the ACL permissions of each file. I need to compare the source file with target file and list down the difference as specified below in required output. Can someone help me on this ? Source File ************* # file: /local/test_1 # owner: own #... (4 Replies)
Discussion started by: sarathy_a35
4 Replies

2. Shell Programming and Scripting

awk common between files

Hello there: I want to find common among files. They all have one column. Format for data: CEU_snp_CHR21.txt 21:10758305 21:10827533 21:10913441 21:10920098 21:10952160 21:10966322 21:10985991 NAT_CHR21_variants.txt 21:10971951 (3 Replies)
Discussion started by: genome
3 Replies

3. Shell Programming and Scripting

Common file from two files

Need Help file1 A.txt B.txt file2 A.txt B.txt C.txt output: Similar file in two files A.txt B.txt (1 Reply)
Discussion started by: satish1222
1 Replies

4. Shell Programming and Scripting

Compare multiple files, and extract items that are common to ALL files only

I have this code awk 'NR==FNR{a=$1;next} a' file1 file2 which does what I need it to do, but for only two files. I want to make it so that I can have multiple files (for example 30) and the code will return only the items that are in every single one of those files and ignore the ones... (7 Replies)
Discussion started by: castrojc
7 Replies

5. Shell Programming and Scripting

Common lines from files

Hello guys, I need a script to get the common lines from two files with a criteria that if the first two columns match then I keep the maximum value of the 5th column.(tab separated columns) . 3rd and 4th columns corresponds to the row which has highest value for the 5th column. Sample... (2 Replies)
Discussion started by: jaysean
2 Replies

6. Shell Programming and Scripting

Common lines from files

Hello guys, I need a script to get the common lines from two files with a criteria that if the first two columns match then I keep the maximum value of the 3rd column.(tab separated columns) Sample input: file1: 111 222 0.1 333 444 0.5 555 666 0.4 file 2: 111 222 0.7 555 666... (5 Replies)
Discussion started by: jaysean
5 Replies

7. Solaris

Common Home directory for different users??

Hi Guys, I have a problem with configuring a server. this is a solaris 10 with sparc platform. I have setup so that the server is Authenticating through NIS but I dont want the server to Mount the Home directories. The users need to logged in through the CDE/display. I have over 200 users... (2 Replies)
Discussion started by: Luky
2 Replies

8. Shell Programming and Scripting

Getting Common value in three files

I have 3 files 1.csv abc def 2.csv abc xyb 3.csv abc e23 frw I need to search for the common word in all the three files. How do i do that in awk ? (10 Replies)
Discussion started by: nuthalapati
10 Replies

9. Shell Programming and Scripting

Merge files of differrent size with one field common in both files using awk

hi, i am facing a problem in merging two files using awk, the problem is as stated below, file1: A|B|C|D|E|F|G|H|I|1 M|N|O|P|Q|R|S|T|U|2 AA|BB|CC|DD|EE|FF|GG|HH|II|1 .... .... .... file2 : 1|Mn|op|qr (2 Replies)
Discussion started by: shashi1982
2 Replies

10. UNIX for Dummies Questions & Answers

Get un common numbers from two files

Hi, I have two files: abc : 50040 123123 31703 cde: 104 97 50040 123123 31703 36609 50534 (3 Replies)
Discussion started by: jingi1234
3 Replies
Login or Register to Ask a Question