Getting Common value in three files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Getting Common value in three files
# 1  
Old 10-19-2009
Getting Common value in three files

I have 3 files

1.csv
abc
def

2.csv
abc
xyb

3.csv
abc
e23
frw

I need to search for the common word in all the three files. How do i do that in awk ?
# 2  
Old 10-19-2009
Code:
awk ' FILENAME=="1.csv"  {arr[$0]++}
      FILENAME=="2.csv"  {arr[$0]++}
      FILENAME=="3.csv"  {arr[$0]++}
      END {for (i in arr) 
      {if(arr[i]==3) {print i, arr[i] } } } ' 1.csv 2.csv 3.csv

# 3  
Old 10-19-2009
Code:
nawk ' {
  a[$0]++
}
END{
  for ( i in a )
    if( a[i] >= 3)
      print i
} ' 1.csv 2.csv 3.csv

# 4  
Old 10-19-2009
Use gawk, nawk or /usr/xpg4/bin/awk on Solaris.

This should handle repeating values in the same file correctly and variable number of arguments.

Code:
awk 'END { 
  for (V in v) 
    if (v[V] >= ARGC - 1)
      print V
    }
{ v[$0] += ++f[FILENAME,$0] }
' *csv 

Edit: wrong result, see below.

Last edited by radoulov; 10-20-2009 at 05:26 AM..
# 5  
Old 10-19-2009
Awk solution is ok except that if each file contains 'abc' more than once.

So below complicated one should be ok :

Code:
cat <(sort a|uniq) <(sort b | uniq) <(sort c |uniq) | sort | uniq -c |awk '$1==3 {print $2}'

Or try below perl script:
Code:
sub fun
{
  open FH,"<$_[0]";
  while(<FH>){
   chomp;
   $hash{$_}->{$_[0]}=1;
  }
  close FH;
}
fun('a.txt');
fun('b.txt');
fun('c.txt');
print join "\n", grep {(keys %{$hash{$_}})==3} keys %hash;

# 6  
Old 10-20-2009
assuming files are sorted beforehand
Code:
paste -d";" 1.csv 2.csv 3.csv | while IFS=";" read a b c
do 
   if [ "$a" == "$b" -a "$b" == "$c" ];then  
       echo "$a"
   fi
done

or bash
Code:
exec 4<"1.csv"
exec 5<"2.csv"
exec 6<"3.csv"
while read line1 <&4
do
    read line2 <&5
    read line3 <&6    
    if [ "$line1" == "$line2" -a "$line2" == "$line3" ];then
        echo "$line1 is common"
    fi
done
exec >&4-
exec >&5-
exec >&6-


Last edited by ghostdog74; 10-20-2009 at 12:25 AM..
# 7  
Old 10-20-2009
This awk script will find common strings among an arbitrary number of files and does not get bothered by multiple instances of strings in individual files.

Code:
# This inits the file counter
BEGIN {
        f_num=0
}

# This increments the file counter when we start an input file
(FNR == 1) {
  f_num++
}

# This clause says to add an entry to the table of strings if this is the first file and it is not in the table.
# The value of the entry is the number of the file it was found (always 1 here)
# We also skip the rest of the clauses in the script and go to the next line
(NR == FNR) && !($0 in a) {
        a[$0] = f_num
        next
}

# This clause is executed if the string is in the table and in the last file.  It really means all previous files....
# If it was, we mark as being in this file.  If a string is not in the current file, its value will never be updated again.
($0 in a) && (a[$0] == f_num - 1){
        a[$0] = f_num
}

# Only print the entries that were in all the files (have a value of f_num)
END {
        for (i in a)
                if( a[i] == f_num)
                print i
}

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Comparing two files and list the difference with common first line content of both files

I have two file as given below which shows the ACL permissions of each file. I need to compare the source file with target file and list down the difference as specified below in required output. Can someone help me on this ? Source File ************* # file: /local/test_1 # owner: own #... (4 Replies)
Discussion started by: sarathy_a35
4 Replies

2. Shell Programming and Scripting

awk common between files

Hello there: I want to find common among files. They all have one column. Format for data: CEU_snp_CHR21.txt 21:10758305 21:10827533 21:10913441 21:10920098 21:10952160 21:10966322 21:10985991 NAT_CHR21_variants.txt 21:10971951 (3 Replies)
Discussion started by: genome
3 Replies

3. Shell Programming and Scripting

Common file from two files

Need Help file1 A.txt B.txt file2 A.txt B.txt C.txt output: Similar file in two files A.txt B.txt (1 Reply)
Discussion started by: satish1222
1 Replies

4. Shell Programming and Scripting

Compare multiple files, and extract items that are common to ALL files only

I have this code awk 'NR==FNR{a=$1;next} a' file1 file2 which does what I need it to do, but for only two files. I want to make it so that I can have multiple files (for example 30) and the code will return only the items that are in every single one of those files and ignore the ones... (7 Replies)
Discussion started by: castrojc
7 Replies

5. UNIX for Dummies Questions & Answers

A directory of common files from two others

Hi all, that' my problem dir1: fileA fileB fileC dir2: fileA fileC I need a command in shell to obtain a dir with common files: dir_common: fileA fileC thank u all (6 Replies)
Discussion started by: pilloso
6 Replies

6. Shell Programming and Scripting

Common lines from files

Hello guys, I need a script to get the common lines from two files with a criteria that if the first two columns match then I keep the maximum value of the 5th column.(tab separated columns) . 3rd and 4th columns corresponds to the row which has highest value for the 5th column. Sample... (2 Replies)
Discussion started by: jaysean
2 Replies

7. Shell Programming and Scripting

Common lines from files

Hello guys, I need a script to get the common lines from two files with a criteria that if the first two columns match then I keep the maximum value of the 3rd column.(tab separated columns) Sample input: file1: 111 222 0.1 333 444 0.5 555 666 0.4 file 2: 111 222 0.7 555 666... (5 Replies)
Discussion started by: jaysean
5 Replies

8. Shell Programming and Scripting

common contents of two files

I have two files: file a with contents 1 2 3 4 5 file b with contents 6 3 5 8 9 10 i want go get file c which has the common contents of both files so file c should have contents 3 5 (9 Replies)
Discussion started by: tomjones
9 Replies

9. Shell Programming and Scripting

Merge files of differrent size with one field common in both files using awk

hi, i am facing a problem in merging two files using awk, the problem is as stated below, file1: A|B|C|D|E|F|G|H|I|1 M|N|O|P|Q|R|S|T|U|2 AA|BB|CC|DD|EE|FF|GG|HH|II|1 .... .... .... file2 : 1|Mn|op|qr (2 Replies)
Discussion started by: shashi1982
2 Replies

10. UNIX for Dummies Questions & Answers

Get un common numbers from two files

Hi, I have two files: abc : 50040 123123 31703 cde: 104 97 50040 123123 31703 36609 50534 (3 Replies)
Discussion started by: jingi1234
3 Replies
Login or Register to Ask a Question