find common elements in 2 files (for loop)


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting find common elements in 2 files (for loop)
# 1  
Old 10-28-2007
Bug find common elements in 2 files (for loop)

Hi,
i'm new here (and to scripting too).

I was hoping for some help in comparing two files.

i have a file called 'file1' with a list of names in the following format:
adam jones
paul higgins
kelly lowe


i also have another file which may contain some of the names but with a lot of other stuff, this is called 'file2'. it contains info such as:
aaa aaaa paul higgins aaaaa
aaaaa aaaaa aaaa aa aa aaa
aaaa aaaa kelly lowe aaaaa
asdsaad adsadsa asdas
assadasd
sa etc

(where aaaa is just random stuff html)

i want to find the common names in both files and produce an output in the form of:
0
1
1


this is with reference to the 'master' file (file1). ie. it has found in the 'file2' file that there are no adam jones, and 1 each of paul and kelly.

to do this task, i am using the following shell script however it does not seem to work:

#!/bin/sh

for i in file1

do

grep -c $i file2

done


it seems like quite a simple script, but i'm not sure why it does not work?(it only seems to grep for the first name, but does not loop through)

thanks for any helpSmilie
# 2  
Old 10-29-2007
Your code use "file1" as argument, not its content.
Try:

Quote:
while read line; do
grep -c "$line" file2;
done < file1
You will need to protect spaces in $line with quotation marks.
# 3  
Old 10-29-2007
thanks!
that worked perfectly
# 4  
Old 12-11-2007
How to read content from a file and match with another file for operation

Hi,

I have somehow similar case of question like the above with some differences.

Basically I have 2 files, file A and file B.

In file A, I have columns of fields such that:-

aaa 107
bbb 108
ccc 109

In file B, I have columns of fields such that:-
101 2 1
102 3 1
107 2 1
108 3 1
109 2 1

I would like to know, if I would like to extract let say first element of file A and compare with file B elements.

If found, I would like to have the position of element 107 in file B in this case it is on 3rd line. And the same goes for other elements in file A which in the end will pipe to another output file.

Next, I would like to get the $2 and $3 of the 3rd line for some computation.

Currently, I tried to do below:-

#!/bin/sh

for i in a.txt

do

grep -c $i b.txt

cat b.txt| awk '
{
count[$i]=$2+$3;

}
END{
printf("%d",count[$i]);
}'
done

I knew it is not going work well as the second field of A.txt is not extracted.

Please advise. Thanks.


-Jason
# 5  
Old 12-11-2007
Hi, ahjiefreak.

It's good that you posted your scripting effort. However:

1) new problems posted in old threads often get over-looked, and some people consider it rude,

2) you've seen requests to put code tags around scripts and data,

3) you didn't post the expected output,

4) it looks like homework

Best wishes ... cheers, drl
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find common files between two directories

I have two directories Dir 1 /home/sid/release1 Dir 2 /home/sid/release2 I want to find the common files between the two directories Dir 1 files /home/sid/release1>ls -lrt total 16 -rw-r--r-- 1 sid cool 0 Jun 19 12:53 File123 -rw-r--r-- 1 sid cool 0 Jun 19 12:53... (5 Replies)
Discussion started by: sidnow
5 Replies

2. Shell Programming and Scripting

Count common elements in a column

HI, I have a 3-column tab separated column (approx 1GB) in which I would like to count and output the frequency of all of the common elements in the 1st column. For instance: If my input was the following: dot is-big 2 dot is-round 3 dot is-gray 4 cat is-big 3 hot in-summer 5 My... (4 Replies)
Discussion started by: owwow14
4 Replies

3. UNIX for Dummies Questions & Answers

Merging tables: identifiying common and unique elements

Hi all, I know how to merge two tables and to remove the duplicated lines based on a field (Column 2) . My next challenge is to be able to identify in a new column those common elements between table A & B, those elements in table A not present in table B and vice versa. A simple count would be... (6 Replies)
Discussion started by: lsantome
6 Replies

4. Shell Programming and Scripting

Find Common Values Across Two Files

Hi All, I have two files like below: File1 MYFILE_28012012_1112.txt|4 MYFILE_28012012_1113.txt|51 MYFILE_28012012_1114.txt|57 MYFILE_28012012_1115.txt|57 MYFILE_28012012_1116.txt|57 MYFILE_28012012_1117.txt|57 File2 MYFILE_28012012_1110.txt|57 MYFILE_28012012_1111.txt|57... (2 Replies)
Discussion started by: angshuman
2 Replies

5. UNIX for Dummies Questions & Answers

Writing a loop to merge multiple files by common column

I have 100 data files labelled 250.1.txt through 250.100.txt. The second column of the data files partially match (there is about %90 overlap). Each data file has 4 columns. I want the merge all these text files by the matching values in the second column. In the output, the first column should... (1 Reply)
Discussion started by: evelibertine
1 Replies

6. Shell Programming and Scripting

Script to find NOT common strings in two files

Hi all, I'd like you to help or give any advise about the following: I have two (2) files, file1 and file2, both files have information common to each other. The contents of file1 is a subset of the contents of file2: file1: errormsgadmin esdp esgservices esignipa iprice ipvpn irm... (18 Replies)
Discussion started by: hnux
18 Replies

7. UNIX for Dummies Questions & Answers

how to find common words and take them out from two files

Hi, everyone, Let's say, we have xxx.txt A 1 2 3 4 5 C 1 2 3 4 5 E 1 2 3 4 5 yyy.txt A 1 2 3 4 5 B 1 2 3 4 5 C 1 2 3 4 5 D 1 2 3 4 5 E 1 2 3 4 5 First I match the first column I find intersection (A,C, E), then I want to take those lines with ACE out from yyy.txt, like A 1... (11 Replies)
Discussion started by: kaixinsjtu
11 Replies

8. Shell Programming and Scripting

Files common in two sets ??? How to find ??

Suppose we have 2 set of files set 1 set 2 ------ ------ abc hgb def ppp mgh vvv nmk sdf hgb ... (1 Reply)
Discussion started by: skyineyes
1 Replies

9. Shell Programming and Scripting

To find all common lines from 'n' no. of files

Hi, I have one situation. I have some 6-7 no. of files in one directory & I have to extract all the lines which exist in all these files. means I need to extract all common lines from all these files & put them in a separate file. Please help. I know it could be done with the help of... (11 Replies)
Discussion started by: The Observer
11 Replies
Login or Register to Ask a Question