Visit Our UNIX and Linux User Community

finding common numbers (contents) across 2 or 3 files

Thread Tools Search this Thread
Top Forums Shell Programming and Scripting finding common numbers (contents) across 2 or 3 files
# 1  
Old 05-25-2011
finding common numbers (contents) across 2 or 3 files

I have 3 files which are tab delimited and have numbers in it.

file 1

File 2

File 3

The contents in each file is sorted from lowest to highest

I want to compare each of the files together and get the common numbers across 3 files

Resultant file

My files have large amount numbers and it would be great if I could do this comparison using awk or sed in such a way that I could compare as many files I want and out put the common number to another file

awk' comand file 1 file outputfile
awk' command file 1 file 2 file 3 outputfile

Please let me know.
# 2  
Old 05-25-2011
x=`cat file_1`; for i in file_2 file_3; do x=`comm -12 $i <(echo "$x")`; done; echo "$x"

# 3  
Old 05-25-2011
assuming all the numbers are unique within a given file:
nawk '{a[$0]++} END{for (i in a) if (a[i]==ARGC-1) print i}' file1 file2 file3 ... fileN

# 4  
Old 05-27-2011
Thanks. It worked.

I would like to know how to modify the above awk comand if I have more than one column in the above files but still find the common numbers only in column 1 across all files.
For example: Column 1 for file 1 file 2 file3 are the same as before but now file 1 has 4 columns, file 2 has 3 columns and file 3 has 1 column.

Please let me know
# 5  
Old 05-27-2011
vgersh's code modified to find the first column, regardless of the number of columns.
nawk '{a[$1]++} END{for (i in a) if (a[i]==ARGC-1) print i}' file1 file2 file3 ... fileN

This User Gave Thanks to jim mcnamara For This Post:

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

If contents of A are in B then move the common contents to C

Hallo Team, I have 2 .csv files file A has 47600 lines and file B has 67000 lines FILEA SD0o9rb01-1d320ddbcc8d220f572739ebed5f58d1-v300g00 SD8bt0101-a0810bfe0e3396060126ec51b30dac0a-v300g00 SD05sce01-cb056af347ed4651f29eb3c3e9addbd6-v300g00... (3 Replies)
Discussion started by: kekanap
3 Replies

2. Shell Programming and Scripting

Finding most common substrings

Hello, I would like to know what is the three most abundant substrings of length 6 from col2. The file is quite large and looks like this col1 col2 EN03 typehellobyedogcatcatdog EN09 typehellobyebyebyebye EN08 dogcatcatdogbyebyebyebye EN09 catcattypehellobyebyebyebye... (9 Replies)
Discussion started by: verse123
9 Replies

3. Shell Programming and Scripting

Finding out the common lines in two files using 4 fields with the help of awk and UNIX

Dear All, I have 2 files. If field 1, 2, 4 and 5 matches in both file1 and file2, I want to print the whole line of file1 and file2 one after another in my output file. File1: sc2/80 20 . A T 86 F=5;U=4 sc2/60 55 . G T ... (1 Reply)
Discussion started by: NamS
1 Replies

4. UNIX for Dummies Questions & Answers

Find common numbers from two very large files using awk or the like

I've got two files that each contain a 16-digit number in positions 1-16. The first file has 63,120 entries all sorted numerically. The second file has 142,479 entries, also sorted numerically. I want to read through each file and output the entries that appear in both. So far I've had no... (13 Replies)
Discussion started by: Scottie1954
13 Replies

5. Shell Programming and Scripting

Find common numbers and print yes or no

Hi I have 2 files with following data First file, sp|Q676U5|A16L1_HUMAN, Autophagy-related protein 16-1 OS=Homo sapiens GN=ATG16L1 PE=1 SV=2, Maximum coiled-coil residue probability: 0.657 in position 163. Maximum dimeric residue probability: 0.288 in position 163. ... (1 Reply)
Discussion started by: manigrover
1 Replies

6. Shell Programming and Scripting

common contents of two files

I have two files: file a with contents 1 2 3 4 5 file b with contents 6 3 5 8 9 10 i want go get file c which has the common contents of both files so file c should have contents 3 5 (9 Replies)
Discussion started by: tomjones
9 Replies

7. UNIX for Dummies Questions & Answers

finding files with numbers in the file name???

Hello all, New to this forum! I got a Q: i want to find all files with numbers in the file name. e.g. or action i took: ls | egrep '+' ls | egrep ls | egrep + ls | egrep '' ls | egrep '(+)' ls | egrep '()' ls | egrep '(.*.*)' ls | egrep '.*.*' ls | grep... (2 Replies)
Discussion started by: RedGrinGo
2 Replies

8. Shell Programming and Scripting

Join file contents via common field

I have 2 files with a common parm - Jobname File 1 0507 1202 JOBA 0507 1302 JOBB 0507 1452 JOBC 0507 1552 JOBA 0507 1553 JOBA File2 JOBA abcdefg server4 JOBB defghij server22 JOBC vwxyz12 server55 I would like to take each line from File1 and match the jobname with the jobname... (8 Replies)
Discussion started by: Northerner
8 Replies

9. UNIX for Dummies Questions & Answers

Get un common numbers from two files

Hi, I have two files: abc : 50040 123123 31703 cde: 104 97 50040 123123 31703 36609 50534 (3 Replies)
Discussion started by: jingi1234
3 Replies

Featured Tech Videos