08-09-2007
Grep Line with Matching Fields
Below is the scenario. Help is appreciated.
File1: ( 500,000 lines ) : Three fields comma delimited : Not sorted
1234FAA,435612,88975
1224FAB,12345,212356
File2: ( 4,000,000 lines ) : Six fields comma delimited (Last 3 field should match the 3 fields of File1) : Not Sorted :
0123456abcd,12345,abcdef,1234FAA,435612,88975
0123456wxyz,11234,lmnopq,1224FAB,12345,212356
I need to grab all the six fields for file2 when there is a match of first 3 fields of file1 and last 3 fields of file2.
I wrote a small script but seems like it might take days to complete :-)
1 #!/bin/ksh
2
3 while read record
4 do
5 cat file2 | grep "$record" >> final.list
6 done < file1
Can someone help me with a faster solution?
Thanks in advance.
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi all,
I am trying to zgrep / grep list of files so that it displays only the matching filename:line number and does not display the whole line, like:
(echo "1.txt";echo "2.txt") | xargs zgrep -no STRING
If I use -o option, it displays the matching STRING and if not used, displays the... (3 Replies)
Discussion started by: vvaidyan
3 Replies
2. Shell Programming and Scripting
Dear experts,
I have a file1 that looks like
60127930928 2091
60129382039 2092
60126382937 2091
60128937928 2061
60127329389 2062
60123748730 2061
60128730293 2061
and file 2 that looks like
60127930928 2091
60129382039 2092
60126382937 2093
60128937928 2061
60127329389... (2 Replies)
Discussion started by: aismann
2 Replies
3. Shell Programming and Scripting
I have a file (key.dat) that contains two columns:
AA|1234|
BB|567|
CC|8910|
I have another file (extract.dat) that contains some data:
SD|458|John|Smith|
AA|3345|Frank|Williams|
AA|1234|Bill|Garner|
BD|0098|Yu|Lin|
BB|567|Gail|Hansen|
CC|8910|Ken|Nielsen|
I want to compare the... (5 Replies)
Discussion started by: ChicagoBlues
5 Replies
4. Shell Programming and Scripting
Hi all,
I want to display line number for matching string in a file. can anyone please help me.
I used
grep -n "ABC" file
so it displays
6 ABC.
But i only want to have line number,i don't want that it should prefix matching context with line number.
Actually my original... (10 Replies)
Discussion started by: sarbjit
10 Replies
5. Linux
Hi
I am having 2 fields and if f1=f2 i wanna print that line
eg
1 2
1 3
1 9
2 2
3 5
9 9
In the abov eg. the highlighted lines shud be printed
2 2
9 9
Thanking u (3 Replies)
Discussion started by: binnybio
3 Replies
6. Shell Programming and Scripting
Hello,
I have two files file 1 and file 2 each having result of a query on certain database tables and need to compare for Col1 in file1 with Col3 in file2, compare Col2 with Col4 and output the value of Col1 from File1 which is a) not present in Col3 of File2 b) value of Col2 is different from... (2 Replies)
Discussion started by: RasB15
2 Replies
7. Shell Programming and Scripting
Hi,
I have a log file that looks like this
"delete" : {
"_type" : "cl",
"_id" : "1000600000000562636",
"_version" : 1,
"status" : 200,
"found" : false
}
}, {
"delete" : {
"_type" : "cl",
"_id" : "1000600000000562643",
... (4 Replies)
Discussion started by: wahi80
4 Replies
8. Shell Programming and Scripting
In the below I am trying to use awk to match all the $13 values in input, which is tab-delimited,
that are in $1 of gene which is just a single column of text.
However only the line with the greatest $9 value in input needs to be printed.
So in the example below all the MECP2 and LTBP1... (0 Replies)
Discussion started by: cmccabe
0 Replies
9. UNIX for Beginners Questions & Answers
I have a file with a list of references towards the end and want to apply a grep for some string.
text ....
@unnumbered References
@sp 1
@paragraphindent 0
2017. @strong{Chalenski, D.A.}; Wang, K.; Tatanova, Maria; Lopez,
Jorge L.; Hatchell, P.; Dutta, P.; @strong{Small airgun... (1 Reply)
Discussion started by: kristinu
1 Replies
10. Shell Programming and Scripting
I have multiple (~80) files (some can be as big as 30GB of >1 billion of lines!) to grep on a pattern, and piped the match to a single file. I have a 96-core machine so that each grep job was sent to the background to speed up the search:
file1.tab
chr1A_part1 123241847 123241848... (6 Replies)
Discussion started by: yifangt
6 Replies
JOIN(1) General Commands Manual JOIN(1)
NAME
join - relational database operator
SYNOPSIS
join [-an] [-e s] [-o list] [-tc] file1 file2
DESCRIPTION
Join forms, on the standard output, a join of the two relations specified by the lines of file1 and file2. If file1 is `-', the standard
input is used.
File1 and file2 must be sorted in increasing ASCII collating sequence on the fields on which they are to be joined, normally the first in
each line.
There is one line in the output for each pair of lines in file1 and file2 that have identical join fields. The output line normally con-
sists of the common field, then the rest of the line from file1, then the rest of the line from file2.
Fields are normally separated by blank, tab or newline. In this case, multiple separators count as one, and leading separators are dis-
carded.
These options are recognized:
-an In addition to the normal output, produce a line for each unpairable line in file n, where n is 1 or 2.
-e s Replace empty output fields by string s.
-o list
Each output line comprises the fields specified in list, each element of which has the form n.m, where n is a file number and m is a
field number.
-tc Use character c as a separator (tab character). Every appearance of c in a line is significant.
SEE ALSO
sort(1), comm(1), awk(1).
BUGS
With default field separation, the collating sequence is that of sort -b; with -t, the sequence is that of a plain sort.
The conventions of join, sort, comm, uniq, look and awk(1) are wildly incongruous.
7th Edition April 29, 1985 JOIN(1)