Visit Our UNIX and Linux User Community


Strings from one file which exactly match to the 1st column of other file and then print lines.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Strings from one file which exactly match to the 1st column of other file and then print lines.
# 1  
Old 08-19-2010
Strings from one file which exactly match to the 1st column of other file and then print lines.

Hi,
I have two files. 1st file has 1 column (huge file containing ~19200000 lines) and 2nd file has 2 columns (small file containing ~6000 lines).
#################################
huge_file.txt
Code:
a
a
ab
b

##################################
small_file.txt
Code:
a       1.5
b       2.5
ab      7.5

###################################
Script I am using :
Code:
BEGIN { cnt=0;
  while ( getline line < "small_file.txt" > 0 )
      n[++cnt]=line
}
{
for(i=1; i<=cnt; i++)
   if(match(n[i],$0) > 0)
      print n[i]
}

####################################
and output I am having :
Code:
a       1.5
ab      7.5
a       1.5
ab      7.5
ab      7.5
b       2.5
ab      7.5

####################################
But the desired output is :
Code:
a     1.5
a     1.5
ab     7.5
b     2.5

#####################################


I understand that I am getting this output since I am using MATCH function here. But I want exact matches only. can I use some regular expression around $0 or any other better way.


Any help would be highly appreciated.


Moderator's Comments:
Mod Comment Please use code tags, thank you

Last edited by Franklin52; 08-19-2010 at 06:27 AM..
# 2  
Old 08-19-2010
Hope this is what you were expecting

Code:
$ cat huge_file.txt
a
a
ab
b
hh
$ cat small_file.txt
a 1.5
b 2.5
ab 7.5
cd 1.1
$ awk 'NR==FNR{buff[$1]=$0;next} {if(buff[$1]!="") print buff[$1]}' small_file.txt huge_file.txt
a 1.5
a 1.5
ab 7.5
b 2.5
$

This User Gave Thanks to ranjithpr For This Post:
# 3  
Old 08-19-2010
Code:
awk 'NR==FNR{a[$1]=$2;next}{print $1,a[$1]}' small_file.txt huge_file.txt

This User Gave Thanks to rdcwayx For This Post:
# 4  
Old 08-19-2010
ranjithpr and rdcwayx, thanks to both of you. Its running very fast and fine. I am wondering if you could spare few minutes and explain me the logic. Once again, many many thanks for saving my day.
# 5  
Old 08-19-2010
Code:
awk 'NR==FNR{a[$1]=$2;next}         # save first file "small_file.txt" into array a. column 1 as array index, column 2 as the array value. 
{print $1,a[$1]}' small_file.txt huge_file.txt    # print each line of huge_file.txt, and the related value in array a

This User Gave Thanks to rdcwayx For This Post:

Previous Thread | Next Thread
Test Your Knowledge in Computers #199
Difficulty: Easy
The TIOBE Index for October 2019 proclaimed that Python is becoming the new big language in favor of languages such as PHP and Perl.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Compare 1st column from 2 file and if match print line from 1st file and append column 7 from 2nd

hi I have 2 file with more than 10 columns for both 1st file apple,0,0,0...... orange,1,2,3..... mango,2,4,5..... 2nd file apple,2,3,4,5,6,7... orange,2,3,4,5,6,8... watermerlon,2,3,4,5,6,abc... mango,5,6,7,4,6,def.... (1 Reply)
Discussion started by: tententen
1 Replies

2. UNIX for Beginners Questions & Answers

Use strings from nth field from one file to match strings in entire line in another file, awk

I cannot seem to get what should be a simple awk one-liner to work correctly and cannot figure out why. I would like to use patterns from a specific field in one file as regex to search for matching strings in the entire line ($0) of another file. I would like to output the lines of File2 which... (1 Reply)
Discussion started by: jvoot
1 Replies

3. UNIX for Beginners Questions & Answers

Match Strings between two files, print portions of each file together when matched ([g]awk)

I have two files and desire to use the strings from $1 of file 1 (file1.txt) as search criteria to find matches in $2 of file 2 (file2.txt). If matches are found I want to output the entire line of file 2 (file2.txt) followed by fields $2-$11 of file 1 (file1.txt). I can find the matches, I cannot... (7 Replies)
Discussion started by: jvoot
7 Replies

4. Shell Programming and Scripting

Input file needs to match a column and print the entire line

I have a file with class c IP addresses that I need to match to a column and print the matching lines of another file. I started playing with grep -if file01.out file02.out but I am stuck as to how to match it to a column and print the matching lines; cat file01.out 10.150.140... (5 Replies)
Discussion started by: lewk
5 Replies

5. Shell Programming and Scripting

awk Print New Column For Every Two Lines and Match On Multiple Column Values to print another column

Hi, My input files is like this axis1 0 1 10 axis2 0 1 5 axis1 1 2 -4 axis2 2 3 -3 axis1 3 4 5 axis2 3 4 -1 axis1 4 5 -6 axis2 4 5 1 Now, these are my following tasks 1. Print a first column for every two rows that has the same value followed by a string. 2. Match on the... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

6. Shell Programming and Scripting

Print only lines where fields concatenated match strings

Hello everyone, Maybe somebody could help me with an awk script. I have this input (field separator is comma ","): 547894982,M|N|J,U|Q|P,98,101,0,1,1 234900027,M|N|J,U|Q|P,98,101,0,1,1 234900023,M|N|J,U|Q|P,98,54,3,1,1 234900028,M|H|J,S|Q|P,98,101,0,1,1 234900030,M|N|J,U|F|P,98,101,0,1,1... (2 Replies)
Discussion started by: Ophiuchus
2 Replies

7. Shell Programming and Scripting

awk strings search + print next column after match

Hi, I have a file filled with search strings which have a blank in between and look like this: S. g. Ehr. o. Jg. v. d. Chijs g. Ehr. Now i would like to search for the strings and it also shall return the next column after the match. awk -v FILE="search_strings.txt" 'BEGIN {... (10 Replies)
Discussion started by: sdf
10 Replies

8. Shell Programming and Scripting

print when column match with other file

Hello all, please help. There are two file like this: file1: 1197510.0 294777.7 9666973.0 21.6 1839.8 1197510.0 294777.7 9666973.0 413.2 2075.9 1197510.0 294777.7 9666973.0 689.3 2260.0 ... (1 Reply)
Discussion started by: attila
1 Replies

9. Shell Programming and Scripting

print column value after exact match of variables in file

I have file like below summit hvar_rgrpd_10d_hvams17_ _kgr_prod.rec checksum checksum us europe summit hvar_rgrpd_10d_hvams17_ _kgr_prod.xml var summit us ... (5 Replies)
Discussion started by: manas_ranjan
5 Replies

10. Shell Programming and Scripting

Match a line in File 1 with Column in File 2 and print whole line in file 2 when matched

Hi Experts, I am very new to scripting and have a prb since few days and it is urgent to solve so much appreciated if u help me. i have 2 files file1.txt 9647810043118 9647810043126 9647810043155 9647810043161 9647810043166 9647810043185 9647810043200 9647810043203 9647810043250... (22 Replies)
Discussion started by: mustafa.abdulsa
22 Replies

Featured Tech Videos