Visit The New, Modern Unix Linux Community

how to fgrep -f two files and get only one instance of each matched line

Thread Tools Search this Thread
Top Forums Shell Programming and Scripting how to fgrep -f two files and get only one instance of each matched line
# 1  
how to fgrep -f two files and get only one instance of each matched line


I have two files:

and not

fgrep -f file1 file2 gives me what I dont want


Last edited by smarones; 11-16-2011 at 06:20 AM..
# 2  
at the cost of performance:

while read searchstring ; do grep -m 1 ${searchstring} file2 ; done < file1

This User Gave Thanks to funksen For This Post:
# 3  
awk -F , 'NR==FNR{a[$1];next} $2 in a{print;delete a[$2]}' file1 file2

This User Gave Thanks to rdcwayx For This Post:
# 4  
Hi rdcwayx,
I think your solution is what I need however my lines are a bit more complicated than the example I gave and your solution is based on that which is my fault. But I was looking for something general and it seems like there is not something you can apply to everycase.

the expressions I want is more like this:

and I want what ever comes after the second equal sign before the last comma, in this case would be GG123.

Hi funksen,

I have thousands of lines not sure if while would be a good idea as you mentioned.

Last edited by smarones; 11-17-2011 at 05:10 AM..
# 5  
Try with this ..
$ fgrep -f file1 file2 | nawk -F'[=,]' '!x[$4]++'

# 6  
Hi jayan jay,

this is the error I get when running your solutions:

user> fgrep -f not_upgraded_sites.txt not_upgraded.sel | nawk -F'[=,]' '!x[$4]++'
x[: Event not found.

# 7  
Pls try with double quotes ..

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #996
Difficulty: Medium
Major Unix vendors, including Sun Microsystems, HP, IBM, and SGI, have been selling virtualized hardware since before 1990.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to print previous line of multiple pattern matched line?

Hello, I have below format log file, Comparing csv_converted_files/2201/9747.1012H67126.5077292103609547345.csv and csv_converted_files/22019/97447.1012H67126.5077292103609547345.csv Comparing csv_converted_files/2559/9447.1012H67126.5077292103609547345.csv and... (6 Replies)
Discussion started by: arvindshukla81
6 Replies

2. Shell Programming and Scripting

awk line instance counter

I Have a text file with several thousand lines of text. Occasionally there will be a "sysAlive" line of text (every so often) What would be an awk command to print every line of text, and to put in incrementing counter ONLY on the "sysAlive" lines For example: >cat file.txt lineAAA a b c d... (4 Replies)
Discussion started by: ajp7701
4 Replies

3. Shell Programming and Scripting

Help required on joining one line above & below to the pattern matched string line.

Hi Experts, Help needed on joining one line above & below to the pattern matched string line. The input file, required output is mentioned below Input file ABCD DEFG5 42.0.1-63.38.31 KKKK iokl IP Connection Available ABCD DEFG5 42.0.1-63.38.31 ... (7 Replies)
Discussion started by: krao
7 Replies

4. Shell Programming and Scripting

Using sed can you specify the last instance of a character on a line?

I was told a way to do this with awk earlier today but is there a way with sed to specify the last instance of a character on a line? You will know what character you're looking for but there could be none or one hundred instances of it on a line say and you ONLY want to specify the last one for... (3 Replies)
Discussion started by: Bashingaway
3 Replies

5. Shell Programming and Scripting

Replace all but skip first instance in a line

I have a record like the one given below. 010000306551~IN ~N~ |WINDWARD PK|Alpharetta| If ~ is present more than instance in a line,then I need to delete those instances. Any ideas? I am working in Solaris (7 Replies)
Discussion started by: prasperl
7 Replies

6. Shell Programming and Scripting

awk get matched line's previous line

hi everyone, a b in c d e f in g output is: a e so awk search for "in", then print out the matched line's previuos line. Please advice. (11 Replies)
Discussion started by: jimmy_y
11 Replies

7. Shell Programming and Scripting

Append instance count to each line

Hello forum, I need help with a script for displaying the number of instances/times a particular line appears in a tab-delimited file and append that number to the end of the line. Example text file: aaa bbb ccc ddd ggg hhh kkk nnn aaa bbb ccc ddd aaa bbb ccc ddd ppp qqq nnn sss ggg... (1 Reply)
Discussion started by: jaysean
1 Replies

8. Shell Programming and Scripting

awk script to move a line after the matched pattern line

I have the following text format in a file which lists the question first and then 5 choices after that the explanantion and finally the answer. 1.The amount of time it takes for most of a worker’s occupational knowledge and skills to become obsolete has been declining because of the... (2 Replies)
Discussion started by: nanchil_guy
2 Replies

9. Shell Programming and Scripting

replace first instance(not first instance in line)

Alright, I think I know what I am doing with sed(which probably means I don't). But I cant figure out how to replace just the first occurance of a string. I have tried sed, ed, and grep but can't seem to figure it out. If you have any suggestions I am open to anything! (3 Replies)
Discussion started by: IronHorse7
3 Replies

10. Shell Programming and Scripting

How to get a next line of a matched word?

Hi , If I match a word in a file, I want to pick the next line of that matched word. My file is a below format- The ntrag trace has auditError Line5005 contains transaction Ntrag data ------------ Here I wanted if I match a word auditError, I need to get the next line "Line5005... (10 Replies)
Discussion started by: Muktesh
10 Replies

Featured Tech Videos