Thread Tools Search this Thread
# 1  


I am trying to exclude certain rows in File 1, based on File 2. i.e. in the example below, File 3 should be the output; i.e. the rows in File 2 are excluded from File 1. Any hints/suggestions would be highly appreciated.

Input File 1:

a	1	xxx
b	1	wje
c	5	ere
c	2	qqq
d	3	yyy

Input File 2:

b	1
c	5

Output File 3:

a	1	xxx
c	2	qqq
d	3	yyy


Last edited by Don Cragun; 10-24-2014 at 12:50 AM.. Reason: Add CODE tags.
# 2  
awk 'FNR==NR {a[$1$2]++; next} {if (! a[$1$2])print}' file2 file1 > file3

This User Gave Thanks to Aia For This Post:
# 3  
Thanks Aia!

It would be great if you could elaborate a bit on the structure and syntax of the code!
# 4  
Originally Posted by pc2001
Thanks Aia!

It would be great if you could elaborate a bit on the structure and syntax of the code!
FNR==NR : FNR and NR are built-in management variables in awk. FNR keeps a tally of the records read from the current file, NR keeps a tally of every record so far read from any files given.
If FNR is equal to NR, then we are dealing with the first file, in this case file2 and it execute the following part:
{a[$1$2]++; next} : That creates an array named a, which index is the combination of fields 1 and 2 and then skip to next line. This will happen only with first file given as argument

{if (! a[$1$2])print} : When the program gets here, it is dealing with the second file, and if it has not seen fields 1 and 2 before, it displays it.
This User Gave Thanks to Aia For This Post:
# 5  

It is best to use a subscript separator and not just glue the index fields together to avoid possible index creep:

For example, suppose that you have the fields A1 and 1 and also the the fields A and 11. They would both produce the index "A11", which would corrupt the results.

With the standard awk SUBSEP subscript separator , (character 034 by default) this can be avoided:
awk 'NR==FNR{A[$1,$2]; next} !(($1,$2) in A)' file2 file1

Last edited by Scrutinizer; 10-24-2014 at 06:57 AM..
These 2 Users Gave Thanks to Scrutinizer For This Post:
# 6  
So, to get things straight:

NR==FNR{A[$1,$2]; next}

- this reads file 2 and creates an array for each row in file 2

!(($1,$2) in A)

- after file 2 has been read, file 1 is read and output is printed only if the fields are not in the array A.

Is this correct? Does file 2 get read before file 1?

thanks! This has been very instructive.
# 7  
You are welcome... Correct, and yes file 2 gets read before file 1.

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #704
Difficulty: Medium
Under the same-origin policy, a web browser permits scripts contained in a first web page to access data in a second web page, but only if both web pages have the same origin.
True or False?

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Join, merge, fill NULL the void columns of multiples files like sql "LEFT JOIN" by using awk

Hello, This post is already here but want to do this with another way Merge multiples files with multiples duplicates keys by filling "NULL" the void columns for anothers joinning files file1.csv: 1|abc 1|def 2|ghi 2|jkl 3|mno 3|pqr file2.csv: 1|123|jojo 1|NULL|bibi... (2 Replies)
Discussion started by: yjacknewton
2 Replies

2. UNIX for Dummies Questions & Answers

How to use the the join command to join multiple files by a common column

Hi, I have 20 tab delimited text files that have a common column (column 1). The files are named GSM1.txt through GSM20.txt. Each file has 3 columns (2 other columns in addition to the first common column). I want to write a script to join the files by the first common column so that in the... (5 Replies)
Discussion started by: evelibertine
5 Replies

3. UNIX for Dummies Questions & Answers

how to join two files using "Join" command with one common field in this problem?

file1: Toronto:12439755:1076359:July 1, 1867:6 Quebec City:7560592:1542056:July 1, 1867:5 Halifax:938134:55284:July 1, 1867:4 Fredericton:751400:72908:July 1, 1867:3 Winnipeg:1170300:647797:July 15, 1870:7 Victoria:4168123:944735:July 20, 1871:10 Charlottetown:137900:5660:July 1, 1873:2... (2 Replies)
Discussion started by: mindfreak
2 Replies

4. Shell Programming and Scripting

Join help

im trying to join 3 files together sort -n ${ASS1_DATA_DIR}Prac1 | sed '/^#/d' > ${ASS1_OUTPUT_DIR}Prac1temp sort -n ${ASS1_DATA_DIR}Prac2 | sed '/^#/d' > ${ASS1_OUTPUT_DIR}Prac2temp join -a1 -a2 -1 1 -2 1 -o "0 1.2 2.2" -e " " ${ASS1_OUTPUT_DIR}Prac1temp ${ASS1_OUTPUT_DIR}Prac2temp >... (6 Replies)
Discussion started by: bigubosu
6 Replies

5. UNIX for Dummies Questions & Answers


Hi.. Just a question: Let's say we have three files: test1.txt --------- 11111111 6 12121212 7 12345678 4 11112222 8 test2.txt --------- 11111111 5 12345678 6 11112222 6 test3.txt --------- 12345678 6 12121212 8 (3 Replies)
Discussion started by: nelsonandwee
3 Replies

6. UNIX for Dummies Questions & Answers

Join 2 files with multiple columns: awk/grep/join?

Hello, My apologies if this has been posted elsewhere, I have had a look at several threads but I am still confused how to use these functions. I have two files, each with 5 columns: File A: (tab-delimited) PDB CHAIN Start End Fragment 1avq A 171 176 awyfan 1avq A 172 177 wyfany 1c7k A 2 7... (3 Replies)
Discussion started by: InfoSeeker
3 Replies

7. Shell Programming and Scripting


joining two files: File1: ----- 1|M 2|M 3|F File2: ----- 1|abc|def 3|xyz|pqr join -t '|' 1.txt 2.txt gives(Itried many other ways) 1|M|abc|def 3|F|xyz|pqr but I need the result like following (14 Replies)
Discussion started by: greenworld
14 Replies

8. Programming

sql,multiple join,outer join issue

example sql: select a.a1,b.b1,c.c1,d.d1,e.e1 from a left outer join b on a.x=b.x left outer join c on b.y=c.y left outer join d on d.z=a.z inner join a.t=e.t I know how single outer or inner join works in sql. But I don't really understand when there are multiple of them. can... (0 Replies)
Discussion started by: robbiezr
0 Replies

9. Shell Programming and Scripting

join (pls help on join command)

Hi, I am a new learner of join command. Some result really make me confused. Please kindly help me. input: file1: LEO oracle engineer 210375 P.Jones Office Runner ID897 L.Clip Personl Chief ID982 S.Round UNIX admin ID6 file2: Dept2C ID897 6 years Dept5Z ID982 1 year Dept3S ID6 2... (1 Reply)
Discussion started by: summer_cherry
1 Replies

Featured Tech Videos