Visit Our UNIX and Linux User Community

Delete records based on a text file from a text file

Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Delete records based on a text file from a text file
# 1  
Old 10-03-2014
Delete records based on a text file from a text file

Hi Folks,

I am a novice and need to build a script in bash. I have 2 text files data.txt file is big file, column 2 is the we need to search and delete in the output. The filter file contains the rows to be deleted.

state 	city	zone
Alabama	Huntsville	4
California	SanDiego	3
Arizona	Tuscon	5
Texas	Dallas	7
Texas	Austin	9
California	SanJose	1
California	Bakersfield	3


state 	city	zone
Alabama	Huntsville	4
Texas	Dallas	7
California	SanJose	1
California	Bakersfield	3

Thanks appreciate your help.

Last edited by rbatte1; 10-03-2014 at 12:54 PM.. Reason: Capital letters
# 2  
Old 10-03-2014
awk 'FNR==NR{A[$1];next}FNR==1 || !($2 in A)' Filter.txt Data.txt

# 3  
Old 10-03-2014
This should work (except for Washington Smilie )

grep -v -f Filter.txt Data.txt

# 4  
Old 10-03-2014
If states like New York are entered in the 1st field without spaces, Akshay's code would work correctly; whether or not there are any spaces in the 1st field, the following minor change should work:
awk -F'\t' 'FNR==NR{A[$1];next}FNR==1 || !($2 in A)' Filter.txt Data.txt


Previous Thread | Next Thread
Test Your Knowledge in Computers #515
Difficulty: Easy
A variable's type (or data type) is the characterization of the data that it represents.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Match text to lines in a file, iterate backwards until text or text substring matches, print to file

hi all, trying this using shell/bash with sed/awk/grep I have two files, one containing one column, the other containing multiple columns (comma delimited). file1.txt abc12345 def12345 ghi54321 ... file2.txt abc1,text1,texta abc,text2,textb def123,text3,textc gh,text4,textd... (6 Replies)
Discussion started by: shogun1970
6 Replies

2. Shell Programming and Scripting

Delete the records in file based on lookup file.

Hi I have two files one.txt and two.txt one.txt 123 324 456 235 456 two txt abc one 000 123 abc abc one 000 456 abc abc one 000 122 abc abc one 000 111 abc My question here is, the records which are present in one.txt has to deleted in second file two.txt my output result... (2 Replies)
Discussion started by: Ganesh L
2 Replies

3. UNIX for Dummies Questions & Answers

Filter records in a huge text file from a filter text file

Hi Folks, I have a text file with lots of rows with duplicates in the first column, i want to filter out records based on filter columns in a different filter text file. bash scripting is what i need. Data.txt Name OrderID Quantity Sam 123 300 Jay 342 498 Kev 78 2500 Sam 420 50 Vic 10... (3 Replies)
Discussion started by: tech_frk
3 Replies

4. Shell Programming and Scripting

Splitting records in a text file based on delimiter

A text file has 2 fields (Data, Filename) delimited by # as below, Data,Filename Row1 -> abc#Test1.xml Row2 -> xyz#Test2.xml Row3 -> ghi#Test3.xml The content in first field has to be written into a file where filename should be considered from second field. So from... (4 Replies)
Discussion started by: jayakkannan
4 Replies

5. Shell Programming and Scripting

How to delete lines of a text file based on another text file?

I have 2 TXT files with with 8 columns in them(tab separated). First file has 2000 entries whereas 2nd file has 300 entries. The first file has ALL the lines of second file. Now I need to remove those 300 lines (which are in both files) from first file so that first file's line count become... (2 Replies)
Discussion started by: prvnrk
2 Replies

6. UNIX for Dummies Questions & Answers

Extracting lines from a text file based on another text file with line numbers

Hi, I am trying to extract lines from a text file given a text file containing line numbers to be extracted from the first file. How do I go about doing this? Thanks! (1 Reply)
Discussion started by: evelibertine
1 Replies

7. UNIX for Dummies Questions & Answers

Delete records from a big file based on some condition

Hi, To load a big file in a table,I have a make sure that all rows in the file has same number of the columns . So in my file if I am getting any rows which have columns not equal to 6 , I need to delete it . Delimiter is space and columns are optionally enclosed by "". This can be ... (1 Reply)
Discussion started by: hemantraijain
1 Replies

8. Shell Programming and Scripting

Parallel delete based flag from text file

Hi, I need a unix shell script for this requirement and is URGENT My input text file contains A-1 B-1 C-1 D-2 E-2 F-3 G-3 H-3 I-3 J-4 K-4 L-5 My expected result should be: if flag is 1, it has to delete A, B, C if flag is 2, it has to delete D,E if flag is 3, it has to delete... (1 Reply)
Discussion started by: moses_a
1 Replies

9. Shell Programming and Scripting

Delete block of text in one file based on list in another file

Hi all I currently use the following in shell. #!/bin/sh while read LINE do perl -i -ne "$/ = ''; print if !m'Using archive: ${LINE}'ms;" "datafile" done < "listfile" NOTE the single quote delimiters in the expression. It's highly likely the 'LINE' may very well have characters in it... (3 Replies)
Discussion started by: Festus Hagen
3 Replies

10. Shell Programming and Scripting

Bash script to delete folder based on text file information

I have been working on a script to list all the name's of a subfolder in a text file then edit that text file and then delete the subfolder base on the edited text file so far I have been able to do every thing I just talked about but can't figure out how to delete the subfolers base on a text file... (8 Replies)
Discussion started by: bone11409
8 Replies

Featured Tech Videos