How to delete lines of a text file based on another text file?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to delete lines of a text file based on another text file?
# 1  
Old 03-07-2013
How to delete lines of a text file based on another text file?

I have 2 TXT files with with 8 columns in them(tab separated). First file has 2000 entries whereas 2nd file has 300 entries.

The first file has ALL the lines of second file. Now I need to remove those 300 lines (which are in both files) from first file so that first file's line count become 1700.

I tried with sort and comm commands but it worked only on single-column files. As my files are multi column files, Its not working.

any help would highly be appreciated.

Thanks very much.
# 2  
Old 03-07-2013
Code:
awk 'FILENAME=="file2" {arr[$0]++}
       FILENAME=="file1"  {if($0 in arr) {next}; print $0 }' file2 file1 > outputfile

# 3  
Old 03-07-2013
I would simply:
Code:
grep -v -x -f 300_file 2000_file > 1700_file

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Match text to lines in a file, iterate backwards until text or text substring matches, print to file

hi all, trying this using shell/bash with sed/awk/grep I have two files, one containing one column, the other containing multiple columns (comma delimited). file1.txt abc12345 def12345 ghi54321 ... file2.txt abc1,text1,texta abc,text2,textb def123,text3,textc gh,text4,textd... (6 Replies)
Discussion started by: shogun1970
6 Replies

2. UNIX for Dummies Questions & Answers

Delete records based on a text file from a text file

Hi Folks, I am a novice and need to build a script in bash. I have 2 text files data.txt file is big file, column 2 is the we need to search and delete in the output. The filter file contains the rows to be deleted. Data.txt state city zone Alabama Huntsville 4 California SanDiego 3... (3 Replies)
Discussion started by: tech_frk
3 Replies

3. UNIX for Dummies Questions & Answers

Extracting lines from a text file based on another text file with line numbers

Hi, I am trying to extract lines from a text file given a text file containing line numbers to be extracted from the first file. How do I go about doing this? Thanks! (1 Reply)
Discussion started by: evelibertine
1 Replies

4. Shell Programming and Scripting

How to delete lines from text file?

hi guys, I have very large txt files (200GB) and just want to to delete the first two lines (headers). So far I used sed -i '1,2d' infile.txtbut this command always takes extremely long as it writes all again. Is there a better way to do it (ie just to delete the lines without writing all... (2 Replies)
Discussion started by: TuAd
2 Replies

5. Shell Programming and Scripting

looking for a script that will delete lines in a text file

it will grep for a line and then delete these line. how do i begin to write this script if theres no available one? (3 Replies)
Discussion started by: garfish
3 Replies

6. UNIX for Dummies Questions & Answers

Delete vertical lines in an text file

Hi everybody! I need to delete several vertical lines in a huge text file. It should work like the example below. Delete the vertical lines 2 and 8. 123456789 masldfjla afsajfwel sajfljsaf safsarfrl sajfeljwq 1345679 msldfja asajfwl sjfljsf sfsarfl sjfeljq Is there a... (11 Replies)
Discussion started by: relaxo
11 Replies

7. Shell Programming and Scripting

Parallel delete based flag from text file

Hi, I need a unix shell script for this requirement and is URGENT My input text file contains A-1 B-1 C-1 D-2 E-2 F-3 G-3 H-3 I-3 J-4 K-4 L-5 My expected result should be: if flag is 1, it has to delete A, B, C if flag is 2, it has to delete D,E if flag is 3, it has to delete... (1 Reply)
Discussion started by: moses_a
1 Replies

8. Shell Programming and Scripting

Delete block of text in one file based on list in another file

Hi all I currently use the following in shell. #!/bin/sh while read LINE do perl -i -ne "$/ = ''; print if !m'Using archive: ${LINE}'ms;" "datafile" done < "listfile" NOTE the single quote delimiters in the expression. It's highly likely the 'LINE' may very well have characters in it... (3 Replies)
Discussion started by: Festus Hagen
3 Replies

9. Shell Programming and Scripting

Delete blocks of lines from text file

Hello, Hello Firends, I have file like below. I want to remove selected blocks say abc,pqr,lst. how can i remove those blocks from file. zone abc { blah blah blah } zone xyz { blah blah blah } zone pqr { blah blah blah } (4 Replies)
Discussion started by: nrbhole
4 Replies

10. Programming

Delete specific lines in a text file

Hi, experts, I would like to create a function that can calculate the total number of lines in a saved text file and delete specific lines in that particular file (I only want the last few lines). Hav anybody have the experience and giv me a hand in this? (9 Replies)
Discussion started by: dniz
9 Replies
Login or Register to Ask a Question