Sponsored Content
Top Forums Shell Programming and Scripting How to delete lines of a text file based on another text file? Post 302777371 by prvnrk on Thursday 7th of March 2013 08:13:17 PM
Old 03-07-2013
How to delete lines of a text file based on another text file?

I have 2 TXT files with with 8 columns in them(tab separated). First file has 2000 entries whereas 2nd file has 300 entries.

The first file has ALL the lines of second file. Now I need to remove those 300 lines (which are in both files) from first file so that first file's line count become 1700.

I tried with sort and comm commands but it worked only on single-column files. As my files are multi column files, Its not working.

any help would highly be appreciated.

Thanks very much.
 

10 More Discussions You Might Find Interesting

1. Programming

Delete specific lines in a text file

Hi, experts, I would like to create a function that can calculate the total number of lines in a saved text file and delete specific lines in that particular file (I only want the last few lines). Hav anybody have the experience and giv me a hand in this? (9 Replies)
Discussion started by: dniz
9 Replies

2. Shell Programming and Scripting

Delete blocks of lines from text file

Hello, Hello Firends, I have file like below. I want to remove selected blocks say abc,pqr,lst. how can i remove those blocks from file. zone abc { blah blah blah } zone xyz { blah blah blah } zone pqr { blah blah blah } (4 Replies)
Discussion started by: nrbhole
4 Replies

3. Shell Programming and Scripting

Delete block of text in one file based on list in another file

Hi all I currently use the following in shell. #!/bin/sh while read LINE do perl -i -ne "$/ = ''; print if !m'Using archive: ${LINE}'ms;" "datafile" done < "listfile" NOTE the single quote delimiters in the expression. It's highly likely the 'LINE' may very well have characters in it... (3 Replies)
Discussion started by: Festus Hagen
3 Replies

4. Shell Programming and Scripting

Parallel delete based flag from text file

Hi, I need a unix shell script for this requirement and is URGENT My input text file contains A-1 B-1 C-1 D-2 E-2 F-3 G-3 H-3 I-3 J-4 K-4 L-5 My expected result should be: if flag is 1, it has to delete A, B, C if flag is 2, it has to delete D,E if flag is 3, it has to delete... (1 Reply)
Discussion started by: moses_a
1 Replies

5. UNIX for Dummies Questions & Answers

Delete vertical lines in an text file

Hi everybody! I need to delete several vertical lines in a huge text file. It should work like the example below. Delete the vertical lines 2 and 8. 123456789 masldfjla afsajfwel sajfljsaf safsarfrl sajfeljwq 1345679 msldfja asajfwl sjfljsf sfsarfl sjfeljq Is there a... (11 Replies)
Discussion started by: relaxo
11 Replies

6. Shell Programming and Scripting

looking for a script that will delete lines in a text file

it will grep for a line and then delete these line. how do i begin to write this script if theres no available one? (3 Replies)
Discussion started by: garfish
3 Replies

7. Shell Programming and Scripting

How to delete lines from text file?

hi guys, I have very large txt files (200GB) and just want to to delete the first two lines (headers). So far I used sed -i '1,2d' infile.txtbut this command always takes extremely long as it writes all again. Is there a better way to do it (ie just to delete the lines without writing all... (2 Replies)
Discussion started by: TuAd
2 Replies

8. UNIX for Dummies Questions & Answers

Extracting lines from a text file based on another text file with line numbers

Hi, I am trying to extract lines from a text file given a text file containing line numbers to be extracted from the first file. How do I go about doing this? Thanks! (1 Reply)
Discussion started by: evelibertine
1 Replies

9. UNIX for Dummies Questions & Answers

Delete records based on a text file from a text file

Hi Folks, I am a novice and need to build a script in bash. I have 2 text files data.txt file is big file, column 2 is the we need to search and delete in the output. The filter file contains the rows to be deleted. Data.txt state city zone Alabama Huntsville 4 California SanDiego 3... (3 Replies)
Discussion started by: tech_frk
3 Replies

10. Shell Programming and Scripting

Match text to lines in a file, iterate backwards until text or text substring matches, print to file

hi all, trying this using shell/bash with sed/awk/grep I have two files, one containing one column, the other containing multiple columns (comma delimited). file1.txt abc12345 def12345 ghi54321 ... file2.txt abc1,text1,texta abc,text2,textb def123,text3,textc gh,text4,textd... (6 Replies)
Discussion started by: shogun1970
6 Replies
PRUNEHISTORY(8) 					      System Manager's Manual						   PRUNEHISTORY(8)

NAME
prunehistory - remove file names from Usenet history file SYNOPSIS
prunehistory [ -f filename ] [ -p ] [ input ] DESCRIPTION
Prunehistory modifies the history(5) text file to ``remove'' a set of filenames from it. The filenames are removed by overwriting them with spaces, so that the size and position of any following entries does not change. Prunehistory reads the named input file, or standard input if no file is given. The input is taken as a set of lines. Blank lines and lines starting with a number sign (``#'') are ignored. All other lines are should consist of a Message-ID followed by zero or more file- names. The Messge-ID is used as the dbz(3) key to get an offset into the text file. If no filenames are mentioned on the input line, then all filenames in the text are ``removed.'' If any filenames are mentioned, they are converted into the history file notation. If they appear in the line for the specified Message-ID then they are removed. Since innd(8) only appends to the text file, prunehistory does not need to have any interaction with it. OPTIONS
-p Prunehistory will normally complain about lines that do not follow the correct format. If the ``-p'' flag is used, then the program will silently print any invalid lines on its standard output. (Blank lines and comment lines are also passed through.) This can be useful when prunehistory is used as a filter for other programs such as reap. -f The default name of the history file is /var/lib/news/history; to specify a different name, use the ``-f'' flag. EXAMPLES
It is a good idea to delete purged entries and rebuild the dbz database every so often by using a script like the following: ctlinnd throttle "Rebuilding history database" cd /var/lib/news awk 'NF > 2 { printf "%s %s %s", $1, $2, $3; for (i = 4; i <= NF; i++) printf " %s", $i; print " "; }' <history >history.n if makehistory -r -f history.n ; then mv history.n history mv history.n.pag history.pag mv history.n.dir history.dir else echo 'Problem rebuilding history; old file not replaced' fi ctlinnd go "Rebuilding history database" Note that this keeps no record of expired articles. HISTORY
Written by Rich $alz <rsalz@uunet.uu.net> for InterNetNews. This is revision 1.9, dated 1996/10/29. SEE ALSO
dbz(3), history(5), innd(8). PRUNEHISTORY(8)
All times are GMT -4. The time now is 05:06 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy