Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: awk function to remove lines that contain contents of another file
Top Forums UNIX for Beginners Questions & Answers awk function to remove lines that contain contents of another file Post 303003691 by jim mcnamara on Tuesday 19th of September 2017 08:52:14 AM
Old 09-19-2017
This requires a lot of memory depending on what you have in reference.txt
Simple awk which can be rewritten as something difficult to read for non-awkers.
We have posters who do that, which is okay as long as you can get what they show you.
Code:
# code assumes that the reference.txt file has field #1 from inputfile

awk ' FILENAME=="reference.txt" {! arr[$0]++; next}  # create an array of values 
         FILENAME=="inputfile" { if(! $1 in arr) {print $0}; next} ' reference.txt inputfile > outputfile

This User Gave Thanks to jim mcnamara For This Post:
aberg
 

9 More Discussions You Might Find Interesting

1. Solaris

remove the contents of a file

Hi Let say a flat file contains 1000 lines. The cursor is at the 530 line number. Now I like to delete all the line at one ahot. how it can be done? (2 Replies)
Discussion started by: surjyap
2 Replies

2. Shell Programming and Scripting

Compare two files and remove all the contents of one file from another

Hi, I have two files, in which the second file has exactly the same contents of the first file with some additional records. Now, if I want to remove those matching lines from file2 and print only the extra contents which the first file does not have, I could use the below unsophisticated... (3 Replies)
Discussion started by: royalibrahim
3 Replies

3. Shell Programming and Scripting

Remove lines based on contents of another file

So, this issue is driving me nuts! I was hoping to get a lending hand here... I have 2 files: file1.txt contains: this is example1 this is example2 this is example3 this is example4 this is example5 file2.txt contains: example3 example5 Basically, I need a script or command to... (4 Replies)
Discussion started by: bashshadow1979
4 Replies

4. Shell Programming and Scripting

Perl script for Calling a function and writing all its contents to a file

I have a function which does awk proceessing sub mergeDescription { system (q@awk -F'~' ' NR == FNR { A = $1 B = $2 C = $0 next } { n = split ( C, V, "~" ) if... (3 Replies)
Discussion started by: crypto87
3 Replies

5. Shell Programming and Scripting

awk remove/grab lines from file with pattern from other file

Sorry for the weird title but i have the following problem. We have several files which have between 10000 and about 500000 lines in them. From these files we want to remove lines which contain a pattern which is located in another file (around 20000 lines, all EAN codes). We also want to get... (28 Replies)
Discussion started by: SDohmen
28 Replies

6. Shell Programming and Scripting

awk to remove lines in file if specific field matches

I am trying to remove lines in the target.txt file if $5 before the - in that file matches sorted_list. I have tried grep and awk. Thank you :). grep grep -v -F -f targets.bed sort_list grep -vFf sort_list targets awk awk -F, ' > FILENAME == ARGV {to_remove=1; next} > ! ($5 in... (2 Replies)
Discussion started by: cmccabe
2 Replies

7. Shell Programming and Scripting

Using awk to remove lines from file that match text

I am trying to remove each line in which $2 is FP or RFP. I believe the below will remove one instance but not both. Thank you :). file 12 123 FP 11 10 RFP awk awk -F'\t' ' $2 != "FP"' file desired output 12 11 (6 Replies)
Discussion started by: cmccabe
6 Replies

8. Shell Programming and Scripting

awk to remove lines that do not start with digit and combine line or lines

I have been searching and trying to come up with an awk that will perform the following on a converted text file (original is a pdf). 1. Since the first two lines are (begin with) text they are removed 2. if $1 is a number then all text is merged (combined) into one line until the next... (3 Replies)
Discussion started by: cmccabe
3 Replies

9. Shell Programming and Scripting

How to remove contents from file which are under bracket?

hello Friend, In hostgroup file, i have define lots of hostgroups. I need to remove few of them without manually editing file. Need script or syntax. I want to search particular on hostgroup_members and delete hostgoup defination of it. for example. define hostgroup{ hostgroup_name... (8 Replies)
Discussion started by: ghpradeep
8 Replies

LEARN ABOUT HPUX

fmt

fmt(1)							      General Commands Manual							    fmt(1)

NAME

       fmt - format text

SYNOPSIS

       width] [file...]

DESCRIPTION

       The  command is a simple text formatter that fills and joins lines to produce output lines of (up to) the number of characters specified in
       the width option.  The default width is 72.  concatenates the arguments.  If none are given, formats text from the standard input.

       Blank lines are preserved in the output, as is the spacing between words.  does not fill lines beginning with a	period	for  compatibility
       with Nor does it fill lines starting with

       Indentation is preserved in the output and input lines with differing indentation are not joined (unless is used).

       can also be used as an in-line text filter for the command:

       reformats the text between the cursor location and the end of the paragraph.

   Options
       recognizes the following options:

	      Crown margin mode.
		    Preserve the indentation of the first two lines within a paragraph and align the left margin of each subsequent line with that
		    of the second line.  This is useful for tagged paragraphs.

	      Split lines only.
		    Do not join short lines to form longer ones.  This prevents sample lines of code, and other such "formatted" text, from  being
		    unduly combined.

	      Fill output lines to up to
		    width columns.

WARNINGS

       The width option is acceptable for BSD compatibility, but it may go away in future releases.

SEE ALSO

       nroff(1), vi(1).

																	    fmt(1)
All times are GMT -4. The time now is 11:53 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy