Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: awk function to remove lines that contain contents of another file
Top Forums UNIX for Beginners Questions & Answers awk function to remove lines that contain contents of another file Post 303003691 by jim mcnamara on Tuesday 19th of September 2017 08:52:14 AM
Old 09-19-2017
This requires a lot of memory depending on what you have in reference.txt
Simple awk which can be rewritten as something difficult to read for non-awkers.
We have posters who do that, which is okay as long as you can get what they show you.
Code:
# code assumes that the reference.txt file has field #1 from inputfile

awk ' FILENAME=="reference.txt" {! arr[$0]++; next}  # create an array of values 
         FILENAME=="inputfile" { if(! $1 in arr) {print $0}; next} ' reference.txt inputfile > outputfile

This User Gave Thanks to jim mcnamara For This Post:
aberg
 

9 More Discussions You Might Find Interesting

1. Solaris

remove the contents of a file

Hi Let say a flat file contains 1000 lines. The cursor is at the 530 line number. Now I like to delete all the line at one ahot. how it can be done? (2 Replies)
Discussion started by: surjyap
2 Replies

2. Shell Programming and Scripting

Compare two files and remove all the contents of one file from another

Hi, I have two files, in which the second file has exactly the same contents of the first file with some additional records. Now, if I want to remove those matching lines from file2 and print only the extra contents which the first file does not have, I could use the below unsophisticated... (3 Replies)
Discussion started by: royalibrahim
3 Replies

3. Shell Programming and Scripting

Remove lines based on contents of another file

So, this issue is driving me nuts! I was hoping to get a lending hand here... I have 2 files: file1.txt contains: this is example1 this is example2 this is example3 this is example4 this is example5 file2.txt contains: example3 example5 Basically, I need a script or command to... (4 Replies)
Discussion started by: bashshadow1979
4 Replies

4. Shell Programming and Scripting

Perl script for Calling a function and writing all its contents to a file

I have a function which does awk proceessing sub mergeDescription { system (q@awk -F'~' ' NR == FNR { A = $1 B = $2 C = $0 next } { n = split ( C, V, "~" ) if... (3 Replies)
Discussion started by: crypto87
3 Replies

5. Shell Programming and Scripting

awk remove/grab lines from file with pattern from other file

Sorry for the weird title but i have the following problem. We have several files which have between 10000 and about 500000 lines in them. From these files we want to remove lines which contain a pattern which is located in another file (around 20000 lines, all EAN codes). We also want to get... (28 Replies)
Discussion started by: SDohmen
28 Replies

6. Shell Programming and Scripting

awk to remove lines in file if specific field matches

I am trying to remove lines in the target.txt file if $5 before the - in that file matches sorted_list. I have tried grep and awk. Thank you :). grep grep -v -F -f targets.bed sort_list grep -vFf sort_list targets awk awk -F, ' > FILENAME == ARGV {to_remove=1; next} > ! ($5 in... (2 Replies)
Discussion started by: cmccabe
2 Replies

7. Shell Programming and Scripting

Using awk to remove lines from file that match text

I am trying to remove each line in which $2 is FP or RFP. I believe the below will remove one instance but not both. Thank you :). file 12 123 FP 11 10 RFP awk awk -F'\t' ' $2 != "FP"' file desired output 12 11 (6 Replies)
Discussion started by: cmccabe
6 Replies

8. Shell Programming and Scripting

awk to remove lines that do not start with digit and combine line or lines

I have been searching and trying to come up with an awk that will perform the following on a converted text file (original is a pdf). 1. Since the first two lines are (begin with) text they are removed 2. if $1 is a number then all text is merged (combined) into one line until the next... (3 Replies)
Discussion started by: cmccabe
3 Replies

9. Shell Programming and Scripting

How to remove contents from file which are under bracket?

hello Friend, In hostgroup file, i have define lots of hostgroups. I need to remove few of them without manually editing file. Need script or syntax. I want to search particular on hostgroup_members and delete hostgoup defination of it. for example. define hostgroup{ hostgroup_name... (8 Replies)
Discussion started by: ghpradeep
8 Replies

LEARN ABOUT FREEBSD

colrm

COLRM(1)						    BSD General Commands Manual 						  COLRM(1)

NAME

     colrm -- remove columns from a file

SYNOPSIS

     colrm [start [stop]]

DESCRIPTION

     The colrm utility removes selected columns from the lines of a file.  A column is defined as a single character in a line.  Input is read
     from the standard input.  Output is written to the standard output.

     If only the start column is specified, columns numbered less than the start column will be written.  If both start and stop columns are spec-
     ified, columns numbered less than the start column or greater than the stop column will be written.  Column numbering starts with one, not
     zero.

     Tab characters increment the column count to the next multiple of eight.  Backspace characters decrement the column count by one.

ENVIRONMENT

     The LANG, LC_ALL and LC_CTYPE environment variables affect the execution of colrm as described in environ(7).

EXIT STATUS

     The colrm utility exits 0 on success, and >0 if an error occurs.

SEE ALSO

     awk(1), column(1), cut(1), paste(1)

HISTORY

     The colrm command appeared in 3.0BSD.

BSD
								  August 4, 2004							       BSD
All times are GMT -4. The time now is 07:17 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy