Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: awk function to remove lines that contain contents of another file
Top Forums UNIX for Beginners Questions & Answers awk function to remove lines that contain contents of another file Post 303003691 by jim mcnamara on Tuesday 19th of September 2017 08:52:14 AM
Old 09-19-2017
This requires a lot of memory depending on what you have in reference.txt
Simple awk which can be rewritten as something difficult to read for non-awkers.
We have posters who do that, which is okay as long as you can get what they show you.
Code:
# code assumes that the reference.txt file has field #1 from inputfile

awk ' FILENAME=="reference.txt" {! arr[$0]++; next}  # create an array of values 
         FILENAME=="inputfile" { if(! $1 in arr) {print $0}; next} ' reference.txt inputfile > outputfile

This User Gave Thanks to jim mcnamara For This Post:
aberg
 

9 More Discussions You Might Find Interesting

1. Solaris

remove the contents of a file

Hi Let say a flat file contains 1000 lines. The cursor is at the 530 line number. Now I like to delete all the line at one ahot. how it can be done? (2 Replies)
Discussion started by: surjyap
2 Replies

2. Shell Programming and Scripting

Compare two files and remove all the contents of one file from another

Hi, I have two files, in which the second file has exactly the same contents of the first file with some additional records. Now, if I want to remove those matching lines from file2 and print only the extra contents which the first file does not have, I could use the below unsophisticated... (3 Replies)
Discussion started by: royalibrahim
3 Replies

3. Shell Programming and Scripting

Remove lines based on contents of another file

So, this issue is driving me nuts! I was hoping to get a lending hand here... I have 2 files: file1.txt contains: this is example1 this is example2 this is example3 this is example4 this is example5 file2.txt contains: example3 example5 Basically, I need a script or command to... (4 Replies)
Discussion started by: bashshadow1979
4 Replies

4. Shell Programming and Scripting

Perl script for Calling a function and writing all its contents to a file

I have a function which does awk proceessing sub mergeDescription { system (q@awk -F'~' ' NR == FNR { A = $1 B = $2 C = $0 next } { n = split ( C, V, "~" ) if... (3 Replies)
Discussion started by: crypto87
3 Replies

5. Shell Programming and Scripting

awk remove/grab lines from file with pattern from other file

Sorry for the weird title but i have the following problem. We have several files which have between 10000 and about 500000 lines in them. From these files we want to remove lines which contain a pattern which is located in another file (around 20000 lines, all EAN codes). We also want to get... (28 Replies)
Discussion started by: SDohmen
28 Replies

6. Shell Programming and Scripting

awk to remove lines in file if specific field matches

I am trying to remove lines in the target.txt file if $5 before the - in that file matches sorted_list. I have tried grep and awk. Thank you :). grep grep -v -F -f targets.bed sort_list grep -vFf sort_list targets awk awk -F, ' > FILENAME == ARGV {to_remove=1; next} > ! ($5 in... (2 Replies)
Discussion started by: cmccabe
2 Replies

7. Shell Programming and Scripting

Using awk to remove lines from file that match text

I am trying to remove each line in which $2 is FP or RFP. I believe the below will remove one instance but not both. Thank you :). file 12 123 FP 11 10 RFP awk awk -F'\t' ' $2 != "FP"' file desired output 12 11 (6 Replies)
Discussion started by: cmccabe
6 Replies

8. Shell Programming and Scripting

awk to remove lines that do not start with digit and combine line or lines

I have been searching and trying to come up with an awk that will perform the following on a converted text file (original is a pdf). 1. Since the first two lines are (begin with) text they are removed 2. if $1 is a number then all text is merged (combined) into one line until the next... (3 Replies)
Discussion started by: cmccabe
3 Replies

9. Shell Programming and Scripting

How to remove contents from file which are under bracket?

hello Friend, In hostgroup file, i have define lots of hostgroups. I need to remove few of them without manually editing file. Need script or syntax. I want to search particular on hostgroup_members and delete hostgoup defination of it. for example. define hostgroup{ hostgroup_name... (8 Replies)
Discussion started by: ghpradeep
8 Replies

LEARN ABOUT DEBIAN

timblclient

timblclient(1)						      General Commands Manual						    timblclient(1)

NAME

       timblclient - a simple client to access the Tilburg Memory Based Learner Server

SYNOPSYS

       timblclient -h host -p port [-i inputfile] [-i outputfile] [--batch] [-b basename]

DESCRIPTION

       timblclient connects to a TimblServer on 'host':'port' and sends it the normal instructions. (see timblserver )

OPTIONS

       The timblclient options are

       -i inputfile
	      a series of timblserver commands.

	      When  BATCH  is  specified, inputfile is supposed to be a 'normal' Timbl testfile. Every line is send to the timblserver as a 'clas-
	      sify' command.  Default is stdin.

       -o outputfile
	      Where to put the results. Default is stdout.

       --batch
	      When --batch is specified, the input from 'inputfile' is not interpreted as timblserver commands, but as instances that will be send
	      'as is' prepended by a classify instruction.

       -h host
	      connect to the server on 'host'

       -p port
	      connect to 'port' on 'host'

       -b basename
	      set the base in TimblServer to 'base'

BUGS

       This is just a simple testprogram, and rather clumsy.

AUTHORS

       Ko van der Sloot timbl@uvt.nl

SEE ALSO

       timbl(1) timblserver(1)

								   2011 march 28						    timblclient(1)
All times are GMT -4. The time now is 11:39 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy