awk delete line if $5 contains string from list


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk delete line if $5 contains string from list
# 1  
Old 12-21-2013
awk delete line if $5 contains string from list

# 2  
Old 12-21-2013
Hi,
Could you show input and output example, please ?
Regards.
# 3  
Old 12-21-2013
Code:
awk ' FILENAME=="strings.txt" {arr[$0]++; next}
        FILENAME=="data.txt" { if( $5 in arr){next}; print$0} ' strings.txt data.txt>proc_data.txt

The order of the files as arguments is important - you have to read the strings file first to build the array.
# 4  
Old 12-21-2013
Code:
awk 'NR==FNR{A[$0]++;next}!(A[$5])' strings.txt file.txt

# 5  
Old 12-21-2013
If strings.txt is edited manually, then the use of $1 will be more reliable, because this will discard spurious spacing and $5 in A is more efficient than A[$5], since this would create unnecessary array elements (minor improvement).

Code:
awk 'NR==FNR{A[$1]; next}!($5 in A[$5])' strings.txt file.txt

Field separators can be added of course, with the default field separators, strings.txt could be more aptly named words.txt, since that would mean the strings cannot contain spacing

Last edited by Scrutinizer; 12-21-2013 at 01:16 PM..
# 6  
Old 12-24-2013
Quote:
Originally Posted by Scrutinizer
If strings.txt is edited manually, then the use of $1 will be more reliable, because this will discard spurious spacing and $5 in A is more efficient than A[$5], since this would create unnecessary array elements (minor improvement).

Code:
awk 'NR==FNR{A[$1]; next}!($5 in A[$5])' strings.txt file.txt

Field separators can be added of course, with the default field separators, strings.txt could be more aptly named words.txt, since that would mean the strings cannot contain spacing
I think Scrutinizer accidentally copied more of Yoda's code than he intended to. I believe that what Scrutinizer meant was:
Code:
awk 'NR==FNR{A[$1]; next}!($5 in A)' strings.txt file.txt

This User Gave Thanks to Don Cragun For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to find string and delete before just in line?

Hello, When my lines contain question mark, I use below command to delete the portion of the matching line coming after question mark: sed 's/?.*//' SampleFile SampleFile: helloworldfirstline?mdksmyymsss hellosecondlineworld?mdksmkkmsss thirdhelloworld?mdksmccmsss Output:... (2 Replies)
Discussion started by: baris35
2 Replies

2. Shell Programming and Scripting

awk find and delete line from list

good day i have a list of numbers in input.txt, i would like to compare to file.txt and delete the line that number appears in file.txt . input.txt: 4558980 5525628 3595233 2650083 2219411 3529741 4675897 3070869 0014685 6365902 file.txt: one-two-three-4558980.txt... (7 Replies)
Discussion started by: klein
7 Replies

3. UNIX for Dummies Questions & Answers

Search for a string,delete the line and replace with new string in a file

Hi Everyone, I have a requirement in ksh where i have a set of files in a directory. I need to search each and every file if a particular string is present in the file, delete that line and replace that line with another string expression in the same file. I am very new to unix. Kindly help... (10 Replies)
Discussion started by: Pradhikshan
10 Replies

4. UNIX for Advanced & Expert Users

How to find a string in a line in UNIX file and delete that line and previous 3 lines ?

Hi , i have a file with data as below.This is same file. But actual file contains to many rows. i want to search for a string "Field 039 00" and delete that line and previous 3 lines in that file.. Can some body suggested me how can i do using either sed or awk command ? Field 004... (7 Replies)
Discussion started by: vadlamudy
7 Replies

5. Shell Programming and Scripting

Delete the last line if match the string

Hi, How to delete the last line if is match the below string else no action... String checking "END OF FILE. ROW COUNT: " 9f680174-cb87-4f71-887a-93b6f62fa5aa|20077337254|2 9f680174-cb87-4f71-887a-93b6f62fa5aa|20077337254|0 229f680174-cb87-4f71-887a-93b6f62fa5aa|20077337254|3 END OF... (2 Replies)
Discussion started by: bmk
2 Replies

6. Shell Programming and Scripting

bash: delete a string from last line

I am using a while loop to read a file and i have to delete a string from the last line. Here is the code so far: 1. Read line by line from file new.html using while loop and store in a variable data 2.Now, need to replace '||chr( from last line in data I am able to replace that character from... (12 Replies)
Discussion started by: Chella15
12 Replies

7. Shell Programming and Scripting

Grep a string from input file and delete next three lines including the line contains string in xml

Hi, 1_strings file contains $ cat 1_strings /home/$USER/Src /home/Valid /home/Review$ cat myxml <projected value="some string" path="/home/$USER/Src"> <input 1/> <estimate value/> <somestring/> </projected> <few more lines > <projected value="some string" path="/home/$USER/check">... (4 Replies)
Discussion started by: greet_sed
4 Replies

8. Shell Programming and Scripting

Search for a line, delete a string in it

let me start out by saying i have ZERO exp with any kind of scripting, so sorry if this is really basic stuff..... For example, I need to have a script that will search a file and find this line in the file: *.cat;dog;kennel;house;barn;horse;hay;coat hat and remove the "coat" from the... (12 Replies)
Discussion started by: skunky
12 Replies

9. Shell Programming and Scripting

delete a string from line

hi i have a file contains data session terminated Line timeout: 120 Change state START->SIGNON_REPLY, RC=0 Change state SIGNON_REPLY->SIGNON, RC=0 i need to remove "Change state" and write to the same file please help thanks Satya (3 Replies)
Discussion started by: Satyak
3 Replies

10. Shell Programming and Scripting

Delete a Line with a string X

Hi guys, I am new here. I have done many scripts in VisualBasic and batch. Now going to *inux and learning bash and perl, quite amazing commands I find in linux, like sed, grep, awk and so on... the task is simple, find a string in folders and remove the line where that string exists. I can... (3 Replies)
Discussion started by: 4scriptmoni
3 Replies
Login or Register to Ask a Question