Remove words from file


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Remove words from file
# 1  
Old 12-19-2008
Remove words from file

Hello,

I have a question:
I have two different files, let's call them file1 and file2. file1 contains a list of words, the words are on seperate lines:

word1
word2
word3
word4
etc...

file2 also contains a list of words, seperated in the same way as file1.

What I want to do is remove the words that are in both file1 and file2 from file2. Does anyone know if this is possible?

I tried some sed stuff, but I just can't get the desired result.

Many thanks!
# 2  
Old 12-19-2008
Hi,

Code:
grep -v -f file2 file1

-v -- print only lines not matching pattern
-f file2 -- get the list of possible matches from file2.

HTH Chris

Last edited by vgersh99; 12-19-2008 at 11:23 AM.. Reason: VbCode fixes
# 3  
Old 12-19-2008
Code:
$ cat file1
word1
word2
word3
word4
word5
word6

$ cat file2
word1
word2
word3
word4
word7
word8
word9

$ comm -13 file1 file2
word7
word8
word9

# 4  
Old 12-19-2008
Thanks for your replies, but I still dont get the desired result.

I still see the words that occur in file1 in file2 after using these commands.

file1 contains for example words that occur often in a text like:
a
the
I
an
to
be

Those words also occur in file2, but I want to strip them out of file2, so I have a list of words that don't occur that much.

I actually don't see why comm won't work.

Are there other options to solve my problem?
# 5  
Old 12-19-2008
Code:
nawk 'FNR==NR {a[$0];next} !($0 in a)' file1 file2

# 6  
Old 12-19-2008
Thanks for your help! You just solved my problem Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replace particular words in file based on if finds another words in that line

Hi All, I need one help to replace particular words in file based on if finds another words in that file . i.e. my self is peter@king. i am staying at north sydney. we all are peter@king. How to replace peter to sham if it finds @king in any line of that file. Please help me... (8 Replies)
Discussion started by: Rajib Podder
8 Replies

2. UNIX for Dummies Questions & Answers

Replace the words in the file to the words that user type?

Hello, I would like to change my setting in a file to the setting that user input. For example, by default it is ONBOOT=ON When user key in "YES", it would be ONBOOT=YES -------------- This code only adds in the entire user input, but didn't replace it. How do i go about... (5 Replies)
Discussion started by: malfolozy
5 Replies

3. Shell Programming and Scripting

How count the number of two words associated with the two words occurring in the file?

Hi , I need to count the number of errors associated with the two words occurring in the file. It's about counting the occurrences of the word "error" for where is the word "index.js". As such the command should look like. Please kindly help. I was trying: grep "error" log.txt | wc -l (1 Reply)
Discussion started by: jmarx
1 Replies

4. Shell Programming and Scripting

Splitting concatenated words in input file with words from the same file

Dear all, I am working with names and I have a large file of names in which some words are written together (upto 4 or 5) and their corresponding single forms are also present in the word-list. An example would make this clear annamarie mariechristine johnsmith johnjoseph smith john smith... (8 Replies)
Discussion started by: gimley
8 Replies

5. Shell Programming and Scripting

remove words

All, I have a file with below entries. /java/usr/abc/123 /java/usr/xyz/123_21 /java/usr/ab12/345/234 ......... ......... And I need entry as /java/usr/abc/config /java/usr/xyz/config /java/usr/ab12/config ......... ......... Actually, I need to remove all other entries... (2 Replies)
Discussion started by: anshu ranjan
2 Replies

6. Shell Programming and Scripting

Splitting Concatenated Words in Input File with Words from a Master File

Hello, I have a complex problem. I have a file in which words have been joined together: Theboy ranslowly I want to be able to correctly split the words using a lookup file in which all the words occur: the boy ran slowly slow put child ly The lookup file which is meant for look up... (21 Replies)
Discussion started by: gimley
21 Replies

7. UNIX for Dummies Questions & Answers

Remove words beginning with a certain character from a file

Hi, how could you go about removing words that begin with a certain character. assuming that this character is '-' I currently have echo "-hello" | sed s/-/""/ which replaces the leading dash with nothing but I want to remove the whole word, even if there are multiple words beginning... (3 Replies)
Discussion started by: skinnygav
3 Replies

8. Shell Programming and Scripting

Need to remove the words

Hi folks, I have file with the below 1245633505 +manual mroennfeldt@news.com.au 1245633506 +manual sal@bynews.com.au 1245633506 +manual whson@btimes.com 1245633507 +manual karla.marsden@tnews.com.au 1245633508 +manual king@netn.com.au Now, I need the output of the files only with... (4 Replies)
Discussion started by: gsiva
4 Replies

9. Shell Programming and Scripting

how to remove words between /* and */ in several lines of file

hi, I want to remove comments in somany files. coments started from /* and ends with */. so i wnt to remove words /* to */ how to remove words between /* and */ in several lines of file EX : cat haha.txt "HI",/*hk*/ob1,raju,/*hjh*/boju,beeju output should be : "HI",ob1,raju,boju,beeju ... (8 Replies)
Discussion started by: spc432
8 Replies

10. Shell Programming and Scripting

remove first few words from a line

Hi All, Sample: 4051 Oct 4 10:03:36 AM 2008: TEST: end of testcase Checking Interface after reload, result fail I need to remove first 10 words of the above line and output should be like Checking Interface after reload, result fail Please help me in this regard. Thanks, (4 Replies)
Discussion started by: shellscripter
4 Replies
Login or Register to Ask a Question