Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Find common terms in two text file, xargs, grep

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Find common terms in two text file, xargs, grep
# 8  
Old 10-12-2011
Can you paste file1 and file2? (need not be the entire file)

--ahamed
# 9  
Old 10-12-2011
Quote:
Originally Posted by ahamed101
Can you paste file1 and file2? (need not be the entire file)

--ahamed
I'd rather not, they are emails. Smilie

Edit: I made two mock up files with just random words in them, and it now work as intended with them. Very strange, are there any special meaning to @ for bash that could mess this up?

There is nothing more than one email address per line at this stage.
Last edited by eon; 10-12-2011 at 04:19 AM..
# 10  
Old 10-12-2011
Quote:
Originally Posted by eon
That still does not work I'm afraid.

file1 contains 99 lines which all appear in file2.

file2 contains 109 lines.

What I want is to get the 10 lines that does not appear in file1.

Doing this:

Code:
grep -vFf file1 file2 | wc -l

gives me: 109

but switching the order gives me: 0



Code:
$ wc -l fff1
99 fff1
$ wc -l fff2
109 fff2
$ fgrep -vxf fff1 fff2
100
101
102
103
104
105
106
107
108
109
$

# 11  
Old 10-12-2011
@anchal_khare I also got it working with other files, but not with emails.
# 12  
Old 10-12-2011
Even with email id's also, it is working for me!
Something fishy in your emails then Smilie

check the output of those mails with cat -E fishy_email if it has any special characters! or take a octal dump
od -bc fishy_email

You didn't edit those files in windows, did you?

--ahamed
This User Gave Thanks to ahamed101 For This Post:
eon
# 13  
Old 10-12-2011
Quote:
Originally Posted by eon
I made two mock up files with just random words in them, and it now work as intended with them. Very strange, are there any special meaning to @ for bash that could mess this up?

There is nothing more than one email address per line at this stage.
If the domain name are same in one or more email addresses, grep would match that line always. In that case you must use "-x" flag.
Code:
           -x                  (eXact) Matches are recognized only when the
                               entire input line matches the fixed string or
                               regular expression.

# 14  
Old 10-12-2011
Quote:
Originally Posted by ahamed101
Even with email id's also, it is working for me!
Something fishy in your emails then Smilie

check the output of those mails with cat -E fishy_email if it has any special characters! or take a octal dump
od -bc fishy_email

You didn't edit those files in windows, did you?

--ahamed
Thanks, I tried the octal dump and it appears that the line endings are: \r \n

Could that be the problem? I could try to strip of '\r' with sed I suppose.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Find a file with common initials and last words

Hi, I have a requirement like i have to find out files and remove them on a daily basis. The files are generated as abc_jnfn_201404230004.csv abc_jnfo_201404230004.csv abc_jnfp_201404230004.csv abc_jnfq_201404230004.csv abd_jnfn_201404220004.csv abe_jnfn_201404220004.csv i want to... (1 Reply)
Discussion started by: Mohammed_Tabish
1 Replies

2. UNIX for Dummies Questions & Answers

Grep -B used with -f? (Searching a file using a list of terms, output is lines before each match)

(1 Reply)
Discussion started by: Twinklefingers
1 Replies

3. UNIX for Dummies Questions & Answers

Grep? - using a file of terms to search another file when the information is on a different line

I have a flat file that looks like this, let's call it Chromosome_9.txt: FT /Gene_Name="Guanyl-Acetylase 9" FT /Gene_Number"36952" FT /Gene_Name="Endoplasmic Luciferase" FT /Gene_Number"36953" FT ... (4 Replies)
Discussion started by: Twinklefingers
4 Replies

4. Shell Programming and Scripting

Xargs or Find with output to a file

Hi, I've got to setup a script that will run daily, and find a log file of a certain age, and then compress and transfer this file to a new location. so far i've been able to specify the file i want with: find . -name 'filename.*.log' -mtime 14 -exec compress -vf {} \; this prints out... (4 Replies)
Discussion started by: horhif
4 Replies

5. Shell Programming and Scripting

Grep multiple search terms with context

I have a file that is a sort library in the format: ##def title1 content1 stuff1 content2 stuff2 ##enddef ##def title2 etc.. I want to grep def and content and pull some trailing context from content so the result would look something like: (1 Reply)
Discussion started by: Moe.Wilensky
1 Replies

6. OS X (Apple)

Need Help with GREP REGEX scripts for common BB-EDIT text-editing

Hi Everybody.. I'm a "newbie" to using Command-line... A few half-remembered DOS commands from 30 years ago, and the very handy "Sudo rm -R pathname" REMOVE command... I do a lot of "cleaning" of plain-text OCR text files. with assorted common line-break, punctuation and capitalization... (1 Reply)
Discussion started by: TheMacGuy
1 Replies

7. UNIX for Dummies Questions & Answers

find/xargs/*grep: find multi-line empty "try-catch" blocks - eg, missing ; not in a commented block

How can I recursively find all files in a directory and print out the file and first line number of any text blocks that match the below cases? This would seem to involve find, xargs, *grep, regex, etc. In summary, I want to find so-called empty "try-catch blocks" that do not contain code... (0 Replies)
Discussion started by: lifechamp
0 Replies

8. UNIX for Dummies Questions & Answers

find . -name "*.*" | xargs grep "help"

Hi all, I am a unix noob. Need some basic help. I have tried using google, but not able to figure this out. Here are the scenarios: 1. How do I find a directory with a particular name, say "Merlin" in the entire file system? I tried : find / -type d -name "dir_name" The problem is I'm... (3 Replies)
Discussion started by: neil.k
3 Replies

9. Shell Programming and Scripting

File find | xargs grep for pattern file

Folks I've been struggling this with for far too liong now and need your help! I've been happily using grep for a search of a directory, to list the files which contain a string: find . -type f -mtime -5 -print | xargs grep -l 'invoiceID=\"12345\"' Now the list of 'invoiceID' I am... (4 Replies)
Discussion started by: daveaasmith
4 Replies

10. Shell Programming and Scripting

How to strip out common terms in string

Hi, I have this kinda of data:- 0,0,0,0,1,2,0,4,5,6,7,foo 0,0,0,0,1,4,0,5,5,5,5,foo1 0,0,6,0,1,6,0,6,1,2,3,orange etc... I wanted to remove the 0 which occur on the same rows of foo,foo1 and orange in this case. Desired output is:- 0,1,2,4,5,6,7,foo 0,1,4,5,5,5,5,foo1... (9 Replies)
Discussion started by: ahjiefreak
9 Replies
Login or Register to Ask a Question