I have a couple of things I got stuck on

1)
I have a text file containing 25k search string that I need to search against compressed file. I have used this command but somehow it doesn't seems to use all the search terms.

I have put one search string per line in the txt file (I clean up the domain name down to just the word ie abc.com to abc).

I have used zgrep --colour=always -nf [name of txt file] name of compressed file


2)
With the same text file 1) , I need to search against a folder containing 50k email messages (.eml format). If the .eml file contains match search string, it will move to another folder. So I can run a batch print later on.


I have both type of email message file, 1) one single file containing multiple messages and 2) 50000 individual email message in .eml format.

The problem with the search, it doesn't seem to run all the search string contains in the txt file against the target file. It seems only a few lines of search string is used.

Thanks in advance for any tips.

A 25k long search string? Is your text so similar that you need to search 25000 characters to be sure of a match. Assuming the regex engine can even deal with that, the search efficiency is going to be pretty poor.

Or do you mean 25000 different strings? That has problems as well.

I think, rather than you tell us how you think it should be done, give us the problem you are trying to solve - in detail. This whole thing sounds wrong from the start, as I read it.

There are a lot of other tools out there like comm for example.

No duplicate or cross-posting, please read the rules.

This thread is similar to:

Help!!

Thread closed.