But hey,
worked *perfectly* for anything I put in my "exclude" file. That's the solution. Bingo.
But why? What does that "F" (that I wasn't originally using) do??? "Fixed strings"? That's a handy option.
Using fixed strings (-F option) instead of basic regular expressions (default) or extended regular expression (-E option) is faster when there aren't any characters that are special in a regular expression; but in this case (with or without -F), the results should be the same except for how long it takes to complete as long as the exclude file and all files being processed are proper text files. Either one of your files isn't a text file, there is a bug in the version of grep you're using, or some hidden characters in your exclude file are affecting regular expression parsing. It would still be interesting to see the output of:
for a version of exclude that causes:
to go into never-never land.
This User Gave Thanks to Don Cragun For This Post:
That's for an "exclude" file that sends grep -vf into never-never land, but one that works with grep -Fvf.
Also, FWIW, I wrote those "exclude" files both with the Mac text editor *and* with vi. No difference. So I can't see why there might be hidden characters.
---------- Post updated at 05:54 PM ---------- Previous update was at 05:44 PM ----------
Quote:
Originally Posted by alister
With -F, exclude contains literal strings. Without it, regular expressions.
I have to assume that was what was special about the alphabetic (as opposed to numeric) characters in the "exclude" file. With grep- vf, grep wanted to interpret those alphabetic character strings as expressions. The numbers were obviously not expressions, so it didn't get confused with those.
That's what seemed so screwy. grep -vf could exclude numbers successfully, but not alphabetic strings. Now, in my book, both are alphanumerics, so if one worked, the other should as well.
... ... ...
That's for an "exclude" file that sends grep -vf into never-never land, but one that works with grep -Fvf.
Also, FWIW, I wrote those "exclude" files both with the Mac text editor *and* with vi. No difference. So I can't see why there might be hidden characters.
Wrong. There is a HUGE difference.
You didn't write the above exclude file with vi. It is not a text file. (It doesn't have a <newline> character at the end of the last line.) The behavior of grep is undefined when the input files you give it are not text files.
If you add the missing <newline> character to that file, I would be very surprised if grep -vf doesn't start working as you expected. There are several ways to fix that file. Among them are:
but don't do that if there is a <newline> at the end of the file or it will add an empty line to the end of the file; and
which will probably include a note in the status line when you open it something like:
where the noeol means that there is no end-of-line marker at the end of the last line. Using the vi command :wq will add the missing <newline> character to the buffer, rewrite the file with the missing <newline>, and quit. You can repeat this as many times as you want and it won't add empty lines to the end of the file.
Note that the standards don't require vi to work when given a file that is not a text file either; but the vi (or vim) on OS X will do what you need in this case.
"The behavior of grep is undefined when the input files you give it are not text files."
So, um, here's a dumb question -- how is it that a file produced by Mac "TextEdit" is not a "text file"? But indeed, if the filename thus produced doesn't have a .txt on the end, it doesn't seem to have a <newline> at the end. In fact, if I open such a file with vi, it says at the bottom "[noeol]", like you said it would. I save it with vi, and from then on, "[noeol]" isn't reported. vi inserts that <newline> when it saves it and makes it into a real live text file, I guess. I can also just change the file name from "exclude" to "exclude.txt", and the OS sticks a <newline> on, it seems. Wow.
So a real "text file" has to have a <newline> character at the end, and Mac TextEdit doesn't put it there, if you don't specify a .txt suffix. I never knew that. I naively thought that, well, text is text.
Now, having done that, grep -vf still doesn't work on that file, once it has a <newline> on it.
"The behavior of grep is undefined when the input files you give it are not text files."
So, um, here's a dumb question -- how is it that a file produced by Mac "TextEdit" is not a "text file"? But indeed, if the filename thus produced doesn't have a .txt on the end, it doesn't seem to have a <newline> at the end. In fact, if I open such a file with vi, it says at the bottom "[noeol]", like you said it would. I save it with vi, and from then on, "[noeol]" isn't reported. vi inserts that <newline> when it saves it and makes it into a real live text file, I guess. I can also just change the file name from "exclude" to "exclude.txt", and the OS sticks a <newline> on, it seems. Wow.
So a real "text file" has to have a <newline> character at the end, and Mac TextEdit doesn't put it there, if you don't specify a .txt suffix. I never knew that. I naively thought that, well, text is text.
Now, having done that, grep -vf still doesn't work on that file, once it has a <newline> on it.
The Mac OS X TextEdit application processes several file formats that are text files and several file formats that are not text files. If the name of a file opened (or created) by TextEdit ends with ".txt", it will treat it as a text file; if it ends with ".rft", it will treat it as a rich text file; if it ends with ".doc", it will handle some of the text formatting done by Microsoft Word (and note that most Microsoft Word files ARE NOT text files). If there is no filename extension on the file, the preferences you have set in TextEdit will determine how it treats that file.
If you have a file (say xxx) that is not a text file and you rename the file xxx.txt, that doesn't change the format or contents of the file. (Although TextEdit might try to turn it into a text file if you use it to edit that file after you rename it.) Most UNIX utilities that take a filename as an operand could care less what the name of the file is. The filename extensions like .txt, .sh, .mp3, .rtf, et cetera provide a useful convention to help humans (and a few applications) make good guesses about what should be inside that file.
If you have turned exclude into a real text file and:
still goes to never-never land, I would assume that (even though the filename ends in .txt and has a <newline> at the end of the file) it is not a text file as defined by the standards. The most likely problems would be that one or more "lines" in log.txt are longer than LINE_MAX (2048 on recent Mac OS X systems) bytes or it contains one or more null bytes (i.e., a byte with all bits set to 0).
Yes, that is, of course, very true about different kinds of text files. But it is fascinating that a <newline> character, which isn't displayed in even vi, determines whether grep thinks of the file as truly text. That octal dump command is handy in that regard, as I suppose is the status line in vi.
And yes, that's right about just changing the suffix. I thought that worked, but it really doesn't. My mistake.
That's interesting speculation about why a file that looks like a real text file isn't doing the job with grep -vf. But at least with the F option, it's all fine and I can make it work. So I'll think about that and, in the meantime, let me thank everyone here for their very prompt and helpful comments.
Dear Ladies & Gents,
I have a requirement to delete all the log files in /var/log/test directory that are older than 10 days and their first line begin with "MSH" or "<?xml" or "FHS". I've put together the following BASH script, but it's erroring out:
for filename in $(find /var/log/test... (2 Replies)
Hello.
Following recommendations for one of my threads, this is working perfectly :
#!/bin/bash
CNT=$( grep -c -e "some text 1" -e "some text 2" -e "some text 3" "/tmp/log_file.txt" )
Now I need a grep success for some thing like :
#!/bin/bash
CNT=$( grep -c -e "some text_1... (4 Replies)
Hi,
I have line in input file as below:
3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL
My expected output for line in the file must be :
"1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL"
Can someone... (7 Replies)
Hi all,
can any one suggest me the script to grep multiple strings from ps -ef
pls correct the below script . its not working/
i want to print OK if all the below process are running in my solaris system. else i want to print NOT OK.
bash-3.00$ ps -ef | grep blu
lscpusr 48 42 ... (11 Replies)
Hi All,
I have working (Perl) code to combine 2 input files into a single output file using the join function that works to a point, but has the following limitations:
1. I am restrained to 2 input files only.
2. Only the "matched" fields are written out to the "matched" output file and... (1 Reply)
AIX 4.2
I am trying to do an rsh grep to search for date records inside server logs by doing this :
xx=`date +"%a %b %d"`
rsh xxx grep "^$XX" zzz
gives :
grep: 0652-033 Cannot open Jun.
grep: 0652-033 Cannot open 11.
But if I do :
xx=`date +"%a %b %d"`
grep "^$XX" zzz
it works... (2 Replies)
Hi Team,
I am new to this forum and also trying to learn Unix.
I will highly appriciate your help if you can help me to get the right command .
{{{
I use the command " today | egrep '(10:| 11: )' | grep ERROR " to grep all the files that has been error betweeen 10 to 11... (6 Replies)
Hi,
I don't know hot to make this command work:
ls -laR | grep "^-" | awk '{print $9}'| grep "$.txt"
It should return the list of file .txt
It's important to search .txt at the end of the line, becouse some file name have "txt" in their name but have other extensions (13 Replies)
Hi Friends,
Can any of you explain me about the below line of code?
mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'`
Im not able to understand, what exactly it is doing :confused:
Any help would be useful for me.
Lokesha (4 Replies)