Problems with "grep -vf", and exclusion files Post: 302877486

Sponsored Content

Top Forums UNIX for Dummies Questions & Answers Problems with "grep -vf", and exclusion files Post 302877486 by Don Cragun on Sunday 1st of December 2013 07:17:51 PM

12-01-2013

Registered User

Quote:

Originally Posted by Doug Lassiter

"The behavior of grep is undefined when the input files you give it are not text files."

So, um, here's a dumb question -- how is it that a file produced by Mac "TextEdit" is not a "text file"? But indeed, if the filename thus produced doesn't have a .txt on the end, it doesn't seem to have a <newline> at the end. In fact, if I open such a file with vi, it says at the bottom "[noeol]", like you said it would. I save it with vi, and from then on, "[noeol]" isn't reported. vi inserts that <newline> when it saves it and makes it into a real live text file, I guess. I can also just change the file name from "exclude" to "exclude.txt", and the OS sticks a <newline> on, it seems. Wow.

So a real "text file" has to have a <newline> character at the end, and Mac TextEdit doesn't put it there, if you don't specify a .txt suffix. I never knew that. I naively thought that, well, text is text.

Now, having done that, grep -vf still doesn't work on that file, once it has a <newline> on it.

The Mac OS X TextEdit application processes several file formats that are text files and several file formats that are not text files. If the name of a file opened (or created) by TextEdit ends with ".txt", it will treat it as a text file; if it ends with ".rft", it will treat it as a rich text file; if it ends with ".doc", it will handle some of the text formatting done by Microsoft Word (and note that most Microsoft Word files ARE NOT text files). If there is no filename extension on the file, the preferences you have set in TextEdit will determine how it treats that file.

If you have a file (say xxx) that is not a text file and you rename the file xxx.txt, that doesn't change the format or contents of the file. (Although TextEdit might try to turn it into a text file if you use it to edit that file after you rename it.) Most UNIX utilities that take a filename as an operand could care less what the name of the file is. The filename extensions like .txt, .sh, .mp3, .rtf, et cetera provide a useful convention to help humans (and a few applications) make good guesses about what should be inside that file.

If you have turned exclude into a real text file and:

Code:

grep -vf exclude log.txt

still goes to never-never land, I would assume that (even though the filename ends in .txt and has a <newline> at the end of the file) it is not a text file as defined by the standards. The most likely problems would be that one or more "lines" in log.txt are longer than LINE_MAX (2048 on recent Mac OS X systems) bytes or it contains one or more null bytes (i.e., a byte with all bits set to 0).

Don Cragun

View Public Profile for Don Cragun

Find all posts by Don Cragun

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

grep to find content in between curly braces, "{" and "},"

problem String ~~~~~~~~~~~~~~~~~~ icecream= { smart peopleLink "good" LC "happy" , smartpeopleLink "dull" LC "sad" } aend = {smart vc4 eatr kalu} output needed ~~~~~~~~~~~~~~~~~~ smart peopleLink "good" LC "happy" , smartpeopleLink "dull" LC "sad" smart vc4...

2. UNIX for Dummies Questions & Answers

Explain the line "mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'`"

Hi Friends, Can any of you explain me about the below line of code? mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'` Im not able to understand, what exactly it is doing :confused: Any help would be useful for me. Lokesha

3. Shell Programming and Scripting

ls -laR | grep "^-" | awk '{print $9}'| grep "$.txt"

Hi, I don't know hot to make this command work: ls -laR | grep "^-" | awk '{print $9}'| grep "$.txt" It should return the list of file .txt It's important to search .txt at the end of the line, becouse some file name have "txt" in their name but have other extensions

4. UNIX for Dummies Questions & Answers

How to use the "grep/egrep" command to search files.

Hi Team, I am new to this forum and also trying to learn Unix. I will highly appriciate your help if you can help me to get the right command . {{{ I use the command " today | egrep '(10:| 11: )' | grep ERROR " to grep all the files that has been error betweeen 10 to 11...

5. AIX

xx=`date +"%a %b %d"`;rsh xxx grep "^$XX" zzz ?

AIX 4.2 I am trying to do an rsh grep to search for date records inside server logs by doing this : xx=`date +"%a %b %d"` rsh xxx grep "^$XX" zzz gives : grep: 0652-033 Cannot open Jun. grep: 0652-033 Cannot open 11. But if I do : xx=`date +"%a %b %d"` grep "^$XX" zzz it works...

6. Shell Programming and Scripting

"Join" or "Merge" more than 2 files into single output based on common key (column)

Hi All, I have working (Perl) code to combine 2 input files into a single output file using the join function that works to a point, but has the following limitations: 1. I am restrained to 2 input files only. 2. Only the "matched" fields are written out to the "matched" output file and...

7. Shell Programming and Scripting

ps -ef | grep "string1" "string2" " "string3"

Hi all, can any one suggest me the script to grep multiple strings from ps -ef pls correct the below script . its not working/ i want to print OK if all the below process are running in my solaris system. else i want to print NOT OK. bash-3.00$ ps -ef | grep blu lscpusr 48 42 ...

8. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone...

9. Shell Programming and Scripting

grep with "[" and "]" and "dot" within the search string

Hello. Following recommendations for one of my threads, this is working perfectly : #!/bin/bash CNT=$( grep -c -e "some text 1" -e "some text 2" -e "some text 3" "/tmp/log_file.txt" ) Now I need a grep success for some thing like : #!/bin/bash CNT=$( grep -c -e "some text_1...

10. Shell Programming and Scripting

Delete all log files older than 10 day and whose first string of the first line is "MSH" or "<?xml"

Dear Ladies & Gents, I have a requirement to delete all the log files in /var/log/test directory that are older than 10 days and their first line begin with "MSH" or "<?xml" or "FHS". I've put together the following BASH script, but it's erroring out: for filename in $(find /var/log/test...

LEARN ABOUT XFREE86

zgrep

ZGREP(1)						      General Commands Manual							  ZGREP(1)

NAME

       zgrep - search possibly compressed files for a regular expression

SYNOPSIS

       zgrep [ grep_options ] [ -e ] pattern filename...

DESCRIPTION

       Zgrep  invokes  grep  on  compressed  or  gzipped  files.   These  grep	options  will  cause  zgrep  to  terminate  with  an  error  code:
       (-[drRzZ]|--di*|--exc*|--inc*|--rec*|--nu*).  All other options specified are passed directly to grep.  If no file is specified,  then  the
       standard input is decompressed if necessary and fed to grep.  Otherwise the given files are uncompressed if necessary and fed to grep.

       If the GREP environment variable is set, zgrep uses it as the grep program to be invoked.

EXIT CODE

       2 - An option that is not supported was specified.

AUTHOR

       Charles Levert (charles@comm.polymtl.ca)

SEE ALSO

       grep(1), gzexe(1), gzip(1), zdiff(1), zforce(1), zmore(1), znew(1)

																	  ZGREP(1)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

grep to find content in between curly braces, "{" and "},"

Discussion started by: keshav_rk

2. UNIX for Dummies Questions & Answers

Explain the line "mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'`"

Discussion started by: Lokesha

3. Shell Programming and Scripting

ls -laR | grep "^-" | awk '{print $9}'| grep "$.txt"

Discussion started by: DNAx86

4. UNIX for Dummies Questions & Answers

How to use the "grep/egrep" command to search files.

Discussion started by: rkhanal