Grepping file and returning passed variable if the value does not exist in file at all.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grepping file and returning passed variable if the value does not exist in file at all.
# 1  
Old 04-26-2011
Grepping file and returning passed variable if the value does not exist in file at all.

I have a list of fields that I want to check a file for, returning that field if it not found at all in the file. Is there a way to do a grep -lc and return the passed variable too rather then just the count?

I am doing some crappy work-around now but I was not sure how to regrep this for :0 so that it only displays the fields not in the file at all.

there must be a simpler way... Smilie
Code:
    a=OPEN
    echo "$a:\c"
    grep -lc "'$a'" /data/data.xml
    a=CLOSE
    echo "$a:\c"
    grep -lc "'$a'" /data/data.xml


Last edited by Franklin52; 04-26-2011 at 12:39 PM.. Reason: Please use code tags
# 2  
Old 04-26-2011
Well you could do something like this:
Code:
awk '/OPEN:/ {o++} /CLOSE:/ {c++} END {print "OPEN:" o "\nCLOSE:" c}' infile

Depends how many different fields you want to check for. Also, if we knew your exact file format it might be easier to give you something more optimal...
# 3  
Old 04-26-2011
Thank you very much...

I actually have a few hundred fields to check. Since I am scripting the scripts I ended up converting this to following. I also made change to only search for instances when my field is wrapped in single quotes. And then to re-grep to only print instances where the field was not found at least once.

I would assume it would be more efficient to open the file once but I only need to run this script once a day prior to file check in and for my source file it runs in 15 seconds. The infile format is a pseudo xml file.
Code:
awk '/'\''OPEN'\''/ {o++} END {print "OPEN:" o }' infile | grep -v '[0-9]'
awk '/'\''CLOSE'\''/ {o++} END {print "CLOSE:" o }' infile | grep -v '[0-9]'
awk '/'\''HIGH'\''/ {o++} END {print "HIGH:" o }' infile | grep -v '[0-9]'
..........
..........
.........


Last edited by Scott; 04-26-2011 at 01:32 PM..
# 4  
Old 04-28-2011
Your solution can be optimised significantly. You don't need to invoke AWK more than once, and piping the output of AWK to grep is a faux paus I witness far too many times ... it's simply unnecessary.

The following will achieve the same result and will be much faster:
Code:
awk -v sq="'" -v fields="OPEN CLOSE HIGH" '
    BEGIN { fn=split(fields, fs) }
    { for (i=1; i<=fn; i++) if ($0 ~ sq fs[i] sq) fc[i]++ }
    END { for (i=1; i<=fn; i++) print fs[i] ":" (fc[i] ? fc[i] : 0) }' infile

The fields are passed in via the fields variable and are separated by whitespace (spaces, tabs, newlines). I don't know how you're getting your list of a few hundred fields, if they're in a file already separated by spaces or newlines then you could do:
Code:
-v fields="$(<fieldfile)"

If you only want a list of fields where the count is zero (which is what I believe your intent was behind using grep in your earlier example), then amend the last line as follows:
Code:
END { for (i=1; i<=fn; i++) if (!fc[i]) print fs[i]}' infile

Don't forget if you're on Solaris to use /bin/nawk or /usr/xpg4/bin/awk instead of /bin/awk.

Best regards,
Mark.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

ksh Script, Reading A File, Grepping A File Contents In Another File

So I'm stumped. First... APOLOGIES... my work is offline in an office that has zero internet connectivity, as required by our client. If need be, I could print out my script attempts and retype them here. But on the off chance... here goes. I have a text file (file_source) of terms, each line... (3 Replies)
Discussion started by: Brusimm
3 Replies

2. Shell Programming and Scripting

Grepping the file into a variable after SSH into a server

Hi All, When i am logged into a server , i am able to assign grep value to a variable as follows. VAR=`grep 'Listener stopped' /logs/perf.log` However, when i log out of the server, and try to execute the following command by first SSHing into server, it throws error. $ VAR=`ssh Server... (4 Replies)
Discussion started by: srkmish
4 Replies

3. Shell Programming and Scripting

Display file date after grepping a string in the file

Hi All, I need to recursively grep several folders for a MAC address and display the results with the date of the file name at the start. Even better would be if the final results were displayed chronologically so the newest file is always at the end. Oldest at the top, regardless of what... (8 Replies)
Discussion started by: quemalr
8 Replies

4. Shell Programming and Scripting

In the sh file variable is not being passed on.

I am having difficulties with the fllowing script: !/bin/sh voicemaildir=/var/spool/asterisk/voicemail/$1/$2/INBOX/ echo `date` ':' $voicemaildir >> /var/log/voicemail-notify.log for audiofile in `ls $voicemaildir/*.wav`; do transcriptfile=${audiofile/wav/transcript} ... (4 Replies)
Discussion started by: ghurty
4 Replies

5. Shell Programming and Scripting

Grepping from a point in a file to the end of the file

does any one know how to turn the equivalent of this command: awk '/2011 John Doe 8344/,0' /tmp/ops.log | egrep -c "received request" to something that would use egrep instead of awk? What the awk command does is, it searches the ops.log file for "2011 John Doe 8344". When it finds it,... (4 Replies)
Discussion started by: SkySmart
4 Replies

6. UNIX for Dummies Questions & Answers

update value in a file using variable passed via unix script

I have a file in my unix directory called "restart_job1.job". Below is a sample of the script where I am doing a 'grep' to check specifically for an oracle error. If the value 'ORA-" is found, I set a variable to hold the return code value (job1_return_code). If the return code is non zero, I... (2 Replies)
Discussion started by: ncsthbell_dr
2 Replies

7. Shell Programming and Scripting

Returning only part of a line when grepping

I want to grep out a part of a snort rule based on the SID given, but all i want as the output is the part in the quotes after the msg: An example line looks something like this: alert tcp any any -> 127.0.0.1 any (msg:"Example Message"; classtype:Example; sid:123456;) I would want it to... (7 Replies)
Discussion started by: riott
7 Replies

8. Shell Programming and Scripting

Grepping a file based on input from a second file

how to grep a file based on another input file File1 ashu 1 ninetwo hari qwer 6 givefour jan fghj 8 noeight mar vbmi 7 noput feb -- --- File2 noput noeight --- -- Taking the input of grep as File2, a search need to be made in File1 giving File3 as output: (7 Replies)
Discussion started by: er_ashu
7 Replies

9. Shell Programming and Scripting

Grepping for two strings that MUST exist on the same line

Trying to find a way to grep for two names on a line. Both names must appear on the same line so '|' / OR is out. So far, I'm just messing around and I've got find . -name "*" | xargs grep "Smith" Let me explain. I'm at a top level and need to know all the names of the files that... (6 Replies)
Discussion started by: Rally_Point
6 Replies

10. Shell Programming and Scripting

How to Check whether list file present in TXT file exist or not

Hi All, I have txt file which has list of files. I have to check whether these files exist or not. Thanks supriya (6 Replies)
Discussion started by: supriyabv
6 Replies
Login or Register to Ask a Question