Parsing a file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parsing a file
# 8  
Old 10-14-2008
Quote:
Originally Posted by mirusko
sorry Andrew! :-)
What about if I have more references to MIN_PASSWORD_LENGTH in the file?
like
MIN_PASSWORD_LENGTH=4
MIN_PASSWORD_LENGTH=7
..
MIN_PASSWORD_LENGTH=x
?
How could I handle this?
Thanks again
LOL, no worries.

The command that I first posted will apply to all uncommented lines. So, suppose you have a file like this:

Code:
OTHER_STUFF_1=foo
MIN_PASSWORD_LENGTH=7
#MIN_PASSWORD_LENGTH=2
OTHER_STUFF_2=bar
MIN_PASSWORD_LENGTH=3
OTHER_STUFF_3=baz
OTHER_STUFF_4=qux
#MIN_PASSWORD_LENGTH=6
MIN_PASSWORD_LENGTH=8
MIN_PASSWORD_LENGTH=11

The grep portion of the command given earlier will filter out everything but:

Code:
MIN_PASSWORD_LENGTH=7
MIN_PASSWORD_LENGTH=3
MIN_PASSWORD_LENGTH=8
MIN_PASSWORD_LENGTH=11

So, those 4 lines will be passed on to the awk statement. So, the output of the original command (adding in the -F= part...) would be:

Code:
Value is good
Danger, Will Robinson!
Value is good
Value is good

So, that's what it would do with multiple matching lines. The real question, though, is what do you want it to do?

Last edited by treesloth; 10-14-2008 at 06:57 PM..
# 9  
Old 10-14-2008
well, in that case I only want the last
MIN_PASSWORD_LENGTH=11
how can I tell grep to give me the last value only?
Thanks Andrew!
K.
# 10  
Old 10-14-2008
Quote:
Originally Posted by mirusko
how can I tell grep to give me the last value only?
Ah, good question. There are a couple of ways. On some systems, you can use either the 'tac' (that is, 'cat' backwards) or the tail -r commands. That takes the input and, in a manner of speaking, simply flips it upside down. Then, the command 'head -n 1' keeps only the first line. I mention that only because it's nice to have in the command line toolbox. In your situation, though, I'd probably use this approach:

Code:
grep ^MIN_PASSWORD_LENGTH security | awk -F= 'END { if ($2 > 6) print "Value is good"; else print "Danger, Will Robinson!" }'

For testing, though, you might want to have it include the actual line in the output so you can be sure that it's giving you what you want; then just use the command above once that's proven:

Code:
grep ^MIN_PASSWORD_LENGTH security | awk -F= 'END { if ($2 > 6) print $0 " Value is good"; else print $0 " Danger, Will Robinson!" }'

Hope that helps.
# 11  
Old 10-14-2008
cool, I got the tail and head commands ( and piping it as output from the grep command ) but what does END in your example do? That's the first time I used it
Thanks Andrew!
K.
# 12  
Old 10-15-2008
hm, one more thing, what about if the MIN_PASSWORD_LENGTH is not at the beginning of the line ( if there are white spaces at the beginning ... ) ? then your grep won't return anything, how could I fix this?

Thank you
K.
# 13  
Old 10-15-2008
Quote:
Originally Posted by mirusko
hm, one more thing, what about if the MIN_PASSWORD_LENGTH is not at the beginning of the line ( if there are white spaces at the beginning ... ) ? then your grep won't return anything, how could I fix this?
I'd bet that there are loads of ways to do this, probably searchable with the keywords "remove leading spaces" or "remove leading whitespace". My usual way to do it is:

Code:
sed 's/^\ *//'

So, this full expression should give you what you need:

Code:
sed 's/^\ *//' security | \
grep ^MIN_PASSWORD_LENGTH security | \
awk -F= 'END { if ($2 > 6) print "Value is good"; else print "Danger, Will Robinson!" }'

You might want to go googling for a while to see if there are expressions to remove leading whitespace that better meet your needs. One should start to get a little nervous as more and more external commands are piled on. On the other hand, if the performance is satisfactory, go for it.

Regarding the END expression in the awk statement... There are both BEGIN and END in awk. BEGIN essentially provides to awk a set of instructions that it is to run before it looks at whatever it will be operating on. END provides instructions that are evaluated after the arguments are evaluated. The practical effect of the END statement in this case, though, is that only the last relevant line ("relevant" meaning those lines that got through the sed and grep filtering) is processed to stdout. My understanding is that the last line, and only the last line, is kept by awk and remains available even to instructions with the END section. An awk wizard might be able to shed more light on the particulars.

BEGIN is illustrated here:

Code:
sed 's/^\ *//' security | \ 
grep ^MIN_PASSWORD_LENGTH security | \
awk -F= 'BEGIN {print "Are you ready?"} ; END { if ($2 > 6) print "Value is good"; else print "Danger, Will Robinson!" }'

Awk is being told to execute the command {print "Are you ready?"} before it even looks at the lines being passed by the sed and grep combo.

Last edited by treesloth; 10-15-2008 at 03:07 PM..
# 14  
Old 10-17-2008
Great! Thank you so much Andrew, I learned so much!
K.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

File Parsing

Hi Gurus, i have files like this and i want to rename it. server1_0_Log0000597500 server1_0_Log0000597501 server1_0_Log0000597502 server1_0_Log0000597503 server1_0_Log0000597504 server1_0_Log0000597505 server1_0_Log0000597506 server1_0_Log0000597507 server1_0_Log0000597508... (7 Replies)
Discussion started by: fedora132010
7 Replies

2. Shell Programming and Scripting

parsing data from a big file using keys from another smaller file

Hi, I have 2 files format of file 1 is: a1 b2 a2 c2 d1 f3 format of file 2 is (tab delimited): a1 1.2 0.5 0.06 0.7 0.9 1 0.023 a3 0.91 0.007 0.12 0.34 0.45 1 0.7 a2 1.05 2.3 0.25 1 0.9 0.3 0.091 b1 1 5.4 0.3 9.2 0.3 0.2 0.1 b2 3 5 7 0.9 1 9 0 1 b3 0.001 1 2.3 4.6 8.9 10 0 1 0... (10 Replies)
Discussion started by: Lucky Ali
10 Replies

3. Shell Programming and Scripting

Parsing of file for Report Generation (String parsing and splitting)

Hey guys, I have this file generated by me... i want to create some HTML output from it. The problem is that i am really confused about how do I go about reading the file. The file is in the following format: TID1 Name1 ATime=xx AResult=yyy AExpected=yyy BTime=xx BResult=yyy... (8 Replies)
Discussion started by: umar.shaikh
8 Replies

4. Shell Programming and Scripting

Parsing file, yaml file? Extracting specific sections

Here is a data file, which I believe is in YAML. I am trying to retrieve just the 'addon_domains" section, which doesnt seem to be as easy as I had originally thought. Any help on this would be greatly appreciated!! I have been trying to do this in awk and mostly bash scripting instead of perl... (3 Replies)
Discussion started by: Rhije
3 Replies

5. UNIX for Dummies Questions & Answers

Script for parsing details in a log file to a seperate file

Hi Experts, Im a new bee for scripting, I would ned to do the following via linux shell scripting, I have an application which throws a log file, on each action of a particular work with the application, as sson as the action is done, the log file would vanish or stops updating there, the... (2 Replies)
Discussion started by: pingnagan
2 Replies

6. Shell Programming and Scripting

File Parsing Help

Hello, I have a file which contains groups of fields. These groups are separated by a blank line, to form a logical record. Each line consists of a field-value pair. If want to find all records where field 'd' has a value of '4' and if it does, I want the value of field 'a' (from the... (4 Replies)
Discussion started by: brawnr
4 Replies

7. Shell Programming and Scripting

Perl parsing compared to Ksh parsing

#! /usr/local/bin/perl -w $ip = "$ARGV"; $rw = "$ARGV"; $snmpg = "/usr/local/bin/snmpbulkget -v2c -Cn1 -Cn2 -Os -c $rw"; $snmpw = "/usr/local/bin/snmpwalk -Os -c $rw"; $syst=`$snmpg $ip system sysName sysObjectID`; sysDescr.0 = STRING: Cisco Internetwork Operating System Software... (1 Reply)
Discussion started by: popeye
1 Replies

8. Shell Programming and Scripting

need help in Parsing a CSV file and generate a new output file

Hi Scripting Gurus, I am trying to parse a csv file and generate a new output file. The input file will be a variable length in turns of rows and columns. output file will have 8 columns. we have three columns from the header for each set. just to give little bit more clarification each row... (15 Replies)
Discussion started by: vkr
15 Replies

9. Shell Programming and Scripting

Finding & Moving Oldest File by Parsing/Sorting Date Info in File Names

I'm trying to write a script that will look in an /exports folder for the oldest export file and move it to a /staging folder. "Oldest" in this case is actually determined by date information embedded in the file names themselves. Also, the script should only move a file from /exports to... (6 Replies)
Discussion started by: nikosey
6 Replies

10. Shell Programming and Scripting

Help me with parsing this file

Hi, I need a shell script that would parse this file /usr/share/i18n/locales/aa_DJ:title "Afar language locale for Djibouti (Cadu/Laaqo Dialects)." /usr/share/i18n/locales/aa_ER:title "Afar language locale for Eritrea (Cadu/Laaqo Dialects)." /usr/share/i18n/locales/aa_ER@saaho:title... (2 Replies)
Discussion started by: eamani_sun
2 Replies
Login or Register to Ask a Question