Awk and Variables


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Awk and Variables
# 1  
Old 11-15-2007
Awk and Variables

Hi,
I have some files in metrica (assume a pre-defined format) which i need to process and have some values in a .csv file. The script that does this, part of which is:

procMetricaOMData()
{
host=$1
fileIN=$2
fileOUT=$3

$CATCMD $fileIN | \
awk -f scripts/OMParser.awk -v host=$host >> $fileOUT
}

.....
host=`hostname`
procMetricaOMData $host $fileIN output/om/$om-$app.csv

The Awk script goes like this:

BEGIN {FS=" "; printf("%s,", host)}
$1 !~ /^#/ && $1 !~ /{/ && $1 !~ /}/ && $2 !~ /{/ {
if ($1 == "start") { printf("%s:%s,", substr($2,1,2),substr($2,3,2)) }
else if ($1 == "end") { printf("%s:%s,", substr($2,1,2),substr($2,3,2)) }
else { printf("%s,",$2) }
}
END {printf("\n")}

The expected output should be something like:

HOST,WEEK,WEEKDAY,day,node,HOUR,start,end,SAMPLING INTERVAL,TotalRequest,MSRequest,NWIRequest,SMSBrequest,SMSBack,SMSBNack,MSRequestSuccess,NWIRequestS uccess,SMSCCongestion,ApplicationCongestion,InvalidParameters,MSIllegalSubscriber,MWIllegalSubscribe r,ApplicationUnavailble,UCPError,SMPPError,

mo1emg1,"=weeknum(d2,2)","=weekday(d2,2)",07Oct2007,iHUB-MO1EMG,"=time(hour(g2),0,0)",03:55,03:59,"=h2-g2",17286,12921,4365,0,0,0,12607,4363,0,0,124,4,0,0,0,0,

mo1emg1,"=weeknum(d3,2)","=weekday(d3,2)",07Oct2007,iHUB-MO1EMG,"=time(hour(g3),0,0)",04:00,04:04,"=h3-g3",16353,12217,4136,0,0,0,11839,4136,0,0,122,4,0,0,0,0,

mo1emg1,"=weeknum(d4,2)","=weekday(d4,2)",07Oct2007,iHUB-MO1EMG,"=time(hour(g4),0,0)",04:05,04:09,"=h4-g4",15762,11684,4078,0,0,0,11359,4077,0,0,123,8,0,0,0,0,

When i run my scripts under Cygwin, everything works fine and the output is as expected.

When i run this script under Solaris OS using /usr/bin/awk, the output is:

HOST,WEEK,WEEKDAY,day,node,HOUR,start,end,SAMPLING INTERVAL,TotalRequest,MSRequest,NWIRequest,SMSBrequest,SMSBack,SMSBNack,MSRequestSuccess,NWIRequestS uccess,SMSCCongestion,ApplicationCongestion,InvalidParameters,MSIllegalSubscriber,MWIllegalSubscribe r,ApplicationUnavailble,UCPError,SMPPError,

,"=weeknum(d2,2)","=weekday(d2,2)",07Oct2007,iHUB-MO1EMG,"=time(hour(g2),0,0)",03:55,03:59,"=h2-g2",17286,12921,4365,0,0,0,12607,4363,0,0,124,4,0,0,0,0,

,"=weeknum(d3,2)","=weekday(d3,2)",07Oct2007,iHUB-MO1EMG,"=time(hour(g3),0,0)",04:00,04:04,"=h3-g3",16353,12217,4136,0,0,0,11839,4136,0,0,122,4,0,0,0,0,

,"=weeknum(d4,2)","=weekday(d4,2)",07Oct2007,iHUB-MO1EMG,"=time(hour(g4),0,0)",04:05,04:09,"=h4-g4",15762,11684,4078,0,0,0,11359,4077,0,0,123,8,0,0,0,0,

Here, looking at the 2,3,4th line, the first field (hostname) is not populated.

When i run the script using /usr/xpg4/bin/awk, i get the following error:

FIVEMIN_APPSMSHUB_OM200711071126.log
/usr/xpg4/bin/awk: file "/opt/redknee/traffic/scripts/OMParser.awk": line 2: /{/: syntax error Context is:
>>> =" "; printf("%s,", host)}
>>> $1 !~ /^#/ && $1 !~ /{/ <<<

Looking at the man pages for awk, i could make out that /usr/bin/awk does not support variable assignment (guessing). But when i tried to use /usr/xpg4/bin/awk, i get this error. I do not have an option to use gawk and have to use the awk that is available by default in SUN OS. Any idea what might be going wrong? I wish to have the value of a variable (hostname in my case) i use in my script to be populated within the AWK script. Is there any other way to do that?

Help is much appreciated.
-Deepak
# 2  
Old 11-15-2007
Enclose the variables in single quotes.

Regards
# 3  
Old 11-15-2007
On Solaris the default awk is /usr/bin/awk i.e the original (and old) awk, AKA oawk on some systems. The default awk does not support the -v assignment option.

To use a more modern version of awk on Solaris, you can use either /usr/bin/nawk (new awk) or /usr/xpg4/bin/awk.

The problem with you awk script is that you are not escaping the left brace in your ERE (extended regular expression). Braces have special meaning in EREs.

Change

$1 !~ /^#/ && $1 !~ /{/ && $1 !~ /}/ && $2 !~ /{/ {

to

$1 !~ /^#/ && $1 !~ /\{/ && $1 !~ /}/ && $2 !~ /\{/ {
# 4  
Old 11-15-2007
In nawk you should take care that -v should be entered after '' and not before.
# 5  
Old 11-15-2007
MySQL Wow!

That was awesome and quick! All i had to do was fix the opening braces and everything is working fine with /usr/xpg4/bin/awk!

Really appreciate your help guys. Thanx to everyone.
-Deepak
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Passing awk variables to bash variables

Trying to do so echo "111:222:333" |awk -F: '{system("export TESTO=" $2)}'But it doesn't work (2 Replies)
Discussion started by: urello
2 Replies

2. Shell Programming and Scripting

awk - Why can't value of awk variables be passed to external functions ?

I wrote a very simple script to understand how to call user-defined functions from within awk after reading this post. function my_func_local { echo "In func $1" } export -f my_func_local echo $1 | awk -F"/" '{for (k=1;k<=NF;k++) { if ($k == "a" ) { system("my_local_func $k") } else{... (19 Replies)
Discussion started by: sreyan32
19 Replies

3. Shell Programming and Scripting

AWK - variables

I have a data file like this: 49960 1157 32390 1227 1268 31 8 21 12 115 18493 67 250 2 2 237704 369658 52 21 312 38 27746 3174 19 160 9 555 6337 6071 43 33 I want to separate the field to three groups, $1 and $2 are the first group, $3 and $4 are the second group, $5 and $6... (1 Reply)
Discussion started by: xshang
1 Replies

4. Shell Programming and Scripting

ksh passing to awk multiple dyanamic variables awk -v

Using ksh to call a function which has awk script embedded. It parses a long two element list file, filled with text numbers (I want column 2, beginning no sooner than line 45, that's the only known thing) . It's unknown where to start or end the data collection, dynamic variables will be used. ... (1 Reply)
Discussion started by: highnthemnts
1 Replies

5. UNIX Desktop Questions & Answers

Variables within awk

Hi, I'm having trouble getting awk to read a variable with spaces in it. Input: vendorName="Bob's Steakhouse" awk -F":" '$2 ~ /'$vendorName'/ {print $1}' Purchases.dat Error: awk: $2 ~ /Bob's awk: ^ unterminated regexp The awk command isn't recognizing the entire string. It... (2 Replies)
Discussion started by: Cablephish
2 Replies

6. Shell Programming and Scripting

Using AWK variables.

Hi all, I am new to the forum and Shell Script programming. The problem is: I need to do a script to search all system processes and show me hierarchical way the number of bytes occupied by each of the regions of the memory map of each process. Today I got to show me the number of regions in... (3 Replies)
Discussion started by: cougar_rea
3 Replies

7. UNIX for Dummies Questions & Answers

Using variables in AWK

Hi, I'm pretty new to AWK and was wondering if someone could let me know how to execute varibles within an AWK statement. An example is below: NO=6 end=25 awk = 'NR == $NO, NR == $end' file1 > file2 I'm currently attempting to use this within a script but awk seems to read $NO and... (2 Replies)
Discussion started by: chris01010
2 Replies

8. Shell Programming and Scripting

awk variables

hi i am using the following awk code for some calculations: awk '/,1,/' "$LINE" >> "$count".dat awk '/,1,/ { i++ } END { print i }' "$count".dat awk '{sum+=$3} END {print sum}' 1.spd > test awk '{average=sum/i} END {print average}' 1.spd >> test can i use a variable created in one... (12 Replies)
Discussion started by: npatwardhan
12 Replies

9. Shell Programming and Scripting

OLD value for awk variables

I want to print old and new values of some field in Awk script. This is wat I done! Sample file is as follows But the output I god is bit different What must be the problem here? I am using bash. (2 Replies)
Discussion started by: yogesh_powar
2 Replies

10. Shell Programming and Scripting

Awk with variables

Hi I have a file with a line like this, wich is read by an awk script : Logical Name | Server Type | Server1 | DB1 | User1 | $PASSWORD | Serv2 |DB2 | User2 | $PASSWORD Awk reads it fine, but...it doesnt interpret the variable $PASSWORD as I wish. How do I tell awk to substitute $PASSWORD... (3 Replies)
Discussion started by: Scarlos
3 Replies
Login or Register to Ask a Question