bash variable (set via awk+sed) not working as expected


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting bash variable (set via awk+sed) not working as expected
# 1  
Old 04-18-2011
bash variable (set via awk+sed) not working as expected

Hi!

Been working on a script and I've been having a problem. I've finally narrowed it down to this variable I'm setting:
Code:
servername=$(awk -v FS=\/ '{ print $7 } blah.txt | sed 's\/./-/g' | awk -v FS=\- '{print $1}')"

This will essentially pare down a line like this:
Code:
http://1.2.3.4/blah/moreblah/soo.blah/myserver1.domain.com/index.html#anchor>link

To simply this: myserver1

It's a bit involved, but it works.

The issue I'm having is that I've placed this within a loop (for x in 1 2... while 1 do ...) and the variable SERVERNAME parses through all the data on the first pass through so when the loop comes around a second time, there's no data and the script poops out saying "awk: can't open file XXXX" [where XXXX equals a myserver value]

If I set SERVERNAME=myserver1 manually everything works as expected.

Anyone know why setting this variable as command output is executing multiple times rather than just the single iteration I'm looking for? This behavior is showing up in awk and gawk.

Cheers!

-cd
ps. I've also used backticks, pushing the contents of this command to a temp file from which I "<temp.txt" the data into the variable, but it is always the same result.

Moderator's Comments:
Mod Comment Please use [code] and [/code] tags when posting code, data or logs etc. to preserve formatting and enhance readability, thanks.

Last edited by zaxxon; 04-18-2011 at 06:18 AM.. Reason: code tags
# 2  
Old 04-18-2011
Can you post your script..at least the loop portion..?
# 3  
Old 04-18-2011
Hard to find out with just those few lines. Maybe post the whole script or/and use a set -x to show debugging output of the shell.
There seems a leading " missing at
Code:
servername="$(awk
           ^

# 4  
Old 04-18-2011
the loop bit of the script

Here's the loop bit of the script: (The missed quotation mark was a typo..I wasn't able to cut/paste.. had to manually type the example in)

Code:
for x in warn crit

do
echo "- - - - - - - - - - -"
curl -s | grep $x > temp-data.txt

awk -v FS=\" '{ for (i = 1; i <= NF; i++)
       if ($i ~ /'"$x"'/)
          print $i,$(i+3),$(i+2) }' temp-data.txt > blah.txt

# Let's strip off all the malformed HTML from the results (blah.txt)...
sed 's/>//g' blah.txt | sed 's/<\/a>//g' | sed 's/<\/a//g' > blah2.txt

# Now let's add properly formatted HTML to the results so we get valid weblinks to each server.
sed 's/<a   href=\ /<a href=http:\/\/1.2.3.4\/directory\/archive\//g' blah2.txt | sed 's/$/>link<\/a>/' > blah3.txt

# Strip out a servername from the URL to make for easy readability...

This is the line that seems to go into its own nested loop before proceeding to the next step below which prints the single line of output I'm looking for.
Code:
servername=$(awk -v FS=\/ '{ print $8 }' blah3.txt | sed 's/\./-/g' | awk -v FS=\- '{ print $1 }')

# Now printing data out again with our new column added.
awk -v serv=$servername -v mtime=$mtime -v mdate=$mdate '{ print mdate,mtime,$1,$2,serv,$3,$4 }' blah3.txt

done

Thanks!

-cd
# 5  
Old 04-19-2011
Just to clarify, if I set SERVERNAME like this, everything works. If I set it as shown in the previous post, it errors out.

Code:
servername=1234

Still looking for a hand with this if anyone has ideas...

Thanks!
-cd
# 6  
Old 04-20-2011
Quote:
Originally Posted by creativedynamo
servername=$(awk -v FS=\/ '{ print $8 }' blah3.txt | sed 's/\./-/g' | awk -v FS=\- '{ print $1 }')

# Now printing data out again with our new column added.
awk -v serv=$servername -v mtime=$mtime -v mdate=$mdate '{ print mdate,mtime,$1,$2,serv,$3,$4 }' blah3.txt

done
Are you sure that your printing the right column. The above awk command prints the 8th column.Also the command could be shortened as
Code:
servername=$(awk -F\/ '{ print $7 }' blah3.txt | awk  FS=\. '{ print $1 }')

Also as you have said that there is no value while it loops the 2nd time check the for loop values' 2nd iteration i.e crit and additionally before running the whole script try each command after curl manually and check if it gives the expected output or not. Alternatively you can also use set -x on top of the script to view the script output in de-bug mode.
# 7  
Old 04-21-2011
Thanks for the tip on shortening the command.

It seems that the variable declaration evaluates all data on the first time through. I was only looking for a single iteration so I could nicely insert the variable in the last awk command. It's looking like I'll need to store all these iterations in an array and call it as a variable in the last awk statement, but I'm weak on arrays and I'm not sure it'll actually work.

If it won't work that way, perhaps I can just save all the SERVERNAME variable values to another text file then attempt to join them?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash read input in case statement not working as expected

I'm having an issue with bash read input when using a case statement. The script halts and doesn't read the input on the first loop. if I hit enter then the scripts starts to respond as expected. Need some help here. defaultans=8hrs read -e -i $defaultans -p "${bldwht}How long would you like... (5 Replies)
Discussion started by: woodson2
5 Replies

2. Shell Programming and Scripting

awk matching script not working as expected

This is my ubuntu version: $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04.4 LTS Release: 16.04 Codename: xenial $ /bin/awk -V | head -n1 bash: /bin/awk: No such file or directory I have gotten a script that helps me to parse,... (14 Replies)
Discussion started by: delbroooks
14 Replies

3. Shell Programming and Scripting

Bash variable assignment failure/unary operator expected

I have a little code block (executing on AIX 7.1) that I cannot understand why the NOTFREE=0 does not appear to be assigned even though it goes through that block. This causes a unary operator issue. #!/bin/bash PLATFORM="AIX" NEEDSPC=3000 set -x if ; then lsvg | grep -v rootvg | while... (6 Replies)
Discussion started by: port43
6 Replies

4. Shell Programming and Scripting

awk gsub not working as expected

Hi Experts, Need your kind help with gsub awk. Below is my pattern:"exec=1_host_cnt=100_dup=4_NameTag=targetSrv_500.csv","'20171122112948"," 100"," 1"," 1"," 4","400","","", " aac sample exec ""hostname=XXXXX commandline='timeout 10 openssl speed -multi 2 ; exit 0'"" ","-1","-1","1","... (6 Replies)
Discussion started by: pradyumnajpn10
6 Replies

5. Shell Programming and Scripting

awk command not working as expected

Following one line of awk code removes first 3 characters from each line but when I run the same code on another linux platform it doesn't work and only prints blank lines for each record. Can anyone please explain why this doesn't work? (31 Replies)
Discussion started by: later_troy
31 Replies

6. Shell Programming and Scripting

Bash question: working with an array of previously set variable strings

while i've used arrays to work with variables, i've never used them to loop through a set of strings and wanted to ask the community for some feedback or assistance. let me be specific. here's my code: # URL port Variables port2195=`nc -z $url2195 2195` port2196=`nc -z $url2196 2196`... (5 Replies)
Discussion started by: hungryd
5 Replies

7. Shell Programming and Scripting

awk not working as expected in script

Dear all, I had script which used to work, but recently it is not working as expected. I have command line in my shell script to choose the following format from the output_elog and perform some task afterwards on As you see, I want all numbers in foramt following RED mark except for... (12 Replies)
Discussion started by: emily
12 Replies

8. UNIX for Advanced & Expert Users

sed not working as I expected. What can be the possible reason?

# cat test <abc> </abc> <abc> </abc> # sed 's/abc/checker/' test>test1 # cat test1 <checker> </checker> <checker> </checker> # I want the first instance of abc to be changed to checker. But all(rest 3 also) are changed. I din't used global (g) option too.. What can be the... (5 Replies)
Discussion started by: Ugenther
5 Replies

9. Shell Programming and Scripting

Var substitution in awk - not working as expected

countA=`awk '/X/''{print substr($0,38,1)}' fName | wc -l` countB=`wc -l fName | awk '{print int($1)}'` echo > temp ratio=`awk -va=$countA -vc=$countB '{printf "%.4f", a/c}' temp` After running script for above I am getting an error as : awk: 0602-533 Cannot find or open file -vc=25. The... (3 Replies)
Discussion started by: videsh77
3 Replies

10. Shell Programming and Scripting

awk not working as expected with BIG files ...

I am facing some strange problem. I know, there is only one record in a file 'test.txt' which starts with 'X' I ensure that with following command, awk /^X/ test.txt | wc -l This gives me output = '1'. Now I take out this record out of the file, as follows : awk /^X/ test.txt >... (1 Reply)
Discussion started by: videsh77
1 Replies
Login or Register to Ask a Question