Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk search pattern in column

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk search pattern in column
# 8  
Old 08-12-2014
I'd guess the vars are unset right after login. So you need to return to the unquoted EOF to have the vars expanded by your local shell, but have e.g. all the $ signs escaped so they are delivered to awk on the remote system.
# 9  
Old 08-12-2014
Quote:
Originally Posted by Roozo
Thanks a lot Don.

The highlighted first and second variables return empty.

Code:
ssh   user@box -P password  <<-"EOF"
awk  -vvar1=”${VAR}” -vvar2=”${SYSTEM}”   -F”\|”    ‘($1 == var1 ) &&  ($3 == var2) {print $1,$2}'   file  | while read first second 
do 
echo “${first}”    “${second}” 
                if [ $? -eq 0 ]
                                then 
                                echo “true”
                else 
                                echo “false”
                fi
done  
EOF

Could you please let know how to resolve this?
Have you use a windows editor (word processor) to write the script?
You're using wrong quotes, “” instead of "" and ‘' instead of ''.
# 10  
Old 08-12-2014
I guess I should have looked closer at what you were doing instead of how you were doing it. It appears that you want the shell to expand "${VAR}" and "${SYSTEM}" in the here-document, but you don't want the shell to eland "${first}" or "${second}". I was looking at the expansions of first and second when I suggested quoting EOF.

Furthermore, unless stdout is unwriteable, your if statement can be replaced by echo true since the exit status from the 1st echo will always be zero unless stdout is not writeable. (And, in that case, the echo false will also fail.) What are the true and false supposed to signify?

What directory will ssh place you in on the system named box? Is file in that directory?

Will VAR and SYSTEM have defined values on box in the environment in which this script will be run?

On some systems, awk requires a space between -v and the following variable assignment.

See if the following will come closer to what you want:
Code:
ssh   user@box –P password  <<-EOF
awk -v var1="${VAR}" –v var2="${SYSTEM}" -F"\|" '($1 == var1 ) &&  ($3 == var2) {print $1,$2}' file |
while read first second 
do 
        echo "\${first}" "\${second}" 
        if [ $? –eq 0 ]
        then 
                echo "true"
        else 
                echo "false"
        fi
done  
EOF

If the expansions of VAR, SYSTEM, first, or second contain any whitespace characters this may need more work.
Last edited by Don Cragun; 08-12-2014 at 07:55 AM.. Reason: Fix single and double quotes.
# 11  
Old 08-12-2014
Thanks for the post Don.

I created the post when search were created problem now while execution there were issue, so I will explain what I'm doing inside the shell.

I'm trying to log in as remote user using (ssh) on a UX-HP server and trying to recycle topends for particular user on box.

Code:
#!/bin/ksh 
TPFILE="/directories/topend.ini"
TPCFG="/directories/tp_recycle.cfg"
TPLOG="/directories/tp_recycle.log"
BOX=`hostname`

PASS=`stanzaget ${TPCFG} SERVERPASS ${BOX}`

echo "Enter USER" && read USER; USER=`echo $USER | tr 'A-Z' 'a-z'`

ssh ${USER}@${BOX} –P ${PASS} <<-"EOF"         # 2>>/dev/null   shall we use this here to avoid warnings?
awk  -F"\|" '$1 ~ /'"${BOX}"'/ && $2 ~ /'"${USER}"'/ {print $2,$3,$4}' ${TPFILE} | while read subsystem process actualcount 
do 
	sleep 1 
	recycle.sh  ${subsystem} ${process}       ## This program will recycle the topend.

		if [ $? -eq 0 ]
		then 
			Recyclecount=`ps -ef | grep -c ${process}`		

                	if [ ${Recyclecount} -lt ${actualcount} ]
			then 
				echo "Only ${Recyclecount} running for ${process} on ${subsystem}" >>${TPLOG}
			else 
				echo "TP_RECYCLE success for ${process} on ${subsystem}" >>${TPLOG}
			fi
		else 
			echo "Recycle failed for  ${process} on ${subsystem}" >>${TPLOG}
		fi 	
done  
EOF

cat /directories/topend.ini
box1|user1|process1|3|
box1|user1|process2|3|
box1|user2|process1|4|
box1|user3|process1|2|
box1|user4|process1|3|
box2|user1|process1|3|
box2|user2|process2|3|
box2|user2|process3|4|
box2|user3|process4|2|
box2|user4|process5|3|

cat /directories/tp_recycle.cfg
SERVERPASS:
box1=password
box2=password

I'm using ssh to avoid log in as each user to recycle. There are number of user on particular box. If issue exist on certain users we can provide the users one by one in prompt and recycle the topends on the particular user.

---------- Post updated at 11:29 PM ---------- Previous update was at 05:58 PM ----------

Don,

What directory will ssh place you in on the system named box ? Is file in that directory?
/bin/ssh
It is a file

On some systems, awk requires a space between -v and the following variable assignment.
ksh accepts without space between -v and also will use ~ for search. Only during ssh "EOF" execution it return empty.

How to resolve the issue on the above code?

Thanks in advance.
Last edited by Roozo; 08-12-2014 at 02:50 PM..
# 12  
Old 08-12-2014
This is untested, but I think it has a chance of working. The EOF has been unquoted and expansions of variables that are defined inside the here-document are escaped. This will allow ksh to expand variables defined before you enter the here-document to be expanded before ssh sees them and should have the shell started by ssh expand the variables that are defined by the script that is in the here-document. I also converted `command` forms of command substitution to the $(command) form of command substitution, removed some trailing spaces and tabs, and rearranged some code to reduce some line lengths:
Code:
#!/bin/ksh 
TPFILE="/directories/topend.ini"
TPCFG="/directories/tp_recycle.cfg"
TPLOG="/directories/tp_recycle.log"
BOX=$(hostname)

PASS=$(stanzaget ${TPCFG} SERVERPASS ${BOX})

echo "Enter USER" && read USER; USER=$(echo $USER | tr 'A-Z' 'a-z')

ssh ${USER}@${BOX} –P ${PASS} <<-EOF	# 2>>/dev/null   shall we use this here to avoid warnings?  NO!
awk  -F"\|" '$1 ~ /'"${BOX}"'/ && $2 ~ /'"${USER}"'/ {print $2,$3,$4}' ${TPFILE} |
while read subsystem process actualcount 
do 
	sleep 1 
	## This program will recycle the topend.
	if recycle.sh  \${subsystem} \${process}
	then 
		Recyclecount=$(ps -ef | grep -c \${process})
		if [ \${Recyclecount} -lt \${actualcount} ]
		then 
			echo "Only \${Recyclecount} running for \${process} on \${subsystem}" >>${TPLOG}
		else 
			echo "TP_RECYCLE success for \${process} on \${subsystem}" >>${TPLOG}
		fi
	else 
		echo "Recycle failed for \${process} on \${subsystem}" >>${TPLOG}
	fi 
done
EOF

I hope this helps...
# 13  
Old 08-12-2014
Thanks Don,

If we use EOF without quote it will create a below error right, so I have moved the awk out of loop
Code:
The error context is

Code:
                           ( >>>  ~  <<<

Code:
#!/bin/ksh 
TPFILE="/directories/topend.ini"
TPCFG="/directories/tp_recycle.cfg"
TPLOG="/directories/tp_recycle.log"
TEMP="/directories/temp"
BOX=$(hostname)
 
PASS=$(stanzaget ${TPCFG} SERVERPASS ${BOX})
 
echo "Enter USER" && read USER; USER=$(echo $USER | tr 'A-Z' 'a-z')
 
awk -F"\|" '$1 ~ /'"${BOX}"'/ && $2 ~ /'"${USER}"'/ {print $2,$3,$4}' ${TPFILE} >${TEMP}
 
ssh ${USER}@${BOX} –P ${PASS} <<-EOF 
cat ${TEMP} | while read subsystem process actualcount 
do 
 
           sleep 1 
           ## This program will recycle the topend.
           if recycle.sh \${subsystem} \${process}
           then 
                    Recyclecount=$(ps -ef | grep -c \${process})
                    if [ \${Recyclecount} -lt \${actualcount} ]
                    then 
                          echo "Only \${Recyclecount} running for \${process} on \${subsystem}" >>${TPLOG}
                    else 
                          echo "TP_RECYCLE success for \${process} on \${subsystem}" >>${TPLOG}
                     fi
           else 
                 echo "Recycle failed for \${process} on \${subsystem}" >>${TPLOG}
           fi 
done
EOF

And it is working
# 14  
Old 08-13-2014
Great!

Just to make me happy, please get rid of the unneeded pipeline and use of cat. It will produce the same results, but run faster and use fewer system resources. Change:
Code:
cat ${TEMP} | while read subsystem process actualcount 
do
        ... ... ... 
done

to:
Code:
while read subsystem process actualcount 
do
        ... ... ... 
done < ${TEMP}
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to find string based on pattern and search for its corresponding rows in column

Experts, Need your support for this awk script. we have only one input file, all these column 1 and column 2 are in same file and have to do lookup for values in one file(column1 and column2) but output we need in another file Need to grep row whose string contains 9K from column 1. When found... (6 Replies)
Discussion started by: as7951
6 Replies

2. UNIX for Beginners Questions & Answers

If pattern in column 3 matches pattern in column 2 (any row), print value in column 1

Hi all, I have searched and searched, but I have not found a solution that quite fits what I am trying to do. I have a long list of data in three columns. Below is a sample: 1,10,8 2,12,10 3,13,12 4,14,14 5,15,16 6,16,18 Please use code tags What I need to do is as follows: If a... (4 Replies)
Discussion started by: bleedingturnip
4 Replies

3. UNIX for Beginners Questions & Answers

Grep/awk using a begin search pattern and end search pattern

I have this fileA TEST FILE ABC this file contains ABC; TEST FILE DGHT this file contains DGHT; TEST FILE 123 this file contains ABC, this file contains DEF, this file contains XYZ, this file contains KLM ; I want to have a fileZ that has only (begin search pattern for will be... (2 Replies)
Discussion started by: vbabz
2 Replies

4. Shell Programming and Scripting

Search Pattern and Print lines in Single Column

Hi Experts I have small query where I request the into a single file Suppose: File1: {Unique entries} AA BB CC DD FileB: AA, 123 AA, 234 AA, 2345 CC, 123 CC, 5678 DD,123 BB, 7890 (5 Replies)
Discussion started by: navkanwal
5 Replies

5. Shell Programming and Scripting

awk with multiple pattern search

Want to fetch a column with multiple pattern using awk. How to achieve the same. Tried cat test address : 10.63.20.92/24 address : 10.64.22.93/24 address : 10.53.40.91/24 cat test | awk '{print $3}' |awk -F "/" '{print $1}' 10.63.20.92 10.64.22.93 10.53.40.91 Is there any... (2 Replies)
Discussion started by: Manasa Pradeep
2 Replies

6. Shell Programming and Scripting

Awk to match a pattern and perform a search after the first pattern

Hello Guyz I have been following this forum for a while and the solutions provided are super useful. I currently have a scenario where i need to search for a pattern and start searching by keeping the first pattern as a baseline ABC DEF LMN EFG HIJ LMN OPQ In the above text i need to... (8 Replies)
Discussion started by: RickCharles
8 Replies

7. UNIX for Dummies Questions & Answers

Pattern search using awk

Hi All, I am trying to find numbers with balance greater than 1 and less than equal 2 from the below file using awk Input file num ,bal 100199,1.708 100225,0 100226,0 100228,0.771166 100232,2 output file 100199,1.708 100232,2 I am using the following command for this... (2 Replies)
Discussion started by: pistachio
2 Replies

8. Shell Programming and Scripting

Pattern Search using AWK

Hi All, I have the below file data.txt.Using awk i want to grep all the zone data.Form the below command i can extact data upto of zone i give but i want it should print until next pattern. awk '/^Total Collection /{c=5;next}c-->0' zin45srs08.tools_utilization instead of c=5 is it possible... (5 Replies)
Discussion started by: ajaincv
5 Replies

9. Shell Programming and Scripting

awk search pattern

Hi, I have a log file which contains lines like below: 2010-07-19 07:13:19,021 ERROR system ...(text) 2010-07-19 07:22:03,427 ERROR system ...(text) class com... (text) 2010-07-19 07:23:19,026 ERROR system ...(text) class com... (text) each line is a separate line... I am given the a... (14 Replies)
Discussion started by: a27wang
14 Replies

10. UNIX for Dummies Questions & Answers

Search Pattern and add column

Hi, I have two files. file1 contents: aaa bbb ccc ddd eee fff ggg ddd www eee ggg dde qqq zzz hhh ddd file2 contents: mmm mmm mmm mmm Now I want to add file2 contents to end of lines in file1 where a line contains pattern "ddd" and it should look like this: file3 contents: aaa... (3 Replies)
Discussion started by: harjitsingh
3 Replies
Login or Register to Ask a Question