The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
awk out an unknown column ? awk out an unknown column ?
.
google unix.com

Forums Register Forum RulesLinksAlbums FAQ Members List Calendar Search Today's Posts Mark Forums Read


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Changing one column of delimited file column to fixed width column manneni prakash Shell Programming and Scripting 5 06-22-2009 06:27 AM
PING - Unknown host 127.0.0.1, Unknown host localhost - Solaris 10 Przemek SUN Solaris 4 05-26-2008 01:11 AM
how to read the column and print the values under that column gemini106 Shell Programming and Scripting 6 03-28-2008 07:05 AM
How to check Null values in a file column by column if columns are Not NULLs Mandab Shell Programming and Scripting 7 03-15-2008 09:57 AM
Replace 10th column with a new column--- Terriblly hurry ahmedwaseem2000 Shell Programming and Scripting 2 09-06-2005 02:10 AM

Closed Thread
Page 1 of 3 1 23
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Bulgarian Greek Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 08-02-2008
i9300 i9300 is offline
Registered User
  
 

Join Date: Jan 2008
Posts: 14
awk out an unknown column ?
I need a solution to awk out an unknown column. I am unable to say '{print $x}' because the location changes. I would like to find a perl or awk solution to this. I do not know either very well but am trying to delve deeper into both.
I am looking for the version of pkg8 in this example. Please keep in mind the location of pkg8 will change depending...

Here is the line I am looking at :


Code:
# grep pkg8 /tmp/trash
XXX Packages             : XXXpkg1(6.2.3.1) XXXpkg2(1.0.0.3) XXXpkg3(4.0.0.5) XXXpkg4(2.0.0.1) XXXpkg5(A0)  XXXpkg6(2.0.1.1)  XXXpkg8(5.7.0.10) XXXpkg9(1.5) XXXpkg10(2.1.2.3)

I was able to find a UGLY solution which includes a for loop and if statement. But this is very slow. I am looking at this file on 200 + systems, so the execution does matter.

Here is the entire script for this :


Code:

#!/bin/ksh
HOSTNAME=`uname -n`
OS_VER=`uname -r`

PKG=XXXpkg8
CORRECT_VER=5.7.0.10
COLLECT=/tmp/trash
for x in `grep $PKG $COLLECT|awk -F: '{print $2}'`;do
    if [ `echo $x|grep "XXXpkg8("` ];then
        PKG_VER=`echo $x|grep "XXXpkg8("`
    else
        continue
    fi
done
VER=`echo $PKG_VER|awk -F"(" '{print $2}'|sed -e 's/)//'`



if [[ $VER = $CORRECT_VER ]];then
        if [[ $STATUS = NO ]];then
            continue
        else
            STATUS=YES
        fi
else
    STATUS=NO
    MESSAGE="$MESSAGE XXXpkg8:$VER"
fi

echo "$STATUS $MESSAGE"

I would like to replace the entire for loop with a perl/awk one liner. I have searched for days and frankly my head hurts already. I have seen grep and for loop used inside of awk but I really don't understand what it is doing.
problems in grep inside awk - Computing.Net
[url=http://www.linuxquestions.org/questions/programming-9/can-i-use-grep-inside-awk-174287[/url]


If anyone can offer some help I would appreciate it.
  #2 (permalink)  
Old 08-02-2008
ghostdog74 ghostdog74 is offline Forum Advisor  
Registered User
  
 

Join Date: Sep 2006
Posts: 2,560

Code:
awk '{
  for( ......) #go through all fields
  {
   if( field contains what i want ){
     print field
   }
  }

}' file
  #3 (permalink)  
Old 08-03-2008
aigles's Avatar
aigles aigles is offline Forum Advisor  
Registered User
  
 

Join Date: Apr 2004
Location: Bordeaux, France
Posts: 1,433
Replace the following lines :
Code:
for x in `grep $PKG $COLLECT|awk -F: '{print $2}'`;do
    if [ `echo $x|grep "XXXpkg8("` ];then
        PKG_VER=`echo $x|grep "XXXpkg8("`
    else
        continue
    fi
done
VER=`echo $PKG_VER|awk -F"(" '{print $2}'|sed -e 's/)//'`

by :

Code:
VER=$( sed -n 's/.*'"$PKG"'(\([^)]*\)).*/\1/p'  $COLLECT )

Jean-Pierre.
  #4 (permalink)  
Old 08-04-2008
i9300 i9300 is offline
Registered User
  
 

Join Date: Jan 2008
Posts: 14
ghostdog74,

While I appreciate your post, I am not an expert with awk. I was unable to get your example to work. This is what I tried.

Code:
awk '{
  for( `grep $PKG $COLLECT|awk -F: '{print $2}'` ) #go through all fields
  {
   if( XXXpkg8 ){
     print field
   }
  }

}' $COLLECT

I am not sure how this will populate the $VER which is what I am after.

Aigles,

Thanks for the example. I was able to drop the one line in and it worked perfectly. Since I am no expert with sed/awk/perl, I really appreciate you providing an example that I could just drop in. However it turned out to be very slow. Since I will be running this within another for loop (about 200 times), I need it to run faster.

I ran your sed in ./check_pkg and my code was in check_pkg2.

Code:
root@smsdr2# time ./check_pkg
YES

real    0m1.737s
user    0m1.500s
sys     0m0.000s
root@smsdr2# time ./check_pkg2
YES

real    0m0.365s
user    0m0.040s
sys     0m0.060s


Code:
root@smsdr2# diff check_pkg check_pkg2
8,16c8,15
< #for x in `grep $PKG $COLLECT|awk -F: '{print $2}'`;do
< #     if [ `echo $x|grep "XXXpkg8("` ];then
< #             PKG_VER=`echo $x|grep "XXXpkg8("`
< #     else
< #             continue
< #     fi
< #done
< #VER=`echo $PKG_VER|awk -F"(" '{print $2}'|sed -e 's/)//'`
< VER=$( sed -n 's/.*'"$PKG"'(\([^)]*\)).*/\1/p'  $COLLECT )
---
> for x in `grep $PKG $COLLECT|awk -F: '{print $2}'`;do
>       if [ `echo $x|grep "XXXpkg8("` ];then
>               PKG_VER=`echo $x|grep "XXXpkg8("`
>       else
>               continue
>       fi
> done
> VER=`echo $PKG_VER|awk -F"(" '{print $2}'|sed -e 's/)//'`
17a17,18
>
>

Any other ideas?
  #5 (permalink)  
Old 08-04-2008
aigles's Avatar
aigles aigles is offline Forum Advisor  
Registered User
  
 

Join Date: Apr 2004
Location: Bordeaux, France
Posts: 1,433
Quote:
Originally Posted by i9300 View Post
ghostdog74,

While I appreciate your post, I am not an expert with awk. I was unable to get your example to work. This is what I tried.

Code:
awk '{
  for( `grep $PKG $COLLECT|awk -F: '{print $2}'` ) #go through all fields
  {
   if( XXXpkg8 ){
     print field
   }
  }

}' $COLLECT

I am not sure how this will populate the $VER which is what I am after.

Aigles,

Thanks for the example. I was able to drop the one line in and it worked perfectly. Since I am no expert with sed/awk/perl, I really appreciate you providing an example that I could just drop in. However it turned out to be very slow. Since I will be running this within another for loop (about 200 times), I need it to run faster.

I ran your sed in ./check_pkg and my code was in check_pkg2.

Code:
root@smsdr2# time ./check_pkg
YES

real    0m1.737s
user    0m1.500s
sys     0m0.000s
root@smsdr2# time ./check_pkg2
YES

real    0m0.365s
user    0m0.040s
sys     0m0.060s


Code:
root@smsdr2# diff check_pkg check_pkg2
8,16c8,15
< #for x in `grep $PKG $COLLECT|awk -F: '{print $2}'`;do
< #     if [ `echo $x|grep "XXXpkg8("` ];then
< #             PKG_VER=`echo $x|grep "XXXpkg8("`
< #     else
< #             continue
< #     fi
< #done
< #VER=`echo $PKG_VER|awk -F"(" '{print $2}'|sed -e 's/)//'`
< VER=$( sed -n 's/.*'"$PKG"'(\([^)]*\)).*/\1/p'  $COLLECT )
---
> for x in `grep $PKG $COLLECT|awk -F: '{print $2}'`;do
>       if [ `echo $x|grep "XXXpkg8("` ];then
>               PKG_VER=`echo $x|grep "XXXpkg8("`
>       else
>               continue
>       fi
> done
> VER=`echo $PKG_VER|awk -F"(" '{print $2}'|sed -e 's/)//'`
17a17,18
>
>

Any other ideas?
A little modification to the sed command will increase performances :
Code:
VER=$( sed -n "/$PKG/"'s/.*'"$PKG"'(\([^)]*\)).*/\1/p'  $COLLECT )

I have tested the three solutions with a file containing 6900 lines (package versions on last line) on my PC (cygwin) and on my AIX box.

The initial version run faster under AIX only (i am very surprise by the difference between Cygwin and AIX results).
The second sed version is the fastest in the two case.

Cygwin times :
Code:
---- Initial version ---
YES  
real    0m3.140s
user    0m1.047s
sys     0m0.719s
---- sed version -------
YES  
real    0m0.406s
user    0m0.215s
sys     0m0.137s
---- sed version 2 -----
YES  

real    0m0.360s
user    0m0.231s
sys     0m0.090s

AIX times :
Code:
---- Initial version ---
YES  
réel    0m0,14s
util    0m0,00s
sys     0m0,09s
---- sed version -------
YES  
réel    0m1,55s
util    0m1,52s
sys     0m0,03s
---- sed version 2 -----
YES  

réel    0m0,03s
util    0m0,02s
sys     0m0,01s

Jean-Pierre.
  #6 (permalink)  
Old 08-04-2008
ghostdog74 ghostdog74 is offline Forum Advisor  
Registered User
  
 

Join Date: Sep 2006
Posts: 2,560
@OP
to go through the fields in awk

Code:
awk '
{
 for(i=1;i<=NF;i++ ){
   if ( $i == "what i want" ) {
      # do something.
   } 
 } 

}

' file
  #7 (permalink)  
Old 08-04-2008
era era is offline Forum Advisor  
Herder of Useless Cats (On Sabbatical)
  
 

Join Date: Mar 2008
Location: /there/is/only/bin/sh
Posts: 3,652
Why do you need to run it in another loop? Perhaps you could show us the whole code so we can see what to optimize. (I'm thinking maybe you could run the whole loop in sed.)
Closed Thread
Page 1 of 3 1 23

Bookmarks

« check the file in multiple Directory in a script | Find a string under a directory that is contained in another file »
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Hybrid Mode Hybrid Mode
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On



All times are GMT -4. The time now is 06:19 AM.

The UNIX and Linux Forums - Learn UNIX and UNIX Commands RSS Feeds - Contact Us - The UNIX and Linux Forums - Learn UNIX and UNIX Commands - Archive - Top

Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0