gzcat into awk and then change FILENAME and process new FILENAME

timj123 · #1 (**permalink**) 06-21-2008

I am trying to write a script that prompts users for date and time, then process the gzip file into awk. During the ksh part of the script another file is created and needs to be processed with a different set of pattern matches then I need to combine the two in the end. I'm stuck at the part where I have to change the FILENAME and FS part. Can anyone shed some light on this for me please.

Thanks

Code:

#!/bin/ksh
tput clear

dir=/my/dir/here/
ifile=/tmp/ifile
ofile=/tmp/ofile

echo "Enter the date and hour to run this script on [YYMMDDHH]: \c"
read file


/usr/bin/gzcat $file | /usr/xpg4/bin/awk '

BEGIN {
print "headers go here"
}


NR == 3 { print $2,$3"  --  "$4,$5 ; print "" ; next }

/^dise / {
        dipce = substr($2,4,1) * 1000 ; next
           }
/^LC/ {
        lec = substr($0,3,3) + dipce
        f++
        s = ""
        for ( i in arr ) { delete arr[i] }
        }
	   f {
		run some more stuff until gzcat is finished
              }

###Here I want to change FILENAME to "'"$ifile"'" ###

#Run some more awk code here while reading in "'"$ifile"'"




END {
Print some stuff from both processed file
}

radoulov · #2 (**permalink**) 06-22-2008

You can use getline to start reading the $ifile whenever you want, but I would prefer something like this: define $ifile as a second argument and then process it as usual: NR == FNR -> action -> next for the first one, action for the second.
FS is not a problem, you can change it anytime you want.

robotronic · #3 (**permalink**) 06-22-2008

To read lines from a different file other than the one you piped through awk, you need "getline" function.

Code:

getline <"'"$ifile"'"

or

getline var <"'"$ifile"'"

In the first case, $0 will be overwritten with the first line read from $ifile, in the second case you retain $0 of the original file (=the output of gzcat) and set a new variable "var" holding the first line of $ifile.

If you want, you can also avoid that pretty ugly syntax for the file name ("'"$ifile"'") by using "-v" option of awk:

Code:

/usr/bin/gzcat $file | /usr/xpg4/bin/awk -v "ifile=$ifile" ' ... '

and then reference your file with the internal awk variable "ifile", like this:

Code:

getline <ifile

Don't miss awk man page for details on getline.

timj123 · #4 (**permalink**) 06-22-2008

Thanks guys, I have learned so much from the both of you. I am still having issues that I can't figure out, here is the script:

Code:

/usr/bin/gzcat $file | /usr/xpg4/bin/awk '

BEGIN {
print "headers go here"
}


NR == 3 { print $2,$3"  --  "$4,$5 ; print "" ; next }

/^dise / {
        dipce = substr($2,4,1) * 1000 ; next
           }
/^LC/ {
        lec = substr($0,3,3) + dipce
        f++
        s = ""
        for ( i in arr ) { delete arr[i] }
        }
	   f {
              if ( $1 == "RRAC" ) {
                s++
                 }
                if ( s ) {
                    if ( $1 == "tsec" ) {
                       lectsec = $2 * 1
                       }
                    if ( $1 == "VAR" ) {
                        car[$2 * 1]+=$8
                        lecCar = $2 * 1
                        c[lec,lectsec,BCar] = 0
                        }
                     c[lec,lectsec,BCar]++
                     if ( c[lec,lectsec,BCar] == 8 ) {
                        car[BCar]+=$8
                        tst[lec"-"BCar]=sprintf("%5.2f",car[BCar] / 12)
                        }
                    }
                if ( $1 == "RRAC-L" ) {
                        f = "" ; next
                   }
                   }


###Here I want to change FILENAME to "'"$ofile"'" ###
#####################################################

{
while (( getline < "'"$ofile"'" ) > 0 ) {
FS = "\;" ; print $1,$3,$4 ; next }
}
#####################################################
END {
for ( i in tst ) { print i,tst[i] }

    } '

/usr/bin/rm $ifile
/usr/bin/rm $ofile

When I change FS it also changes the output of the following:

Code:

for ( i in tst ) { print i,tst[i] }

I can't figure this out??? Why does this change the output of the array?

robotronic · #5 (**permalink**) 06-23-2008

You need to set the OFS variable with the desired Output Field Separator, by default a space.
Or try using "nawk". I've noticed this behavior on Solaris:

Code:

> echo "aaa:bbb:ccc" | /usr/xpg4/bin/awk '{ FS=":"; print $2,$3 }'
 
> echo "aaa:bbb:ccc" | /bin/awk '{ FS=":"; print $2,$3 }'
bbb ccc

> echo "aaa:bbb:ccc" | /bin/nawk '{ FS=":"; print $2,$3 }'
bbb ccc

The first command returns null!

ghostdog74 · #6 (**permalink**) 06-23-2008

@OP, use split inside you while loop

Code:

...
while (( getline line< "'"$ofile"'" ) > 0 ) {
 n=split(line,temp,"\")
 print temp[1],temp[3] ......
 next }
}
...

timj123 · #7 (**permalink**) 06-23-2008

Thanks guys, I will try them both.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
change the filename by adding up 1 each time, tricky one	netbanker	Shell Programming and Scripting	5	03-07-2008 04:45 PM
Change multiple filename formats with WHILE	ScKaSx	UNIX for Dummies Questions & Answers	5	02-27-2008 07:11 PM
shortcut for tar cvf - [filename] \| gzip > [filename].tar.gz	bcamp1973	UNIX for Dummies Questions & Answers	4	12-11-2007 05:45 PM
script to change filename with numbers	silversurfer202	Shell Programming and Scripting	5	11-10-2006 10:13 AM
Change new filename with date ??	sabercats	Shell Programming and Scripting	9	02-13-2006 05:12 PM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Switch to Linear Mode Hybrid Mode Switch to Threaded Mode	Rate This Thread:


	Powered by