Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

gzcat into awk and then change FILENAME and process new FILENAME

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting gzcat into awk and then change FILENAME and process new FILENAME
# 1  
Old 06-21-2008
gzcat into awk and then change FILENAME and process new FILENAME
I am trying to write a script that prompts users for date and time, then process the gzip file into awk. During the ksh part of the script another file is created and needs to be processed with a different set of pattern matches then I need to combine the two in the end. I'm stuck at the part where I have to change the FILENAME and FS part. Can anyone shed some light on this for me please.

Thanks

Code:
#!/bin/ksh
tput clear

dir=/my/dir/here/
ifile=/tmp/ifile
ofile=/tmp/ofile

echo "Enter the date and hour to run this script on [YYMMDDHH]: \c"
read file


/usr/bin/gzcat $file | /usr/xpg4/bin/awk '

BEGIN {
print "headers go here"
}


NR == 3 { print $2,$3"  --  "$4,$5 ; print "" ; next }

/^dise / {
        dipce = substr($2,4,1) * 1000 ; next
           }
/^LC/ {
        lec = substr($0,3,3) + dipce
        f++
        s = ""
        for ( i in arr ) { delete arr[i] }
        }
	   f {
		run some more stuff until gzcat is finished
              }

###Here I want to change FILENAME to "'"$ifile"'" ###

#Run some more awk code here while reading in "'"$ifile"'"




END {
Print some stuff from both processed file
}

# 2  
Old 06-22-2008
You can use getline to start reading the $ifile whenever you want, but I would prefer something like this: define $ifile as a second argument and then process it as usual: NR == FNR -> action -> next for the first one, action for the second.
FS is not a problem, you can change it anytime you want.
# 3  
Old 06-22-2008
To read lines from a different file other than the one you piped through awk, you need "getline" function.

Code:
getline <"'"$ifile"'"

or

getline var <"'"$ifile"'"

In the first case, $0 will be overwritten with the first line read from $ifile, in the second case you retain $0 of the original file (=the output of gzcat) and set a new variable "var" holding the first line of $ifile.

If you want, you can also avoid that pretty ugly syntax for the file name ("'"$ifile"'") by using "-v" option of awk:

Code:
/usr/bin/gzcat $file | /usr/xpg4/bin/awk -v "ifile=$ifile" ' ... '

and then reference your file with the internal awk variable "ifile", like this:

Code:
getline <ifile

Don't miss awk man page for details on getline.
# 4  
Old 06-22-2008
Thanks guys, I have learned so much from the both of you. I am still having issues that I can't figure out, here is the script:
Code:
/usr/bin/gzcat $file | /usr/xpg4/bin/awk '

BEGIN {
print "headers go here"
}


NR == 3 { print $2,$3"  --  "$4,$5 ; print "" ; next }

/^dise / {
        dipce = substr($2,4,1) * 1000 ; next
           }
/^LC/ {
        lec = substr($0,3,3) + dipce
        f++
        s = ""
        for ( i in arr ) { delete arr[i] }
        }
	   f {
              if ( $1 == "RRAC" ) {
                s++
                 }
                if ( s ) {
                    if ( $1 == "tsec" ) {
                       lectsec = $2 * 1
                       }
                    if ( $1 == "VAR" ) {
                        car[$2 * 1]+=$8
                        lecCar = $2 * 1
                        c[lec,lectsec,BCar] = 0
                        }
                     c[lec,lectsec,BCar]++
                     if ( c[lec,lectsec,BCar] == 8 ) {
                        car[BCar]+=$8
                        tst[lec"-"BCar]=sprintf("%5.2f",car[BCar] / 12)
                        }
                    }
                if ( $1 == "RRAC-L" ) {
                        f = "" ; next
                   }
                   }


###Here I want to change FILENAME to "'"$ofile"'" ###
#####################################################

{
while (( getline < "'"$ofile"'" ) > 0 ) {
FS = "\;" ; print $1,$3,$4 ; next }
}
#####################################################
END {
for ( i in tst ) { print i,tst[i] }

    } '

/usr/bin/rm $ifile
/usr/bin/rm $ofile

When I change FS it also changes the output of the following:
Code:
for ( i in tst ) { print i,tst[i] }

I can't figure this out??? Why does this change the output of the array?
# 5  
Old 06-23-2008
You need to set the OFS variable with the desired Output Field Separator, by default a space.
Or try using "nawk". I've noticed this behavior on Solaris:

Code:
> echo "aaa:bbb:ccc" | /usr/xpg4/bin/awk '{ FS=":"; print $2,$3 }'
 
> echo "aaa:bbb:ccc" | /bin/awk '{ FS=":"; print $2,$3 }'
bbb ccc

> echo "aaa:bbb:ccc" | /bin/nawk '{ FS=":"; print $2,$3 }'
bbb ccc

The first command returns null!
# 6  
Old 06-23-2008
@OP, use split inside you while loop
Code:
...
while (( getline line< "'"$ofile"'" ) > 0 ) {
 n=split(line,temp,"\")
 print temp[1],temp[3] ......
 next }
}
...

# 7  
Old 06-23-2008
Thanks guys, I will try them both.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Programming

to extract all the part of the filename before a particular word in the filename

Hi All, Thanks in Advance I am working on a shell script. I need some assistance. My code: if then set "subscriber" "promplan" "mapping" "dedicatedaccount" "faflistSub" "faflistAcc" "accumulator"\ "pam_account"; for i in 1 2 3 4 5 6 7 8;... (0 Replies)
Discussion started by: aealexanderraj
0 Replies

2. UNIX for Dummies Questions & Answers

to extract all the part of the filename before a particular word in the filename

Hi All, Thanks in Advance I am working on a shell script. I need some assistance. My Requirement: 1) There are some set of files in a directory like given below OTP_UFSC_20120530000000_acc.csv OTP_UFSC_20120530000000_faf.csv OTP_UFSC_20120530000000_prom.csv... (0 Replies)
Discussion started by: aealexanderraj
0 Replies

3. Shell Programming and Scripting

Change filename - increment digit by 60

I have a bunch of files, they have spaces! I am working in windows 7, but I am hoping I could install and use cygwin or some other unix platform to rename my files (suggestions welcome). The files are named: Wind Attn Vol - 474.wrl Wind Attn Vol - 475.wrl Wind Attn Vol - 476.wrl etc.. I... (10 Replies)
Discussion started by: d_sai_kumar
10 Replies

4. UNIX for Dummies Questions & Answers

How to change date in a filename?

Hi i want to list files based on date and change the date alone in the files in a directory abc20120101.txt xyzxyxz20120101.txt ccc20120201.txt ddd20120301.txt In the above i want to select only files having date 20120101 and rename the date for those files like below abc20111231.txt... (3 Replies)
Discussion started by: Dewdrop
3 Replies

5. Shell Programming and Scripting

Change the filename variable value

Hi guys, I have a variable where i am storing the filename (with full path). I just need the value before ".txt". But instead of getting the filename i am getting the contents of the filename. FileName=/appl/data/Input/US/Test.txt a=`awk -F"." '{print $1}' ${FileName}` echo $a... (3 Replies)
Discussion started by: mac4rfree
3 Replies

6. Shell Programming and Scripting

filename change with awk needed

Hi, i have files which contains list of csv file names say temp.txt contains below like data Dns_bangalore_08172011.093033.1139.csv Dns_bangalore_08172011.093133.1139.csv now i want to insert some string before .csv in the filename say i want to insert string _sim1 beofre .csv... (3 Replies)
Discussion started by: raghavendra.nsn
3 Replies

7. Shell Programming and Scripting

Change the filename

I have 100 files in a directory with a.1 a.2 a.3 a.4 How do i remove a. and i need the file names as 1 2 3 4 please help (2 Replies)
Discussion started by: srichunduru
2 Replies

8. Shell Programming and Scripting

Filename from splitting files to have the same filename of the original file with counter value

Hi all, I have a list of xml file. I need to split the files to a different files when see the <ko> tag. The list of filename are B20090908.1100-20090908.1200_CDMA=1,NO=2,SITE=3.xml B20090908.1200-20090908.1300_CDMA=1,NO=2,SITE=3.xml B20090908.1300-20090908.1400_CDMA=1,NO=2,SITE=3.xml ... (3 Replies)
Discussion started by: natalie23
3 Replies

9. UNIX for Dummies Questions & Answers

Change multiple filename formats with WHILE

Hi All, I'm trying to run a simple shell program to change all the files named *.cvs to *.txt. I am trying to use WHILE and this is what I have so far: This changes the first file from *.cvs to *.txt, but it is not cycling through the other files. My suspicion is that I don't have the... (5 Replies)
Discussion started by: ScKaSx
5 Replies

10. Shell Programming and Scripting

Change new filename with date ??

Hi all, I am newbie and hope that you can help me to rename a file If I have a file name Perform.01222006.12345.Log now I would like to backup another file with another name like perform-20060112.dat This is a flat file, and I want to collect some field, then put it in a new file from... (9 Replies)
Discussion started by: sabercats
9 Replies
Login or Register to Ask a Question