Shell Programming and Scripting

Deal All,

I'm writing a simple awk to generate some sort of report. The awk will check 24 files (file generated each one hour in a wholoe day) and then it will print one field to another file for counting purposes.

The script is working fine but the problem is that the CPU load is very high and almost reaching 100% when the script is running.

Is there a way to improve the CPU Load?

The machine I'm using is Sparc SUNW,Sun-Fire-V240 and the awk command I'm suing is:
cat $year$month$day*.TLG | awk 'BEGIN {FS=","} {if (($1)=="203" || ($1)=="204" || ($1)=="205" || ($1)=="206") print $3}' >> file.txt

Should work for you.

Code:

nice awk -F, '$1 > 202 && $1 < 207 {print $3}' $year$month$day*.TLG >> file.txt

---------- Post updated at 08:17 AM ---------- Previous update was at 08:17 AM ----------

To keep the forums high quality for all users, please take the time to format your posts correctly.
Thank You.

The UNIX and Linux Forums
Reply With Quote

Hi.

I can't imagine why that would max out a Sparc!

In any case it's best not to use awk on Solaris. Use nawk or /usr/xpg4/bin/awk

Code:

nawk -F, '$1 ~ /20[3-6]/ {print $3 > "file.txt" }' $year$month$day*.TLG

I have tried both options but unfortunately, the problem is still the same!

Are the variables $year, $month and $day actually set (i.e. otherwise you're generating a report based on possibly every *.TLG file)?

How big is each TLG file?

Are you saying that the CPU usage is low until you run the script?

Does the script finish at all? Is file.txt created? Does it contain the correct data?

Are the variables $year, $month and $day actually set (i.e. otherwise you're generating a report based on possibly every *.TLG file)?

Yes the variables are right. I have checked it again and for sure I am not searching in the whole *.TLG

How big is each TLG file?

The size of each file is not fixed, it depends on the traffic during that hour, sometimes rhe size is 100KB and sometimes it might be over 600KB.

Are you saying that the CPU usage is low until you run the script?

The CPU is fine until you start the script. Before starting the script, if you check CPU load through vmstat command, you will find the idle value varies between 85 & 96%, once the script is started, the idle value drops to 0

Does the script finish at all? Is file.txt created? Does it contain the correct data?

The script is generating the file successfully. One thing to be noted here, is that the same line of this script will have to run in 6 different directories, since I have different 26 files *.TLG generated in 6 different directories.

And the directories are locally mounted or are they network directories?

I honestly have to say I've never known awk to report data from 26 files (even 156 files) of the size you mention throttling a Solaris server.

Are there other I/O- (if not CPU-) intensive jobs running? Is there a problem with your filesystem?

More questions than answers, sorry!