Visit Our UNIX and Linux User Community


awk & CPU Load


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk & CPU Load
# 1  
Old 10-31-2009
awk & CPU Load

Deal All,

I'm writing a simple awk to generate some sort of report. The awk will check 24 files (file generated each one hour in a wholoe day) and then it will print one field to another file for counting purposes.

The script is working fine but the problem is that the CPU load is very high and almost reaching 100% when the script is running.

Is there a way to improve the CPU Load?

The machine I'm using is Sparc SUNW,Sun-Fire-V240 and the awk command I'm suing is:
cat $year$month$day*.TLG | awk 'BEGIN {FS=","} {if (($1)=="203" || ($1)=="204" || ($1)=="205" || ($1)=="206") print $3}' >> file.txt
# 2  
Old 10-31-2009
Should work for you.
Code:
nice awk -F, '$1 > 202 && $1 < 207 {print $3}' $year$month$day*.TLG >> file.txt



---------- Post updated at 08:17 AM ---------- Previous update was at 08:17 AM ----------

To keep the forums high quality for all users, please take the time to format your posts correctly.
  1. Use Code Tags when you post any code or data samples so others can easily read your code.
    You can easily do this by highlighting your code and then clicking on the # in the editing menu. (You can also type code tags [code] and [/code] by hand.)
  2. Avoid adding color or different fonts and font size to your posts.
    Selective use of color to highlight a single word or phrase can be useful at times, but using color, in general, makes the forums harder to read, especially bright colors like red.
  3. Be careful when you cut-and-paste, edit any odd characters and make sure all links are working property.

Thank You.

The UNIX and Linux Forums
Reply With Quote
# 3  
Old 10-31-2009
Hi.

I can't imagine why that would max out a Sparc!

In any case it's best not to use awk on Solaris. Use nawk or /usr/xpg4/bin/awk

Code:
nawk -F, '$1 ~ /20[3-6]/ {print $3 > "file.txt" }' $year$month$day*.TLG

# 4  
Old 10-31-2009
I have tried both options but unfortunately, the problem is still the same!
# 5  
Old 10-31-2009
Are the variables $year, $month and $day actually set (i.e. otherwise you're generating a report based on possibly every *.TLG file)?

How big is each TLG file?

Are you saying that the CPU usage is low until you run the script?

Does the script finish at all? Is file.txt created? Does it contain the correct data?
# 6  
Old 10-31-2009
Are the variables $year, $month and $day actually set (i.e. otherwise you're generating a report based on possibly every *.TLG file)?

Yes the variables are right. I have checked it again and for sure I am not searching in the whole *.TLG

How big is each TLG file?

The size of each file is not fixed, it depends on the traffic during that hour, sometimes rhe size is 100KB and sometimes it might be over 600KB.

Are you saying that the CPU usage is low until you run the script?

The CPU is fine until you start the script. Before starting the script, if you check CPU load through vmstat command, you will find the idle value varies between 85 & 96%, once the script is started, the idle value drops to 0

Does the script finish at all? Is file.txt created? Does it contain the correct data?

The script is generating the file successfully. One thing to be noted here, is that the same line of this script will have to run in 6 different directories, since I have different 26 files *.TLG generated in 6 different directories.
# 7  
Old 10-31-2009
And the directories are locally mounted or are they network directories?

I honestly have to say I've never known awk to report data from 26 files (even 156 files) of the size you mention throttling a Solaris server.

Are there other I/O- (if not CPU-) intensive jobs running? Is there a problem with your filesystem?

More questions than answers, sorry!

Previous Thread | Next Thread
Test Your Knowledge in Computers #16
Difficulty: Easy
There are a total of 25 pins in the traditional parallel port of a computer system.
True or False?

10 More Discussions You Might Find Interesting

1. Solaris

Understanding & Monitoring CPU performance (Load vs SAR)

Hi all, Been reading a lot of the cpu load and its "analogy of it to car traffic path of expressway" From wiki Most UNIX systems count only processes in the running (on CPU) or runnable (waiting for CPU) states. However, Linux also includes processes in uninterruptible sleep states... (13 Replies)
Discussion started by: javanoob
13 Replies

2. Ubuntu

Load Balancing - LB Server & BackEnd1 & BackEnd2

Hello, What I wish to succeed is to setup loadbalancing between two dedicated servers. Here is my case: ======================================================================================================= LB IP (Just an IP address): Provided by hosting company (11.22.33.44) Hosting company... (1 Reply)
Discussion started by: baris35
1 Replies

3. Solaris

Help me Solaris 10&11 cpu load average states for 24 hours report

need to capture the following data on an hourly basis without cronjob scheduling in Solaris 5.10/5.11:- 1. load averages 2. Total no. of processes. 3. CPU state 4. Memory 5. Top 3 process details. any other third-party tool is available? (7 Replies)
Discussion started by: thoranam
7 Replies

4. UNIX for Dummies Questions & Answers

CPU load in video decoding using SAR

Hi, I'm John_giova and I'm new in this Forum. Sorry my english, it's not my first language. So, I'm trying to check the CPU utlization during the video encoding and decoding (making a comparison between SW and HW ) using the SAR tool. According to a past thread I saw as CPU utilization I should... (4 Replies)
Discussion started by: John_giova
4 Replies

5. Shell Programming and Scripting

how to increase cpu load

can someone suggest me some code in any language that will increase CPU and memory both. (3 Replies)
Discussion started by: learnbash
3 Replies

6. Solaris

CPU load -12.50 in server.

Friends I have noticed that the Sun Fire v490 server with Solaris9 OS in my office, is showing a load of 12.50 during peak time and the CPU showing a max of 75% and an average of 60%. The Application running in this machine hung last month(For reasons unknown) and is running fine after... (5 Replies)
Discussion started by: Renjesh
5 Replies

7. Linux

How to find the load on CPU ?

Hi ALL, I have to develop a script which checks for the load on CPU on regular intervals. I created a simple script which uses 'uptime' command to find out the avg load in the last 5 min. I then used grep and put the value of the avg load in a variable OUT. It was working fine till... (5 Replies)
Discussion started by: vikings.svnit
5 Replies

8. Red Hat

High cpu load average

Hi Buddies, Thanx for reading my first post... After googling a lot and searching so many forums I am feeling down a bit... Please don't mind my ignorence, and my grammer ... :) My server is running RHEL 2.6.9-5.EL. The cpu load is going higher than roof, almost 100 sometimes. I am... (2 Replies)
Discussion started by: squid04
2 Replies

9. AIX

Application high CPU load

after a long period of running, the network application's CPU load in our syst em increase slowly, the failed at the end. we use "truss" tool to trace the process, found that it processes something like "semop" ,"semctl","thread_waitlock","kread" kernel call . The trace log file looks like the... (0 Replies)
Discussion started by: Frank2004
0 Replies

10. UNIX for Dummies Questions & Answers

CPU load unit of measure?

If unix says my cpu load is 2.15 exactly what does that mean? --Jason (1 Reply)
Discussion started by: Mac J
1 Replies

Featured Tech Videos