Sponsored Content
Operating Systems Solaris Solaris 10 - script creating huge spike in Kernel CPU Post 302400550 by Rorgg on Wednesday 3rd of March 2010 02:28:02 PM
Old 03-03-2010
Solaris 10 - script creating huge spike in Kernel CPU

I'm running on Solaris 10, and I have a script that's running on several machines. Basically, what it's doing is:

* tail -f | grep one or more log files into a temp file
* Every minute or so, copy that temp file to a second temp and zero the first
* Sed through the 2nd temp to pull out a user ID
* grep through the file for occurences of that ID with some other text (the other text is in a local file that I read from in a loop)
* Output a text record into a log file that basically consists of the User, that text, and a count.

Another server then does a tail -f of the output log file.

The user cpu on this script is small... on the order of 2-4% depending on what box I'm running it on. But the crazy thing is that on my test box, with one input data stream, as soon as I start running the script, my kernal % jumps from like 1% to 60%.

As far as I can see, the issue is NOT with disk i/o wait, or with memory (still half free and very very small paging values).
Code:
# sar -g 5 5

SunOS ssdev01 5.10 138889-03 i86pc    03/03/2010

13:09:01  pgout/s ppgout/s pgfree/s pgscan/s %ufs_ipf
13:09:06     0.20     0.80     0.60     0.00     0.00
13:09:11     0.40     1.00     0.80     0.00     0.00
13:09:16     0.20     0.80     0.80     0.00     0.00
13:09:21     0.20     0.80     0.60     0.00     0.00
13:09:26     0.60     1.80     1.40     0.00     0.00

Average      0.32     1.04     0.84     0.00     0.00

Pre-run top:
Code:
load averages:  0.05,  0.27,  0.66                                                      13:25:39
69 processes:  68 sleeping, 1 on cpu
CPU states: 96.7% idle,  2.5% user,  0.8% kernel,  0.0% iowait,  0.0% swap
Memory: 2048M real, 1126M free, 486M swap in use, 2652M swap free

During run:
Code:
load averages:  0.68,  0.39,  0.68                                                      13:26:19
78 processes:  75 sleeping, 1 running, 1 zombie, 1 on cpu
CPU states: 20.5% idle, 18.3% user, 61.2% kernel,  0.0% iowait,  0.0% swap
Memory: 2048M real, 1123M free, 491M swap in use, 2647M swap free

here's the script in ps:
Code:
# /usr/ucb/ps -aux |more     
USER       PID %CPU %MEM   SZ  RSS TT       S    START  TIME COMMAND
ops      15436  4.7  0.1 1524 1028 ?        S 13:25:51  0:06 /usr/bin/ksh /sscp

Any ideas what I can look at to see what's chewing all the kernel?
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

prngd consumes huge CPU

I've noticed that the prngd process seems to consume lots of CPU every now and again on our Sparc Solaris 8 machines - I've done a google search and others too have experienced this. I have yet to find a reason or a fix. Does anyone here know why this happens? Once I stop ssh and restart it prngd... (1 Reply)
Discussion started by: ozzmosiz
1 Replies

2. Linux

Kernel panic - not syncing: cannot execute a PAE-enabled kernel on PAE-less CPU

ok so I just installed fedora core 6 on my dell inspiron 700m and I go to boot into linux and I get this error. Has anyone seen this before? I also had XP Pro and Vista installed on this pc prior to putting fedora core 6 on the machine. I'm trying to setup a triple boot system. Please Help... (2 Replies)
Discussion started by: dave043
2 Replies

3. Solaris

Find any spike in CPU usage a hour ago

Folks, I need to find out from the system logs ( or elsewhere ) if the CPU utilization reached say 90% an hour ago from now. Is there a way out. (2 Replies)
Discussion started by: kavera
2 Replies

4. Programming

Much time in insmod of huge kernel module

I'm using Linux-2.6.14. My application is having one kernel module of large size(approx 8MB), insmoding of that kernel module is taking pretty much time(approx 8Mins). Is there a way to reduce the insmod time? I tried even by modprobe also. (0 Replies)
Discussion started by: ptprabu
0 Replies

5. Ubuntu

kernel-huge

Hi All, I have a uname -a Linux caws101arop 2.6.9-67.0.20.ELsmp #1 SMP Wed Jun 18 12:35:02 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux And I'm having lots of problem especially when the web server got new two additional website. Before the memory was just 12G but we upgraded it to 65G. Do... (1 Reply)
Discussion started by: itik
1 Replies

6. Programming

Solaris CPU/memory status monitoring (Shell script or c++)

i'm trying to find a way to monitor the CPU/Memory status of a solaris station using vmstat. I like to write a small script to periodically run vmstat and store the output. Can anyone show me how (preferrably in C++ if possible)? Thanks in advance. (6 Replies)
Discussion started by: shingpui
6 Replies

7. UNIX for Dummies Questions & Answers

Apache causing CPU to spike when POSTing

Hello all. I've built a SAMP server on Solaris 10 using Apache 2.2.9 and PHP 5.2.6 For the most part everything seems to be working except when I post form data, Apache causes the CPU to max out. Sometimes the script will complete about 40 seconds later. Other times I have to stop Apache to get... (4 Replies)
Discussion started by: RobertSubnet
4 Replies

8. Solaris

How to check CPU spike between certain priod of time

Hi, Does anyone know how to check which process has been used the most CPU between certain time period? say I have noticed CPU spike happend between 1:00 to 1:30 now it is 3:00 o'clock already. Is there anyway to find out what process/PID use the most CPU then? Thanks a lot (3 Replies)
Discussion started by: uuontario
3 Replies

9. Solaris

Multi CPU Solaris system shows 100% CPU usage.

Hello Friends, On one of my Solaris 10 box, CPU usage shows 100% using "sar", "vmstat". However, it has 4 CPUs and prstat and glance are not showing enough processes to justify high CPU utilization. ========================================================================= $ prstat -a ... (4 Replies)
Discussion started by: mahive
4 Replies

10. UNIX for Advanced & Expert Users

Extremely high kernel CPU Usage (Solaris 10 SPARC)

I've got a domain running on a few boards of a 25k. I'm seeing very high kernel cpu usage in top and cant' quite explain it. System runs a large number of smallish Oracle 10g2 databases (30), used mainly for development. load average: 36.63, 36.68, 37.42 2489 processes: 2452 sleeping, 21... (0 Replies)
Discussion started by: utopiajoe
0 Replies
swap(1M)						  System Administration Commands						  swap(1M)

NAME
swap - swap administrative interface SYNOPSIS
/usr/sbin/swap -a swapname [swaplow] [swaplen] /usr/sbin/swap -d swapname [swaplow] /usr/sbin/swap -l /usr/sbin/swap -s DESCRIPTION
The swap utility provides a method of adding, deleting, and monitoring the system swap areas used by the memory manager. OPTIONS
The following options are supported: -a swapname Add the specified swap area. This option can only be used by the super-user. swapname is the name of the swap file: for example, /dev/dsk/c0t0d0s1 or a regular file. swaplow is the offset in 512-byte blocks into the file where the swap area should begin. swaplen is the desired length of the swap area in 512-byte blocks. The value of swaplen can not be less than 16. For example, if n blocks are specified, then (n-1) blocks would be the actual swap length. swaplen must be at least one page in length. The size of a page of memory can be determined by using the pagesize command. See pagesize(1). Since the first page of a swap file is automatically skipped, and a swap file needs to be at least one page in length, the minimum size should be a multiple of 2 pagesize bytes. The size of a page of memory is machine dependent. swaplow + swaplen must be less than or equal to the size of the swap file. If swaplen is not specified, an area will be added starting at swaplow and extending to the end of the designated file. If neither swaplow nor swaplen are specified, the whole file will be used except for the first page. Swap areas are normally added automatically during system startup by the /sbin/swapadd script. This script adds all swap areas which have been specified in the /etc/vfstab file; for the syntax of these specifications, see vfstab(4). To use an NFS or local file-system swapname, you should first create a file using mkfile(1M). A local file-system swap file can now be added to the running system by just running the swap -a command. For NFS mounted swap files, the server needs to export the file. Do this by performing the following steps: 1. Add the following line to /etc/dfs/dfstab: share -F nfs -o rw=clientname,root=clientname path-to-swap-file 2. Run shareall(1M). 3. Have the client add the following line to /etc/vfstab: server:path-to-swap-file - local-path-to-swap-filenfs --- local-path-to-swap-file -- swap --- 4. Have the client run mount: # mount local-path-to-swap-file 5. The client can then run swap -a to add the swap space: # swap -a local-path-to-swap-file -d swapname Delete the specified swap area. This option can only be used by the super-user. swapname is the name of the swap file: for example, /dev/dsk/c0t0d0s1 or a regular file. swaplow is the offset in 512-byte blocks into the swap area to be deleted. If swaplow is not specified, the area will be deleted starting at the second page. When the command completes, swap blocks can no longer be allocated from this area and all swap blocks previously in use in this swap area have been moved to other swap areas. -l List the status of all the swap areas. The output has five columns: path The path name for the swap area. dev The major/minor device number in decimal if it is a block special device; zeroes otherwise. swaplo The swaplow value for the area in 512-byte blocks. blocks The swaplen value for the area in 512-byte blocks. free The number of 512-byte blocks in this area that are not currently allocated. The list does not include swap space in the form of physical memory because this space is not associated with a particular swap area. If swap -l is run while swapname is in the process of being deleted (by swap -d), the string INDEL will appear in a sixth column of the swap stats. -s Print summary information about total swap space usage and availability: allocated The total amount of swap space in bytes currently allocated for use as backing store. reserved The total amount of swap space in bytes not currently allocated, but claimed by memory mappings for possi- ble future use. used The total amount of swap space in bytes that is either allocated or reserved. available The total swap space in bytes that is currently available for future reservation and allocation. These numbers include swap space from all configured swap areas as listed by the -l option, as well swap space in the form of physical memory. USAGE
On the 32-bit operating system, only the first 2 Gbytes -1 are used for swap devices greater than or equal to 2 Gbytes in size. On the 64-bit operating system, a block device larger than 2 Gbytes can be fully utilized for swap up to 2**63 -1 bytes. ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of swap: LC_CTYPE and LC_MESSAGE. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ SEE ALSO
pagesize(1), mkfile(1M), shareall(1M), getpagesize(3C), vfstab(4), attributes(5), largefile(5) WARNINGS
No check is done to determine if a swap area being added overlaps with an existing file system. SunOS 5.10 20 Jan 2004 swap(1M)
All times are GMT -4. The time now is 12:47 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy