Login or Register to Ask a Question and Join Our Community


HP-UX

Process going to 100% CPU

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Operating Systems HP-UX Process going to 100% CPU
# 1  
Old 12-05-2013
Process going to 100% CPU
I have a httpd process which uses up 100% cpu. This makes my application to stop working and it just gets hung.

I tried getting the tusc output looks something like this:
Code:
# /usr/local/bin/tusc -pl 26516
( Attached to process 26516 ("/opt/hpws22/apache/bin/httpd -d /opt/hpws22/apache -k start") [64-bit] )
[26516]{579822} ksleep(PTH_CONDVAR_OBJECT, 0x6000000000177090, 0x6000000000347640, NULL) .............. [sleeping]
[26516]{579821} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579820} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579819} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579818} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579817} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579816} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579815} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579814} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579813} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579812} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579811} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579810} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579809} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579808} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579807} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579806} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579805} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579804} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579803} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579802} In user-mode .......................................................................... [running]
[26516]{579801} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579800} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579799} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579798} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579797} ksleep(PTH_MUTEX_OBJECT, 0x6000000000027df8, 0x6000000000347140, NULL) ................ [sleeping]
[26516]{579795} read(8, 0x9fffffffffff7080, 1) ........................................................ [sleeping]
( Detaching from process 26516 ("/opt/hpws22/apache/bin/httpd -d /opt/hpws22/apache -k start") )

I am using HP-UX B.11.31 and my apache version is B.2.2.15.15.
Any idea how can I proceed further on my analysis?
# 2  
Old 12-05-2013
I have had similar issues and the end result on many are the http server isn't tuned to handle the load being placed on it.

How many total http process are there?

There is a parent and it spawns off a number of child processes. You can elect to have the children start at startup or get started dynamically based on need. I think the default is to have an intial number of children already running.

I never adjust anything until I understand the load being placed on the http server...the bigger picture rather than the detailed look you are doing.

------------------------------------------------------------
netstat -an ....

I look at the state of the send and receive buffers of all the clients (netstat -an | grep 80).

Look at how many are established and if any or in other states. Look at the Recv-q and Send-Q for each connections and see if they are stuck...or what those values are...0 0 means that particluar connection doesn't have any transactions in progress. If you see one of the buffers with a value that isn't changing, that particular connection may be hanging.

------------------------------------------------------------
access.log

I also go through my http logs and look to see how much data and the fequency of of the requests. So you can look in the access log and see the ip addresses of clients, their frequency, and the size of the data involved in a get for example. On one occassion we had someone using the server as a patch server (the clients would connect over http). I grep'd through the logs and counted the bytes over time and eventually found 12 clients hitting the server (http) with high frequency and large data requests and the http server just couldn't handle it.

grep through your access.log for ipaddress, gets and puts, and the amount of the transfer...see the format of the access log to determine the fields of an access log entry

So...after such an analysis I was able to determine the frequency of when the http process got to 100 percent and I exampled the
data accumulated during this frequency and then I stopped some of the clients I saw making frequent requests to test my theory.

---------------------------------------------------------------------------------------------------------
How to solve or provide relief for overworked HTTP server...

So once you understand if this is a load issue, that doesn't mean you automatically start tweaking the httpd server. When I saw the same clients frequently hittng the http server I reconfigured those clients to use something else. I could also have elected to do a hard-temp fix by restarting my http server every so often. These things are easier to do if you're too busy to get to the perfomance adjustments you need to make to resolve an overworked web server.

In short, I counted the amount of data that was sent through the webserver each time it went to 100 percent and I knew how long the webserver would run from startup of the webserver to the time of the 100 precent cpu usage.
Last edited by blackrageous; 12-05-2013 at 12:23 PM..
This User Gave Thanks to blackrageous For This Post:
chacko193
# 3  
Old 12-06-2013
I tried getting the Recv-q and Send-Q for each connections and they are 0.

And this issue is coming even when the we are giving very small load also.

There is nothing in the access.log which suggest there is any heavy data transfer.
# 4  
Old 12-06-2013
What is trying to connect to your Apache?
Windows PC?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Emergency UNIX and Linux Support

CPU and memory utilization of a process, by process name

Can someone please help me with a script that will help in identifying the CPU & memory usage by a process name, rather than a process id.This is to primarily analyze the consumption of resources, for performance tweaking. G (4 Replies)
Discussion started by: ggayathri
4 Replies

2. Red Hat

Process by CPU

Hello All, I am preparing a script to capture the processes consuming more CPU. So is there any way that i can sort & redirect to file only those processes consuming more than 5.0 % using ps command itself. Regards Ankit (3 Replies)
Discussion started by: ajaincv
3 Replies

3. UNIX for Dummies Questions & Answers

100% cpu process

hi guys you know we got an issue and my support line is linux I need to make sure my linux is OK...now the issue I have this process db2sysc reaching 10% CPU utilization yeah that's a DB2 process my DB guys wants to make sure linux is not the culprit but I have no idea the situation is... (3 Replies)
Discussion started by: karlochacon
3 Replies

4. Solaris

Multi CPU Solaris system shows 100% CPU usage.

Hello Friends, On one of my Solaris 10 box, CPU usage shows 100% using "sar", "vmstat". However, it has 4 CPUs and prstat and glance are not showing enough processes to justify high CPU utilization. ========================================================================= $ prstat -a ... (4 Replies)
Discussion started by: mahive
4 Replies

5. UNIX for Dummies Questions & Answers

how to get persistant cpu utilization values per process per cpu in linux (! top,ps)

hi, i want to know cpu utilizatiion per process per cpu..for single processor also if multicore in linux ..to use these values in shell script to kill processes exceeding cpu utilization.ps (pcpu) command does not give exact values..top does not give persistant values..psstat,vmstat..does njot... (3 Replies)
Discussion started by: pankajd
3 Replies

6. AIX

cpu utilization of a process

Hi, How can i find out the average cpu utilization of a particular long-running process in AIX? is there some command for this Thanks (2 Replies)
Discussion started by: iam
2 Replies

7. UNIX for Advanced & Expert Users

how to bind a process to a CPU

Hi all please can any body please suggest me how to bind a process to a particular CPU on unix machine. i have a unix machine with 2 CPUs and i wanna have my process running on CPU 0. please suggest. (11 Replies)
Discussion started by: Raom
11 Replies

8. Solaris

How to find which process is using up too much CPU

Hi, I need to find which processes are hogging the cpu up. please advise. Thanks (1 Reply)
Discussion started by: 0ktalmagik
1 Replies

9. UNIX for Dummies Questions & Answers

process not using enough cpu

I am running solaris 9 on a SUn 480r. It is running SAS statistical software, these processes in full flow normally run at about 50-60% cpu (theres nothing else really running on the box) this is fine, and the SAS jobs get run nice and quick. However over the last few weeks everytime a SAS job is... (1 Reply)
Discussion started by: hcclnoodles
1 Replies

10. UNIX for Advanced & Expert Users

process -> 1 cpu

Is there a way I can assign processes to different processors? I know in windows xp you can set process affinity, and wondered if there is a *nix equivelant. (2 Replies)
Discussion started by: Solitare
2 Replies
Login or Register to Ask a Question