Intermittent "slowdowns" on Solaris 2.6


 
Thread Tools Search this Thread
Special Forums Hardware Filesystems, Disks and Memory Intermittent "slowdowns" on Solaris 2.6
# 1  
Old 08-12-2002
Question Intermittent "slowdowns" on Solaris 2.6

We have a Sun Enterprise 450 running Solaris 2.6 that is giving us intermittent performance issues. These problems happen several times a day, usually lasting for a few minutes each time. sar shows that when this occurs, CPU idle time is 0% or close to it, and sys and wio are both high. iostat shows 100% disk busy during these "slowdowns", but it also shows a drop in writes during this time (reads stay about the same). My question is, what is the disk doing to make it 100% busy, if the writes are actually less and reads about the same? I suspect some sort of hardware problem, but Sun have investigated this for us and do not believe it's a hardware fault.

Any ideas?

Thanks

Sean
# 2  
Old 08-14-2002
More information would be required - and even then it's hard to completely diagnose your problem.

1. What application(s) are running on the server?
2. How are the filesystems laid out?
3. How much memory (physical) in the server?
4. How much swap space?
5. What (if any) disk management software is being used?
6. Do you have top installed? What does it show as the top processes during the 'problem'?
7. Has the problem always occurred?
8. What changed recently - more users, more data, more cron jobs, more applications, ....


It comes down to knowing your system - something that may be hard to do if you never looked before. If you don't know how it ran before, you may not know what could have caused the change. Another thing - is this temporary slowdown seen as a problem to the users? If they don't complain, don't fix it. Just start keeping history of how it runs. Then when they do complain, you can show how it's out-grown the server in use.

Remember, adding memory can solve one problem and cause another - same with adding cpu and disk space. Rearranging how your data is laid out may help without buying anything.
# 3  
Old 08-16-2002
1. What application(s) are running on the server?

We have around 100/150 users at any one time running an in-house app written in c, that access's various CISAM database files ranging in size from 100KB to 70MB.

2. How are the filesystems laid out?

The various database files are spread between three file systems. We use Solstice Disk Suite to stripe and mirror the file systems.

3. How much memory (physical) in the server?

2 GB

4. How much swap space?

4GB

5. What (if any) disk management software is being used?

SDS.

6. Do you have top installed? What does it show as the top processes during the 'problem'?

top doesn't show any process using more than 1 or 2 percent CPU during the "slowdown". Also, the %usr figures from sar during these times are very low, so it doesn't seem to be related to a user process.

7. Has the problem always occurred?

The problems has been occuring for a few months now.

8. What changed recently - more users, more data, more cron jobs, more applications, ....

There have been some changes to the applications and increased data over the last 6 months or so. However, my feeling is that these changes have not caused the problem. We have another identicle machine that has had all the same changes, but is not
experiencing any performance issues.

The users experience extremely slow response times when the performance drops. BAsically the system is almost unusable for them for a few minutes.

The main thing I can't understand is why the disks are indicating 100% busy while the performance problems occur, yet there is very little (less than the normal amount) being written during these times.
# 4  
Old 08-16-2002
Sorry, more questions!

The 'identical' machine - how many users does it have on it?

(back to the original server) Are you running NFS on it sharing out drives?

What does swap look like during normal times versus slow times?

Do the users on the normal server have the same paths as on the 'second' server? Do the two server reside on the same network?

Are there any backup processes running (both our Sybase and Oracle DBA run hourly backups - found that the Sybase one ran gzip across a NFS mount which was killing our server for about 3 minutes each hour - combination of the program running from a NFS mount, doing the compression, and the fact it was on the same drive as the data - top didn't show it except for once in a while - it was more guessing than knowing that it was the problem and we turned it off to every two hours during the normal work day )

Are /etc/system the same on both servers?

Do you have any 3rd party monitoring software (such as BMC Patrol or Landmark) that shows anything ?

Are you the only admin who can change things? Are you SURE no one added something - changed when things run -
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. AIX

Apache 2.4 directory cannot display "Last modified" "Size" "Description"

Hi 2 all, i have had AIX 7.2 :/# /usr/IBMAHS/bin/apachectl -v Server version: Apache/2.4.12 (Unix) Server built: May 25 2015 04:58:27 :/#:/# /usr/IBMAHS/bin/apachectl -M Loaded Modules: core_module (static) so_module (static) http_module (static) mpm_worker_module (static) ... (3 Replies)
Discussion started by: penchev
3 Replies

2. Shell Programming and Scripting

Bash script - Print an ascii file using specific font "Latin Modern Mono 12" "regular" "9"

Hello. System : opensuse leap 42.3 I have a bash script that build a text file. I would like the last command doing : print_cmd -o page-left=43 -o page-right=22 -o page-top=28 -o page-bottom=43 -o font=LatinModernMono12:regular:9 some_file.txt where : print_cmd ::= some printing... (1 Reply)
Discussion started by: jcdole
1 Replies

3. UNIX for Dummies Questions & Answers

Using "mailx" command to read "to" and "cc" email addreses from input file

How to use "mailx" command to do e-mail reading the input file containing email address, where column 1 has name and column 2 containing “To” e-mail address and column 3 contains “cc” e-mail address to include with same email. Sample input file, email.txt Below is an sample code where... (2 Replies)
Discussion started by: asjaiswal
2 Replies

4. Solaris

Printer configuration Migration from Solaris 10 "LP" to Solaris 11 "CUPS"

Need to find a way to import an LP printers.conf file to CUPS. I have some new Solaris 11.1 boxes that need to have 300 printers added. (0 Replies)
Discussion started by: os2mac
0 Replies

5. Shell Programming and Scripting

Intermittent "cp: cannot stat" error with nested loop

I have a bash script that has been running (on SUSE 9.3) dozens of times over the past couple of years without error. Recently it has been hitting intermittent “cp: cannot stat FILE: No such file or directory” errors. The script has nested loops that continuously process files in a... (2 Replies)
Discussion started by: jcart
2 Replies

6. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

7. Shell Programming and Scripting

cat $como_file | awk /^~/'{print $1","$2","$3","$4}' | sed -e 's/~//g'

hi All, cat file_name | awk /^~/'{print $1","$2","$3","$4}' | sed -e 's/~//g' Can this be done by using sed or awk alone (4 Replies)
Discussion started by: harshakusam
4 Replies

8. Solaris

Solaris escape my script from "-" to "/226"

Hello everyone. I beg your guys pardon please. I try to ls -al in many path/directories. So, I put the code in text file which look like below; ls -al / ls -al /etc ls -al /etc/default ... however, when I paste it to Solaris over SecureCRT, it seems the code was escaped from "-" to... (0 Replies)
Discussion started by: Smith
0 Replies

9. UNIX for Dummies Questions & Answers

Explain the line "mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'`"

Hi Friends, Can any of you explain me about the below line of code? mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'` Im not able to understand, what exactly it is doing :confused: Any help would be useful for me. Lokesha (4 Replies)
Discussion started by: Lokesha
4 Replies
Login or Register to Ask a Question