Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Temp mysql files jamming CPU resources

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Temp mysql files jamming CPU resources
# 1  
Old 07-20-2010
Temp mysql files jamming CPU resources
Hello Friends

I am currently facing high CPU usage problem which is making my site extremely slow.
Currently I am using a 8GB RAM with 8 cores but, the creation of temporary files is eating away a lot of the CPU resource making the site very slow.
The normal CPU load average remains below 2-3 but, suddenly it shoots to double digits in order of 20-30
On investigating the mysql data files, I could see huge files created as listed below :

Code:
/var/tmp/#sql_25dd_0.MYD 
/var/tmp/#sql_25dd_1.MYD 
/var/lib/mysql/egully1_cpdata2009/isc_product_search.MYD 
/var/lib/mysql/egully1_cpdata2009/isc_product_search.MYD 
/var/lib/mysql/egully1_cpdata2009/isc_product_search.MYD 
/var/lib/mysql/egully1_cpdata2009/isc_product_search.MYD 
/var/lib/mysql/egully1_cpdata2009/isc_product_search.MYI

These files are in order of 0.5-2.5GB.
After restarting mysql all the temp files gets deleted and system gets back to normal. But, after some time the problem again resurfaces.

Can anyone suggest how to deal with this problem and how to flush huge temp files from system to keep the site performance smooth.
Last edited by Scott; 07-20-2010 at 02:32 AM.. Reason: Code tags
# 2  
Old 07-20-2010
I suspect that those temporary tables are being used to store large joins and other queries that you get as you use the site.

This indicates that your my.cnf file needs tuning. I would suggest that you let your site run for a while, and then use one of the many fine mysql tuning suggestion scripts that can help you identify what's going on.
# 3  
Old 07-20-2010
Thanks

Can you suggest me where I can find mysql fine tuning suggestion scripts.


Quote:
Originally Posted by malcolmpdx
I suspect that those temporary tables are being used to store large joins and other queries that you get as you use the site.

This indicates that your my.cnf file needs tuning. I would suggest that you let your site run for a while, and then use one of the many fine mysql tuning suggestion scripts that can help you identify what's going on.
# 4  
Old 07-20-2010
There are a lot you can find by google-ing. The one I like is here:

http://www.day32.com/MySQL/tuning-primer.sh
This User Gave Thanks to malcolmpdx For This Post:
egully
# 5  
Old 07-21-2010
My current my.cnf config is : This may not be the best suited for us. Can you suggest. We have 8GB RAM , Xeon-2 processor and operating system is CentOS Linux 5.4

Code:
[mysqld]
socket = /var/lib/mysql/mysql.sock
datadir=/var/lib/mysql
log-slow-queries = /var/log/slowqueries.log
tmpdir=/tmp
skip-locking
skip-innodb


max_connections=200

key_buffer =512M
table_cache = 1024
open_files_limit = 16344
join_buffer_size = 32M
read_buffer_size = 32M (8M)
sort_buffer_size = 32M
tmp_table_size=256M
read_rnd_buffer_size=16M
max_heap_table_size=256M
myisam_sort_buffer_size=256M
thread_cache_size=8
thread_cache=8192
query_cache_type=1
query_cache_limit=256M
query_cache_size=256M
thread_concurrency=8
wait_timeout = 30
connect_timeout = 30
interactive_timeout = 60
long_query_time=1

max_allowed_packet=128M


Quote:
Originally Posted by malcolmpdx
There are a lot you can find by google-ing. The one I like is here:

http://www.day32.com/MySQL/tuning-primer.sh
Last edited by Scott; 07-21-2010 at 03:42 AM.. Reason: Code tags, please...
# 6  
Old 07-21-2010
Tuning my.cnf settings is not trivial, and is very dependent on the application.

Run that script I linked to. It will suggest a lot of different things you can tune.

Also, most distributions of mysql have sample my.cnf files for various levels of load and use in the docs that come with the distribution, you might look at those to get some ideas.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing large number of temp files

Hi All, I am having a situation now to delete a huge number of temp files created during run times approx. 16700+ files. We have never imagined that we will get this this much big list of files during run time. It worked fine for lesser no of files in the list. But when list is huge we are... (7 Replies)
Discussion started by: mad man
7 Replies

2. Solaris

Sun Fire V210 CPU Fan Temp too high?

Hey, I have a V210 with a failed CPU fan. The temperature is currently at 84C and I've been asked to wait a few weeks before replacing as its a production system and it cant be shut down yet. Is it too hot? Do I risk killing the CPU at this temp? Its been like this for a few weeks now... (5 Replies)
Discussion started by: magarvo
5 Replies

3. Shell Programming and Scripting

RMTF (ReMove Temp Files)

Heyas As some applications (sed,grep,vi, etc) create some tempfiles, i'd changed a script to this: (sadly i cant find the original post (code) anymore (which just removed 2 'diffrent kinds'), just similar ones - forgot that as i was new to all this) :) ~ $ cat $(which rmtf) #!/bin/sh #... (1 Reply)
Discussion started by: sea
1 Replies

4. Shell Programming and Scripting

temp files

Hi there, As a regular unix user I am forever programming on the command line or writing scripts so that I first write a load of data to a file to read from. In the end I am always left with a bundle of .txt, .tmp which is what I usually call them. As a basic programmmer I was wondering is... (6 Replies)
Discussion started by: cyberfrog
6 Replies

5. UNIX for Dummies Questions & Answers

Security issue and temp files

Hello, One of the senior network admins at work told me that I should not hard code temp files into my scripts. Rather I should use the mktemp commands in the script to create them on the fly. His argument was that if a malicious user knew the name of my temp files in the script they could... (6 Replies)
Discussion started by: mojoman
6 Replies

6. Solaris

CPU resources

Hi, I got a solarsi 10 box with 9 zones and the cpu shares as following ID NAME SHARES 0 global 1 1 FMW1 100 2 FMW2 100 3 OID1 100 4 OID2 100 5 OVD1 100 6 OID0 100 7 FMW5 100 8... (2 Replies)
Discussion started by: fugitive
2 Replies

7. Linux

Shared web server resources (CPU&RAM)

Hello, guys ! Long time no seen. I have a question about Linux web servers. I guess it is a more general question, but in my case it is a Linux web server. I want to know how much RAM memory should I put on my servers. Do you have some formula or guidelines on how should I calculate the amount... (1 Reply)
Discussion started by: Sergiu-IT
1 Replies

8. Programming

Application occupying CPU resources

Dear all, I have a pro c application running in the unix environement. This pro c program actually trigger by a java application from sun workstation. Recently, when we released a new proc c application and notice that the application occupying the CPU resources even through we check that the... (1 Reply)
Discussion started by: ghho
1 Replies
Login or Register to Ask a Question