MySQL Performance Problems


 
Thread Tools Search this Thread
The Lounge What is on Your Mind? MySQL Performance Problems
# 8  
Old 11-14-2019
Update:

To be clear, the main reasons I put the man page table in a different database was to make the main database smaller (75%) smaller; and so the daily backups are faster and use less resources. These backups take up less disk space on both the main server and the backup server.

From a DB CPU perspective, there is little to no performance difference having one huge database or two smaller ones.
# 9  
Old 11-14-2019
Seems like the bulk of this MySQL performance issue is solved (fingers crossed).
This User Gave Thanks to Neo For This Post:
# 10  
Old 11-15-2019
Daily backups:

Before moving man pages to new database:

Code:
-rw-r--r-- 1 root root 1292500820 Nov 13 22:31 main_masterdump_2019-11-13-22.sql.gz

After moving man pages to new database:

Code:
-rw-r--r-- 1 root root  449734289 Nov 14 22:30 main_masterdump_2019-11-14-22.sql.gz

Roughly a 65% reduction in the size for full daily backups.
This User Gave Thanks to Neo For This Post:
# 11  
Old 11-22-2019
I just completed (for the most part) a database optimization which was slowing the site down.

In a nutshell, when I created some new code to mitigate against Google soft 404 "errors" (slim content) by adding matching man pages to posts with very little content, I used a "real time" FULLTEXT search of over 350K man pages, and this was causing some performance issues.

Plus, my new code worked, and Google search referrals have just about doubled since I created the solution, but the upside/downside was that the site traffic also doubled (which is a good thing) and the new queries were killing performance (a bad thing).

In the final optimization, I preprocessed all these matches and stored the results in the DB (the ids of all the man pages which match the discussion contents) so, the long MySQL queries doing these FULLTEXT NATURAL LANGUAGE searches on the fly were eliminated (right now about 99.5+ are gone, I have about a half a day of processing to get to 99.999 percent.

The performance improvement is remarkable and the site is much faster, and I have earned my black belt (or at least a brown belt) in MySQL performance tuning, after all these years.

Smilie Smilie Smilie Smilie Smilie Smilie
This User Gave Thanks to Neo For This Post:
# 12  
Old 12-08-2019
Following up on this after a few weeks of testing....

Looks like we have solved any mysql performance issues we had and all is running smooth and fast, as it should be.

There is a single fall back query related to the "similar man pages" and tag searches which has an under 0.4 second "slow query"; and I plan redesign that "fall back query" to get rid of that occasional delay which effects only non registered users and tag searches.
This User Gave Thanks to Neo For This Post:
# 13  
Old 12-10-2019
Quote:
Originally Posted by Neo
There is a single fall back query related to the "similar man pages" and tag searches which has an under 0.4 second "slow query"; and I plan redesign that "fall back query" to get rid of that occasional delay which effects only non registered users and tag searches.
Mitigated this slow query for "tag search" and the associated "similar man page".

Still need fine tuning for bad spelling of tags, etc; but over all the performance is much better (and the quality / spelling of the tags issue it a different issue than DB query performance, generally).
# 14  
Old 12-11-2019
Just shaved about 200ms from the mysql query used on the user badge page for each user by adding a field in the user database to store the total tag count for each user and moving that query to a daily cron file instead of updating this in real time.

Code:
<?php
$sql = "select userid, count(1)  as tagcount  FROM tagthread group by userid order by tagcount desc";
$taginfo = $vbulletin->db->query_read($sql);
while ($tag = $vbulletin->db->fetch_array($taginfo)) {
    $sql2 = "update user set tagcount = " . $tag['tagcount'] . " where userid = " . $tag['userid'];
    $vbulletin->db->query_write($sql2);
}

Login or Register to Ask a Question

Previous Thread | Next Thread

4 More Discussions You Might Find Interesting

1. Programming

Xlib - Rotation and interpolation of pixmap - Performance problems

I need to rotate a pixmap in XLib with some kind of interpolation to reduce the aliasing. I came up with the following code, which uses bilinear interpolation. It works fine: the rotated image looks perfect, but unfortunately it takes 5 or 6 seconds for each rotation. (in a 300x300, 16 colours... (5 Replies)
Discussion started by: mghis
5 Replies

2. AIX

AIX 5.3 performance problems

Hello, I encounter some performance issues on my AIX 5.3 server running in a LPAR on a P520. How do I investigate performance issues in AIX. Is there any kind of procedure that takes me to the steps to investigate my server and find the sub systems that is causing the issues? The performance... (1 Reply)
Discussion started by: petervg
1 Replies

3. SCO

CPU Performance Problems on VMWARE

hi We have migrated SCO 5.0.6 into ESX4, but the VM eats 100% of the virtual CPU. Here is top print from the SCO VM: last pid: 16773; load averages: 1.68, 1.25, 0.98 02:08:41 79 processes: 75 sleeping, 2 running, 1 zombie, 1 onproc CPU states: 0.0% idle, 17.0% user,... (7 Replies)
Discussion started by: ccc
7 Replies

4. AIX

Performance and paging problems

... a disk drive to be 100% busy? hdisk0 100.0 1.3K 342.7 1.3K 22.0 PgspIn 651 % Noncomp 75.5 hdisk1 100.0 1.3K 320.2 1.2K 20.0 PgspOut 6 % Client 75.5 It's really slowing down performance on my system and I would like to know what is causing this. ... (2 Replies)
Discussion started by: bbbngowc
2 Replies
Login or Register to Ask a Question