|
|
Sponsored Content
The Lounge
What is on Your Mind?
MySQL Performance Problems
Post 303041393 by Neo on Friday 22nd of November 2019 06:14:47 AM
11-22-2019
Administrator
I just completed (for the most part) a database optimization which was slowing the site down.
In a nutshell, when I created some new code to mitigate against Google soft 404 "errors" (slim content) by adding matching man pages to posts with very little content, I used a "real time" FULLTEXT search of over 350K man pages, and this was causing some performance issues.
Plus, my new code worked, and Google search referrals have just about doubled since I created the solution, but the upside/downside was that the site traffic also doubled (which is a good thing) and the new queries were killing performance (a bad thing).
In the final optimization, I preprocessed all these matches and stored the results in the DB (the ids of all the man pages which match the discussion contents) so, the long MySQL queries doing these FULLTEXT NATURAL LANGUAGE searches on the fly were eliminated (right now about 99.5+ are gone, I have about a half a day of processing to get to 99.999 percent.
The performance improvement is remarkable and the site is much faster, and I have earned my black belt (or at least a brown belt) in MySQL performance tuning, after all these years.
In a nutshell, when I created some new code to mitigate against Google soft 404 "errors" (slim content) by adding matching man pages to posts with very little content, I used a "real time" FULLTEXT search of over 350K man pages, and this was causing some performance issues.
Plus, my new code worked, and Google search referrals have just about doubled since I created the solution, but the upside/downside was that the site traffic also doubled (which is a good thing) and the new queries were killing performance (a bad thing).
In the final optimization, I preprocessed all these matches and stored the results in the DB (the ids of all the man pages which match the discussion contents) so, the long MySQL queries doing these FULLTEXT NATURAL LANGUAGE searches on the fly were eliminated (right now about 99.5+ are gone, I have about a half a day of processing to get to 99.999 percent.
The performance improvement is remarkable and the site is much faster, and I have earned my black belt (or at least a brown belt) in MySQL performance tuning, after all these years.
This User Gave Thanks to Neo For This Post:
|
|
4 More Discussions You Might Find Interesting
1. AIX
... a disk drive to be 100% busy?
hdisk0 100.0 1.3K 342.7 1.3K 22.0 PgspIn 651 % Noncomp 75.5
hdisk1 100.0 1.3K 320.2 1.2K 20.0 PgspOut 6 % Client 75.5
It's really slowing down performance on my system and I would like to know what is causing this.
... (2 Replies)
Discussion started by: bbbngowc
2 Replies
2. SCO
hi
We have migrated SCO 5.0.6 into ESX4, but the VM eats 100% of the virtual CPU.
Here is top print from the SCO VM:
last pid: 16773; load averages: 1.68, 1.25, 0.98 02:08:41
79 processes: 75 sleeping, 2 running, 1 zombie, 1 onproc
CPU states: 0.0% idle, 17.0% user,... (7 Replies)
Discussion started by: ccc
7 Replies
3. AIX
Hello,
I encounter some performance issues on my AIX 5.3 server running in a LPAR on a P520. How do I investigate performance issues in AIX. Is there any kind of procedure that takes me to the steps to investigate my server and find the sub systems that is causing the issues?
The performance... (1 Reply)
Discussion started by: petervg
1 Replies
4. Programming
I need to rotate a pixmap in XLib with some kind of interpolation to reduce the aliasing.
I came up with the following code, which uses bilinear interpolation. It works fine: the rotated image looks perfect, but unfortunately it takes 5 or 6 seconds for each rotation. (in a 300x300, 16 colours... (5 Replies)
Discussion started by: mghis
5 Replies
LEARN ABOUT OSF1
emacs
emacs(1) General Commands Manual emacs(1) NAME
emacs, xemacs - emacs editor DESCRIPTION
The emacs software is unsupported software that is provided as part of Tru64 UNIX. Compaq will fix problems in this unsupported software only if they are specific to Tru64 UNIX. Compaq will not fix problems that are integral to the software itself or that occur when the com- ponent is used on UNIX systems other than Tru64 UNIX. Compaq will not add functionality to this software. Except for this reference page, other reference pages that Tru64 UNIX supplies for emacs are passed through without changes. The reference pages distributed as part of this software are available in the directories /usr/share/doclib/annex/man/man[1-9]. You should use this directory stem in the man command or add it to the MANPATH environment variable to make these files available to the man command. Note Compaq is not responsible for the content or quality of reference pages and other documents installed under the /usr/share/doclib/annex directory and does not revise this material in response to customer problem reports. Reference pages installed under the /usr/share/doclib/annex/man directory are not available from Compaq in book form; for example, they are not included in the reference manu- als that you receive when you order the Tru64 UNIX documentation set as hard copy books. Problems related to the content or quality of any documentation installed in the /usr/share/doclib/annex directory tree should be sent to the developers of the documentation. The format for changing the search path with the man command is: man -P /usr/share/doclib/annex/man [section] title... If you are using the Bourne, Korn, or POSIX shells, use the following command sequence to modify your environment: MANPATH=$MAN- PATH:/usr/share/doclib/annex/man export MANPATH If you are using the C shell, enter the command: setenv MANPATH `echo $MANPATH`:/usr/share/doclib/annex/man See the reference pages for the man(1) command for additional information on the search path used to locate files. The reference pages associated with this product are not included in the whatis data base created by the catman command. Therefore, the man -k and apropos commands will not locate reference pages included with this product. SEE ALSO
Commands: apropos(1), catman(8), man(1) emacs(1)