MySQL Performance Problems Post: 303041393

Sponsored Content

The Lounge What is on Your Mind? MySQL Performance Problems Post 303041393 by Neo on Friday 22nd of November 2019 06:14:47 AM

11-22-2019

Administrator

I just completed (for the most part) a database optimization which was slowing the site down.

In a nutshell, when I created some new code to mitigate against Google soft 404 "errors" (slim content) by adding matching man pages to posts with very little content, I used a "real time" FULLTEXT search of over 350K man pages, and this was causing some performance issues.

Plus, my new code worked, and Google search referrals have just about doubled since I created the solution, but the upside/downside was that the site traffic also doubled (which is a good thing) and the new queries were killing performance (a bad thing).

In the final optimization, I preprocessed all these matches and stored the results in the DB (the ids of all the man pages which match the discussion contents) so, the long MySQL queries doing these FULLTEXT NATURAL LANGUAGE searches on the fly were eliminated (right now about 99.5+ are gone, I have about a half a day of processing to get to 99.999 percent.

The performance improvement is remarkable and the site is much faster, and I have earned my black belt (or at least a brown belt) in MySQL performance tuning, after all these years.

This User Gave Thanks to Neo For This Post:

Neo

View Public Profile for Neo

Visit Neo's homepage!

Find all posts by Neo

4 More Discussions You Might Find Interesting

1. AIX

Performance and paging problems

... a disk drive to be 100% busy? hdisk0 100.0 1.3K 342.7 1.3K 22.0 PgspIn 651 % Noncomp 75.5 hdisk1 100.0 1.3K 320.2 1.2K 20.0 PgspOut 6 % Client 75.5 It's really slowing down performance on my system and I would like to know what is causing this. ...

2. SCO

CPU Performance Problems on VMWARE

hi We have migrated SCO 5.0.6 into ESX4, but the VM eats 100% of the virtual CPU. Here is top print from the SCO VM: last pid: 16773; load averages: 1.68, 1.25, 0.98 02:08:41 79 processes: 75 sleeping, 2 running, 1 zombie, 1 onproc CPU states: 0.0% idle, 17.0% user,...

3. AIX

AIX 5.3 performance problems

Hello, I encounter some performance issues on my AIX 5.3 server running in a LPAR on a P520. How do I investigate performance issues in AIX. Is there any kind of procedure that takes me to the steps to investigate my server and find the sub systems that is causing the issues? The performance...

4. Programming

Xlib - Rotation and interpolation of pixmap - Performance problems

I need to rotate a pixmap in XLib with some kind of interpolation to reduce the aliasing. I came up with the following code, which uses bilinear interpolation. It works fine: the rotated image looks perfect, but unfortunately it takes 5 or 6 seconds for each rotation. (in a 300x300, 16 colours...

LEARN ABOUT LINUX

ndb_delete_all

NDB_DELETE_ALL(1)					       MySQL Database System						 NDB_DELETE_ALL(1)

NAME

       ndb_delete_all - delete all rows from an NDB table

SYNOPSIS

       ndb_delete_all options

DESCRIPTION

       ndb_delete_all deletes all rows from the given NDB table. In some cases, this can be much faster than DELETE or even TRUNCATE TABLE.
	       Usage

	   ndb_delete_all -c connect_string tbl_name -d db_name

       This deletes all rows from the table named tbl_name in the database named db_name. It is exactly equivalent to executing TRUNCATE
       db_name.tbl_name in MySQL.

       The following table includes options that are specific to ndb_delete_all. Additional descriptions follow the table. For options common to
       most MySQL Cluster programs (including ndb_delete_all), see Options Common to MySQL Cluster Programs(1).

       Table 17.16. ndb_delete_all Options and Variables: MySQL Cluster NDB 7.2
       +--------------------------+-----------------------------------+-------------------------------------+
       |Format			  | Description 		      | Added / Removed 		    |
       +--------------------------+-----------------------------------+-------------------------------------+
       |			  | Name of the database in which the | 				    |
       |       --database=dbname, | table is found		      |        All MySQL 5.5 based releases |
       |			  |				      | 				    |
       |       -d		  |				      | 				    |
       +--------------------------+-----------------------------------+-------------------------------------+
       |			  | Perform the delete in a single    | 				    |
       |       --transactional,   | transaction (may run out of       |        All MySQL 5.5 based releases |
       |			  | operations) 		      | 				    |
       |       -t		  |				      | 				    |
       +--------------------------+-----------------------------------+-------------------------------------+
       |			  | Run tup scan		      | 				    |
       |       --tupscan	  |				      |        All MySQL 5.5 based releases |
       +--------------------------+-----------------------------------+-------------------------------------+
       |			  | Run disk scan		      | 				    |
       |       --diskscan	  |				      |        All MySQL 5.5 based releases |
       +--------------------------+-----------------------------------+-------------------------------------+

       o   --transactional, -t

	   Use of this option causes the delete operation to be performed as a single transaction.

	       Warning
	       With very large tables, using this option may cause the number of operations available to the cluster to be exceeded.

COPYRIGHT

       Copyright (C) 1997, 2014, Oracle and/or its affiliates. All rights reserved.

       This documentation is free software; you can redistribute it and/or modify it only under the terms of the GNU General Public License as
       published by the Free Software Foundation; version 2 of the License.

       This documentation is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
       MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

       You should have received a copy of the GNU General Public License along with the program; if not, write to the Free Software Foundation,
       Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA or see http://www.gnu.org/licenses/.

SEE ALSO

       For more information, please refer to the MySQL Reference Manual, which may already be installed locally and which is also available online
       at http://dev.mysql.com/doc/.

AUTHOR

       Oracle Corporation (http://dev.mysql.com/).

MySQL 5.5							    01/30/2014							 NDB_DELETE_ALL(1)