Sponsored Content
Top Forums UNIX for Advanced & Expert Users mysqldump slowing down the process? Post 302305002 by DILEEP410 on Tuesday 7th of April 2009 05:58:34 PM
Old 04-07-2009
Java mysqldump slowing down the process?

Hi All,

I have a data calculation process-a perl script running each and every hour which will do some calculations on the data stored in a mysql server. Normally it tooks around 2minutes (max) to complete.

But in case if i did any actions on the linux box where the database is hosted, it is slowing down the process so as to finish the job in 30-50 minutes. I am just doing a mysqldump in the early hours of morning which is slowing down the very next process run. The process run is starting slowly but picking up it's speed and will finish up in it's own fashion, but overall 30-50 minutes it will consume.

I tried to do some optimization on the db server, moved the db server to a different box apart from where the perl script is running ie, loadserver but no change. I also noticed the io% consumed while the job is running and it is too high initially when the process start, then slows down gradually.

Here goes the version information:

Code:
MySQL:
        version    :   5.0.51a-24
        version_comment   :  (Debian)
        version_compile_machine :  i486
        version_compile_os  : debian-linux-gnu

Kernel version: 2.6.26-1-686 #1 SMP
Operating System: Debian 5.0 (lenny)

Can anyone face the similar scenario? Any ideas on this? Your help is very much appreciated!

With Regards
Dileep Pattayath
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

mysqldump script without hardcode password

OS: Linux ambglx02 2.6.16.60-0.21-default #1 Tue May 6 12:41:02 UTC 2008 i686 i686 i386 GNU/Linux Shell: bash Currently I have a mysqldump script to backup my mysql database, the command is as below: /opt/novell/mysql/bin/mysqldump --add-drop-table -u root -p -h mydb > /home/john/mydb.sql ... (5 Replies)
Discussion started by: bulkbiz
5 Replies

2. Shell Programming and Scripting

Help needed for mysqldump command

I want to take a backup of a database and redirect the output of the whole process to a log file. I am using the below command: mysqldump -A --add-drop-table > mysql-daily-backup.sql &> /tmp/backup_log/mysql.log Is there anything wrong with the syntax? ---------- Post updated at 08:32 PM... (0 Replies)
Discussion started by: proactiveaditya
0 Replies

3. UNIX for Dummies Questions & Answers

Problem with my crontab file, using mysqldump

My crontab file tells cron to run a certain shell script at 10:30 AM every day. The shell script backs up my database with mysqldump and then runs a sed script that does some editing of the backup file. I have programmed the shell script to write an error message to a file I have in my home... (1 Reply)
Discussion started by: Ultrix
1 Replies

4. UNIX for Dummies Questions & Answers

Sendmail process "Toomany" system slowing down

Hello Experts I have M4000 Solaris 10 server, from few many days there are too many sendmail and mail.local process starting on server and each time i need to kill mannualy using pkill send mail, some time there will 600 of them taking 30mb memory for each and hence slowing down the server,... (2 Replies)
Discussion started by: karghum
2 Replies

5. UNIX for Dummies Questions & Answers

Mysqldump certain tables

Hi, I have to upload part of my database periodically when i make changes to product data etc. However I only want to upload certain tables. We suffer from bandwidth chock here, so i want to write a couple of separate scripts that upload parts of the database that changed. The database is large... (5 Replies)
Discussion started by: timgolding
5 Replies

6. Shell Programming and Scripting

Expect slowing down / missing characters

Im writing an expect program to connect to cisco routers and run commands. my commands file has only two entries show version show running-config when I run the script, the first command is run without a problem. The second command isn't. The "s" is missing at the device command line,... (1 Reply)
Discussion started by: popeye
1 Replies

7. Shell Programming and Scripting

Mysqldump rotate backup

I have a very simple script that uses a cron job to take a daily backup of our orders database. echo "Dumping ORDERS database"; mysqldump -u root --password='mypassword' -h '1.1.1.1' --opt --compress ORDERS $tbl_names > /Volumes/Files_Backup_1/db_backups/orders.sql echo "Copied database to... (2 Replies)
Discussion started by: timgolding
2 Replies

8. UNIX for Advanced & Expert Users

awk slowing down -- why?

I have an awk script that extracts data from log files and generates a report. The script is incrementing elements in arrays, summing fields depending on contents of other fields, and then in END performing some operations on those arrays. It seemed to be taking longer than it should on my... (6 Replies)
Discussion started by: treesloth
6 Replies

9. Shell Programming and Scripting

Need Help with automatically Import from special mysqldump

Hi @ all I need a little bit help with a tricky problem ... Here´s the situation: We´ve 2 MySQL-Servers, one is productive, the other is Backup. At the productive Server there runs every 2 hours a cron Job which does a Dump from MySQL-DB with script 'automysqlbackup.sh' and copy it then... (7 Replies)
Discussion started by: jackcracker
7 Replies
SPEEDY(1p)																SPEEDY(1p)

NAME
speedy - a persistent Perl interpreter SYNOPSIS
speedy [ <perl options> ] [ -- <speedy options> ] [ <filename> ] DESCRIPTION
speedy, short for SpeedyCGI, is a way to run perl scripts persistently, which can make them run much more quickly. The most common way to make a script run persistently is by changing the interpreter line at the top of the script from: #!/usr/bin/perl to #!/usr/bin/speedy After the script is initially run, instead of exiting, the perl interpreter is kept running. During subsequent runs, this interpreter is used to handle new executions instead of starting a new perl interpreter each time. A very fast frontend program, written in C, is exe- cuted for each request. This fast frontend then contacts the persistent Perl process, which is usually already running, to do the work and return the results. Each perl script runs in its own Unix process, so one perl script can't interfere with another. Command line options can also be used to deal with programs that have memory leaks or other problems that might keep them from otherwise running persistently. Although SpeedyCGI is especially useful for CGI scripts, it can be used to keep any perl script running persistently. See CGI::Speedy- CGI(3.pm) for a complete description of SpeedyCGI, including further details on CGI execution and Apache issues. This manual page is based on that, but concentrates on running speedy from the command line. OPTIONS
The speedy command line is the same as for regular perl, with the exception that SpeedyCGI specific options can be passed in after a "--". For example the line: #!/usr/bin/speedy -w -- -t300 at the top of your script will set the perl option `-w' and will pass the `-t' option to SpeedyCGI, setting the Timeout value to 300 sec- onds. The options can also be set at run-time from the perl script using the CGI::SpeedyCGI module. OPTIONS AVAILABLE See CGI::SpeedyCGI(3.pm) for a complete description of the options. -p<string> BackendProg : Path to the speedy backend program. (Default: /usr/bin/speedy_backend) -B<number> BufsizGet : Use <number> bytes as the maximum size for the buffer that receives data from the perl backend. (Default: 131072) -b<number> BufsizPost : Use <number> bytes as the maximum size for the buffer that sends data to the perl backend. (Default: 131072) -g<string> Group : Allow a single perl interpreter to run multiple scripts. See CGI::SpeedyCGI(3.pm) for details. (Default: `none') -M<number> MaxBackends : If non-zero, limits the number of speedy backends running for this perl script to <number>. (Default: 0) -r<number> MaxRuns : Once the perl interpreter has run <number> times, re-exec the backend process. Zero indicates no maximum. This option is useful for processes that tend to consume resources over time. (Default: 500) -t<number> Timeout : If no new requests have been received after <number> seconds, exit the persistent perl interpreter. Zero indicates no timeout. (Default: 3600) -T<string> TmpBase : Use the given prefix for creating temporary files. This must be a filename prefix, not a directory name. (Default: `/tmp/speedy') -v Version : Print the SpeedyCGI version and exit. ENVIRONMENT
Environment variables can also be used to pass in options. This can only be done before the initial execution, not from within the script itself. The name of the environment variable is always SPEEDY_ followed by the option name in upper-case. For example to set the speedy Timeout option, use the environment variable named SPEEDY_TIMEOUT. FILES
/tmp/speedy* A unix socket used to connect to the backend process. See speedy_backend(1) for more information. AUTHOR
Sam Horrocks http://daemoninc.com sam@daemoninc.com NOTES
This manual page was created by Niko Tyni <ntyni@iki.fi> for Debian GNU/Linux, because the original program does not have one. It is based on the original and more complete CGI::SpeedyCGI(3pm) manual page. BUGS
There are command-line parsing incompatibilities with the real Perl. These aren't very easy to fix, as even the perlrun manpage isn't quite accurate on which parameters can be separated (like '-I') and which can't (like '-C'). speedy doesn't allow any of them to be sepa- rated. It considers the first option without a leading dash as the script filename. SEE ALSO
perl(1), CGI::SpeedyCGI(3pm), speedy_backend(1) SPEEDY(1p)
All times are GMT -4. The time now is 04:12 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy