Sponsored Content
Top Forums UNIX for Advanced & Expert Users mysqldump slowing down the process? Post 302305002 by DILEEP410 on Tuesday 7th of April 2009 05:58:34 PM
Old 04-07-2009
Java mysqldump slowing down the process?

Hi All,

I have a data calculation process-a perl script running each and every hour which will do some calculations on the data stored in a mysql server. Normally it tooks around 2minutes (max) to complete.

But in case if i did any actions on the linux box where the database is hosted, it is slowing down the process so as to finish the job in 30-50 minutes. I am just doing a mysqldump in the early hours of morning which is slowing down the very next process run. The process run is starting slowly but picking up it's speed and will finish up in it's own fashion, but overall 30-50 minutes it will consume.

I tried to do some optimization on the db server, moved the db server to a different box apart from where the perl script is running ie, loadserver but no change. I also noticed the io% consumed while the job is running and it is too high initially when the process start, then slows down gradually.

Here goes the version information:

Code:
MySQL:
        version    :   5.0.51a-24
        version_comment   :  (Debian)
        version_compile_machine :  i486
        version_compile_os  : debian-linux-gnu

Kernel version: 2.6.26-1-686 #1 SMP
Operating System: Debian 5.0 (lenny)

Can anyone face the similar scenario? Any ideas on this? Your help is very much appreciated!

With Regards
Dileep Pattayath
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

mysqldump script without hardcode password

OS: Linux ambglx02 2.6.16.60-0.21-default #1 Tue May 6 12:41:02 UTC 2008 i686 i686 i386 GNU/Linux Shell: bash Currently I have a mysqldump script to backup my mysql database, the command is as below: /opt/novell/mysql/bin/mysqldump --add-drop-table -u root -p -h mydb > /home/john/mydb.sql ... (5 Replies)
Discussion started by: bulkbiz
5 Replies

2. Shell Programming and Scripting

Help needed for mysqldump command

I want to take a backup of a database and redirect the output of the whole process to a log file. I am using the below command: mysqldump -A --add-drop-table > mysql-daily-backup.sql &> /tmp/backup_log/mysql.log Is there anything wrong with the syntax? ---------- Post updated at 08:32 PM... (0 Replies)
Discussion started by: proactiveaditya
0 Replies

3. UNIX for Dummies Questions & Answers

Problem with my crontab file, using mysqldump

My crontab file tells cron to run a certain shell script at 10:30 AM every day. The shell script backs up my database with mysqldump and then runs a sed script that does some editing of the backup file. I have programmed the shell script to write an error message to a file I have in my home... (1 Reply)
Discussion started by: Ultrix
1 Replies

4. UNIX for Dummies Questions & Answers

Sendmail process "Toomany" system slowing down

Hello Experts I have M4000 Solaris 10 server, from few many days there are too many sendmail and mail.local process starting on server and each time i need to kill mannualy using pkill send mail, some time there will 600 of them taking 30mb memory for each and hence slowing down the server,... (2 Replies)
Discussion started by: karghum
2 Replies

5. UNIX for Dummies Questions & Answers

Mysqldump certain tables

Hi, I have to upload part of my database periodically when i make changes to product data etc. However I only want to upload certain tables. We suffer from bandwidth chock here, so i want to write a couple of separate scripts that upload parts of the database that changed. The database is large... (5 Replies)
Discussion started by: timgolding
5 Replies

6. Shell Programming and Scripting

Expect slowing down / missing characters

Im writing an expect program to connect to cisco routers and run commands. my commands file has only two entries show version show running-config when I run the script, the first command is run without a problem. The second command isn't. The "s" is missing at the device command line,... (1 Reply)
Discussion started by: popeye
1 Replies

7. Shell Programming and Scripting

Mysqldump rotate backup

I have a very simple script that uses a cron job to take a daily backup of our orders database. echo "Dumping ORDERS database"; mysqldump -u root --password='mypassword' -h '1.1.1.1' --opt --compress ORDERS $tbl_names > /Volumes/Files_Backup_1/db_backups/orders.sql echo "Copied database to... (2 Replies)
Discussion started by: timgolding
2 Replies

8. UNIX for Advanced & Expert Users

awk slowing down -- why?

I have an awk script that extracts data from log files and generates a report. The script is incrementing elements in arrays, summing fields depending on contents of other fields, and then in END performing some operations on those arrays. It seemed to be taking longer than it should on my... (6 Replies)
Discussion started by: treesloth
6 Replies

9. Shell Programming and Scripting

Need Help with automatically Import from special mysqldump

Hi @ all I need a little bit help with a tricky problem ... Here´s the situation: We´ve 2 MySQL-Servers, one is productive, the other is Backup. At the productive Server there runs every 2 hours a cron Job which does a Dump from MySQL-DB with script 'automysqlbackup.sh' and copy it then... (7 Replies)
Discussion started by: jackcracker
7 Replies
MYSQLDIFF(1p)						User Contributed Perl Documentation					     MYSQLDIFF(1p)

NAME
mysql-schema-diff - compare MySQL database schemas SYNOPSIS
mysql-schema-diff [B<options>] B<database1> B<database2> mysql-schema-diff --help DESCRIPTION
mysql-schema-diff is a Perl script front-end to the CPAN <http://www.perl.com/CPAN> module MySQL::Diff <http://search.cpan.org/search?module=MySQL::Diff> which compares the data structures (i.e. schema / table definitions) of two MySQL <http://www.mysql.com/> databases, and returns the differences as a sequence of MySQL commands suitable for piping into mysql which will transform the structure of the first database to be identical to that of the second (c.f. diff and patch). Database structures can be compared whether they are files containing table definitions or existing databases, local or remote. N.B. The program makes no attempt to compare any of the data which may be stored in the databases. It is purely for comparing the table definitions. I have no plans to implement data comparison; it is a complex problem and I have no need of such functionality anyway. However there is another program coldiff <http://rossbeyer.net/software/mysql_coldiff/> which does this, and is based on an older program called datadiff which seems to have vanished off the 'net. For PostgreSQL there are similar tools such as pgdiff <http://pgdiff.sourceforge.net/> and apgdiff <http://apgdiff.startnet.biz/>. EXAMPLES
# compare table definitions in two files mysql-schema-diff db1.mysql db2.mysql # compare table definitions in a file 'db1.mysql' with a database 'db2' mysql-schema-diff db1.mysql db2 # interactively upgrade schema of database 'db1' to be like the # schema described in the file 'db2.mysql' mysql-schema-diff -A db1 db2.mysql # compare table definitions in two databases on a remote machine mysql-schema-diff --host=remote.host.com --user=myaccount db1 db2 # compare table definitions in a local database 'foo' with a # database 'bar' on a remote machine, when a file foo already # exists in the current directory mysql-schema-diff --host2=remote.host.com --password=secret db:foo bar OPTIONS
More details to come; for now run "mysql-schema-diff --help". INTERNALS
For both of the database structures being compared, the following happens: o If the argument is a valid filename, the file is used to create a temporary database which "mysqldump -d" is run on to obtain the table definitions in canonicalised form. The temporary database is then dropped. (The temporary database is named "test_mysqldiff_temp_something" because default MySQL permissions allow anyone to create databases beginning with the prefix "test_".) o If the argument is a database, "mysqldump -d" is run directly on it. o Where authentication is required, the hostname, username, and password given by the corresponding options are used (type "mysql-schema-diff --help" for more information). o Each set of table definitions is now parsed into tables, and fields and index keys within those tables; these are compared, and the differences outputted in the form of MySQL statements. BUGS, DEVELOPMENT, CONTRIBUTING See <http://software.adamspiers.org/wiki/mysqldiff>. COPYRIGHT AND LICENSE
Copyright (c) 2000-2011 Adam Spiers. All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. SEE ALSO
MySQL::Diff, MySQL::Diff::Database, MySQL::Diff::Table, MySQL::Diff::Utils, mysql, mysqldump, mysqlshow AUTHOR
Adam Spiers <mysqldiff@adamspiers.org> perl v5.14.2 2012-04-06 MYSQLDIFF(1p)
All times are GMT -4. The time now is 08:45 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy