I have a data calculation process-a perl script running each and every hour which will do some calculations on the data stored in a mysql server. Normally it tooks around 2minutes (max) to complete.
But in case if i did any actions on the linux box where the database is hosted, it is slowing down the process so as to finish the job in 30-50 minutes. I am just doing a mysqldump in the early hours of morning which is slowing down the very next process run. The process run is starting slowly but picking up it's speed and will finish up in it's own fashion, but overall 30-50 minutes it will consume.
I tried to do some optimization on the db server, moved the db server to a different box apart from where the perl script is running ie, loadserver but no change. I also noticed the io% consumed while the job is running and it is too high initially when the process start, then slows down gradually.
Here goes the version information:
Can anyone face the similar scenario? Any ideas on this? Your help is very much appreciated!
Location: Asia Pacific, Cyberspace, in the Dark Dystopia
Posts: 19,118
Thanks Given: 2,351
Thanked 3,359 Times in 1,878 Posts
No problem at all. I just wanted to get your ideas first, so we can have a two-way dialog, not just one way.
First, I suggest you decrease your maximum number of SQL connections to 100 to save memory, since you are very low on memory, according to the tuning scripts.
At the same time you do this increase your query cache size to 128 MB, as a first step.
ok, i will increase my query cache to 128mb as you said.The number of connections was 100 before and i modified it few days ago after i faced an issue with the connection limit. The figure the script is showing (74) is because i restarted my server yesterday early morning. What else i was supposed to do?
Hi @ all
I need a little bit help with a tricky problem ...
Here´s the situation:
We´ve 2 MySQL-Servers, one is productive, the other is Backup.
At the productive Server there runs every 2 hours a cron Job which does a Dump from MySQL-DB with script 'automysqlbackup.sh' and
copy it then... (7 Replies)
I have an awk script that extracts data from log files and generates a report. The script is incrementing elements in arrays, summing fields depending on contents of other fields, and then in END performing some operations on those arrays. It seemed to be taking longer than it should on my... (6 Replies)
I have a very simple script that uses a cron job to take a daily backup of our orders database.
echo "Dumping ORDERS database";
mysqldump -u root --password='mypassword' -h '1.1.1.1' --opt --compress ORDERS $tbl_names > /Volumes/Files_Backup_1/db_backups/orders.sql
echo "Copied database to... (2 Replies)
Im writing an expect program to connect to cisco routers and run commands.
my commands file has only two entries
show version
show running-config
when I run the script, the first command is run without a problem.
The second command isn't.
The "s" is missing at the device command line,... (1 Reply)
Hi,
I have to upload part of my database periodically when i make changes to product data etc. However I only want to upload certain tables. We suffer from bandwidth chock here, so i want to write a couple of separate scripts that upload parts of the database that changed. The database is large... (5 Replies)
Hello Experts
I have M4000 Solaris 10 server, from few many days there are too many sendmail and mail.local process starting on server and each time i need to kill mannualy using pkill send mail, some time there will 600 of them taking 30mb memory for each and hence slowing down the server,... (2 Replies)
My crontab file tells cron to run a certain shell script at 10:30 AM every day. The shell script backs up my database with mysqldump and then runs a sed script that does some editing of the backup file.
I have programmed the shell script to write an error message to a file I have in my home... (1 Reply)
I want to take a backup of a database and redirect the output of the whole process to a log file. I am using the below command:
mysqldump -A --add-drop-table > mysql-daily-backup.sql &> /tmp/backup_log/mysql.log
Is there anything wrong with the syntax?
---------- Post updated at 08:32 PM... (0 Replies)
OS: Linux ambglx02 2.6.16.60-0.21-default #1 Tue May 6 12:41:02 UTC 2008 i686 i686 i386 GNU/Linux
Shell: bash
Currently I have a mysqldump script to backup my mysql database, the command is as below:
/opt/novell/mysql/bin/mysqldump --add-drop-table -u root -p -h mydb > /home/john/mydb.sql
... (5 Replies)