Sponsored Content
Top Forums Web Development MySQL Replication Issues: Duplicate Key Error Post 302377903 by Neo on Saturday 5th of December 2009 06:45:36 PM
Old 12-05-2009
MySQL Replication Issues: Duplicate Key Error

Lately my attention has been focused on distributed MySQL database replication to other web servers. All was going well, but then I noticed the replication would stop and there were there "Duplicate Primary Key" errors in SHOW SLAVE STATUS;

I started to trace these down and noticed that this is a very common issue in MySQL replication with many of these distributed web applications. It seems that the slave has dB tables of it's own like log files, authorization files, statistics. A mind of it's own! What kind of a slave is that!? Ha Ha...

Anyway, after doing the google-research-thing (GRT) for nearly an hour, I finally made the decision to add this line to my.cnf:

Code:
slave-skip-errors = 1062       #skip duplicate key errors in slave

.... and I feel "bad" about it.

I always thought dB replication was dB replication, pure-and-simple, but now there is a more complex side.

It seems to actually be a "good replicator" we need to know which tables in the database to replicate, heaven forbid!

Most folks on the net, based on my GRT, use the slave-skip-errors = 1062 solution (where 1062 is the error number of the duplicate key insert error), but I feel like I've just struck a deal with the devil.

Now, I have no idea what is going on... because the longer I run in this slave-skip-errors = 1062 mode, the further and further the dBs will lose consistency.

GRT yielded no good solution.

I'm lazy to test, test, test... find 1062 errors .... and then see if the table in question should be excluded from the replicate list.

Surely there must be a better way!?
 

5 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to delete duplicate records based on key

For example suppose I have a file which contains data as: $cat data 800,2 100,9 700,3 100,9 200,8 100,3 Now I want the output as 200,8 700,3 800,2 Key is first three characters, I don't want any reords which are having duplicate keys. Like sort +0.0 -0.3 data can we use... (9 Replies)
Discussion started by: sumitc
9 Replies

2. UNIX for Dummies Questions & Answers

forming duplicate rows based on value of a key

if the key (A or B or ...others) has 4 in its 3rd column the 1st A row has to form 4 dupicates along with the all the values of A in 4th column (2.9, 3.8, 4.2) . Hope I explain the question clearly. Cheers Ruby input "A" 1 4 2.9 "A" 2 5 ... (7 Replies)
Discussion started by: ruby_sgp
7 Replies

3. Shell Programming and Scripting

Strategical Shell Scripting For MySql Replication

Hi to all guyz , As i'm new to Shell scripting i was been working out to write a shell script for mysql Replication for multiple slave master architecture.i have done the process up with a script of 200 lines but still there are some things which can make my script more effective so i want your... (5 Replies)
Discussion started by: kgrvamsi
5 Replies

4. Programming

MySQL auto_increment, primary key

Hello, I want to create a table in mysql database by loading the local file. I am confused with the auto_increment column (say Run_ID) that I want to set it as primary key. My questions are: 1) Do I need add this Run_ID in my file ready? then this does not make any sense to the auto_increment;... (5 Replies)
Discussion started by: yifangt
5 Replies

5. Linux

Mysql replication

Need to recover a replication broken error on the mysql slave server. I want to force it to resend the binlog file from the begining. What is the correct value for the: MASTER_LOG_POS ? 0, 1 or 107 ? (2 Replies)
Discussion started by: andriesh
2 Replies
OCF_HEARTBEAT_MYSQL(7)						OCF resource agents					    OCF_HEARTBEAT_MYSQL(7)

NAME
ocf_heartbeat_mysql - Manages a MySQL database instance SYNOPSIS
mysql [start | stop | status | monitor | promote | demote | notify | validate-all | meta-data] DESCRIPTION
Resource script for MySQL. May manage a standalone MySQL database, a clone set with externally managed replication, or a complete master/slave replication setup. SUPPORTED PARAMETERS
binary Location of the MySQL server binary (optional, string, default /usr/bin/mysqld_safe) client_binary Location of the MySQL client binary (optional, string, default mysql) config Configuration file (optional, string, default /etc/my.cnf) datadir Directory containing databases (optional, string, default /var/lib/mysql) user User running MySQL daemon (optional, string, default mysql) group Group running MySQL daemon (for logfile and directory permissions) (optional, string, default mysql) log The logfile to be used for mysqld. (optional, string, default /var/log/mysqld.log) pid The pidfile to be used for mysqld. (optional, string, default /var/lib/mysql/mysqld.pid) socket The socket to be used for mysqld. (optional, string, default /var/lib/mysql/mysql.sock) test_table Table to be tested in monitor statement (in database.table notation) (optional, string, default mysql.user) test_user MySQL test user (optional, string, default root) test_passwd MySQL test user password (optional, string, no default) enable_creation If the MySQL database does not exist, it will be created (optional, integer, default 0) additional_parameters Additional parameters which are passed to the mysqld on startup. (e.g. --skip-external-locking or --skip-grant-tables) (optional, string, no default) replication_user MySQL replication user. This user is used for starting and stopping MySQL replication, for setting and resetting the master host, and for setting and unsetting read-only mode. Because of that, this user must have SUPER, REPLICATION SLAVE, REPLICATION CLIENT, and PROCESS privileges on all nodes within the cluster. (optional, string, no default) replication_passwd MySQL replication password. Used for replication client and slave. (optional, string, no default) replication_port The port on which the Master MySQL instance is listening. (optional, string, default 3306) max_slave_lag The maximum number of seconds a replication slave is allowed to lag behind its master. Do not set this to zero. What the cluster manager does in case a slave exceeds this maximum lag is determined by the evict_outdated_slaves parameter. (optional, integer, default 3600) evict_outdated_slaves If set to true, any slave which is more than max_slave_lag seconds behind the master has its MySQL instance shut down. If this parameter is set to false in a primitive or clone resource, it is simply ignored. If set to false in a master/slave resource, then exceeding the maximum slave lag will merely push down the master preference so the lagging slave is never promoted to the new master. (optional, boolean, default false) SUPPORTED ACTIONS
This resource agent supports the following actions (operations): start Starts the resource. Suggested minimum timeout: 120. stop Stops the resource. Suggested minimum timeout: 120. status Performs a status check. Suggested minimum timeout: 60. monitor Performs a detailed status check. Suggested minimum timeout: 30. Suggested interval: 10. promote Promotes the resource to the Master role. Suggested minimum timeout: 120. demote Demotes the resource to the Slave role. Suggested minimum timeout: 120. notify Suggested minimum timeout: 90. validate-all Performs a validation of the resource configuration. Suggested minimum timeout: 5. meta-data Retrieves resource agent metadata (internal use only). Suggested minimum timeout: 5. EXAMPLE
The following is an example configuration for a mysql resource using the crm(8) shell: primitive example_mysql ocf:heartbeat:mysql op monitor depth="0" timeout="30" interval="10" SEE ALSO
http://www.linux-ha.org/wiki/mysql_(resource_agent) AUTHOR
Linux-HA contributors (see the resource agent source for information about individual authors) resource-agents 1.0.3 07/05/2010 OCF_HEARTBEAT_MYSQL(7)
All times are GMT -4. The time now is 04:49 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy