Sponsored Content
Top Forums Web Development MySQL Replication Issues: Duplicate Key Error Post 302377903 by Neo on Saturday 5th of December 2009 06:45:36 PM
Old 12-05-2009
MySQL Replication Issues: Duplicate Key Error

Lately my attention has been focused on distributed MySQL database replication to other web servers. All was going well, but then I noticed the replication would stop and there were there "Duplicate Primary Key" errors in SHOW SLAVE STATUS;

I started to trace these down and noticed that this is a very common issue in MySQL replication with many of these distributed web applications. It seems that the slave has dB tables of it's own like log files, authorization files, statistics. A mind of it's own! What kind of a slave is that!? Ha Ha...

Anyway, after doing the google-research-thing (GRT) for nearly an hour, I finally made the decision to add this line to my.cnf:

Code:
slave-skip-errors = 1062       #skip duplicate key errors in slave

.... and I feel "bad" about it.

I always thought dB replication was dB replication, pure-and-simple, but now there is a more complex side.

It seems to actually be a "good replicator" we need to know which tables in the database to replicate, heaven forbid!

Most folks on the net, based on my GRT, use the slave-skip-errors = 1062 solution (where 1062 is the error number of the duplicate key insert error), but I feel like I've just struck a deal with the devil.

Now, I have no idea what is going on... because the longer I run in this slave-skip-errors = 1062 mode, the further and further the dBs will lose consistency.

GRT yielded no good solution.

I'm lazy to test, test, test... find 1062 errors .... and then see if the table in question should be excluded from the replicate list.

Surely there must be a better way!?
 

5 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to delete duplicate records based on key

For example suppose I have a file which contains data as: $cat data 800,2 100,9 700,3 100,9 200,8 100,3 Now I want the output as 200,8 700,3 800,2 Key is first three characters, I don't want any reords which are having duplicate keys. Like sort +0.0 -0.3 data can we use... (9 Replies)
Discussion started by: sumitc
9 Replies

2. UNIX for Dummies Questions & Answers

forming duplicate rows based on value of a key

if the key (A or B or ...others) has 4 in its 3rd column the 1st A row has to form 4 dupicates along with the all the values of A in 4th column (2.9, 3.8, 4.2) . Hope I explain the question clearly. Cheers Ruby input "A" 1 4 2.9 "A" 2 5 ... (7 Replies)
Discussion started by: ruby_sgp
7 Replies

3. Shell Programming and Scripting

Strategical Shell Scripting For MySql Replication

Hi to all guyz , As i'm new to Shell scripting i was been working out to write a shell script for mysql Replication for multiple slave master architecture.i have done the process up with a script of 200 lines but still there are some things which can make my script more effective so i want your... (5 Replies)
Discussion started by: kgrvamsi
5 Replies

4. Programming

MySQL auto_increment, primary key

Hello, I want to create a table in mysql database by loading the local file. I am confused with the auto_increment column (say Run_ID) that I want to set it as primary key. My questions are: 1) Do I need add this Run_ID in my file ready? then this does not make any sense to the auto_increment;... (5 Replies)
Discussion started by: yifangt
5 Replies

5. Linux

Mysql replication

Need to recover a replication broken error on the mysql slave server. I want to force it to resend the binlog file from the begining. What is the correct value for the: MASTER_LOG_POS ? 0, 1 or 107 ? (2 Replies)
Discussion started by: andriesh
2 Replies
SLURPD(8C)																SLURPD(8C)

NAME
slurpd - Standalone LDAP Update Replication Daemon SYNOPSIS
/usr/sbin/slurpd [-d debug-level] [-f slapd-config-file] [-r slapd-replog-file] [-t temp-dir] [-o] [-k srvtab-file] DESCRIPTION
Slurpd is used to propagate changes from one slapd database to another. If slapd is configured to produce a replication log, slurpd reads that replication log and sends the changes to the slave slapd instances via the LDAP protocol. slurpd is typically invoked at boot time, usually out of /etc/rc.local. Upon startup, slurpd normally forks and disassociates itself from the invoking tty, then reads the replication log (given either by the replogfile directive in the slapd config file, or by the -r command-line option). If the replication log file does not exist or is empty, slurpd goes to sleep. It periodically wakes up and checks to see if there are any changes to be propoagated. When slurpd notices that there are changes to propagate to slave slapd instances, it locks the replication log, makes its own private copy, releases the lock, and forks one copy of itself for each replica slapd to be updated. Each child process binds to the slave slapd as the DN given by the binddn option to the replica directive in the slapd config file, and sends the changes. See slapd(8) for details on the standalone LDAP daemon. OPTIONS
-d debug-level Turn on debugging as defined by debug-level. If this option is specified, even with a zero argument, slurpd will not fork or disas- sociate from the invoking terminal. Some general operation and status messages are printed for any value of debug-level. debug-level is taken as a bit string, with each bit corresponding to a different kind of debugging information. See <ldap.h> for details. -f slapd-config-file Specifies the slapd configuration file. The default is /etc/openldap/slapd.conf. -r slapd-replog-file Specifies the name of the slapd replication logfile. Normally, the name of the replication log file is read from the slapd configu- ration file. The file should be located in a directory with limited read/write/execute access. The -r option allows you to over- ride this. In conjunction with the -o option, you can process a replication log file in a "one-shot" mode. For example, if slurpd has encountered errors in processing a replication log, you can run it in one-shot mode and give the rejection file name as the argument to the -r option, once you've resolved the problem which caused the replication to fail. -o Run in "one-shot" mode. Normally, slurpd processes the replog file and then watches for more replication entries to be appended. In one-shot mode, slurpd processes a replication log and exits. -t temp-dir slurpd copies the replication log to a working directory before processing it. The directory permissions should limit read/write/execute access as temporary files may contain sensitive information. This option allows you to specify the location of these temporary files. The default is /var/lib/ldap. -k srvtab-file Specify the location of the kerberos srvtab file which contains keys for the replica slapd instances. Overrides the srvtab argument to the replica directive in the slapd configuration file. EXAMPLES
To start slurpd and have it fork and detach from the terminal and process the replication logs generated by slapd, just type: /usr/sbin/slurpd To start slurpd with an alternate slapd configuration file, and turn on voluminous debugging which will be printed on standard error, type: /usr/sbin/slurpd -f /etc/openldap/slapd.conf -d 255 SEE ALSO
ldap(3), slapd.replog(5), slapd(8) "OpenLDAP Administrator's Guide" (http://www.OpenLDAP.org/doc/admin/) ACKNOWLEDGEMENTS
OpenLDAP is developed and maintained by The OpenLDAP Project (http://www.openldap.org/). OpenLDAP is derived from University of Michigan LDAP 3.3 Release. OpenLDAP 2.0.27-Release 20 August 2000 SLURPD(8C)
All times are GMT -4. The time now is 05:49 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy