Sponsored Content
Full Discussion: Lsyncd Configuration
Top Forums UNIX for Beginners Questions & Answers Lsyncd Configuration Post 303038591 by stomp on Friday 6th of September 2019 04:54:45 PM
Old 09-06-2019
I used lsyncd to migrate(p2v) servers to minimize downtime when migrating. Had lot's of files(1K-1M).

Like plain rsync it's good for a lot of things. It's very bad for syncing big files which change often(-> big database files), because lsyncd will constantly syncing those big files if only one byte changes. ( I had ~100 smaller mysql databases on some server. This worked fine.)

Worked like charm. You can apply rsync args with rsync/_extra:

Code:
sync {
    default.rsync,
    source    = "/",
    target    = "1.2.3.42:/disk/",
    exclude   = {
                "/proc/**",
                "/sys/**",
                "/dev/**"
                },
    rsync     = {
        archive  = true,
        _extra  = { "-v", "-H", "--numeric-ids" }
    }
}

Update:
Since you copy files from nfs to nfs I recommend to check disabling the Delta-Transfer with -W.

If not lsyncd/rsync will read the whole file from the source-nfs before transferring the 5 (Bytes? MBytes? GBytes?) that have changed.


The above is default, when - like in this case - both source and target path are local(begin with /).

If you have so many files and access it via nfs the initial phase is of course taking long. (I had backup issues of servers with 4-8 million files, took 10-30 hours for an rsync backup to take place even if little changed - and that was with a local source). This should be either with lsyncd or rsync. But I'm curious if plain rsync is faster.

If you just have vanishing many files, this should not cause heavy load.

I think there's not so much difference between lsyncd and rsync. Because the former is just like a wrapper of the latter.

Last edited by stomp; 09-07-2019 at 09:09 AM..
 

8 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Configuration

Can anyone please tell me anything about the following: AIX O/S - IBM RS6000 server I want to know where I can go to check and see how much RAM is intalled in the server, how many Processors are installed in the server, and how I can run a sar command to show me processor statistics? I... (4 Replies)
Discussion started by: Docboyeee
4 Replies

2. BSD

X configuration

HEllo, I try to Launch OOO, but I have X configuration problem. I 'm searching in man pages but if someone can help me... Of course I set the DISPLAY as explain in the man page but with no more result: setenv DISPLAY myws:0 Thanks in advance (1 Reply)
Discussion started by: SoulCoder
1 Replies

3. Post Here to Contact Site Administrators and Moderators

configuration

how to configure servor dns on windows servor 2003 (0 Replies)
Discussion started by: djest
0 Replies

4. Linux

configuration

hello, I don't know how to configure servor dns on linux,please help I thanck YOU (1 Reply)
Discussion started by: djest
1 Replies

5. AIX

Need help on IP Configuration

I have an RS/6000 running AIX 5.3. I would like to get either dhcp client working or assigning it an ip would be fine as well. I have tried using smit to do it many times trying different things, and I can get an ip assigned but it doesn't communicate with the network or internet. I haven't been... (3 Replies)
Discussion started by: izzzy
3 Replies

6. SCO

printer configuration

Dear all i have install printer in sco open 5 , i m not getting print out , but printing is showing in que plz help (7 Replies)
Discussion started by: sudhir69
7 Replies

7. Red Hat

LDAP configuration

I have installed openldap-servers package in server machine i have edited vi /etc/openldap/ldap.conf i have added following line BASE dc=abcd,dc=com URI ldap://ldap.abcd.com ldap://ldap-master.abcd.com:666 i have restarted the ldap service then client pc i have installed... (0 Replies)
Discussion started by: ainstin
0 Replies

8. UNIX for Advanced & Expert Users

Postfix Configuration

Hi, i have to configure postfix in this mode: - if i receive a mail from a@a.a relay to - if i receive a mail NOT from a@a.a relay the mail to user "test" that is on localhost For the first rule i do this in main.cf: sender_dependent_default_transport_maps = hash:/etc/postfix/sender_check... (0 Replies)
Discussion started by: certo85
0 Replies
SVN::Notify::Mirror::Rsync(3pm) 			User Contributed Perl Documentation			   SVN::Notify::Mirror::Rsync(3pm)

NAME
SVN::Notify::Mirror::Rsync - Mirror a repository path via Rsync SYNOPSIS
Use svnnotify in post-commit: svnnotify --repos-path "$1" --revision "$2" --handler Mirror::Rsync --to "/path/to/local/htdocs" [--svn-binary /full/path/to/svn] --rsync-host remote_server [--rsync-delete=[yes|no]] [--rsync-dest "/path/on/remote/server"] [--rsync-args arg1 [--rsync-args arg2...]] [[--rsync-ssh] [--ssh-user remote_user] [--ssh-identity /home/user/.ssh/id_rsa]] or better yet, use SVN::Notify::Config for a more sophisticated setup: #!/usr/bin/perl -MSVN::Notify::Config=$0 --- #YAML:1.0 '': PATH: "/usr/bin:/usr/local/bin" 'path/in/repository': handler: Mirror to: "/path/to/www/htdocs" 'some/other/path/in/repository': handler: Mirror to: "/path/to/local/www/htdocs" rsync-host: "remote_host" rsync-dest: "/path/on/remote/www/htdocs" ssh-user: "remote_user" ssh-identity: "/home/user/.ssh/id_rsa" DESCRIPTION
Keep a directory in sync with a portion of a Subversion repository. Typically used to keep a development web server in sync with the changes made to the repository. This directory can either be on the same box as the repository itself, or it can be remote (via SSH connection). USAGE
Depending on whether the target is a "Local Mirror" or a Remote Mirror, there are different options available. All options are available either as a commandline option to svnnotify or as a hash key in SVN::Notify::Config (see their respective documentation for more details). Working Copy on Local host Because 'svn export' is not able to be consistently updated, the local rsync'd directory must be a full working copy. The remote server will only contain the ordinary files (no Subversion admin files). The files in the working copy must be writeable (preferrably owned) by the user identity executing the hook script (this is the user identity that is running Apache or svnserve respectively). Local Mirror Please see " SVN::Notify::Mirror " for details. Remote Mirror Used for directories not located on the same machine as the repository itself. Typically, this might be a production web server located in a DMZ, so special consideration must be paid to security concerns. In particular, the remote mirror server may not be able to directly access the repository box. o rsync-host This value is required and must be the hostname or IP address of the remote host (where the mirror directories reside). o rsync-delete The default mode of operation is to delete remote files which are not present in the local working copy. NOTE: this will delete any unversioned files in the remote directory tree. Unless you have all of your files under version control, you should pass the "--no-rsync-delete" or "--rsync-delete no" option. o rsync-dest This optional value specifies the path to update on the remote host. If you do not specify this value, the same path as passed in as the "--to" parameter will be used (this may not be what you meant to do). o rsync-args This optional parameter can be used to pass additional commandline options to the rsync command. You can use this multiple times in order to pass multiple options. The default args are "--archive --compress". See the "rsync-ssh" options for using SSH instead of RSH (rather than pass those commands via "--rsync-args" o rsync-ssh This optional parameter signals that you wish to use SSH instead of whatever the default remote shell program is configured in your copy of rsync. You may need to set one or more of the "ssh-*" parameters as well. o ssh-user If the remote user is different than the local user executing the postcommit script, you can specify it with this parameter. You would often use this in conjunction with the next parameter. o ssh-identity This value may be optional and should be the full path to the local identity file being used to authenticate with the remote host. If you are setting the ssh-user to be something other than the local user name, you will typically also have to set the ssh-identity. AUTHOR
John Peacock <jpeacock@cpan.org> COPYRIGHT
Copyright (c) 2005-2008 John Peacock This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. The full text of the license can be found in the LICENSE file included with this module. SEE ALSO
SVN::Notify, SVN::Notify::Config, SVN::Notify::Mirror perl v5.14.2 2012-07-04 SVN::Notify::Mirror::Rsync(3pm)
All times are GMT -4. The time now is 06:18 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy