10-11-2012
On restart: rsync takes a list of source files and looks at file times. Then compares those times to the existing destination filetimes. If you already copied 8000 files and those 8000 files match what is in the new directory, rsync can figure that out in a few minutes, tops. It then goes on to copy the files it has not already done. That takes time. rsync has to read and hash every block, send it write it and verify using the hash.
It really sounds like you need to segment your operation if you want to max I/O throughput. Of course if this is production, then you cannot eat the box alive just for rsync.
9 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
I guess I posted in wrong forum before. How do I pause another process and then restart it on linux? The other process doesn't listen for anything.
Thanks for any help you can offer.
Dane :confused: (1 Reply)
Discussion started by: daneensign
1 Replies
2. UNIX for Dummies Questions & Answers
well is gets stuck and i dont know why.......
pid=fork();
if(pid==0)
{
pause();
write(1,"child",5);
exit(0);
}
else
{
sleep(1);
kill(pid,SIGCONT);
write(1,"parent",5);
wait(0);
}
all=1; (1 Reply)
Discussion started by: IdleProc
1 Replies
3. Shell Programming and Scripting
All,
I have one script which gives me the O/P of "percentage of filesystems utilization". we have four filesystem for which i want to check and get the mail when utilization is more than 40%. below are the filesystems.
/AB/Filesy1
/AB/Filesy2
/AB/Filesy3
/AB/Filesy4
Below script is working... (14 Replies)
Discussion started by: anshu ranjan
14 Replies
4. Shell Programming and Scripting
Hi All,
I have searched this forum for related posts but could not find one that fits mine. I have a shell script which removes all the XML tags including the text inside the tags from some 4 million XML files.
The shell script looks like this (MODIFIED):
find . "*.xml" -print | while read... (6 Replies)
Discussion started by: shoaibjameel123
6 Replies
5. Red Hat
Hi,
I've some directory that I used as working directory for a program. At the end of the procedure, the content is deleted. This directory, when I do a ls -l, appears to still take up some space. After a little research, I've seen on a another board of this forum that it's not really taking... (5 Replies)
Discussion started by: bdx
5 Replies
6. Red Hat
Hi. My "/usr" folder is running out of space. My "/home" folder is quite large and has a lot of free space. As follows:
Filesystem Type Size Used Avail Use% Mounted on
...
/dev/sda5 ext3 9.7G 2.6G 6.7G 28% /
/dev/sda7 ext3 152G 16G 128G 11% /home
/dev/sda3 ... (7 Replies)
Discussion started by: pkiula
7 Replies
7. Shell Programming and Scripting
Hi all,
I need a script to delete a large set of files from a directory under / based on an input file and want to redirect errors into separate file.
I have already prepared a list of files in the input file.
Kndly help me.
Thanks,
Prash (36 Replies)
Discussion started by: prash358
36 Replies
8. Shell Programming and Scripting
I have constant trouble with XCOPY/s for multi-gigabyte transfers.
I need a utility like XCOPY/S that remembers where it left off if I reboot. Is there such a utility? How about a free utility (free as in free beer)?
How about an md5sum sanity check too?
I posted the above query in another... (3 Replies)
Discussion started by: siegfried
3 Replies
9. Shell Programming and Scripting
Hi.
I found many scripts in the web of achieving this.
But I like to use this one
find /EDWH-DMT03 -xdev -size +10000 -exec ls -la {} \;|sort -n -k 5 > LARGE.rst
But the problem is, why it still list out files with 89 bytes as the output? Is there anything wrong with the command?
My... (7 Replies)
Discussion started by: aimy
7 Replies
LEARN ABOUT DEBIAN
slack.conf
slack.conf(5) File Formats Manual slack.conf(5)
NAME
slack.conf - configuration file for slack
DESCRIPTION
The file /etc/slack.conf contains configuration information for slack(8) and its backends. It should contain one keyword-value pair per
line, separated by an '=' sign. Keywords must consist solely of capital letters and underscores. Values may take any appropriate format,
but must not begin with a space. Comments start with '#', and all text from the '#' to the end of a line is ignored. Trailing whitespace
on lines is ignored. Empty lines or lines consisting of only whitespace and comments are ignored.
Valid keywords are:
SOURCE The master source for slack roles. It can be in one of four forms:
o /path/to/dir
Use a local directory.
o somehost:/path/to/dir
Use given directory on a remote host via rsync over SSH.
o rsync://somehost/module
Use module on a remote rsyncd server (directly over the network).
o somehost::module
Use the rsync daemon protocol over SSH to the given host. See "USING RSYNC-DAEMON FEATURES VIA A REMOTE-SHELL CONNECTION" in
rsync(1)
All forms of SOURCE are passed directly to rsync, so you can do things like add "user@" before the host on any remote forms. For
more about what rsync can do, see its manual page, of course.
For the last form, however, we do a little magic. rsync treats the last two forms equivalently, so we overload the last form by
automatically passing "-e ssh" to rsync when we see it. This hack lets us tell slack to use this nice feature of rsync just using
the SOURCE config option.
ROOT The root filesystem into which to install slack roles. Usually '/'.
ROLE_LIST
The location of the role list, which lists the roles to be installed by default on each host.
This can be a path relative to the source, or can be an entirely separate location if it starts with a slash or a hostname (option-
ally preceeded by user@).
CACHE A local cache directory, used as a local mirror of the SOURCE.
STAGE A local staging directory, used as an intermediate stage when installing files.
BACKUP_DIR
A directory in which to keep dated backups for rollbacks.
EXAMPLE
A typical file might look like this:
# slack.conf configuration file
SOURCE=slack-master:/slack # source is on a remote
# host named "slack-master"
ROLE_LIST=slack-master:/roles.conf
ROOT=/
CACHE=/var/cache/slack
STAGE=/var/lib/slack/stage
BACKUP_DIR=/var/lib/slack/backups
FILES
/etc/slack.conf
SEE ALSO
slack(8), rsync(1)
File formats 2005-05-23 slack.conf(5)