Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Is it better/possible to pause the rsyncing of a very large directory? Post 302713527 by jim mcnamara on Wednesday 10th of October 2012 11:53:24 PM
Old 10-11-2012
On restart: rsync takes a list of source files and looks at file times. Then compares those times to the existing destination filetimes. If you already copied 8000 files and those 8000 files match what is in the new directory, rsync can figure that out in a few minutes, tops. It then goes on to copy the files it has not already done. That takes time. rsync has to read and hash every block, send it write it and verify using the hash.


It really sounds like you need to segment your operation if you want to max I/O throughput. Of course if this is production, then you cannot eat the box alive just for rsync.
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

how to pause another process?

I guess I posted in wrong forum before. How do I pause another process and then restart it on linux? The other process doesn't listen for anything. Thanks for any help you can offer. Dane :confused: (1 Reply)
Discussion started by: daneensign
1 Replies

2. UNIX for Dummies Questions & Answers

pause() problems

well is gets stuck and i dont know why....... pid=fork(); if(pid==0) { pause(); write(1,"child",5); exit(0); } else { sleep(1); kill(pid,SIGCONT); write(1,"parent",5); wait(0); } all=1; (1 Reply)
Discussion started by: IdleProc
1 Replies

3. Shell Programming and Scripting

script to check large directory--help

All, I have one script which gives me the O/P of "percentage of filesystems utilization". we have four filesystem for which i want to check and get the mail when utilization is more than 40%. below are the filesystems. /AB/Filesy1 /AB/Filesy2 /AB/Filesy3 /AB/Filesy4 Below script is working... (14 Replies)
Discussion started by: anshu ranjan
14 Replies

4. Shell Programming and Scripting

Using find in a directory containing large number of files

Hi All, I have searched this forum for related posts but could not find one that fits mine. I have a shell script which removes all the XML tags including the text inside the tags from some 4 million XML files. The shell script looks like this (MODIFIED): find . "*.xml" -print | while read... (6 Replies)
Discussion started by: shoaibjameel123
6 Replies

5. Red Hat

Empty directory, large size and performance

Hi, I've some directory that I used as working directory for a program. At the end of the procedure, the content is deleted. This directory, when I do a ls -l, appears to still take up some space. After a little research, I've seen on a another board of this forum that it's not really taking... (5 Replies)
Discussion started by: bdx
5 Replies

6. Red Hat

On CentOS, moving space from large free directory to another

Hi. My "/usr" folder is running out of space. My "/home" folder is quite large and has a lot of free space. As follows: Filesystem Type Size Used Avail Use% Mounted on ... /dev/sda5 ext3 9.7G 2.6G 6.7G 28% / /dev/sda7 ext3 152G 16G 128G 11% /home /dev/sda3 ... (7 Replies)
Discussion started by: pkiula
7 Replies

7. Shell Programming and Scripting

Need to delete large set of files (i.e) close to 100K from a directory based on the input file

Hi all, I need a script to delete a large set of files from a directory under / based on an input file and want to redirect errors into separate file. I have already prepared a list of files in the input file. Kndly help me. Thanks, Prash (36 Replies)
Discussion started by: prash358
36 Replies

8. Shell Programming and Scripting

How to copy very large directory trees

I have constant trouble with XCOPY/s for multi-gigabyte transfers. I need a utility like XCOPY/S that remembers where it left off if I reboot. Is there such a utility? How about a free utility (free as in free beer)? How about an md5sum sanity check too? I posted the above query in another... (3 Replies)
Discussion started by: siegfried
3 Replies

9. Shell Programming and Scripting

Find Large Files Recursively From Specific Directory

Hi. I found many scripts in the web of achieving this. But I like to use this one find /EDWH-DMT03 -xdev -size +10000 -exec ls -la {} \;|sort -n -k 5 > LARGE.rst But the problem is, why it still list out files with 89 bytes as the output? Is there anything wrong with the command? My... (7 Replies)
Discussion started by: aimy
7 Replies
stopping(7)                                              Miscellaneous Information Manual                                              stopping(7)

NAME
stopping - event signalling that a job is stopping SYNOPSIS
stopping JOB=JOB INSTANCE=INSTANCE RESULT=RESULT [PROCESS=PROCESS] [EXIT_STATUS=STATUS] [EXIT_SIGNAL=SIGNAL] [ENV]... DESCRIPTION
The stopping event is generated by the Upstart init(8) daemon when an instance of a job begins stopping. The JOB environment variable con- tains the job name, and the INSTANCE environment variable contains the instance name which will be empty for single-instance jobs. If the job is stopping normally, the RESULT environment variable will be ok, otherwise if the job is stopping because it has failed it will be failed. When the job has failed, the process that failed will be given in the PROCESS environment variable. This may be pre-start, post-start, main, pre-stop or post-stop; it may also be the special value respawn to indicate that the job is stopping because it hit the respawn limit. Finally in the case of a failed job, one of either EXIT_STATUS or EXIT_SIGNAL may be given to indicate the cause of the stop. Either EXIT_STATUS will contain the exit status code of the process, or EXIT_SIGNAL will contain the name of the signal that the process received. The normal exit job configuration stanza can be used to prevent particular exit status values or signals resulting in a failed job, see init(5) for more information. If neither EXIT_STATUS or EXIT_SIGNAL is given for a failed process, it is because the process failed to spawn (for example, file not found). See the system logs for the error. init(8) will wait for all services started by this event to be running, all tasks started by this event to have finished and all jobs stopped by this event to be stopped before allowing the job to continue stopping. This allows jobs to depend on other jobs, safely stopping themselves before their dependency goes away. This event is typically combined with the started(7) event by services. Job configuration files may use the export stanza to export environment variables from their own environment into the stopping event. See init(5) for more details. EXAMPLE
A service that wishes to depend on another service might use: start on started apache stop on stopping apache A task that must be run before another task or service is stopped might use: start on stopping postgresql RESULT=ok SEE ALSO
starting(7) started(7) stopped(7) init(5) Upstart 2009-07-09 stopping(7)
All times are GMT -4. The time now is 07:31 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy