Sponsored Content
Top Forums Shell Programming and Scripting Copying large files in a bash script stops execution Post 302886597 by drl on Sunday 2nd of February 2014 09:17:48 AM
Old 02-02-2014
Hi.

My guess -- there is some computing done at both ends of the rsync transaction. From: rsync - Wikipedia, the free encyclopedia:
Quote:
The recipient splits its copy of the file into fixed-size non-overlapping chunks and computes two checksums for each chunk ... It sends these checksums to the sender.

The sender then compares its rolling checksums with the set sent by the recipient to determine if any matches exist. If they do, it verifies the match by computing the hash for the matching block and by comparing it with the hash for that block sent by the recipient.

The sender then sends the recipient those parts of its file that did not match the recipient's blocks, along with information on where to merge these blocks into the recipient's version. This makes the copies identical.
So the longer the file, the more time it takes to do the checksums.

Best wishes ... cheers, drl
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Copying of large files fail

Hi, I have a process which duplicates files for different environments. As the files arrive, my script (korn shell) makes copies of them (giving a unique name) and then renames the original file so that my process won't get triggered again. I don't like it either, but it's what we were told to... (4 Replies)
Discussion started by: GoldenEye4ever
4 Replies

2. Shell Programming and Scripting

Bash folder manipulation - selecting/copying specified files

Bash/scripting newbie here - I feel this might be a trivial problem, but I'm not sure how to tackle it. I've got a folder of a year's worth of files, with some random number of files generated every day of the year (but at least one per day). I'm writing a script to automatically grab the file with... (6 Replies)
Discussion started by: WildGooseChased
6 Replies

3. Shell Programming and Scripting

Execution Problems with bash script

Hello, can someone please help me to fix this script, I have a 2 files, one file has hostname information and second file has console information of the hosts in each line, I have written a script which actually reads each line in hostname file and should grep in the console file and paste the... (8 Replies)
Discussion started by: bobby320
8 Replies

4. SCO

Need advice: Copying large CSV report files off SCO system

I have a SCO Unix server from 1999 running SCO 5.0.5 and some ancient accounting software called Real World A report writer program on the system is used to generate CSV files from accounting that we write with DOSCOPY commands to 3.5" floppies In the next 60 days we will be decommissioning... (11 Replies)
Discussion started by: magnetman
11 Replies

5. Shell Programming and Scripting

Execution problems with BASH Shell Script

Hi I need help with my coding , first time I'm working with bash . What i must do is check if there is 3 .txt files if there is not 3 of them i must give an error code , if al three is there i must first arrange them in alphabetical order and then take the last word in al 3 of the .txt files... (1 Reply)
Discussion started by: linux newb
1 Replies

6. Shell Programming and Scripting

Script stops working after copying it

Hi all, When I cp a script, this section of the code stops reading from a file. if ; then while read dirtoclean do DIRSTOCLEAN=${DIRSTOCLEAN}' '$dirtoclean done < ${BASEDIR}test.conf fi ${DIRSTOCLEAN} doesn't return anything, even though it will work right before... (6 Replies)
Discussion started by: nicksantos1
6 Replies

7. Shell Programming and Scripting

Mount fails (because remote machine is off) bash script stops

Hi, I'm trying to automate a couple of mounts. (I would do this in fstab, but the auto feature causes the virtual machine I'm running in to freeze when it boots up) If the machine I'm trying to connect to is OFF, I get the host unreachable error and then the bash script stops. The problem is... (3 Replies)
Discussion started by: jdilts
3 Replies

8. Shell Programming and Scripting

Need help in finding and copying list of files using bash shell script

Dear All, I have a situation where I want to copy some files of type .txt. These files are o/p from one program. Some of the files are named as fileName .txt instead of fileName.txt after fileName by mistake I have specified "space". Now I want to move these files as follows. mv fileName*... (13 Replies)
Discussion started by: linuxUser_
13 Replies

9. Shell Programming and Scripting

Bash: copying lines with specific character to files with same name as copied line.

I am trying to make my script as simple as a possible but, I am not sure if the way I am approaching is necessarily the most efficient or effective it can be. What I am mainly trying to fix is a for loop to remove a string from the specified files and within this loop I am trying to copy the lines... (2 Replies)
Discussion started by: Allie_gastrator
2 Replies

10. Shell Programming and Scripting

Bash script search, improve performance with large files

Hello, For several of our scripts we are using awk to search patterns in files with data from other files. This works almost perfectly except that it takes ages to run on larger files. I am wondering if there is a way to speed up this process or have something else that is quicker with the... (15 Replies)
Discussion started by: SDohmen
15 Replies
talk(1) 						      General Commands Manual							   talk(1)

NAME
talk - talk to another user SYNOPSIS
talk_party [ttyname] DESCRIPTION
The utility is a two-way, screen-oriented communication program. The command argument talk_party can take one the following forms: user where user is a login name and host is a host name. The optional command argument, ttyname, can be used to specify the terminal to be used when contacting a user who is logged in more than once. In absence of this argument, will try to contact the user on the user's most recently used terminal. When first invoked, sends the following message to the party it tries to connect to (callee): ... At this point, the recipient of the message can reply by typing: Once communication is established, the two parties can type simultaneously, with their output displayed in separate regions of the screen. Characters are processed as follows: o Typing characters from LC_CTYPE classifications print or space will cause those characters to be sent to the recipient's terminal. o Typing <control>-L will cause the sender's screen to be refreshed. o Typing the erase, kill or kill word character will delete the last character, line or word on the sender's terminal, with the action propagated to the recipient's terminal. o Typing the interrupt character will terminate the local talk utility. Once the talk session has been terminated on one side, the other side of the talk session will be notified that the session has been terminated and will be able to do nothing except exit. o Other non-printable characters typed on the sender's terminal are converted to printable characters before they are sent to the recipi- ent's terminal. Permission to be a recipient of a message can be denied or granted by using the utility. However, a user may need other privileges to be able to access other users' terminals. The utility will fail when the user lacks the appropriate privileges. SEE ALSO
mesg(1), who(1), write(1). STANDARDS CONFORMANCE
talk(1)
All times are GMT -4. The time now is 12:59 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy