Sponsored Content
Top Forums Shell Programming and Scripting How can we overcome Broken pipe error during scp,SFTP,Rsync while transferring big files.? Post 302954977 by Upendra Bhushan on Monday 14th of September 2015 03:33:00 AM
Old 09-14-2015
How can we overcome Broken pipe error during scp,SFTP,Rsync while transferring big files.?

Hello All,

Hope all are doing well.

We use scp (some times sftp and rsync also) for transferring big files (around 2GB each ) from 1 Network to another Network.
The Issues which we face :-

During transfer some times( Once in 1 week (or twice)) , the speed of transfer gets down to 30 kb/s, which is usually 500kb/s during normal days.And most times when speed become low after few min. it failed with BROKEN PIPE ERROR.

Is there any way we can transfer files by avoiding above error
OR
any way in unix we can transfer the file(usually jars ) by breaking them into parts
OR is there any way ,if we can continue the file copy from the same point where it failed last time.

Many Thanks in Advance.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Error transferring large file using sftp protocol

Hello All, I am not able to transfer large file completely (more than 60 MB) through automated SFTP script. It transfers only the partial file (approximately 55 MB) and it stops. I use "expect" command to transfer file in a automatic mode through script. The file transfer rate i get is around... (0 Replies)
Discussion started by: prashant.patel
0 Replies

2. UNIX for Dummies Questions & Answers

Transferring hidden files with scp

Is there a way to scp hidden files only ( with regex ) without specifying the name of the hidden files ? Many thanks in advance ! :) (13 Replies)
Discussion started by: matrixmadhan
13 Replies

3. AIX

broken pipe error

Hi, I am working on AIX 5.3 . I have client-server program which is in ProC.while sending packet to server i am getting error as broken pipe and program exiting. please help?/? (1 Reply)
Discussion started by: ajaysahoo
1 Replies

4. Emergency UNIX and Linux Support

rsync transferring multiple files issue

Hi, I want to specify multiple remote directories but want to transfer them in a single command with one connection with remote server. This avoids entering passwords repeatedly and is also efficient. e.g. rsync -vrt --size-only --delete user@host:/home/user1/dir1... (9 Replies)
Discussion started by: sardare
9 Replies

5. Programming

Broken Pipe error

All, I am using the below code The C code : if ((fp2=fopen(szout_fname,"r"))==NULL) { sprintf(stream_ptr1,"cat %s | sort -t, -rn -k 11,11 | awk -F\",\" '{ \ if ( \$3 ==\"%s\" ) {print... (0 Replies)
Discussion started by: arunkumar_mca
0 Replies

6. Shell Programming and Scripting

Error copying files from Unix (Solaris10) to Windows Server 2003 using scp/sftp

Hi, I have generated a Public/Private Key Pair in Solaris Unix (source) server and deployed the Public key in Windows 2003(target) server .ssh directory of user profile. When i try to connect(ssh, scp, sftp) from Unix, i'm getting below error message. Sun_SSH_1.1, SSH protocols 1.5/2.0,... (0 Replies)
Discussion started by: ajaykumarb
0 Replies

7. UNIX for Advanced & Expert Users

sftp error:- Couldn't send packet: Broken pipe

Hi, I am trying to sftp a large file(appx. 2 GB size) from solaris 10 to a Linux server. While sftping(with no options), connection is terminated after some time with broken pipe error. sftp with option -R1 or -B1500 is successful. Linux server ssh Version: OpenSSH_3.9p1, OpenSSL 0.9.7a... (3 Replies)
Discussion started by: brij123
3 Replies

8. Shell Programming and Scripting

Broken Pipe error

Hello while doing sftp over server "A" , i am getting a broken pipe error i.e cat: write error: Broken pipe what does that mean? please let me know if you want any other info on this.. (3 Replies)
Discussion started by: urfrnddpk
3 Replies

9. UNIX for Dummies Questions & Answers

broken pipe error

I'm new to scripting, and this forum has been invaluable in helping me out. I'm hoping I can get some personal help now though. I have a korn script that takes a list of servers and either telnets or sshs into it (only some are set up for ssh). What I'm doing now is trying to telnet first, and... (10 Replies)
Discussion started by: aimeet
10 Replies

10. UNIX for Dummies Questions & Answers

SFTP error between UNIX to iSeries (Write Failed Broken Pipe)

Hi, I am iseries resource and having Issue in one of the sFTP failures between one of my job (Unix --> iSeries). The response to sending machine (Unix) is "Write Failed Broken Pipe". Appreciate any help Available on why this Issue happens, how can we replicate the same, what fixes can... (1 Reply)
Discussion started by: hamelchauhan
1 Replies
RSSH.CONF(5)							  Derek D. Martin						      RSSH.CONF(5)

NAME
/etc/rssh.conf - configuration file for rssh OVERVIEW
rssh.conf is the configuration file for rssh. It allows the system administrator to control the behavior of the shell. Configuration key- words are either used by themselves on a line, or followed by an equal sign ('=') and a configuration value. Comments start with a hash ('#') and can occur anywhere on the line. Configuration options are case insensitive. Spaces at the beginning or end of line, or between the equal sign and the configuration keywords or values are ignored. If the value of a configuration option contains spaces, it (or at least the space) must be enclosed in either single or double quotes. A default configuration file is provided with the source distribution of rssh. If the configuration file is missing or contains errors, ssh will lock out all users. If a config file is present, the default is to lock out users if no services have been explicitly allowed. New in v2.1 is the ability to configure options on a per-user basis, using the user keyword. More details are below. CONFIGURATION KEYWORDS
allowscp Tells the shell that scp is allowed. allowsftp Tells the shell that sftp is allowed. allowcvs Tells the shell that cvs is allowed. allowrdist Tells the shell that rdist is allowed. allowrsync Tells the shell that rsync is allowed. allowsvnserve Tells the shell that svnserve is allowed. umask Sets the umask value for file creations in the scp/sftp session. This is normally set at login time by the user's shell. In order not to use the system default, rssh must set the umask. logfacility Allows the system administrator to control what syslog facility rssh logs to. The facilities are the same as those used by sys- logd.conf(5), or the C macros for the facilities can be used instead. For example: logfacility=user logfacility=LOG_USER are equivalent, and tell rssh to use the user facility for logging to syslog. chrootpath Causes rssh (actually a helper program) to call the chroot() system call, changing the root of the file system to whatever directory is specified. Note that the value on the right hand side of the equal sign is the name of a directory, not a command. For example: chrootpath=/usr/chroot will change the root of the virtual file system to /usr/chroot, preventing the user from being able to access anything below /usr/chroot in the file system, and making /usr/chroot appear to be the root directory. Care must be taken to set up a proper chroot jail; see the file CHROOT in the rssh source distribution for hints about how to do this. See also the chroot(2) man page. If the user's home directory (as specified in /etc/passwd) is underneath the path specified by this keyword, then the user will be chdir'd into their home directory. If it is not, then they will be chdir'd to the root of the chroot jail. In other words, if the jail is /chroot, and your user's home directory is /chroot/home/user, then once rssh_chroot_helper changes the root of the system, it will cd into /home/user inside the jail. However, if your user's home directory is given as /home/user in /etc/passwd, then even if that directory exists in the jail, the chroot helper will not try to cd there. The user's normal home directory must live inside the jail for this to work. user The user keyword allows for the configuration of options on a per-user basis. THIS KEYWORD OVERRIDES ALL OTHER KEYWORDS FOR THE SPECIFIED USER. That is, if you use a user keyword for user foo, then foo will use only the settings in that user line, and not any of the settings set with the keywords above. The user keyword's argument consists of a group of fields separated by a colon (':'), as shown below. The fields are, in order: username The username of the user for whom the entry provides options umask The umask for this user, in octal, just as it would be specified to the shell access bits Six binary digits, which indicate whether the user is allowed to use rsync, rdist, cvs, sftp, scp and svnserve, in that order. One means the command is allowed, zero means it is not. path The directory to which this user should be chrooted (this is not a command, it is a directory name). See chroot_path above for complete details. For example, you might have something like this: user = luser:022:000010: This does the following: for the user with the username "luser", set the umask to 022, disallow sftp, and allow scp. Because there is no chroot path specified, the user will not be chrooted, regardless of default options set with the keywords above. If you wanted this user to be chrooted, you would need to specify the chroot path explicitly, even if it should be the same as that set using the chrootpath keyword. Remember that if there are spaces in the path, you need to quote it, something like this: user = "luser:022:000010:/usr/local/chroot dir" See the default rssh.conf file for more examples. SEE ALSO
rssh(1), sshd(8), ssh(1), scp(1), sftp(1), svnserve(8), syslogd.conf(5), chroot(2). man pages 7 Jul 2003 RSSH.CONF(5)
All times are GMT -4. The time now is 11:13 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy