Sponsored Content
Top Forums Shell Programming and Scripting How can we overcome Broken pipe error during scp,SFTP,Rsync while transferring big files.? Post 302954977 by Upendra Bhushan on Monday 14th of September 2015 03:33:00 AM
Old 09-14-2015
How can we overcome Broken pipe error during scp,SFTP,Rsync while transferring big files.?

Hello All,

Hope all are doing well.

We use scp (some times sftp and rsync also) for transferring big files (around 2GB each ) from 1 Network to another Network.
The Issues which we face :-

During transfer some times( Once in 1 week (or twice)) , the speed of transfer gets down to 30 kb/s, which is usually 500kb/s during normal days.And most times when speed become low after few min. it failed with BROKEN PIPE ERROR.

Is there any way we can transfer files by avoiding above error
OR
any way in unix we can transfer the file(usually jars ) by breaking them into parts
OR is there any way ,if we can continue the file copy from the same point where it failed last time.

Many Thanks in Advance.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Error transferring large file using sftp protocol

Hello All, I am not able to transfer large file completely (more than 60 MB) through automated SFTP script. It transfers only the partial file (approximately 55 MB) and it stops. I use "expect" command to transfer file in a automatic mode through script. The file transfer rate i get is around... (0 Replies)
Discussion started by: prashant.patel
0 Replies

2. UNIX for Dummies Questions & Answers

Transferring hidden files with scp

Is there a way to scp hidden files only ( with regex ) without specifying the name of the hidden files ? Many thanks in advance ! :) (13 Replies)
Discussion started by: matrixmadhan
13 Replies

3. AIX

broken pipe error

Hi, I am working on AIX 5.3 . I have client-server program which is in ProC.while sending packet to server i am getting error as broken pipe and program exiting. please help?/? (1 Reply)
Discussion started by: ajaysahoo
1 Replies

4. Emergency UNIX and Linux Support

rsync transferring multiple files issue

Hi, I want to specify multiple remote directories but want to transfer them in a single command with one connection with remote server. This avoids entering passwords repeatedly and is also efficient. e.g. rsync -vrt --size-only --delete user@host:/home/user1/dir1... (9 Replies)
Discussion started by: sardare
9 Replies

5. Programming

Broken Pipe error

All, I am using the below code The C code : if ((fp2=fopen(szout_fname,"r"))==NULL) { sprintf(stream_ptr1,"cat %s | sort -t, -rn -k 11,11 | awk -F\",\" '{ \ if ( \$3 ==\"%s\" ) {print... (0 Replies)
Discussion started by: arunkumar_mca
0 Replies

6. Shell Programming and Scripting

Error copying files from Unix (Solaris10) to Windows Server 2003 using scp/sftp

Hi, I have generated a Public/Private Key Pair in Solaris Unix (source) server and deployed the Public key in Windows 2003(target) server .ssh directory of user profile. When i try to connect(ssh, scp, sftp) from Unix, i'm getting below error message. Sun_SSH_1.1, SSH protocols 1.5/2.0,... (0 Replies)
Discussion started by: ajaykumarb
0 Replies

7. UNIX for Advanced & Expert Users

sftp error:- Couldn't send packet: Broken pipe

Hi, I am trying to sftp a large file(appx. 2 GB size) from solaris 10 to a Linux server. While sftping(with no options), connection is terminated after some time with broken pipe error. sftp with option -R1 or -B1500 is successful. Linux server ssh Version: OpenSSH_3.9p1, OpenSSL 0.9.7a... (3 Replies)
Discussion started by: brij123
3 Replies

8. Shell Programming and Scripting

Broken Pipe error

Hello while doing sftp over server "A" , i am getting a broken pipe error i.e cat: write error: Broken pipe what does that mean? please let me know if you want any other info on this.. (3 Replies)
Discussion started by: urfrnddpk
3 Replies

9. UNIX for Dummies Questions & Answers

broken pipe error

I'm new to scripting, and this forum has been invaluable in helping me out. I'm hoping I can get some personal help now though. I have a korn script that takes a list of servers and either telnets or sshs into it (only some are set up for ssh). What I'm doing now is trying to telnet first, and... (10 Replies)
Discussion started by: aimeet
10 Replies

10. UNIX for Dummies Questions & Answers

SFTP error between UNIX to iSeries (Write Failed Broken Pipe)

Hi, I am iseries resource and having Issue in one of the sFTP failures between one of my job (Unix --> iSeries). The response to sending machine (Unix) is "Write Failed Broken Pipe". Appreciate any help Available on why this Issue happens, how can we replicate the same, what fixes can... (1 Reply)
Discussion started by: hamelchauhan
1 Replies
Net::SFTP::Foreign::Compat(3pm) 			User Contributed Perl Documentation			   Net::SFTP::Foreign::Compat(3pm)

NAME
Net::SFTP::Foreign::Compat - Adaptor for Net::SFTP compatibility SYNOPSIS
use Net::SFTP::Foreign::Compat; my $sftp = Net::SFTP::Foreign::Compat->new($host); $sftp->get("foo", "bar"); $sftp->put("bar", "baz"); use Net::SFTP::Foreign::Compat ':supplant'; my $sftp = Net::SFTP->new($host); DESCRIPTION
This package is a wrapper around Net::SFTP::Foreign that provides an API (mostly) compatible with that of Net::SFTP. Methods on this package are identical to those in Net::SFTP except that Net::SFTP::Foreign::Attributes::Compat objects have to be used instead of Net::SFTP::Attributes. If the ":supplant" tag is used, this module installs also wrappers on the "Net::SFTP" and Net::SFTP::Attributes packages so no other parts of the program have to modified in order to move from Net::SFTP to Net::SFTP::Foreign. Setting defaults The hash %Net::SFTP::Foreign::DEFAULTS can be used to set default values for Net::SFTP::Foreign methods called under the hood and otherwise not accesible through the Net::SFTP API. The entries currently supported are: new => @opts extra options passed to Net::SFTP::Foreign constructor. get => @opts extra options passed to Net::SFTP::Foreign::get method. put => @opts extra options passed to Net::SFTP::Foreign::put method. ls => @opts extra options passed to Net::SFTP::Foreign::ls method. COPYRIGHT
Copyright (c) 2006-2008, 2011 Salvador Fandin~o All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.14.2 2012-05-04 Net::SFTP::Foreign::Compat(3pm)
All times are GMT -4. The time now is 04:24 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy