10-20-2010
You can only run so many streams over the net until you saturate it. Sending compressed streams is good, and long ones. Writing any extra files is evil wasted disk I/O bandwidth, never mind the space.
I would use my xdemux tool with find to divide the file names found into N parallel streams, where n is 2-4 times the cpu core/thread count of the sending system. Then use rsh to leap the gap and cpio to pack and unpack. You can use ssh/ssh2, eating more cpu to crypt/decrypt, plus optionally get the compression in the same process, which might be slower and harder to control. The gzip might work better than compress, if you have lots of sending CPU and relatively low net speed, and vice versa. There is a gap between compress (16 bit LZ)and gzip -1 in both speed and compression. The bzip2 is almost always too slow, and rzip demands seekable files. It might work best to lay down empty directories first, so cpio does not create them with the wrong permissions.
(
find ... -type d
find ... -type l
find ... -type p
find ... -type f
)|xdemux 16 'cpio -oaH crc | gzip -9 |rsh other_host "gzcat | cpio -idmH crc" '
https://www.unix.com/shell-programmin...data-file.html
Now, if rsh won't bridge the root id, you might need to add a named pipe in each stream, where root has find to N instances that write the cpio data to the named pipe, not-root gzips and rsh's and gzcat's between named pipes, and root over there invokes N instances of cpio reading from that named pipe. It's a bit manual, but fairly secure. The named pipes can be owned by not-root accessible to himself only, and root should be able to use them because root is root, but I have no system to tinker with and check that. I guess root can group permit root in on a root-exclusive group.
My named pipes are a bit tricky: you want to be spawning N read openers first else all the writers data goes to the first reader! Not my usual medium! Funny command, too, infix pipe path: mknod pipe_path p
Last edited by DGPickett; 10-20-2010 at 04:53 PM..
10 More Discussions You Might Find Interesting
1. IP Networking
The problem I am facing now is that the QNX host could not ping the SCO host and vice versa. They are in the same domain, ie, 172.20.3.xx. As I am very new to Unix, I guess I must have missed out some important steps. Pls help... Thanx alot (2 Replies)
Discussion started by: gavon
2 Replies
2. UNIX for Advanced & Expert Users
Hello,
I am working on HP-UX , and in the /etc/hosts file we have setup an alias:
aa.bb.cc.dd devmach2.unix.org devmach2 devma2v
The alias devma2v does not work. Error when pinging devma2v
ping: unknown host devma2v
For devmach2 the ping works fine , returning the correct IP... (4 Replies)
Discussion started by: FunnyCats
4 Replies
3. Solaris
Hi,
I´m having a problem here. I have a directory that holds many subdirs and files (by many I mean MANY thousands).
What I want to do is make blocks of these files and transfer them block by block to a remote host; but once a given file is already placed in the remote host, the script must... (0 Replies)
Discussion started by: Dago
0 Replies
4. Solaris
Hi all,
i am copying .gz files from production server to development server using
"scp" command.my requirement is after copying .gz files i want to delete old
.gz files(two days back) in development server from production server.
like this way i need to delelte .log ,.z and .dmp files... (3 Replies)
Discussion started by: krishna176
3 Replies
5. Shell Programming and Scripting
Gurus/Experts
We have a centralized UNIX/Solaris server from where we can actually ssh to all other UNIX/Solaris servers...I need to write a script that reside on this centerlized server and do FileSystem monitoring (basically run df -h or -k) of other remote servers and then send an email to me... (6 Replies)
Discussion started by: anjum.suri
6 Replies
6. UNIX for Advanced & Expert Users
I do a ssh to remote host(A1) from local host(L1). I then ssh to another remote(A2) from A1.
When I do a who -m from A2, I see the "connected from" as "A1".
=> who -m
userid pts/2 2010-03-27 08:47 (A1)
I want to identify who is the local host who initiated the connection to... (3 Replies)
Discussion started by: gomes1333
3 Replies
7. IP Networking
Hi,
I have a weird problem.
when ever I do ping command like for example
ping unix.comI get the following message:
# ping unix.com
ping: unknown host unix.com
but when I use host the computer is able to know the host.
# host unix.com
unix.com has address 81.17.242.186
unix.com mail is... (2 Replies)
Discussion started by: programAngel
2 Replies
8. Shell Programming and Scripting
Hi ,
I have to pass a string through a variable from one host to another. Is that any any way to do it . Please help me (4 Replies)
Discussion started by: arukuku
4 Replies
9. Solaris
Hi All
I am having VxVm on two Solaris hosts. host1 is using disk group dgHR. right now this server went down due to hardware fault. Not I need to import this dgHR into host2 server. Please let me know the procedure for the same. (1 Reply)
Discussion started by: amity
1 Replies
10. Shell Programming and Scripting
Hi All,
Need to connect to target host and execute a command and connect back to source host to continue with next set of command execution.
Is there a utility/command using which we can connect to target host ?
Please suggest.
Note: Netezza database is instaled on Linux server. ... (4 Replies)
Discussion started by: Nagaraja Akkiva
4 Replies
LEARN ABOUT OPENDARWIN
tftp
tftp(1) User Commands tftp(1)
NAME
tftp - trivial file transfer program
SYNOPSIS
tftp [ host [port]]
DESCRIPTION
tftp is the user interface to the Internet TFTP (Trivial File Transfer Protocol), which allows users to transfer files to and from a remote
machine. The remote host and optional port may be specified on the command line, in which case tftp uses host as the default host, and if
specified, port as the default port, for future transfers. See the connect command below.
USAGE
Once tftp is running, it issues the prompt tftp> and recognizes the following commands:
Commands
connect host-name [ port ]
Set the host, and optionally port, for transfers. The TFTP protocol, unlike the FTP protocol, does not maintain connections between
transfers; thus, the connect command does not actually create a connection, but merely remembers what host is to be used for transfers.
You do not have to use the connect command; the remote host can be specified as part of the get or put commands.
mode transfer-mode
Set the mode for transfers; transfer-mode may be one of ascii or binary. The default is ascii.
put filename
put localfile remotefile
put filename1 filename2 ... filenameN remote-directory
Transfer a file, or a set of files, to the specified remote file or directory. The destination can be in one of two forms: a filename
on the remote host if the host has already been specified, or a string of the form:
host:filename
to specify both a host and filename at the same time. If the latter form is used, the specified host becomes the default for future
transfers. If the remote-directory form is used, the remote host is assumed to be running the UNIX system.
The host can be a host name (see hosts(4) or ipnodes(4)) or an IPv4 or IPv6 address string (see inet(7P) or inet6(7P)). Since IPv6
addresses already contain ":"s, the host should be enclosed in square brackets when an IPv6 address is used. Otherwise, the first
occurrence of a colon will be interpreted as the separator between the host and the filename. For example,
[1080::8:800:200c:417A]:myfile
Files may be written only if they already exist and are publicly writable. See in.tftpd(1M).
get filename
get remotename localname
get filename1 filename2 filename3 ... filenameN
Get a file or set of files (three or more) from the specified remote sources. source can be in one of two forms: a filename on the
remote host if the host has already been specified, or a string of the form:
host:filename
to specify both a host and filename at the same time. If the latter form is used, the last host specified becomes the default for
future transfers. See the put command regarding specifying a host.
quit
Exit tftp. An EOF also exits.
verbose
Toggle verbose mode.
trace
Toggle packet tracing.
status
Show current status.
rexmt retransmission-timeout
Set the per-packet retransmission timeout, in seconds.
timeout total-transmission-timeout
Set the total transmission timeout, in seconds.
ascii
Shorthand for mode ascii.
binary
Shorthand for mode binary.
blksize transfer-blocksize
The value of the transfer blocksize option to negotiate with the server. A value of 0 disables the negotiation of this option.
srexmt server-retransmission-timeout
The value of the retransmission timeout option to request that the server uses. A value of 0 disables the negotiation of this option.
tsize
A toggle that sends the transfer size option to the server. By default, the option is not sent. The transfer size option is not sent
with a write request when the transfer-mode is ascii.
? [ command-name ... ]
Print help information.
ATTRIBUTES
See attributes(5) for descriptions of the following attributes:
+-----------------------------+-----------------------------+
| ATTRIBUTE TYPE | ATTRIBUTE VALUE |
+-----------------------------+-----------------------------+
|Availability |SUNWtftp |
+-----------------------------+-----------------------------+
SEE ALSO
in.tftpd(1M), hosts(4), ipnodes(4),attributes(5),inet(7P), inet6(7P)
Malkin, G. and Harkin, A. RFC 2347, TFTP Option Extension. The Internet Society. May 1998
Malkin, G. and Harkin, A. RFC 2348, TFTP Blocksize Option. The Internet Society. May 1998
Malkin, G. and Harkin, A. RFC 2349, TFTP Timeout Interval and Transfer Size Options. The Internet Society. May 1998
Sollins, K.R. RFC 1350, The TFTP Protocol (Revision 2). Network Working Group. July 1992.
NOTES
The default transfer-mode is ascii. This differs from pre-SunOS 4.0 and pre-4.3BSD systems, so explicit action must be taken when transfer-
ring non-ASCII binary files such as executable commands.
Because there is no user-login or validation within the TFTP protocol, many remote sites restrict file access in various ways. Approved
methods for file access are specific to each site, and therefore cannot be documented here.
When using the get command to transfer multiple files from a remote host, three or more files must be specified. If two files are speci-
fied, the second file is used as a local file.
SunOS 5.10 2 Jan 2002 tftp(1)