Sponsored Content
Top Forums UNIX for Beginners Questions & Answers scp of multiple files to remote server Post 302991934 by jim mcnamara on Saturday 18th of February 2017 09:30:36 PM
Old 02-18-2017
Sure. Two logins okay?
Code:
# create a single tar file , scp the one file to the remote
cd /path/my/directories  # local place where directories are
tar cvf myfiles.tar  x_1 x_2 x_3 x_4 x_5 x_6 x_7 x_8 x_9 x_10 x_11 x_12 x_13 x_14 x_15 x_16 x_17 x_18 x_19 x_20 x_21 x_22 x_23 x_24 x_25 x_26 x_27 x_28 x_29 x_30
# scp tar file onto the remote box
scp myfiles.tar remote_server@remote.edu:/me/my_data/ 
#  extract all of the directories to the place they belong
ssh remote_server@remote.edu 'cd remote_server@remote.edu && tar xf myfiles.tar

If the tar file is gigantic you may want to compress it first, send it, uncompress on the other side, extract the new re-sized tar file.

There is actually another way to move the files directly, more complicated but for long transmissions runs into issues with crummy connections. This is my plan A when I know enough to feel okay about moving lots of data. I gave you plan B because it is less likely to bomb from a dropped connection.

The reason for this is that some ISP's in my area 're-establish' DHCP periodically, dropping the old connection.

Example:
Code:
cd /path/to/local/directories
tar cvf - . | ssh -e none  me@remote 'cd /path/to/remote  && ( tar xf - )'


Last edited by jim mcnamara; 02-18-2017 at 10:45 PM..
This User Gave Thanks to jim mcnamara For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

FTP multiple files from remote server to local server

Hi, I am facing a weired problem in my FTP script. I want to transfer multiple files from remote server to local server everyday, using mget * in my script. I also, want to send an email for successful or failed FTP. My script works for file transfer, but it don't send any mail. There is... (2 Replies)
Discussion started by: berlin_germany
2 Replies

2. Shell Programming and Scripting

SCP multiple files

Hi , I am doing SCP for copying log files from different server(around 24 server) I need to copy these bulk logfiles every 15 min. How can i do multiple SCP at the same time? My current code is like this scp $CUSTCARE_USER@$CUSTCARE_SERVER:$CUSTCARE_HOME/$CUSTCARE_LOG.*... (2 Replies)
Discussion started by: scorpio
2 Replies

3. UNIX for Dummies Questions & Answers

scp multiple files

Hi, I'm new to Linux. I would like to know how to scp a group of files. I have the below command, but it asks for remote password while sending each file. Is there a way to send all files (identified by - $ ls | grep '.*hrs0314a.*' | xargs -I {} grep -l '.*35663.*' {}) in one go? $ ls |... (6 Replies)
Discussion started by: don_tom
6 Replies

4. Shell Programming and Scripting

ftp'ing multiple files to the remote server

unix shell script (2 Replies)
Discussion started by: giridhar276
2 Replies

5. Shell Programming and Scripting

scp or rsync multiple files in parallel from a remote host

Hi. I'm trying to speed up an rsync command by running it in parallel. There's no real option for this other than if the files are in multiple directories (which they're not). And even then there's no way of knowing if rsync has succeeded as the process is running in the background .. and... (4 Replies)
Discussion started by: Big_Jeffrey
4 Replies

6. Shell Programming and Scripting

scp files that are 3 days older from remote server-

hello, i am trying to get a list of files to be scped from the remote server by running the below in my local unix server ( note - there is a passwordless connectivity setup made between the local and remote server) and, we use KSH. --- ssh $scp_host "find /a/b/c/*/ -iname "$remote_file"" >... (4 Replies)
Discussion started by: billpeter3010
4 Replies

7. Solaris

scp multiple files without pattern on Solaris

I need to transfer multiple files using scp between two solaris machines. I could not use pattern since they match with other files in the same directory. I tried the below command, it does not copy any files. $ scp -p user@machine1:/home/fid1/staging/\{a.ksh,b.ksh,c.ksh,d.ksh\} . But... (7 Replies)
Discussion started by: marecar
7 Replies

8. Shell Programming and Scripting

Script connect to remote server, not find files and exit only from remote server, but not from scrip

I have a script, which connecting to remote server and first checks, if the files are there by timestamp. If not I want the script exit without error. Below is a code TARFILE=${NAME}.tar TARGZFILE=${NAME}.tar.gz ssh ${DESTSERVNAME} 'cd /export/home/iciprod/download/let/monthly;... (3 Replies)
Discussion started by: digioleg54
3 Replies

9. UNIX for Beginners Questions & Answers

Automate splitting of files , scp files as each split completes and combine files on target server

i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,... (2 Replies)
Discussion started by: malaika
2 Replies

10. Shell Programming and Scripting

Rename (move) multiple files on remote server using sftp

I want to rename (move) multiple files on remote server. I tried the following command to move all TXT files from my_dir directory to /new_dir. But it does not work. Any help? #!/bin/ksh sftp -dev3 << ABC cd my_dir $(for i in TXT; do echo "ls *.$i" ; rename $x /new_dir/$x;... (1 Reply)
Discussion started by: Soham
1 Replies
GIT-ARCHIVE(1)							    Git Manual							    GIT-ARCHIVE(1)

NAME
git-archive - Create an archive of files from a named tree SYNOPSIS
git archive [--format=<fmt>] [--list] [--prefix=<prefix>/] [<extra>] [-o | --output=<file>] [--worktree-attributes] [--remote=<repo> [--exec=<git-upload-archive>]] <tree-ish> [<path>...] DESCRIPTION
Creates an archive of the specified format containing the tree structure for the named tree, and writes it out to the standard output. If <prefix> is specified it is prepended to the filenames in the archive. git archive behaves differently when given a tree ID versus when given a commit ID or tag ID. In the first case the current time is used as the modification time of each file in the archive. In the latter case the commit time as recorded in the referenced commit object is used instead. Additionally the commit ID is stored in a global extended pax header if the tar format is used; it can be extracted using git get-tar-commit-id. In ZIP files it is stored as a file comment. OPTIONS
--format=<fmt> Format of the resulting archive: tar or zip. If this option is not given, and the output file is specified, the format is inferred from the filename if possible (e.g. writing to "foo.zip" makes the output to be in the zip format). Otherwise the output format is tar. -l, --list Show all available formats. -v, --verbose Report progress to stderr. --prefix=<prefix>/ Prepend <prefix>/ to each filename in the archive. -o <file>, --output=<file> Write the archive to <file> instead of stdout. --worktree-attributes Look for attributes in .gitattributes in working directory too. <extra> This can be any options that the archiver backend understands. See next section. --remote=<repo> Instead of making a tar archive from the local repository, retrieve a tar archive from a remote repository. --exec=<git-upload-archive> Used with --remote to specify the path to the git-upload-archive on the remote side. <tree-ish> The tree or commit to produce an archive for. <path> Without an optional path parameter, all files and subdirectories of the current working directory are included in the archive. If one or more paths are specified, only these are included. BACKEND EXTRA OPTIONS
zip -0 Store the files instead of deflating them. -9 Highest and slowest compression level. You can specify any number from 1 to 9 to adjust compression speed and ratio. CONFIGURATION
tar.umask This variable can be used to restrict the permission bits of tar archive entries. The default is 0002, which turns off the world write bit. The special value "user" indicates that the archiving user's umask will be used instead. See umask(2) for details. If --remote is used then only the configuration of the remote repository takes effect. tar.<format>.command This variable specifies a shell command through which the tar output generated by git archive should be piped. The command is executed using the shell with the generated tar file on its standard input, and should produce the final output on its standard output. Any compression-level options will be passed to the command (e.g., "-9"). An output file with the same extension as <format> will be use this format if no other format is given. The "tar.gz" and "tgz" formats are defined automatically and default to gzip -cn. You may override them with custom commands. tar.<format>.remote If true, enable <format> for use by remote clients via git-upload-archive(1). Defaults to false for user-defined formats, but true for the "tar.gz" and "tgz" formats. ATTRIBUTES
export-ignore Files and directories with the attribute export-ignore won't be added to archive files. See gitattributes(5) for details. export-subst If the attribute export-subst is set for a file then git will expand several placeholders when adding this file to an archive. See gitattributes(5) for details. Note that attributes are by default taken from the .gitattributes files in the tree that is being archived. If you want to tweak the way the output is generated after the fact (e.g. you committed without adding an appropriate export-ignore in its .gitattributes), adjust the checked out .gitattributes file as necessary and use --worktree-attributes option. Alternatively you can keep necessary attributes that should apply while archiving any tree in your $GIT_DIR/info/attributes file. EXAMPLES
git archive --format=tar --prefix=junk/ HEAD | (cd /var/tmp/ && tar xf -) Create a tar archive that contains the contents of the latest commit on the current branch, and extract it in the /var/tmp/junk directory. git archive --format=tar --prefix=git-1.4.0/ v1.4.0 | gzip >git-1.4.0.tar.gz Create a compressed tarball for v1.4.0 release. git archive --format=tar.gz --prefix=git-1.4.0/ v1.4.0 >git-1.4.0.tar.gz Same as above, but using the builtin tar.gz handling. git archive --prefix=git-1.4.0/ -o git-1.4.0.tar.gz v1.4.0 Same as above, but the format is inferred from the output file. git archive --format=tar --prefix=git-1.4.0/ v1.4.0^{tree} | gzip >git-1.4.0.tar.gz Create a compressed tarball for v1.4.0 release, but without a global extended pax header. git archive --format=zip --prefix=git-docs/ HEAD:Documentation/ > git-1.4.0-docs.zip Put everything in the current head's Documentation/ directory into git-1.4.0-docs.zip, with the prefix git-docs/. git archive -o latest.zip HEAD Create a Zip archive that contains the contents of the latest commit on the current branch. Note that the output format is inferred by the extension of the output file. git config tar.tar.xz.command "xz -c" Configure a "tar.xz" format for making LZMA-compressed tarfiles. You can use it specifying --format=tar.xz, or by creating an output file like -o foo.tar.xz. SEE ALSO
gitattributes(5) GIT
Part of the git(1) suite Git 1.7.10.4 11/24/2012 GIT-ARCHIVE(1)
All times are GMT -4. The time now is 09:30 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy