Sponsored Content
Top Forums UNIX for Advanced & Expert Users How to zip/tar millions of files? Post 302478662 by citaylor on Wednesday 8th of December 2010 01:14:04 PM
Old 12-08-2010
When you say somewhere else, I guess you mean a different server ?
I imagine the "gzip" and the ssl encryption in scp are both adding quite an overhead.

There is a tradeoff between network speed, cpu speed and disk speed. You may find it more efficient leaving out the gzip if you have a fast network. Also if you are not worried about security using "rsh" instead of "ssh/scp" will be quicker. Probably the quickest would be:
Code:
tar cf - my/path | rsh -l user host "cd /new/target; tar xfp -"

However if network speed is slower and you need security something like "rsync" with the "-z" option may be better for you.
This User Gave Thanks to citaylor For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

unzip .zip file and list the files included in the .zip archive

Hello, I am trying to return the name of the resulting file from a .zip archive file using unix unzip command. unzip c07212007.cef7081.zip Archive: c07212007.cef7081.zip SecureZIP for z/OS by PKWARE inflating: CEP/CEM7080/PPVBILL/PASS/G0063V00 I used the following command to unzip in... (5 Replies)
Discussion started by: oracledev
5 Replies

2. UNIX Desktop Questions & Answers

Using Tar Zip

Hi, I want to backup my SQL database using tar zip but I'm paranoid that I will archive it. What I mean is I want the files to stay where they are but make a zipped copy of the files as well, I don't want to delete the originals. Is the command? tar -cvzf databasename.tar.gz... (1 Reply)
Discussion started by: chickenhouse
1 Replies

3. Shell Programming and Scripting

To write a shell script which groups files with certain pattern, create a tar and zip

Hi Guru's, I have to write a shell script which groups file names based upon the certain matching string pattern, then creates the Tar file for that particular group of files and then zips the Tar file created for the respective group of files. For example, In the given directory these files... (3 Replies)
Discussion started by: rahu_sg
3 Replies

4. Shell Programming and Scripting

Need script to remove millions of tmp files in /html/cache/ directory

Hello, I just saw that on my vps (centOS) my oscommerce with a seo script has created millions of tmp files inside the /html/cache/ directory. I would need to remove all those files (millions), I tried via shell but the vps loads goes to very high and it hangs, is there some way to do a... (7 Replies)
Discussion started by: andymc1
7 Replies

5. UNIX for Dummies Questions & Answers

TAR and ZIP files

Hi, I need a help with zip and tar. I have no done any scripts before with zip command. What I need to achieve is list files in a directory with a specific name (ID_DATE format- given examples) and then zip (or gunzip which I need to use, I am not sure) with timestamp on the file name and then... (15 Replies)
Discussion started by: Vijay81
15 Replies

6. Shell Programming and Scripting

help with tar & zip only last months(say,Sep) files

Need to 1. archive all the files in a directory from the previous month into a tar/gz file, ignoring all already archived 'tar.gz' files 2. Check created .tar.gz file isnt corrupted and has all the required files in it. and then remove the original files. I am using a function to get the... (1 Reply)
Discussion started by: Prev
1 Replies

7. Shell Programming and Scripting

Zip Multiple files to One .zip file in AIX system

Hi I have a requirement in unix shell where I need to zip multiple files on server to one single .zip file. I dont see zip command in AIX and gzip command not doing completely what I want. One I do .zip file, I should be able to unzip in my local Computer. Here is example what I want... (9 Replies)
Discussion started by: RAMA PULI
9 Replies

8. Shell Programming and Scripting

How to create zip/gz/tar files for if the files are older than particular days in UNIX or Linux?

I need a script file for backup (zip or tar or gz) of old log files in our unix server (causing the space problem). Could you please help me to create the zip or gz files for each log files in current directory and sub-directories also? I found one command which is to create gz file for the... (4 Replies)
Discussion started by: Mallikgm
4 Replies

9. UNIX for Beginners Questions & Answers

How can we Zip multiple files created on the same date into one single zip file.?

Hi all i am very new to shell scripting and need some help from you to learn 1)i have some log files that gets generated on daily basis example: i have abc_2017_01_30_1.log ,2017_01_30_2.log like wise so i want to zip this 4 logs which are created on same date into one zip folder. 2)Post zipping... (2 Replies)
Discussion started by: b.saipriyanka
2 Replies

10. Shell Programming and Scripting

How can we Zip multiple files created on the same date into one single zip file.?

Hi all i am very new to shell scripting and need some help from you to learn 1)i have some log files that gets generated on daily basis example: i have abc_2017_01_30_1.log ,2017_01_30_2.log like wise so i want to zip this 4 logs which are created on same date into one zip folder. 2)Post zipping... (1 Reply)
Discussion started by: b.saipriyanka
1 Replies
GPG-ZIP(1)							 GNU Privacy Guard							GPG-ZIP(1)

NAME
gpg-zip - Encrypt or sign files into an archive SYNOPSIS
gpg-zip [options] filename1 [ filename2, ... ] directory1 [ directory2, ... ] DESCRIPTION
gpg-zip encrypts or signs files into an archive. It is an gpg-ized tar using the same format as used by PGP's PGP Zip. OPTIONS
gpg-zip understands these options: --encrypt -e Encrypt data. This option may be combined with --symmetric (for output that may be decrypted via a secret key or a passphrase). --decrypt -d Decrypt data. --symmetric -c Encrypt with a symmetric cipher using a passphrase. The default symmetric cipher used is CAST5, but may be chosen with the --cipher-algo option to gpg. --sign -s Make a signature. See gpg. --recipient user -r user Encrypt for user id user. See gpg. --local-user user -u user Use user as the key to sign with. See gpg. --list-archive List the contents of the specified archive. --output file -o file Write output to specified file file. --gpg gpgcmd Use the specified command gpgcmd instead of gpg. --gpg-args args Pass the specified options to gpg. --tar tarcmd Use the specified command tarcmd instead of tar. --tar-args args Pass the specified options to tar. --version Print version of the program and exit. --help Display a brief help page and exit. EXAMPLES
Encrypt the contents of directory 'mydocs' for user Bob to file 'test1': gpg-zip --encrypt --output test1 --gpg-args -r Bob mydocs List the contents of archive 'test1': gpg-zip --list-archive test1 DIAGNOSTICS
The program returns 0 if everything was fine, 1 otherwise. SEE ALSO
gpg(1), tar(1), The full documentation for this tool is maintained as a Texinfo manual. If GnuPG and the info program are properly installed at your site, the command info gnupg should give you access to the complete manual including a menu structure and an index. GnuPG 2.0.15 2010-07-05 GPG-ZIP(1)
All times are GMT -4. The time now is 03:42 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy