Sponsored Content
Top Forums UNIX for Advanced & Expert Users How to zip/tar millions of files? Post 302478667 by methyl on Wednesday 8th of December 2010 01:27:01 PM
Old 12-08-2010
Please state what Operating System you have and describe your hardware configuration including memory and discs and enything relevant to performance.

Is it safe to assume that the filesystem will be quiescent?

Do you have spare discs equivalent to say twice the existing space? Personally I would copy the entire filesystem first to produce a defragmented filesystem which runs at a reasonable speed. This is also intended to prove that the original disc can be read from end-to-end.

Please post the current values of:
Code:
df -i
df -k

This User Gave Thanks to methyl For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

unzip .zip file and list the files included in the .zip archive

Hello, I am trying to return the name of the resulting file from a .zip archive file using unix unzip command. unzip c07212007.cef7081.zip Archive: c07212007.cef7081.zip SecureZIP for z/OS by PKWARE inflating: CEP/CEM7080/PPVBILL/PASS/G0063V00 I used the following command to unzip in... (5 Replies)
Discussion started by: oracledev
5 Replies

2. UNIX Desktop Questions & Answers

Using Tar Zip

Hi, I want to backup my SQL database using tar zip but I'm paranoid that I will archive it. What I mean is I want the files to stay where they are but make a zipped copy of the files as well, I don't want to delete the originals. Is the command? tar -cvzf databasename.tar.gz... (1 Reply)
Discussion started by: chickenhouse
1 Replies

3. Shell Programming and Scripting

To write a shell script which groups files with certain pattern, create a tar and zip

Hi Guru's, I have to write a shell script which groups file names based upon the certain matching string pattern, then creates the Tar file for that particular group of files and then zips the Tar file created for the respective group of files. For example, In the given directory these files... (3 Replies)
Discussion started by: rahu_sg
3 Replies

4. Shell Programming and Scripting

Need script to remove millions of tmp files in /html/cache/ directory

Hello, I just saw that on my vps (centOS) my oscommerce with a seo script has created millions of tmp files inside the /html/cache/ directory. I would need to remove all those files (millions), I tried via shell but the vps loads goes to very high and it hangs, is there some way to do a... (7 Replies)
Discussion started by: andymc1
7 Replies

5. UNIX for Dummies Questions & Answers

TAR and ZIP files

Hi, I need a help with zip and tar. I have no done any scripts before with zip command. What I need to achieve is list files in a directory with a specific name (ID_DATE format- given examples) and then zip (or gunzip which I need to use, I am not sure) with timestamp on the file name and then... (15 Replies)
Discussion started by: Vijay81
15 Replies

6. Shell Programming and Scripting

help with tar & zip only last months(say,Sep) files

Need to 1. archive all the files in a directory from the previous month into a tar/gz file, ignoring all already archived 'tar.gz' files 2. Check created .tar.gz file isnt corrupted and has all the required files in it. and then remove the original files. I am using a function to get the... (1 Reply)
Discussion started by: Prev
1 Replies

7. Shell Programming and Scripting

Zip Multiple files to One .zip file in AIX system

Hi I have a requirement in unix shell where I need to zip multiple files on server to one single .zip file. I dont see zip command in AIX and gzip command not doing completely what I want. One I do .zip file, I should be able to unzip in my local Computer. Here is example what I want... (9 Replies)
Discussion started by: RAMA PULI
9 Replies

8. Shell Programming and Scripting

How to create zip/gz/tar files for if the files are older than particular days in UNIX or Linux?

I need a script file for backup (zip or tar or gz) of old log files in our unix server (causing the space problem). Could you please help me to create the zip or gz files for each log files in current directory and sub-directories also? I found one command which is to create gz file for the... (4 Replies)
Discussion started by: Mallikgm
4 Replies

9. UNIX for Beginners Questions & Answers

How can we Zip multiple files created on the same date into one single zip file.?

Hi all i am very new to shell scripting and need some help from you to learn 1)i have some log files that gets generated on daily basis example: i have abc_2017_01_30_1.log ,2017_01_30_2.log like wise so i want to zip this 4 logs which are created on same date into one zip folder. 2)Post zipping... (2 Replies)
Discussion started by: b.saipriyanka
2 Replies

10. Shell Programming and Scripting

How can we Zip multiple files created on the same date into one single zip file.?

Hi all i am very new to shell scripting and need some help from you to learn 1)i have some log files that gets generated on daily basis example: i have abc_2017_01_30_1.log ,2017_01_30_2.log like wise so i want to zip this 4 logs which are created on same date into one zip folder. 2)Post zipping... (1 Reply)
Discussion started by: b.saipriyanka
1 Replies
Z(1)							      General Commands Manual							      Z(1)

NAME
comprez - safely (un)tar and (de)feather files and directories SYNOPSIS
comprez [ -t | -T ] [ -v | -V ] [ -l | -L ] [ -gz | -z | -I | -lz | -Z | -zip | -jar ] [ -# ] [ -s suffix ] [ -m mode ] [ -p | -P ] [ -h ] [ -- ] file ... DESCRIPTION
Comprez is a simple, safe and convenient front-end for the compress(1), uncompress(1), gzip(1), bzip2(1), lzip(1), tar(1), zip(1) and unzip(1) utilities for compressing and uncompressing files and directories. It processes each of its arguments according to the type of the file or directory given: If the argument is a plain file, then the file is compressed, ie, feathered. If the argument is a compressed file with a name ending in .Z, .gz, .z, .bz2, .lz, .zip or .jar, then the file is uncompressed, ie, defeathered. If the argument is a directory, then the directory is archived into one tar or zip file which is then compressed, ie, tarred and feathered. If the argument is a compressed tar or zip archive with a name ending in .{tar.,tar,ta,t}{Z,gz,z,bz2,lz} or .{zip,jar}, then the archive is uncompressed and untarred, ie, untarred and defeathered. The new compressed or uncompressed version will be in the same directory as the original. A compressed file is always uncompressed into a file with the same name sans the feather suffix. A compressed tar or zip archive is always unpacked into a subdirectory with the same name as the archive sans the tar and feather suffix, even if the archive did not itself contain such a subdirectory. OPTIONS
-- Interprets all following arguments as files instead of options. -# Where # is a digit from 1 through 9. This option is passed on to gzip(1), bzip2(1), lzip(1) and zip(1) when feathering with them. --fast may be used instead of -1 and --best instead of -9. -gz Uses gzip(1) and a .gz compression suffix when feathering. -h Prints a helpful usage message. -I Uses bzip2(1) and a .bz2 compression suffix when feathering. -l Lists the file or directory name created resulting from each argument. -L Does not report created files or directories. Default. -lz Uses lzip(1) and a .lz compression suffix when feathering. -m mode Apply the given chmod(1) mode argument to any created tar and feather files. Before this argument is applied, the files have the same read and write permissions as the directories from which they were created. For security, the default argument is go-rwx. If the mode is -, then no argument is applied. See chmod(1) for all other allowable formats of this argument. -p Preserves modes when untarring by giving the p flag to tar(1). Default. -P Doesn't preserve modes when untarring by not giving the p flag to tar(1). This option may be necessary on systems where ordinary users are allowed to run chown(2). -s suffix Creates tar and feather files using the given suffix style. The default suffix style is .tar.Z. Note that the suffix style does not dictate the program used for feathering nor the compression suffix. -t Only lists the table of contents of the given files. Does not make any changes. -T (Un)tars and (de)feathers the given files according to their type. Default. -v Verbose output. For example, reports compression ratios when feathering. -V Non-verbose output. Does not report compression ratios. Default. -q is a synonym. -z Uses gzip(1) and a .z compression suffix when feathering. -Z Uses compress(1) and a .Z compression suffix when feathering. Default. -zip, -jar Uses zip(1) and a .zip or .jar compression suffix when tarring and/or feathering. The zip format combines tarring and feather- ing. WARNING: zip does not preserve complete Unix filesystem information for the files it archives, such as links, some permis- sions, etc. A tar(1)-based format should be used if this is required. ENVIRONMENT VARIABLES
ZOPTS This variable may be set to a string of the above options to supersede the default settings. They may still be overridden by options given on the command line. GZIP This environment variable for gzip(1) is not passed on by comprez so that gzip's behavior is standard and predictable. ZIPOPT, UNZIP These environment variables for zip(1) and unzip(1) are not passed on by comprez so that their behavior is standard and pre- dictable. BUGS
There should be an option to allow the use of another directory for temporary files needed during the (un)tarring and (de)feathering pro- cesses. This would make comprez more useful when the quota or disk is nearly full. There should be -k and -K options for keeping the original input files or not. CAVEATS
Comprez is written to work with gzip versions 1.2.3 and 1.2.4, bzip2 versions 0.9.0 and 1.0.0, Unix zip version 2.0.1 and Unix unzip ver- sion 5.12. Other versions will probably work safely, too, but should still be checked for compatibility. Every effort has been made to assure that the use of this program will not lead to the inappropriate deletion or corruption of any files. However, there are never any guarantees, so please use at your own risk. VERSION
2.6.1 AUTHOR
Steve Kinzler, kinzler@cs.indiana.edu, May 89/Jun 93/Aug 99/Dec 00 URL
http://www.cs.indiana.edu/~kinzler/z SEE ALSO
compress(1), uncompress(1), gzip(1), bzip2(1), lzip(1), tar(1), zip(1), unzip(1) 2.6.1 Z(1)
All times are GMT -4. The time now is 11:55 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy