02-21-2011
Help with zipping files
Hi,
I have come across a requirement in which I need to zip files. This is fine but the requriement has one conditions like below:
One .z file can not have more than 10,000 files
Now in the directory I have several files liek below:
aaa_file_10_00001.txt
aaa_file_10_00002.txt
aaa_file_20_00001.txt
aaa_file_30_00001.txt
aaa_file_10_00003.txt
...........................
...........................
...........................
aaa_file_30_00750.txt
aaa_file_30_10005.txt
If i have to zip the above files, I will have 4 .z files. One for files like aaa_file_20_*.txt, one for files like aaa_file_30_*.txt and two for files like aaa_file_10_*.txt I am planning to move files of different type (diferentiated by 3rd field in the file name) to a temporary directory and zip them. But the problem is I can not zip more than 10,000 files together. How do I move files of type aaa_file_10_*.txt from 1-10000 (this is the value in 4th field of the file name )first, zip them and then move the reamining 10001-10005.
Using a loop is not a good option as per my understanding. Is there any other way of doing it which will not have impact on performance ?
Thanks
Angshuman
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Dear Experts,
I need a script which will zipped the files older than 2 days.
but i dont want to use find . * -mtime 2.
Is there is any other method to achive this task.
i will ececute the script daily.
Regards, (3 Replies)
Discussion started by: shary
3 Replies
2. Solaris
I was trying to remove & archive all the files in the current directory and embed them into the zip file, i mean i don't want to list the files any more after zipping those files. For this i wrote the following code though operation was succeeded zipping all the files and creating a zip file i am... (2 Replies)
Discussion started by: Ariean
2 Replies
3. UNIX for Dummies Questions & Answers
how would i zip a file?
what does zip mean? (4 Replies)
Discussion started by: trob
4 Replies
4. UNIX for Dummies Questions & Answers
Dear all,
I have thousands of log files in my log directory which I need to zip them and archive. I tried using zip command. But it is not allowing me to archive it more that 4GB of file size. So how to archive them. If it is not possible how to zip all files in to multiple archive files which... (3 Replies)
Discussion started by: tvbhkishore
3 Replies
5. Shell Programming and Scripting
Hi,
Is there any difference if files are individually zipped and archived in a directory or if files are moved into archiving directory and zipping that directory. (3 Replies)
Discussion started by: swathich
3 Replies
6. Shell Programming and Scripting
Hi,
i am using the below script to zip the files with respect to their timestamp of the files.It is working fine.
For example lets take I have two files in this directory /path/folder1 with the timestamp as this month and previous month. When i run this script first time it is creating two... (7 Replies)
Discussion started by: velava
7 Replies
7. UNIX for Dummies Questions & Answers
Hi All,
I have a scenario where in am using uuencode to send a txt file as an excel to end users( email attachment).I have 7 different files and these files are sent as emails 7 times...
So my question is, can i not zip all the 7 files at once and attach those files in a single... (9 Replies)
Discussion started by: saggiboy10
9 Replies
8. Shell Programming and Scripting
Hi Guys,
The script below works but it creates a zip folder under
123_arch.zip
-- test1
-- orig1.txt
-- orig2.txt
-- orig3.txt
-- orig4.txt
I don't want the sub directory test1 but everything under the base *arch name
I can not create a long name with... (4 Replies)
Discussion started by: GaryP1973
4 Replies
9. Shell Programming and Scripting
I have files
a_cd_1.csv
a_cd_2.csv
a_cd_3.csv
I need to zip these files into one zip file a_cd.zip but if no data is there in any of these csv then only csv with data should be zipped.
Also these files will always have header record if data is present or not.
Please let me know how this... (9 Replies)
Discussion started by: weknowd
9 Replies
10. UNIX for Beginners Questions & Answers
Hi,
I am trying to removing the old files which were older than 10 days and same g zipping the files using the shell script.
script was return as follows.
find /jboss7_homes/JBOSS7/SKYLIV??/SKYLIV??_CRM/jboss-eap-7.0/standalone/log -mtime +10 -type f | xargs rm -f
find /cer_skyliv??/log... (6 Replies)
Discussion started by: venkat918
6 Replies
LEARN ABOUT DEBIAN
parallel-slurp
PARALLEL-SLURP(1) PARALLEL-SLURP(1)
NAME
parallel-slurp - copy files from listed hosts
SYNOPSIS
parallel-slurp [OPTIONS] -h hosts.txt -L destdir remote local
DESCRIPTION
pssh provides a number of commands for executing against a group of computers, using SSH. It's most useful for operating on clusters of
homogenously-configured hosts.
parallel-slurp gathers specified files from hosts you listed.
OPTIONS
-r --recursive
recusively copy directories (OPTIONAL)
-L --localdir
output directory for remote file copies
-h --hosts
hosts file (each line "host[:port] [user]")
-l --user
username (OPTIONAL)
-p --par
max number of parallel threads (OPTIONAL)
-o --outdir
output directory for stdout files (OPTIONAL)
-e --errdir
output directory for stderr files (OPTIONAL)
-t --timeout
timeout (secs) (-1 = no timeout) per host (OPTIONAL)
-O --options
SSH options (OPTIONAL)
-v --verbose
turn on warning and diagnostic messages (OPTIONAL)
EXAMPLE
An example to copy /home/irb2/foo.txt from each host. Files gathered will be stored in /tmp/outdir/hostname/foo.txt.
# prallel-slurp -h hosts.txt -L /tmp/outdir -l irb2
/home/irb2/foo.txt foo.txt
ENVIRONMENT
All four programs take similar sets of options. All of these options can be set using the following environment variables:
o PSSH_HOSTS
o PSSH_USER
o PSSH_PAR
o PSSH_OUTDIR
o PSSH_VERBOSE
o PSSH_OPTIONS
SEE ALSO
parallel-ssh(1), parallel-scp(1), parallel-nuke(1), parallel-rsync(1), ssh(1)
AUTHOR
Brent N. Chun <bnc@theether.org>
COPYING
Copyright: 2003, 2004, 2005, 2006, 2007 Brent N. Chun
NOTES
1. bnc@theether.org
mailto:bnc@theether.org
03/30/2009 PARALLEL-SLURP(1)