09-28-2012
Hi Pamu,
Thanks for your advice. I have more than 10 files each of 20GB.
I wanted to confirm before trying it. But, anyways I figured out that you first zcat multiple files into another file and gzipping it rather than directly printing into a gz file.
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi all
I have more than 1000 files in a folder and when ever i use a "compress" or "zcat" command it give error
/bin/zcat: Arg list too long. .
any solution for this :o (3 Replies)
Discussion started by: muneebr
3 Replies
2. Linux
Hi
I am trying to extract a <filename>.tar.Z on a SLES machine using zcat.
The command I am using is
zcat <filename>.tar.Z | tar xf -
When I use the above I get the following message
tar: Read 7168 bytes from -
I think the message is benign because I see that my files where... (2 Replies)
Discussion started by: felixmat1
2 Replies
3. Shell Programming and Scripting
Hi I am going to migrate our datawarehouse system from HP Tru 64 Unix to the Red Hat Linux.
Inside the box, it is running around 40 cron jobs; inside each cron job, it is calling other shell script files, and the shell script files may again call other shell script files or ctl files(for... (1 Reply)
Discussion started by: franksubramania
1 Replies
4. Shell Programming and Scripting
I created a backup script that emails all the admins when the backup is complete and attaches a log file of what what backed up. On occasion, something happens in which the backups stop working, I started "grep"ing around /var/log/syslog and I usually find the smoking gun. My goal is to zcat... (8 Replies)
Discussion started by: metallica1973
8 Replies
5. UNIX for Dummies Questions & Answers
Hi,
just i would like to know, how will be the response if you try to read a 40GB compressed file using zcat.
a)Uncompress the 40GB file and have it in the disk. use cat to view the steps.
b)Use zcat directly to view the compressed file?
What are the steps being occurred in step (b)?
Where... (3 Replies)
Discussion started by: pandeesh
3 Replies
6. Shell Programming and Scripting
I have a tar file which i want to read and check some specific fields basis on which i want to get output.
Code
zcat samplefile.tar.gz | awk 'FNR==1 {++counter}
counter ==2 {BB=1;next}
substr($0,26,2) =="01") {next}
(substr($0,28,12) ~ "^") {next}
(substr($0,184,3) in BB) {next}
1
'... (7 Replies)
Discussion started by: siramitsharma
7 Replies
7. Shell Programming and Scripting
Hello,
When i lauches this command
ssh -n server_name gunzip -c "/REP/xxxx.gz" > server.logThere are a few stray characters like NULNULNULNUL100644 NUL000000NUL000000NUL00024002704 12135060747 012607NUL at the beginning of server.log.
In the xxxx.gz there is a tar file
I'm not sure but it... (5 Replies)
Discussion started by: amazigh42
5 Replies
8. Solaris
I have recently built a new server and patched Soalris 10 up tp latest bundle etc...
When I run a decompress using the format zcat fred.Z |tar -xvf - it runs at a very slow rate.
A similiar server which is less powerful runs over twice as quick.
Is there any work arounds to configure decompress... (4 Replies)
Discussion started by: smcart
4 Replies
9. Shell Programming and Scripting
I need a script file for backup (zip or tar or gz) of old log files in our unix server (causing the space problem). Could you please help me to create the zip or gz files for each log files in current directory and sub-directories also?
I found one command which is to create gz file for the... (4 Replies)
Discussion started by: Mallikgm
4 Replies
10. UNIX for Beginners Questions & Answers
i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,... (2 Replies)
Discussion started by: malaika
2 Replies
LEARN ABOUT DEBIAN
bup-margin
bup-margin(1) General Commands Manual bup-margin(1)
NAME
bup-margin - figure out your deduplication safety margin
SYNOPSIS
bup margin [options...]
DESCRIPTION
bup margin iterates through all objects in your bup repository, calculating the largest number of prefix bits shared between any two
entries. This number, n, identifies the longest subset of SHA-1 you could use and still encounter a collision between your object ids.
For example, one system that was tested had a collection of 11 million objects (70 GB), and bup margin returned 45. That means a 46-bit
hash would be sufficient to avoid all collisions among that set of objects; each object in that repository could be uniquely identified by
its first 46 bits.
The number of bits needed seems to increase by about 1 or 2 for every doubling of the number of objects. Since SHA-1 hashes have 160 bits,
that leaves 115 bits of margin. Of course, because SHA-1 hashes are essentially random, it's theoretically possible to use many more bits
with far fewer objects.
If you're paranoid about the possibility of SHA-1 collisions, you can monitor your repository by running bup margin occasionally to see if
you're getting dangerously close to 160 bits.
OPTIONS
--predict
Guess the offset into each index file where a particular object will appear, and report the maximum deviation of the correct answer
from the guess. This is potentially useful for tuning an interpolation search algorithm.
--ignore-midx
don't use .midx files, use only .idx files. This is only really useful when used with --predict.
EXAMPLE
$ bup margin
Reading indexes: 100.00% (1612581/1612581), done.
40
40 matching prefix bits
1.94 bits per doubling
120 bits (61.86 doublings) remaining
4.19338e+18 times larger is possible
Everyone on earth could have 625878182 data sets
like yours, all in one repository, and we would
expect 1 object collision.
$ bup margin --predict
PackIdxList: using 1 index.
Reading indexes: 100.00% (1612581/1612581), done.
915 of 1612581 (0.057%)
SEE ALSO
bup-midx(1), bup-save(1)
BUP
Part of the bup(1) suite.
AUTHORS
Avery Pennarun <apenwarr@gmail.com>.
Bup unknown- bup-margin(1)