Sponsored Content
Full Discussion: How zcat works?
Top Forums UNIX for Dummies Questions & Answers How zcat works? Post 302750175 by pamu on Monday 31st of December 2012 04:51:14 AM
Old 12-31-2012
Quote:
Originally Posted by pandeesh
And also i want to understand the internal implementation of zcat.
i believe zcat should un-compress the original file intermediately somewhere and read.
although we don't see any impact in small files, i want to make sure the impact for the huge files.
Yes. It uses temp memory to perform un compression work.

So the impact might be how much load it will ask to do the things. I believe while performing zcat oparations on a big file, there is large chances that system will throw low memory space error.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

zcat --> Arg list too long

Hi all I have more than 1000 files in a folder and when ever i use a "compress" or "zcat" command it give error /bin/zcat: Arg list too long. . any solution for this :o (3 Replies)
Discussion started by: muneebr
3 Replies

2. UNIX for Dummies Questions & Answers

How ls | wc -l works?

ls displays files in tabbed output. Say a directory contains 3 files. ls will list all 3 in one line. So, I expect ls | wc -l to give 1, but it counts the nr of files and gives 3. Can someone explain how this works? (3 Replies)
Discussion started by: krishmaths
3 Replies

3. Linux

zcat on Linux

Hi I am trying to extract a <filename>.tar.Z on a SLES machine using zcat. The command I am using is zcat <filename>.tar.Z | tar xf - When I use the above I get the following message tar: Read 7168 bytes from - I think the message is benign because I see that my files where... (2 Replies)
Discussion started by: felixmat1
2 Replies

4. Programming

how this works?

pls explain me how this works.... DECODE (SUBSTR (field, 1, 1),'''', '''''' || field || '''','''' || field || '''') here field is a column in an oracle table.... (7 Replies)
Discussion started by: vijay_0209
7 Replies

5. UNIX for Advanced & Expert Users

How this works?

I have a program............ #include<stdio.h> #include<unistd.h> main() { if(fork == 0) { printf("Hi every body:p!!!!!!!!!!"); } } This program works with out any error. here fork is not a system call. It just act as a variable.But how it works without declaring it? What data type it... (19 Replies)
Discussion started by: carolsanjeevi
19 Replies

6. Shell Programming and Scripting

zcat two files

Hi, Like cat file1 file2 > file Can I do zcat file1.gz file2.gz > file.gz (11 Replies)
Discussion started by: jacobs.smith
11 Replies

7. Shell Programming and Scripting

BASH ZCAT EGREP Shell Script

I created a backup script that emails all the admins when the backup is complete and attaches a log file of what what backed up. On occasion, something happens in which the backups stop working, I started "grep"ing around /var/log/syslog and I usually find the smoking gun. My goal is to zcat... (8 Replies)
Discussion started by: metallica1973
8 Replies

8. Shell Programming and Scripting

awk & zcat not working together

I have a tar file which i want to read and check some specific fields basis on which i want to get output. Code zcat samplefile.tar.gz | awk 'FNR==1 {++counter} counter ==2 {BB=1;next} substr($0,26,2) =="01") {next} (substr($0,28,12) ~ "^") {next} (substr($0,184,3) in BB) {next} 1 '... (7 Replies)
Discussion started by: siramitsharma
7 Replies

9. Shell Programming and Scripting

[zcat] [gunzip -c] header

Hello, When i lauches this command ssh -n server_name gunzip -c "/REP/xxxx.gz" > server.logThere are a few stray characters like NULNULNULNUL100644 NUL000000NUL000000NUL00024002704 12135060747 012607NUL at the beginning of server.log. In the xxxx.gz there is a tar file I'm not sure but it... (5 Replies)
Discussion started by: amazigh42
5 Replies

10. Solaris

ZCAT xxx.Z|tar -xvf - decompression slow ?

I have recently built a new server and patched Soalris 10 up tp latest bundle etc... When I run a decompress using the format zcat fred.Z |tar -xvf - it runs at a very slow rate. A similiar server which is less powerful runs over twice as quick. Is there any work arounds to configure decompress... (4 Replies)
Discussion started by: smcart
4 Replies
COMPRESS(1)						      General Commands Manual						       COMPRESS(1)

NAME
compress, uncompress, zcat - compress and expand data (version 4.1) SYNOPSIS
compress [ -f ] [ -v ] [ -c ] [ -V ] [ -r ] [ -b bits ] [ name ... ] uncompress [ -f ] [ -v ] [ -c ] [ -V ] [ name ... ] zcat [ -V ] [ name ... ] DESCRIPTION
Compress reduces the size of the named files using adaptive Lempel-Ziv coding. Whenever possible, each file is replaced by one with the extension .Z, while keeping the same ownership modes, access and modification times. If no files are specified, the standard input is com- pressed to the standard output. Compress will only attempt to compress regular files. In particular, it will ignore symbolic links. If a file has multiple hard links, compress will refuse to compress it unless the -f flag is given. If -f is not given and compress is run in the foreground, the user is prompted as to whether an existing file should be overwritten. Compressed files can be restored to their original form using uncompress or zcat. uncompress takes a list of files on its command line and replaces each file whose name ends with .Z and which begins with the correct magic number with an uncompressed file without the .Z. The uncompressed file will have the mode, ownership and timestamps of the compressed file. The -c option makes compress/uncompress write to the standard output; no files are changed. zcat is identical to uncompress -c. zcat uncompresses either a list of files on the command line or its standard input and writes the uncompressed data on standard output. zcat will uncompress files that have the correct magic number whether they have a .Z suffix or not. If the -r flag is specified, compress will operate recursively. If any of the file names specified on the command line are directories, compress will descend into the directory and compress all the files it finds there. The -V flag tells each of these programs to print its version and patchlevel, along with any preprocessor flags specified during compila- tion, on stderr before doing any compression or uncompression. Compress uses the modified Lempel-Ziv algorithm popularized in "A Technique for High Performance Data Compression", Terry A. Welch, IEEE Computer, vol. 17, no. 6 (June 1984), pp. 8-19. Common substrings in the file are first replaced by 9-bit codes 257 and up. When code 512 is reached, the algorithm switches to 10-bit codes and continues to use more bits until the limit specified by the -b flag is reached (default 16). Bits must be between 9 and 16. The default can be changed in the source to allow compress to be run on a smaller machine. After the bits limit is attained, compress periodically checks the compression ratio. If it is increasing, compress continues to use the existing code dictionary. However, if the compression ratio decreases, compress discards the table of substrings and rebuilds it from scratch. This allows the algorithm to adapt to the next "block" of the file. Note that the -b flag is omitted for uncompress, since the bits parameter specified during compression is encoded within the output, along with a magic number to ensure that neither decompression of random data nor recompression of compressed data is attempted. The amount of compression obtained depends on the size of the input, the number of bits per code, and the distribution of common sub- strings. Typically, text such as source code or English is reduced by 50-60%. Compression is generally much better than that achieved by Huffman coding (as used in pack), or adaptive Huffman coding (compact), and takes less time to compute. Under the -v option, a message is printed yielding the percentage of reduction for each file compressed. Exit status is normally 0; if the last file is larger after (attempted) compression, the status is 2; if an error occurs, exit status is 1. SEE ALSO
pack(1), compact(1) DIAGNOSTICS
Usage: compress [-dfvcVr] [-b maxbits] [file ...] Invalid options were specified on the command line. Missing maxbits Maxbits must follow -b. file: not in compressed format The file specified to uncompress has not been compressed. file: compressed with xx bits, can only handle yy bits File was compressed by a program that could deal with more bits than the compress code on this machine. Recompress the file with smaller bits. file: already has .Z suffix -- no change The file is assumed to be already compressed. Rename the file and try again. file: filename too long to tack on .Z The file cannot be compressed because its name is longer than 12 characters. Rename and try again. This message does not occur on BSD systems. file already exists; do you wish to overwrite (y or n)? Respond "y" if you want the output file to be replaced; "n" if not. uncompress: corrupt input A SIGSEGV violation was detected which usually means that the input file has been corrupted. Compression: xx.xx% Percentage of the input saved by compression. (Relevant only for -v.) -- not a regular file or directory: ignored When the input file is not a regular file or directory, (e.g. a symbolic link, socket, FIFO, device file), it is left unaltered. -- has xx other links: unchanged The input file has links; it is left unchanged. See ln(1) for more information. Use the -f flag to force compression of multiply- linked files. -- file unchanged No savings is achieved by compression. The input remains virgin. BUGS
Although compressed files are compatible between machines with large memory, -b12 should be used for file transfer to architectures with a small process data space (64KB or less, as exhibited by the DEC PDP series, the Intel 80286, etc.) Invoking compress with a -r flag will occasionally cause it to produce spurious error warnings of the form "<filename>.Z already has .Z suffix - ignored" These warnings can be ignored. See the comments in compress.c:compdir() for an explanation. local COMPRESS(1)
All times are GMT -4. The time now is 11:46 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy