Gzip files as they are created Post: 302246630

Sponsored Content

Top Forums Shell Programming and Scripting Gzip files as they are created Post 302246630 by eisenhorn on Tuesday 14th of October 2008 04:57:10 AM

10-14-2008

Registered User

Thanks cfajohnson. I will try to incorporate your recommendations.

However, for my real problem, the oracle export utility (the process that creates the binary files) will create a file and then populate it with data up until it reaches 1Gb in size, then it will create a new file. If we use parallelisation, it will create n number of files (one for each parallel process) and fill them. The final binary files created could and probably would be less the 1Gb.

My thought was to call the gzip func before the export utility and then have it wait for files to gzip, i.e. only gzip files if there are more than the parallel number n. So if parallel was set to 4, only gzip the 5th file.

Thinking it through, I find it hard to identify which file the gzip program should gzip as we can't just zip files of 1Gb in size as it could still be finishing writing to the file, etc. Could I use something like fuser to identify if the export tool has finished with the file? perhaps some form of looping gzip that waits for the fuser to return no pid for an export file and then zips it? I have looked at an export and can see that when the utility is finished writing the file it no longer locks it so this could be feasible.

I would welcome your ideas.

Best Regards.

eisenhorn

View Public Profile for eisenhorn

Find all posts by eisenhorn

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

gzip, multiple files

Hello Everyone, Here is what I am trying to do. I have four text files, I want to gzip them under unix and mail the zipped file via outlook. I am able to do this easily enough, but using winzip or pkunzip to unzip the file, there is only one file. (In essence, all four files were...

2. UNIX for Dummies Questions & Answers

Need to gzip LARGE files

The windows version of gzip supports pretty much unlimited file sizes while the one we have in solaris only goes up to a set size, one or two gigs I think. Is there a new version of gzip I can put on our systems that supports massive file sizes?

3. UNIX for Dummies Questions & Answers

gzip all the files in a directory

Hi, There are multiple files in a directory with different names.How can they be gzipped such that the timestamp of the files is not changed.

4. Shell Programming and Scripting

unzip particular gzip files among the normal data files

Hello experts, I run Solaris 9. I have a below script which is used for gunzip the thousand files from a directory. ---- #!/usr/bin/sh cd /home/thousands/gzipfiles/ for i in `ls -1` do gunzip -c $i > /path/to/file/$i done ---- In my SAME directory there thousand of GZIP file and also...

5. Shell Programming and Scripting

gzip the files with particular extension

Is there any way to compress only the files with .xml extension within a folder which in turn has many sub folders? gzip -r9 path/name/*.xml is not working This compression is done in the Windows server using Batch script.

6. Shell Programming and Scripting

gzip files with extension

Hi, I have 1000 of files in a folder with the file extension as .csv In this some of the files are already zipped and its looks like filename.csv.gz Now i need to zip all the files in the folder to free some disk space. When i give gzip *.csv It prompts me to overwrite filename.csv.gz...

7. UNIX for Dummies Questions & Answers

Gzip files - Excluded directories

Hi, I am using the commande line find . -name "*.nc" -type f -exec gzip -v {} \; to zip all files with the extension " *.nc " in all directories. But I am looking for a way to excluded some directories as the command will recursively check all of them. If somone can help me with some...

8. UNIX for Advanced & Expert Users

gzip vs pipe gzip: produce different file size

Hi All, I have a random test file: test.txt, size: 146 $ ll test.txt $ 146 test.txt Take 1: $ cat test.txt | gzip > test.txt.gz $ ll test.txt.gz $ 124 test.txt.gz Take 2: $ gzip test.txt $ ll test.txt.gz $ 133 test.txt.gz As you can see, gzipping a file and piping into gzip...

9. Shell Programming and Scripting

tar and gzip files

Hi Guys, I am using RHEL5 and Solaris 9 & 10. I want to tar and gzip my files then remove them after a successful tar command... Lets say I have files with extension .arc then I want to tar and gzip these files. After successful tar command I want to remove all these files (i.e .arc). ...

10. Shell Programming and Scripting

Listing the file name and no of records in each files for the files created on a specific day

Hi, I want to display the file names and the record count for the files in the 2nd column for the files created today. i have written the below command which is listing the file names. but while piping the above command to the wc -l command its not working for me. ls -l...

LEARN ABOUT V7

gzforce

GZFORCE(1)						      General Commands Manual							GZFORCE(1)

NAME

       gzforce - force a '.gz' extension on all gzip files

SYNOPSIS

       gzforce [ name ...  ]

DESCRIPTION

       gzforce	forces a .gz extension on all gzip files so that gzip will not compress them twice.  This can be useful for files with names trun-
       cated after a file transfer.  On systems with a 14 char limitation on file names, the original name is truncated to make room for  the  .gz
       suffix. For example, 12345678901234 is renamed to 12345678901.gz. A file name such as foo.tgz is left intact.

SEE ALSO

       gzip(1), gznew(1), gzmore(1), gzgrep(1), gzdiff(1), gzexe(1)

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +--------------------+-----------------+
       |  ATTRIBUTE TYPE    | ATTRIBUTE VALUE |
       +--------------------+-----------------+
       |Availability	    | SUNWgzip	      |
       +--------------------+-----------------+
       |Interface Stability | External	      |
       +--------------------+-----------------+
NOTES

       Source for gzip is available in the SUNWgzipS package.

																	GZFORCE(1)

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

gzip, multiple files

Discussion started by: smbodnar

2. UNIX for Dummies Questions & Answers

Need to gzip LARGE files

Discussion started by: LordJezo

3. UNIX for Dummies Questions & Answers

gzip all the files in a directory

Discussion started by: er_ashu

4. Shell Programming and Scripting

unzip particular gzip files among the normal data files

Discussion started by: thepurple