Compress and logging files from October 6th in a loop


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Compress and logging files from October 6th in a loop
# 1  
Old 10-10-2018
Compress and logging files from October 6th in a loop

Good night, i need your help please

Because there are about 10000 files from October 6th, i need to to compress but i use this command and it does not do anything, in the prompt has no respones and i have to press CRTL+C to goback to the shell

Code:
for file in $(ls -l | grep "Impres" | grep "Oct  6"|grep -v .gz | awk '{print $9}');
do
 gzip $file `
done


but if i list them, there are files from October 6th

Code:
CEL /produccion/explotacion/xpfactur/facturacion/log/ImpresionScl/nociclo # ls -l | grep "Impres" | grep "Oct  6"|grep -v .gz | >
bc
ImpresionScl_11715951.log
ImpresionScl_11762413.log
ImpresionScl_11762472.log
ImpresionScl_11763290.log
ImpresionScl_11763291.log
ImpresionScl_11763292.log
ImpresionScl_11763294.log
ImpresionScl_11797566.log
ImpresionScl_11864274.log

There must be a syntax error or somethink ?
The OS is SunOS

I appreciate your help in advanced

------ Post updated at 09:44 PM ------

I found the problem but i face this while overriding:

Code:
already exists; do you wish to overwrite (y or n)?   not overwritten

# 2  
Old 10-11-2018
Looks like the files in your current working directory are not unique.
Is this true ?

Do you have both (in one moment in time, just an example) :
Code:
ImpresionScl_11763290.log # Oct 6 you wish to compress
ImpresionScl_11763290.log.gz # older date then Oct 6

In working directory when trying to compress ?

Initial problem you found is at the end of following line :
Code:
gzip $file `

If you found the answer yourself, it is always nice to point it out and describe, so others may benefit.

Hope that helps
Regards
Peasant.
This User Gave Thanks to Peasant For This Post:
# 3  
Old 10-11-2018
Yes thank you it was a silly mistake i made, the initial problem was:
Code:
gzip $file `

yes there are not unique there are some from september.

but neither i dont want to overwrite them nor to be asked to overwrite, so i implemented this to skip if it really exists and redirect to a lof file the gzipped files:



Code:
for file in $(ls -l | grep "Impres" | grep "Oct  6"|grep -v .gz | awk '{print $9}');
do
 if [-f $file]
    echo "Skipped"
 else 
   echo "compresing file $file" >> output.log
   gzip $file
done

But its kind of a hassle because at least there are nearly 10000 files to gzip and log.

so is there any better approach to enhance this script and run faster?

I appreciate your help in advanced
# 4  
Old 10-12-2018
Please check this older thread :
gzip parallelized

So i do not rewrite from scratch, seems Scrutinizer already wrote some good code to run parallel gzips, with examples and multiple scenarios.
Of course, it's not one to fit all, you will have to adjust it to meet your needs a bit.

Post here if you get stuck with it or have any questions.

Regards
Peasant.
# 5  
Old 10-15-2018
Personally, I would avoid reading the output of ls -l because you could get false positives too easily. Maybe another approach might be better:-
Code:
touch -mt 201810060000 /tmp/start_point
touch -mt 201810070000 /tmp/end_point

echo "$(date) : Starting"

for file in $(find . -type f -newer /tmp/start_point ! -newer /tmp/end_point ! -name "*.gz")
do
   echo "$(date) : Compressing ${file}"
   echo gzip ${file}
done
echo "$(date) : Complete"

Remove the dark read echo if you are happy that it should work.

Does that help?
Robin
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Listing files from October 4th is not accurate

Good evening: Need your help please Before deleting older logs from octobre 4th i need to identify which files i have to remove, for instance today is octobre 8ht, i need to remove files from October 4th Because there are many files i can not list because error args too long, so ... (5 Replies)
Discussion started by: alexcol
5 Replies

2. UNIX for Dummies Questions & Answers

How to sort the 6th field of tab delimited files?

Here's a sample of the data: NAME BIRTHDAY SEX LOCATION AGE ID Jim 05/11/1986 M Japan 27 86 Rei 08/25/1990 F Korea 24 33 Jane 02/24/1985 F India 29 78 I've been trying to sort files using the... (8 Replies)
Discussion started by: maihani
8 Replies

3. Shell Programming and Scripting

Compress a particular month files

Hi ALL, I am working on a folder where there are lot of files for the past one year. I need to compress a particular month files alone. suppose i need to compress the feb month files alone, what is the script we can use. Thanks in advance (2 Replies)
Discussion started by: acronis.84
2 Replies

4. UNIX for Dummies Questions & Answers

Issue: Compress in unix server and FTP to windows and open the compress file using Winzip

Hi All ! We have to compress a big data file in unix server and transfer it to windows and uncompress it using winzip in windows. I have used the utility ZIP like the below. zip -e <newfilename> df2_test_extract.dat but when I compress files greater than 4 gb using zip utility, it... (4 Replies)
Discussion started by: sakthifire
4 Replies

5. Linux

Compress files >2GB

Hi folks, I'm trying to compress a certain number of files from a cifs mount to a xfs mount, but cannot do it when the total size of the files is bigger than 2GB. Is there any limitation for above 2GB? OS is SLES 64bit The files are maximum 1MB, so there are aprox. 2000 files to compress... (2 Replies)
Discussion started by: xavix
2 Replies

6. Shell Programming and Scripting

compress files

Could someone give me an idea how to compress all files from a given directory that are not of type .z (compressed). Please help. (2 Replies)
Discussion started by: lesstjm
2 Replies

7. UNIX for Dummies Questions & Answers

how to compress html files

Hello, On a Centos 5.0 server, Apache 2.2 delivers static html page. How could I compress those html pages to gain speed and save bandwidth? is there a utility that would be effective and save? Thanks (2 Replies)
Discussion started by: JCR
2 Replies

8. UNIX for Dummies Questions & Answers

Compress files

Hi All, I would like to archive some of the scripts below(USFINUM042006_01.CSV USFINUM042006_02.CSV and USFINUM042006_03.CSV )and also use a wildcard e.g. <command> USFINUM*.CSV. Also there are a lot of similar files but I want only the three latest files to be compressed. Which is the best... (3 Replies)
Discussion started by: indira
3 Replies

9. Shell Programming and Scripting

Compress multiple files

Hi Friends, Can anyone help me out with compressing multiple files. I have multiple files in directory , I have to compress these into a single file, I tried using gzip -r outfile.gz file1 file2 file3. It is not working Thanks in advance for your help S :) (5 Replies)
Discussion started by: sbasetty
5 Replies
Login or Register to Ask a Question