Compressing & removing files in a directory & subdirectory


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compressing & removing files in a directory & subdirectory
# 1  
Old 10-07-2016
Compressing & removing files in a directory & subdirectory

Hi,

I want a simple line of code that will compress files within a directory specified (parameter) and its subdirectories and also i want to remove files which are exactly 365 days old from the sysdate after this compression.
Please help.

Thanks,
JD
# 2  
Old 10-07-2016
the following part of script will
  1. compress files older than 10 days , except already compressed files
  2. delete compressed files older than 365 days

To be adapted :

Code:
#
DIRECTORY=/mydir   #change it to $1 for instance
find $DIRECTORY -type f  ! -name '*.gz'  -mtime +10 -exec gzip {} \;
find $DIRECTORY -name '*.gz' -mtime +365 -exec rm {} \;


Last edited by rbatte1; 10-18-2016 at 06:15 AM.. Reason: Converted to formatted number-list
# 3  
Old 10-07-2016
Hi,
Here, we can use only one find command:
Code:
find $DIRECTORY -type f  ! -name '*.gz'  -mtime +10 -exec gzip {} \; -o -name '*.gz' -mtime +365 -exec rm {} \;

Regards.
This User Gave Thanks to disedorgue For This Post:
# 4  
Old 10-08-2016
Use \+ if you are passing files to utilities which can handle multiple files.

This is valid in both gzip and cp, and you should gain performance with small change.

Also, files with .gz extension gzip will not attempt to compress, but rather print on stderr that file(s) already have gz extension and it will skip it.
Keep in mind that if fileXYZ.gz exists on same location where same fileXYZ is being compressed, gzip will prompt you for overwrite halting your program awaiting user input.

You really don't want to come to a machine after N days/months to see N crons with N finds awaiting user input..

Hope that helps
Regards
Peasant.
This User Gave Thanks to Peasant For This Post:
# 5  
Old 10-08-2016
Quote:
Originally Posted by disedorgue
Hi,
Here, we can use only one find command:
Code:
find $DIRECTORY -type f  ! -name '*.gz'  -mtime +10 -exec gzip {} \; -o -name '*.gz' -mtime +365 -exec rm {} \;

Regards.
The part after the -type f should be in (esccaped) brackets, otherwise it would attempt to rm an old *.gz directory.
Or this version that also eliminates a redundant -name '*.gz'
Code:
find $DIRECTORY -type f -mtime +10 \( ! -name '*.gz' -exec gzip -f {} \; -o -mtime +365 -exec rm {} \; \)

Ok, this will delete all files if +365 days old, including uncompressed ones (where gzip has failed). Rather a feature than a mistake.
The efficient + instead of the \;should work as well.

Last edited by MadeInGermany; 10-08-2016 at 08:19 AM..
This User Gave Thanks to MadeInGermany For This Post:
# 6  
Old 10-08-2016
Exact, if name of subdirectory is the name of file where to launch command then this file will delete.
So, We can just repeat -type f after -o connector:
Code:
find $DIRECTORY -type f  ! -name '*.gz'  -mtime +10 -exec gzip {} \; -o -type f -name '*.gz' -mtime +365 -exec rm {} \;

Regards.

Last edited by vbe; 10-08-2016 at 10:59 AM.. Reason: typo - added the missing i to/icode...
# 7  
Old 10-17-2016
Hi,

When i execute this line of code:

Code:
find $DIRECTORY -type f -mtime +10 \( ! -name '*.gz' -exec gzip -f {} \; -o -mtime +365 -exec rm {} \; \)

i get no error, but the files do not get compressed.
Can anyone help.

Thanks,
Jess
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Check Directory for files & Email

I'm working on a bash script to move files from one location, to two. The first part of my challenge is intended to check a particular directory for contents (e.g. files or other items in it), if files exists, then send the list of names to a txt file and email me the text file. If files do not... (4 Replies)
Discussion started by: Nvizn
4 Replies

2. UNIX for Dummies Questions & Answers

Need Help in reading N days files from a Directory & combining the files

Hi All, Request your expertise in tackling one requirement in my project,(i dont have much expertise in Shell Scripting). The requirement is as below, 1) We store the last run date of a process in a file. When the batch run the next time, it should read this file, get the last run date from... (1 Reply)
Discussion started by: dsfreddie
1 Replies

3. UNIX for Dummies Questions & Answers

How to compare 2 files & get specific value & replace it in other file.

Hiiii Friends I have 2 files with huge data. I want to compare this 2 files & if they hav same set of vales in specific rows & columns i need to get that value from one file & replace it in other. For example: I have few set data of both files here: a.dat: PDE-W 2009 12 16 5 29 11.11 ... (10 Replies)
Discussion started by: reva
10 Replies

4. Shell Programming and Scripting

PHP read large string & split in multidimensional arrays & assign fieldnames & write into MYSQL

Hi, I hope the title does not scare people to look into this thread but it describes roughly what I'm trying to do. I need a solution in PHP. I'm a programming beginner, so it might be that the approach to solve this, might be easier to solve with an other approach of someone else, so if you... (0 Replies)
Discussion started by: lowmaster
0 Replies

5. Shell Programming and Scripting

Just listing size, timestamp & name of files in a directory

How can I list the files in a directory and just show the file size, date stamp, timestamp and file name.. I've been trying to ls -lrt the directory to a file and then use the cut command but I'm not having any luck with getting the proper results.. I thought i could use the -f switch and count... (4 Replies)
Discussion started by: Jazmania
4 Replies

6. Shell Programming and Scripting

Find & Replace string in multiple files & folders using perl

find . -type f -name "*.sql" -print|xargs perl -i -pe 's/pattern/replaced/g' this is simple logic to find and replace in multiple files & folders Hope this helps. Thanks Zaheer (0 Replies)
Discussion started by: Zaheer.mic
0 Replies

7. UNIX for Advanced & Expert Users

Extracting the different files from directory & its sub directories

Hi Everyone, It would be helpful if someone helps me on this. Requirement: I have a directory which includes different types of files(for example *.java,*.class),but not restricted for only these types. I need to find the same types of file extensions from its directories and subdirectories... (3 Replies)
Discussion started by: rcvasu
3 Replies

8. UNIX for Dummies Questions & Answers

Find & Copy Selected files to another Directory

I am wanting to find files within a directory that are over a certain number of days old and copy them to another directory. And unfortunately not having much luck.......is someone able to help. Would also like to add that there are literally thousands of files that I am wanting to copy in one... (3 Replies)
Discussion started by: hellfyre
3 Replies

9. Shell Programming and Scripting

find and replace in subdirectory (filename & content - both)

Hi, I have to rename all occurance of CUST_MST to RESELLER_MST both in filename and file content under a directory (say D0) which contains multiple (2-3 levels) sub directory. Example: D0 -> D1 -> D2 has a file CUST_MST_TEMP.txt this contains : > cat /D0/D1/D2/CUST_MST_TEMP.txt... (3 Replies)
Discussion started by: sabyasm
3 Replies

10. UNIX for Dummies Questions & Answers

list largest files in a directory & its subdirectories

I need to find the largest files in a directory & it's subdirectories. I'm not sure what options on ls -l will work to give me this. or is there another way to do this? Thanks, igidttam (6 Replies)
Discussion started by: igidttam
6 Replies
Login or Register to Ask a Question