File maintenance programs/scripts ala logrotate


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting File maintenance programs/scripts ala logrotate
# 1  
Old 03-27-2011
File maintenance programs/scripts ala logrotate

Hi all,

Before I start writing my own file maintenance script, maybe one such program/scripts already exist somewhere :-)

Am looking for a file maintenance script/application that is configurable that I can use to process files under certain criteria, for example, removing files that are x-number of days old, gzip'ping files if they are core dumped files, removing files if they are zero-sized files etc.

Am not sure logrotate is the solution that am looking for.

Any response or suggestion will be much appreciated. Thanks in advance.
# 2  
Old 03-28-2011
Hi newbie_01,

I don't know about such custom applications, so, a custom script could be adapted to do the job.

As you can see below, for this tasks "find" command would be your friend, and combining it with desired
criterias and some other commands, you'll satisfy your requirements.

You've asked for:


1-) removing files that are x-number of days old,
Code:
# 1-) Looking for txt files older than 365 days and printing them within folder and subfolder
 find . -name "*.txt" -mtime +365

# 2-) Printing txt files modified less than 365 days ago within folder and subfolders
 find . -name "*.txt" -mtime -365

# 3-) Printing txt files modified 365 days ago within folder and subfolders
 find . -name "*.txt" -mtime 365

# 4-) Removing txt files older than 365 days within folder and subfolders
*(if you want to remove files is advisable to be sure about files listed/found by "find...")
 find . -name "*.txt" -mtime +365 -exec /bin/rm -f '{}' +

About -mtime: (-mtime n)
(+) means greater than, (-) means less than, and without any symbol means exactly equal to.

2-) Zipping files "if they are core dumped files"
*(Below examples how to zip files, but which is the characteristic of a core dumped file?)
Code:
#1-) Looking all txt files within folders and its subfolders and compress them in a unique zipped file (Compressed_Files.zip)
 find . -name "*.txt" -print | zip Compressed_Files -@

# 2--)Compressing each txt file within folder and subfolders file, adds to original name ".gz" (filename.txt to filename.txt.gz) 
# and deletes original file
 find . -name "*.txt" -exec gzip -f '{}' +

# 3-)Unzipping each txt files within folder and its subfolders, deleting original gz file.
 find . -name "*.txt.gz" -exec gunzip -f '{}' +

3-) Removing files if they are zero-sized files:
Code:
 # 1-) Removing zero size files within folder and its subfolders.
1-) find . -type f -size 0 -exec /bin/rm -f '{}' +
or
2-) find . -type f -empty -exec /bin/rm -f '{}' +

Hope it helps,

Regards.
# 3  
Old 03-28-2011
Quote:
Originally Posted by cgkmal
Hi newbie_01,

I don't know about such custom applications, so, a custom script could be adapted to do the job.

As you can see below, for this tasks "find" command would be your friend, and combining it with desired
criterias and some other commands, you'll satisfy your requirements.

You've asked for:


1-) removing files that are x-number of days old,
Code:
# 1-) Looking for txt files older than 365 days and printing them within folder and subfolder
 find . -name "*.txt" -mtime +365

# 2-) Printing txt files modified less than 365 days ago within folder and subfolders
 find . -name "*.txt" -mtime -365

# 3-) Printing txt files modified 365 days ago within folder and subfolders
 find . -name "*.txt" -mtime 365

# 4-) Removing txt files older than 365 days within folder and subfolders
*(if you want to remove files is advisable to be sure about files listed/found by "find...")
 find . -name "*.txt" -mtime +365 -exec /bin/rm -f '{}' +

About -mtime: (-mtime n)
(+) means greater than, (-) means less than, and without any symbol means exactly equal to.

2-) Zipping files "if they are core dumped files"
*(Below examples how to zip files, but which is the characteristic of a core dumped file?)
Code:
#1-) Looking all txt files within folders and its subfolders and compress them in a unique zipped file (Compressed_Files.zip)
 find . -name "*.txt" -print | zip Compressed_Files -@

# 2--)Compressing each txt file within folder and subfolders file, adds to original name ".gz" (filename.txt to filename.txt.gz) 
# and deletes original file
 find . -name "*.txt" -exec gzip -f '{}' +

# 3-)Unzipping each txt files within folder and its subfolders, deleting original gz file.
 find . -name "*.txt.gz" -exec gunzip -f '{}' +

3-) Removing files if they are zero-sized files:
Code:
 # 1-) Removing zero size files within folder and its subfolders.
1-) find . -type f -size 0 -exec /bin/rm -f '{}' +
or
2-) find . -type f -empty -exec /bin/rm -f '{}' +

Hope it helps,

Regards.
Hi cgkmal

Thanks for your advise.

Characteristic of a core dump? For my application, core dumps are in the form of directories named core_nnnnn, so I assume I should do find -type d -exec zip -r {} ???

BTW, I've had some instance where I use find and xargs and if nothing matches the criteria, it reverts to the root directory? Have you had that problem before? Although, looking at your example, yuo are not using xargs so I don't it will be problem.

Thanks again
# 4  
Old 03-28-2011
Quote:
Originally Posted by newbie_01
Hi cgkmal

Thanks for your advise.

Characteristic of a core dump? For my application, core dumps are in the form of directories named core_nnnnn, so I assume I should do find -type d -exec zip -r {} ???
Hi newbie_01,

I've tried with find . -type d -name "core_*" -exec zip -r '{}' +, finds correctly all core_xxxx folders, but it zips all within a unique zip file. And doesn't work fine if the folders have spaces in their name.

Instead I did this:

Code:
With tar:
Core_Folders=$(find . -type d -name "core_*")

for each in $Core_Folders
do
tar -zcvf $(basename $each).tar.gz $each
done

 # -It looks for every folder named with pattern core_xxxx within main folder.
# -Only works fine if the folders don't have spaces in the name.
# - All core_xxxx.tar.gz will be stored in main folder. 

Or with zip:
Core_Folders=$(find . -type d -name "core_*")

for each in $Core_Folders
do
zip -r $(basename $each) $each
done
# -It looks for every folder named with pattern core_xxxx within main folder.
  # -Only works fine if the folders don't have spaces in the name.
  # - All core_xxxx.zip will be stored in main folder.

Quote:
Originally Posted by newbie_01
Hi cgkmal
BTW, I've had some instance where I use find and xargs and if nothing matches the criteria, it reverts to the root directory? Have you had that problem before? Although, looking at your example, yuo are not using xargs so I don't it will be problem.
I haven't faced that xargs reverts me to root folder so far, and if that happens not using xargs(as in examples I've posted) we will be avoiding that issue Smilie

Hope it helps

Regards
# 5  
Old 03-28-2011
Xargs is great if you end up finding heaps of files to process, but be carefull, particularly for files with spaces in their name.

I'd recommend using -print0 and xargs -0 together, also use -r to stop rm (or other command you want to use) being run when nothing is found. If your OS dosn't support these options with xargs, you are best of avoid using it all together.
This User Gave Thanks to Chubler_XL For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Logrotate 0 byte file

Hello, For various reasons I decided to rebuild my log server on a new microSD. To simplify matters I restored a backed up copy of the appropriate config files and uploaded them to the new log server once syslog-ng was setup. The issue I am running into now is when logrotate compresses the log... (3 Replies)
Discussion started by: MyUserName7000
3 Replies

2. AIX

Logrotate - /etc/logrotate.conf does't exist

Hi Admins. I have installed logrotate rpm on Aix 6.1. After the installation of rpm, I don't find /etc/logrotate.conf file and /etc/logrotate.d dir . The config file is located in /opt/freeware/etc/logrotate.conf. When I ran logrotate -v /opt/freeware/etc/logrotate.conf I get below... (2 Replies)
Discussion started by: snchaudhari2
2 Replies

3. Shell Programming and Scripting

Logrotate - I am not able to rotate files using logrotate

I have written script which is working in Home directory perfectly and also compressing log files and rotating correctly. But, when i try to run script for /var/log/ i am able to get compressed log files but not able to get rotation of compressed log files. Please suggest. I am using below command... (5 Replies)
Discussion started by: VSom007
5 Replies

4. Red Hat

File is missing after logrotate!!

I am having a requirement to rotate the my application logs dailay as it is keep on writiing to single file and below is the logrotate function which I am using, cat /apps/bmc/bmtm/QPasa_logrotate.conf /apps/bmc/bmtm/all_events.log /apps/bmc/bmtm/history_association.log { missingok ... (1 Reply)
Discussion started by: sandyrajh
1 Replies

5. Shell Programming and Scripting

Automatically determining directory path for scripts and programs

I have some C++ code in the following directory structure /home/chrisd/tatsh/trunk/hstmy/ ├── baseLib ├── bin │ ├── awk │ ├── bash │ ├── diag │ ├── ksh │ │ └── TAG201011 │ ├── old │ ├── perl │ ├── prog │ ├── py │ └── tcsh ├── docs ├── fortran ├── others... (0 Replies)
Discussion started by: kristinu
0 Replies

6. UNIX for Advanced & Expert Users

What files or programs have the ability to change your default network scripts and config

What files or programs have the ability to change your default network scripts and config files? All 3 of these very important files got changed on their own. /etc/sysconfig/network-scripts/ifcfg-wlan0 /etc/sysconfig/networking/devices/ifcfg-wlan0... (4 Replies)
Discussion started by: cokedude
4 Replies

7. OS X (Apple)

Right-Click scripts and programs in finder

How can I add a right-click option where I can add my own programs and script to finder? (4 Replies)
Discussion started by: codecaine
4 Replies

8. UNIX for Dummies Questions & Answers

Are programs like sys_open( ) ,sys_read( ) et al examples of system level programs ?

Are the programs written on schedulers ,thread library , process management, memory management, et al called systems programs ? How are they different from the programs that implement functions like open() , printf() , scanf() , read() .. they have a prefix sys_open, sys_close, sys_read etc , right... (1 Reply)
Discussion started by: vishwamitra
1 Replies

9. UNIX for Advanced & Expert Users

logrotate with /etc/logrotate.conf file

Hi there, I want to rotate the logfiles which are located in /var/log/jboss/tomcat* so I have created a file named as 'tomat' in /etc/logrotate.d/tomcat with the following content. # cat /etc/logrotate.d/tomcat /var/log/jboss/tomcat_access_log*.log { daily nocreate ... (2 Replies)
Discussion started by: skmdu
2 Replies

10. Answers to Frequently Asked Questions

scripts/programs/code posted to www.unix.com

Every now and then our users post complete programs to this site. It is especially important that these contributions don't get lost, so I will collect them here. Some of these programs are intended to demonstrate a programming technique and some are ready to run. As a guideline, the code... (0 Replies)
Discussion started by: Perderabo
0 Replies
Login or Register to Ask a Question