Sponsored Content
Top Forums Shell Programming and Scripting Purging 2000+ directories efficiently Post 302824123 by Apoorvbarwa on Thursday 20th of June 2013 08:02:20 AM
Old 06-20-2013
Getting the mountpoints wont be a problem and can be hardcoded to a certain extent eg. In the path
/XXXX/XXX/XXXX/XXXX/XXXX/<AAAAA>/<aaaa>/duplicate ... /XXXX/XXX/XXXX/XXXX/XXXX/ part is generic and can be hardcoded.
Using the part /<AAAAA>/, /<aaaa>/ can be fetched using a sql query so we can get and traverse into /XXXX/XXX/XXXX/XXXX/XXXX/<AAAAA>/<aaaa> path . Now inside this there
are the following dirs where purging needs to be done.

/XXXX/XXX/XXXX/XXXX/XXXX/<AAAAA>/<aaaa>/duplicate
exception archive
There are 1000+ dirs of this kind (/XXXX/XXX/XXXX/XXXX/XXXX/<AAAAA>) with the same structure as above.

What i am looking for is a way to create parallel processes which can purge these dirs parallely.

A POINT TO REMEMBER IS THAT THE NUMBER OF DIRS IS NOT CONSTANT AND IS ALWAYS INCREASING so if there are 1200 (/XXXX/XXX/XXXX/XXXX/XXXX/<AAAAA>) kind of dirs

then we should not have 3 processes cleaning up 400 dirs each.
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Purging script

Hi, Need to change the following command to also purge the child directories after /data/tmp within one command (recursively check for X number of days old files and purge accordingly)? e.g. /data/tmp/a, /data/tmp/a/b, /data/tmp/log. find /data/tmp -type f -mtime +7 -exec rm -f {} \; ... (2 Replies)
Discussion started by: egls
2 Replies

2. Shell Programming and Scripting

Using xapply efficiently?

Hi all, Were currently using xapply to run multiple ssh instances that then calls a script that returns the PID of a webserver process. Currently we have like 30 xapply statements in a script call checkit which checks various webserver processes on various unix/linux boxes. My question... (0 Replies)
Discussion started by: bdsffl
0 Replies

3. Shell Programming and Scripting

Purging a Set of Files

Hi Frineds, I want to delete a set of files which are older than 7 days from teh current date.I am totally enw to shell scripting, can anyone help me with a sample code to list out the files which are older and then remove them from the directory. Please help THanks Viswa (5 Replies)
Discussion started by: svishh123
5 Replies

4. Shell Programming and Scripting

Error While Purging Files

find /filearchive/ -type f -mtime +7 -exec rm weblogs*.log {} \; This worked only if this comand is executed int he unix comand prompt, but when i put this in the shell script it is not recognizing the file.It says weblogs: No such file or directory Am i doing anything wrong here ? (4 Replies)
Discussion started by: svishh123
4 Replies

5. Shell Programming and Scripting

Shell script for purging the 3 days old files

Hi all, I try to write shell script to the below requirement. I have Hard coded the oratab location and take the list of databases from oratab and find out archive log locations for each database, and list more than 3 days old files for each location and purge those. ... (2 Replies)
Discussion started by: mak_boop
2 Replies

6. Shell Programming and Scripting

purging of Files

Hello All, I want to delete the files based on the days. like, Files available under directory /abc want to delete if they are older than 15 days. Files available under directory /pqr want to delete if they are 7 days old and some files under directory /xyz should get deleted if they are... (5 Replies)
Discussion started by: ssachins
5 Replies

7. Shell Programming and Scripting

Fine Tune - Huge files/directory - Purging

Hi Expert's, I need your assitance in tunning one script. I have a mount point where almost 4848008 files and 864739 directories are present. The script search for specific pattern files and specfic period then delete them to free up space. The script is designed to run daily and its taking around... (19 Replies)
Discussion started by: senthil.ak
19 Replies

8. Shell Programming and Scripting

Disc space issues and purging of files

Hi All, I am looking forward to create a unix shell script to purge the files. The requirement is: 1) Do df -k and check the current space occupied for the /a1 folder. 2) If the space consumed is greater than 90 %, delete all the DEF* files from a subfolder /a1/archive. Example: df... (4 Replies)
Discussion started by: shilpa_acc
4 Replies

9. UNIX for Dummies Questions & Answers

Purging logic

Hi guys i am having following files in a directory a_07:15:13.txt a_07:16:13.txt a_07:17:13.txt a_07:18:13.txt a_07:19:13.txt a_date file will be created on a day to day basics so that it may pileup on a long go.So i need to create a logic so that the files will be gunzipped to a backup... (1 Reply)
Discussion started by: mohanalakshmi
1 Replies
DH_USRLOCAL(1)							     Debhelper							    DH_USRLOCAL(1)

NAME
dh_usrlocal - migrate usr/local directories to maintainer scripts SYNOPSIS
dh_usrlocal [debhelperoptions] [-n] DESCRIPTION
dh_usrlocal is a debhelper program that can be used for building packages that will provide a subdirectory in /usr/local when installed. It finds subdirectories of usr/local in the package build directory, and removes them, replacing them with maintainer script snippets (unless -n is used) to create the directories at install time, and remove them when the package is removed, in a manner compliant with Debian policy. These snippets are inserted into the maintainer scripts by dh_installdeb. See dh_installdeb(1) for an explanation of debhelper maintainer script snippets. When the Rules-Requires-Root field is not (effectively) binary-targets, the directories in /usr/local will have ownership root:staff and the mode will be 02775. These values have been chosen to comply with the recommendations of the Debian policy for directories in /usr/local. When Rules-Requires-Root has an effective value of binary-targets, the owners, groups and permissions will be preserved with one exception. If the directory is owned by root:root, then ownership will be reset to root:staff and mode will be reset to 02775. This is useful, since that is the group and mode policy recommends for directories in /usr/local. OPTIONS
-n, --no-scripts Do not modify postinst/prerm scripts. NOTES
Note that this command is not idempotent. dh_prep(1) should be called between invocations of this command. Otherwise, it may cause multiple instances of the same text to be added to maintainer scripts. CONFORMS TO
Debian policy, version 2.2 SEE ALSO
debhelper(7) This program is a part of debhelper. AUTHOR
Andrew Stribblehill <ads@debian.org> 11.1.6ubuntu2 2018-05-10 DH_USRLOCAL(1)
All times are GMT -4. The time now is 03:50 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy