04-03-2013
Yes, perhaps the find command is the bottleneck. Maybe it has a hard time "dealing with" so many files.
What happens if you run the find command for a minute (forget about the mv part for the time being), saving the output from find to a file, and see how many lines accumulate in the file?
If there are perhaps 120 lines after a minute (two per second), then find is the bottleneck. If there are tens of thousands of lines, then it's still a mystery.
Is there any chance you could just use 'ls' instead of find?
9 More Discussions You Might Find Interesting
1. AIX
my task : tar up large bunch of files(about 10,000 files) in the current directories that created more than 30 days ago
but it come with following error
find ./ -ctime +30 | xargs tar rvf test1.tar
tar: test1.tar: A file or directory in the path name does not exist. (3 Replies)
Discussion started by: darkrainbow
3 Replies
2. UNIX for Dummies Questions & Answers
Can anyone interpret and tell me the way the below command works?
find * -name "*${msgType}" -mtime +${archiveDays} -prune -type f -print 2>/dev/null | xargs rm -f 2> /dev/null
Please tell me the usage of prune and xargs in the above command?
Looking forward your reply.
Thanks in... (1 Reply)
Discussion started by: venkatesht
1 Replies
3. Shell Programming and Scripting
I believe what is happening is rm is executing in the script on every directory and on failure of the first it stops although returns status 0.
find $HOME -name /directory/filename | xargs -l rm
This is the code I use but file remains. I am using sun solaris system which has way limited... (4 Replies)
Discussion started by: Ebodee
4 Replies
4. Solaris
Hi
I need to delete more than 3 million files from /var/spool/clientmqueue. When I give the following command to delete the files, I get the error
# pwd
/var/spool/clientmqueue
# rm -f *
/usr/bin/rm: arg list too long
Please tell me how can I delete the files (5 Replies)
Discussion started by: sb200
5 Replies
5. Shell Programming and Scripting
I need to apply mp3gain (album mode) to all mp3 files in a given directory. Each album is in its own directory under /media/data/music/albums for example:
/media/data/music/albums/foo
/media/data/music/albums/bar
/media/data/music/albums/more
What needs to happen is:
cd... (4 Replies)
Discussion started by: audiophile
4 Replies
6. Shell Programming and Scripting
these are numeric ids..
222932017099186177
222932014385467392
222932017371820032
222932017409556480
I have text file having 300 millions of line as shown above. I want to find duplicates from this file. Please suggest the quicker way..
sort | uniq -d will... (3 Replies)
Discussion started by: pamu
3 Replies
7. UNIX for Dummies Questions & Answers
Hi,
Which way is faster rm -rf /path/ or find / -name -exec rm {} \; and why? (7 Replies)
Discussion started by: cain82
7 Replies
8. Shell Programming and Scripting
I have a script (ksh) which tries to run a function in parallel for performance gains. I am also trying to limit the number of parallel child processes to avoid overloading the system by using a variable to count triggered processes and waiting for completion e.g.
do_something ()
{
...
}
... (9 Replies)
Discussion started by: jawsnnn
9 Replies
9. UNIX for Advanced & Expert Users
Hi
I have task to zip files based on modified time but they are in millions and it is taking lot of time more than 12 hours and also eating up high cpu
is there any other / better way to handle it quickly with less cpu consumptionfind . ! -name \"*.gz\" -mtime +7 -type f | grep -v '/.*/' |... (2 Replies)
Discussion started by: reldb
2 Replies
LEARN ABOUT ULTRIX
crontab
crontab(5) File Formats Manual crontab(5)
Name
crontab - clock daemon table file
Syntax
/usr/lib/crontab
Description
The command executes at specified dates and times according to the instructions in the file. The file consists of lines with six fields
each. The format for a line is as follows:
minute hour day month weekday command
The following list defines each field in the line:
minute (0-59) The exact minute that the command sequence executes.
hour (0-23) The hour of the day that the command sequence executes.
day (1-31) The day of the month that the command sequence executes.
month (1-12) The month of the year that the command sequence executes.
weekday (1-7) The day of the week that the command sequence executes. Monday = 1, Tuesday = 2, and so forth.
command The complete command sequence variable that is to be executed. Note that the command string must conform to Bourne shell
syntax.
The first five integer fields may be specified as follows:
o A single number in the specified range
o Two numbers separated by a minus, meaning a range inclusive
o A list of numbers separated by commas, meaning any of the numbers
o An asterisk meaning all legal values
The sixth field is a string that is executed by the shell at the specified times. A percent sign (%) in this field is translated to a new-
line character. Only the first line of the command field, up to a percent sign (%) or end of line, is executed by the shell. The other
lines are made available to the command as standard input.
Examples
The following example is part of a file:
# periodic things
0,15,30,45 * * * * (echo '^M' `date`; echo '') >/dev/console
0,15,30,45 * * * * /usr/lib/atrun
# daily stuff
5 4 * * * sh /usr/adm/newsyslog
15 4 * * * ( cd /usr/preserve; find . -mtime +7 -a -exec rm -f {} ; )
20 4 * * * find /usr/msgs -mtime +21 -a ! -perm 444 -a ! -name bounds
-a -exec rm -f {} ;
# NOTE: The above line is wrapped.
# local cleanups
30 4 * * * find /usr/spool/mqueue -type f -mtime +5 -name df-exec rm {} ;
35 4 * * * find /usr/spool/mqueue -type f -mtime +5 -name tf-exec rm {} ;
40 4 * * * find /usr/spool/rwho -type f -mtime +21 -exec rm {} ;
#
# redirecting error output
0 17 * * 1,3,5 /bin/tar -cv /usr/sysads/smith > /dev/console 2>&1
#
Files
See Also
sh(1), cron(8)
Guide to System Environment Setup
crontab(5)