faster command than find for sorting?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting faster command than find for sorting?
# 1  
Old 02-09-2010
faster command than find for sorting?

I'm sorting files from a source directory by size into 4 categories then copying them into 4 corresponding folders, just wondering if there's a faster/better/more_elegant way to do this:
Code:
find /home/user/sourcefiles -type f -size -400000k -exec /bin/cp -uv {} /home/user/medfiles/ \;
find /home/user/medfiles -type f -size -200000k -exec /bin/mv {} /home/user/smallfiles/ \;
find /home/user/sourcefiles -type f -size +400000k -exec /bin/cp -uv {} /home/user/reallybigfiles/ \;
find /home/user/reallybigfiles -type f -size -1000000k -exec /bin/mv {} /home/user/bigfiles/ \;

I have 570G of source files and need to sync updates about every hour, I update the sourcefiles using rsync and then run this to sort them.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to make awk command faster?

I have the below command which is referring a large file and it is taking 3 hours to run. Can something be done to make this command faster. awk -F ',' '{OFS=","}{ if ($13 == "9999") print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12 }' ${NLAP_TEMP}/hist1.out|sort -T ${NLAP_TEMP} |uniq>... (13 Replies)
Discussion started by: Peu Mukherjee
13 Replies

2. Shell Programming and Scripting

Recursive folder search faster than find?

I'm trying to find folders created by a propritary data aquisition software with the .aps ending--yes, I have never encountered folder with a suffix before (some files also end in .aps) and sort them by date. I need the whole path ls -dt "$dataDir"*".aps"does exactly what I want except for the... (2 Replies)
Discussion started by: Michael Stora
2 Replies

3. Solaris

Monitoring log file for entries - Find command & sorting

hi, I would like to monitor a log file, which rolls over, everytime a server is restarted. I would like to grep for a string, and to be more efficient i'd like to grep only newly appended data. so something like a 'tail -f' would do, however, as the log rolls over i think a 'tail -F' is... (2 Replies)
Discussion started by: horhif
2 Replies

4. UNIX for Dummies Questions & Answers

A faster equivalent for this sed command

Hello guys, I'm cleaning out big XML files (we're talking about 1GB at least), most of them contain words written in a non-latin alphabet. The command I'm using is so slow it's not even funny: cat $1 | sed -e :a -e 's/&lt;*&gt;//g;/&lt;/N;//ba;s/</ /g;s/>/... (4 Replies)
Discussion started by: bobylapointe
4 Replies

5. Shell Programming and Scripting

Faster way to use this awk command

awk "/May 23, 2012 /,0" /var/tmp/datafile the above command pulls out information in the datafile. the information it pulls is from the date specified to the end of the file. now, how can i make this faster if the datafile is huge? even if it wasn't huge, i feel there's a better/faster way to... (8 Replies)
Discussion started by: SkySmart
8 Replies

6. HP-UX

Faster command for file copy than cp ?

we have 30 GB files on our filesystem which we need to copy daily to 25 location on the same machine (but different filesystem). cp is taking 20 min to do the copy and we have 5 different thread doing the copy. so in all its taking around 2 hr and we need to reduce it. Is there any... (9 Replies)
Discussion started by: shipra_31
9 Replies

7. Shell Programming and Scripting

**HELP** need to split this line faster than cut-command

Hi, A datafile containing lines such as below needs to be split: 500000000000932491683600000000000000000000000000016800000GS0000000000932491683600*HOME I need to get the 2-5, 11-20, and 35-40 characters and I can do it via cut command. cut -c 2-5 file > temp1.txt cut -c 11-20 file >... (9 Replies)
Discussion started by: daytripper1021
9 Replies

8. UNIX for Dummies Questions & Answers

Which command will be faster? y?

i)wc -c/etc/passwd|awk'{print $1}' ii)ls -al/etc/passwd|awk'{print $5}' (4 Replies)
Discussion started by: karthi_g
4 Replies

9. Shell Programming and Scripting

command faster in crontab..

Hi all you enlightened unix people, I've been trying to execute a perl script that contains the following line within backticks: `grep -f patternfile.txt otherfile.txt`;It takes normally 2 minutes to execute this command from the bash shell by hand. I noticed that when i run this command... (2 Replies)
Discussion started by: silverlocket
2 Replies

10. UNIX for Dummies Questions & Answers

sorting files with find command before sending to text file

i need help with my script.... i am suppose to grab files within a certain date range now i have done that already using the touch and find command (found them in other threads) touch -d "$date_start" ./tmp1 touch -d "$date_end" ./tmp2 find "$data_location" -maxdepth 1 -newer ./tmp1 !... (6 Replies)
Discussion started by: deking
6 Replies
Login or Register to Ask a Question
QSORT(3)						   BSD Library Functions Manual 						  QSORT(3)

NAME
qsort, heapsort, mergesort -- sort functions LIBRARY
Standard C Library (libc, -lc) SYNOPSIS
#include <stdlib.h> void qsort(void *base, size_t nmemb, size_t size, int (*compar)(const void *, const void *)); int heapsort(void *base, size_t nmemb, size_t size, int (*compar)(const void *, const void *)); int mergesort(void *base, size_t nmemb, size_t size, int (*compar)(const void *, const void *)); DESCRIPTION
The qsort() function is a modified partition-exchange sort, or quicksort. The heapsort() function is a modified selection sort. The mergesort() function is a modified merge sort with exponential search intended for sorting data with pre-existing order. The qsort() and heapsort() functions sort an array of nmemb objects, the initial member of which is pointed to by base. The size of each object is specified by size. mergesort() behaves similarly, but requires that size be greater than ``sizeof(void *) / 2''. The contents of the array base are sorted in ascending order according to a comparison function pointed to by compar, which requires two arguments pointing to the objects being compared. The comparison function must return an integer less than, equal to, or greater than zero if the first argument is considered to be respec- tively less than, equal to, or greater than the second. The functions qsort() and heapsort() are not stable, that is, if two members compare as equal, their order in the sorted array is undefined. The function mergesort() is stable. The qsort() function is an implementation of C.A.R. Hoare's ``quicksort'' algorithm, a variant of partition-exchange sorting; in particular, see D.E. Knuth's Algorithm Q. qsort() takes O N lg N average time. This implementation uses median selection to avoid its O N**2 worst-case behavior. The heapsort() function is an implementation of J.W.J. William's ``heapsort'' algorithm, a variant of selection sorting; in particular, see D.E. Knuth's Algorithm H. heapsort() takes O N lg N worst-case time. Its only advantage over qsort() is that it uses almost no additional memory; while qsort() does not allocate memory, it is implemented using recursion. The function mergesort() requires additional memory of size nmemb * size bytes; it should be used only when space is not at a premium. mergesort() is optimized for data with pre-existing order; its worst case time is O N lg N; its best case is O N. Normally, qsort() is faster than mergesort() is faster than heapsort(). Memory availability and pre-existing order in the data can make this untrue. RETURN VALUES
The qsort() function returns no value. Upon successful completion, heapsort() and mergesort() return 0. Otherwise, they return -1 and the global variable errno is set to indicate the error. ERRORS
The heapsort() function succeeds unless: [EINVAL] The size argument is zero, or, the size argument to mergesort() is less than ``sizeof(void *) / 2''. [ENOMEM] heapsort() or mergesort() were unable to allocate memory. COMPATIBILITY
Previous versions of qsort() did not permit the comparison routine itself to call qsort(). This is no longer true. SEE ALSO
sort(1), radixsort(3) Hoare, C.A.R., "Quicksort", The Computer Journal, 5:1, pp. 10-15, 1962. Williams, J.W.J, "Heapsort", Communications of the ACM, 7:1, pp. 347-348, 1964. Knuth, D.E., "Sorting and Searching", The Art of Computer Programming, Vol. 3, pp. 114-123, 145-149, 1968. McIlroy, P.M., "Optimistic Sorting and Information Theoretic Complexity", Proceedings of the Fourth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 467-474, 1993. Bentley, J.L. and McIlroy, M.D., "Engineering a Sort Function", Software-Practice and Experience, Vol. 23, pp. 1249-1265, 1993. STANDARDS
The qsort() function conforms to ANSI X3.159-1989 (``ANSI C89''). BSD
June 4, 1993 BSD