Visit Our UNIX and Linux User Community


Find Large Files Recursively From Specific Directory


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Find Large Files Recursively From Specific Directory
# 1  
Old 04-27-2016
RedHat Find Large Files Recursively From Specific Directory

Hi.

I found many scripts in the web of achieving this.

But I like to use this one
Code:
find /EDWH-DMT03 -xdev -size +10000 -exec ls -la {} \;|sort -n -k 5 > LARGE.rst

But the problem is, why it still list out files with 89 bytes as the output? Is there anything wrong with the command?

My UNIX environment is as below
Code:
HP-UX system1 B.11.31 U ia64 0189138652 unlimited-user license

Any help would be much appreciated.

Thank you.
# 2  
Old 04-27-2016
Please show us the ls -la output for the file that has size 89 bytes. That find command shouldn't show you anything for files smaller than 512000 bytes. Unless you have a huge directory, it shouldn't make any difference, but please also try:
Code:
find /EDWH-DMT03 -xdev -size +10000 -exec ls -lad {} +|sort -n -k 5,5 > LARGE.rst

This User Gave Thanks to Don Cragun For This Post:
# 3  
Old 04-27-2016
Thanks Don.

This is some of the output:
Code:
-rw-r--r--   1 ftphsbb    ftpuser         89 Apr 24 21:40 CASS_voice_NON_IMS_20160424_61486.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser         89 Apr 25 23:20 CASS_voice_NON_IMS_20160425_61701.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser         89 Apr 26 05:20 CASS_voice_NON_IMS_20160426_61746.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser        127 Nov 11  2014 CASS_voice_NON_IMS_20141022_59770.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser        174 Nov 11  2014 CASS_voice_NON_IMS_20141021_59453.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser        200 Nov 11  2014 CASS_voice_NON_IMS_20141022_59764.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser        221 Dec  5  2013 CASS_voice_NON_IMS_20131205_02067.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      21832 Sep 15  2015 CASS_voice_NON_IMS_20150915_16466.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      21833 Dec 15  2014 CASS_voice_NON_IMS_20141215_68321.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      21834 Jun 17  2014 CASS_voice_NON_IMS_20140617_41284.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      21834 Nov 11  2014 CASS_voice_NON_IMS_20141001_57072.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      21836 Nov 25  2013 CASS_voice_NON_IMS_20131125_99511.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      21860 Nov 27  2013 CASS_voice_NON_IMS_20131127_00075.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      21861 Jul 28  2014 CASS_voice_NON_IMS_20140728_47098.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      37255 Jul  6  2015 CASS_voice_NON_IMS_20150706_01203.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      37258 May 29  2015 CASS_voice_NON_IMS_20150529_94299.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      37260 Nov 11  2014 CASS_voice_NON_IMS_20141021_59484.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      37264 Nov  6 15:00 CASS_voice_NON_IMS_20151106_27007.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      37272 Sep 15  2015 CASS_voice_NON_IMS_20150915_16414.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      37279 Jan 31  2014 CASS_voice_NON_IMS_20140131_16810.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      37280 Dec  8 10:00 CASS_voice_NON_IMS_20151208_33705.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      37280 Feb 26  2014 CASS_voice_NON_IMS_20140226_23747.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      37281 Apr 12  2014 CASS_voice_NON_IMS_20140412_31888.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      38657 Feb 15 15:20 CASS_voice_NON_IMS_20160215_47855.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      38657 Mar  3 14:40 CASS_voice_NON_IMS_20160303_51340.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      38658 Apr  4  2014 CASS_voice_NON_IMS_20140404_30777.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      38658 Nov 23 11:40 CASS_voice_NON_IMS_20151123_30547.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      38862 Apr 14  2014 CASS_voice_NON_IMS_20140414_32198.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      38862 Jun  1  2015 CASS_voice_NON_IMS_20150601_94805.cdr.ext
-rw-r--r--   1 ftphsbb    ftpuser      39020 Sep 12  2014 CASS_voice_NON_IMS_20140912_54032.cdr.ext

All of them are under /EDWH-DMT03/stgdata/HSBB/USAGE/ directory.

Anyway, your modified script give neat output indeed.

Thank you very much.
# 4  
Old 04-27-2016
Quote:
Originally Posted by aimy
Thanks Don.

This is some of the output:
Code:
-rw-r--r--   1 ftphsbb    ftpuser         89 Apr 24 21:40 CASS_voice_NON_IMS_20160424_61486.cdr.ext
... deleted to save space ...

All of them are under /EDWH-DMT03/stgdata/HSBB/USAGE/ directory.

Anyway, your modified script give neat output indeed.

Thank you very much.
Hi aimy,
You're welcome.

Is the output you showed us from your original script, or from my suggested modification? I'm not clear as to whether or not the "neat output" my suggestion provided solved your problem???

- Don
This User Gave Thanks to Don Cragun For This Post:
# 5  
Old 04-27-2016
Quote:
Originally Posted by Don Cragun
Hi aimy,
You're welcome.

Is the output you showed us from your original script, or from my suggested modification? I'm not clear as to whether or not the "neat output" my suggestion provided solved your problem???

- Don
Sorry to make you confused Sir.

Of course the neat output of your script solved my problem.

The one that I showed you is from the original script output. But what is wrong with the script I posted if you can detect it?

And by the way, doesn't that +10000 indicate the byte size? How could it be equivalent to the 512000 bytes?

Thank you.
# 6  
Old 04-27-2016
Quote:
Originally Posted by aimy
Sorry to make you confused Sir.

Of course the neat output of your script solved my problem.
It isn't a problem; I just wanted to be sure your problem had been fixed. I'm glad my suggestion worked.
Quote:
The one that I showed you is from the original script output. But what is wrong with the script I posted if you can detect it?
The size reported by the lstat() system call for a directory varies depending on filesystem type. Some file systems report the number of files contained in the directory; some report the space needed to hold the i-node numbers and the names of the files contained in the directory; some report the accumulated sizes of the files contained in the directory (which I would guess is what happened in your case); and some report other values. Furthermore, when a file is unlinked from a directory, the size of the directory might or might not shrink. By adding the -d option to ls, the output reported just the directories larger than the size you specified instead of those directories AND the contents of those directories.
Quote:
And by the way, doesn't that +10000 indicate the byte size? How could it be equivalent to the 512000 bytes?
Sorry. The 512000 was a typo. It should have been 5120000. If you look at the find man page's description of the size primary, you'll see that the size specified is the number of 512 byte blocks; not the number of bytes. If you want files with sizes larger than 10,000 bytes (instead of larger than 10,000 512-byte blocks), use -size 10000c.
Quote:
Thank you.
The change from -exec ls -lad {} \; (which causes find to invoke ls for each file found meeting your size limits) to -exec ls -lad + (which causes find to invoke ls with as many arguments as it can without overflowing ARG_MAX limits) just makes your script run a little faster (or, if there are a lot of files meeting your size limits, a lot faster).
This User Gave Thanks to Don Cragun For This Post:
# 7  
Old 04-27-2016
Thanks a lot again Mr. Don.

So meaning if I want get a list of all files that exceed 500MB, this should be the command right:
Code:
$ find 104857600 -xdev -size +104857600c -exec ls -lad {} +|sort -n -k 5,5 > LARGE.rst

But why did I got this error returned:
Code:
find: cannot stat 104857600

Thank you.

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Copy files recursively to one single directory

I need to copy a complete directory structure into a new location. But I want to have all files copied into one directory and leave out the directory structure. So all files must be placed in one directory. (4 Replies)
Discussion started by: ReneVL
4 Replies

2. Shell Programming and Scripting

Copy Specific Files Recursively

Is it possible to only copy selected files+its directories when you are copying recursively? find /OriginalFolder/* -type -d \{ -mtime 1 -o -mtime 2 \ } -exec cp -R {} /CopyTo/'hostname'__CopyTo/ \; -print From the above line, I want to only copy *txt and *ini files from /OriginalFolder/* ... (4 Replies)
Discussion started by: apacheLinux
4 Replies

3. Shell Programming and Scripting

Find specific files only in current directory...not sub directories AIX

Hi, I have to find specific files only in the current directory...not in the sub directories. But when I use Find command ... it searches all the files in the current directory as well as in the subdirectories. I am using AIX-UNIX machine.Please help.. I am using the below command. And i am... (2 Replies)
Discussion started by: aakishore
2 Replies

4. Shell Programming and Scripting

Find all the files under a specific directory and zip them into a single file.

Hi, Is there a way to find all the files from a specific location and then zip them into a single file, even if they are in multiple directories? (3 Replies)
Discussion started by: rudoraj
3 Replies

5. Shell Programming and Scripting

Need to find recursively all shell script in the /xyz directory

Pls. advise how to find or used grep recursively all shell script files. Some files doesnt have a .sh or .ksh extension name. find / -name "*" |xargs grep bin |grep sh ?? TIA (1 Reply)
Discussion started by: budz26
1 Replies

6. Shell Programming and Scripting

Using find in a directory containing large number of files

Hi All, I have searched this forum for related posts but could not find one that fits mine. I have a shell script which removes all the XML tags including the text inside the tags from some 4 million XML files. The shell script looks like this (MODIFIED): find . "*.xml" -print | while read... (6 Replies)
Discussion started by: shoaibjameel123
6 Replies

7. Shell Programming and Scripting

Recursively move directories along with files/specific files

I would like to transfer all files ending with .log from /tmp and to /tmp/archive (using find ) The directory structure looks like :- /tmp a.log b.log c.log /abcd d.log e.log When I tried the following command , it movies all the log files... (8 Replies)
Discussion started by: frintocf
8 Replies

8. Shell Programming and Scripting

delete files recursively in the specified directory

I have to write a shell script which can delete all the files and directories recursively inside the specified directory but should not delete the specified directory. Please some body help me in writing the script. (3 Replies)
Discussion started by: deepthi.s
3 Replies

9. Shell Programming and Scripting

How to find a specific files in a many directory

Dear All, Appreciate some help here. I have a log of report. It located in several directory as below: Directory: mysscpr1 mysscpr2 mysscpr3 my_scnpr4 In the directory it contain hundred of files. i need to find a specific files that contain 'invc2345' in the directory. How... (7 Replies)
Discussion started by: selamba_warrior
7 Replies

10. Shell Programming and Scripting

Recursively copy only specific files from a directory tree

Hi I am a shell-script newbie and am looking to synchronize certain files in two directory structures. Both these directory-trees are in CVS and so I dont want the CVS directory to be copied over. I want only .sh and .pl files in each subdirectory under these directory trees to be... (3 Replies)
Discussion started by: sharpsharkrocks
3 Replies

Featured Tech Videos