The UNIX and Linux Forums  
Hello and Welcome from United States to the UNIX and Linux Forums! Thank You for Visiting and Joining Our Global Community.

Go Back   The UNIX and Linux Forums > Top Forums > UNIX for Dummies Questions & Answers
.
google unix.com



UNIX for Dummies Questions & Answers If you're not sure where to post a UNIX or Linux question, post it here. All UNIX and Linux newbies welcome !!

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Traverse a flatfile and check for errors aravindc Shell Programming and Scripting 1 04-17-2008 02:56 AM
Using File Descriptors, traverse a list MaestroRage UNIX for Dummies Questions & Answers 2 02-12-2008 07:05 PM
Traverse catalogs baghera Shell Programming and Scripting 3 08-30-2007 09:29 AM
Traverse Directory Tree for backup srmadab Shell Programming and Scripting 1 09-11-2006 06:16 PM
Unix File System performance with large directories dive UNIX for Dummies Questions & Answers 3 03-12-2004 05:31 PM

Closed Thread
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Bulgarian Greek Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 12-21-2005
sreedharange sreedharange is offline
Registered User
  
 

Join Date: Dec 2005
Posts: 2
Fastest way to traverse through large directories

Hi!

I have thousands of sub-directories, and hundreds of thousands of files in them. What is the fast way to find out which files are older than a certain date? Is the "find" command the fastest? Or is there some other way?

Right now I have a C script that traverses through and checks each file. But it is taking forever to run! Can somebody tell me a better way?

Thanks
  #2 (permalink)  
Old 12-21-2005
Perderabo's Avatar
Perderabo Perderabo is offline Forum Staff  
Unix Daemon
  
 

Join Date: Aug 2001
Location: Ashburn, Virginia
Posts: 9,111
This is going to take a while no matter what you do. "find" should be able to do at close to the top possible speed. A custom C program should be able to beat "find", but not by much. I would not csh for anything.
  #3 (permalink)  
Old 12-21-2005
Corrail Corrail is offline
Registered User
  
 

Join Date: Oct 2005
Posts: 50
Grep?

Would grep be faster than find or no? I dont know much about any of it but just a thought. if you grepped for 10-2-04 or older *not exactly sure how to do that* but shouldnt turn up files fast?
  #4 (permalink)  
Old 12-21-2005
jim mcnamara jim mcnamara is offline Forum Staff  
...@...
  
 

Join Date: Feb 2004
Location: NM
Posts: 5,719
Corrail -
He wants a -ctime or an -mtime older than a certain date. grep doesn't do file date/times
  #5 (permalink)  
Old 12-21-2005
Corrail Corrail is offline
Registered User
  
 

Join Date: Oct 2005
Posts: 50
Thanks

Ok thanks for clearing that up. I am working on teaching myself unix and grep/awk/sed and different things like that so i bring up questions on here of possible solutions to problems and then ppl can correct me and explain why and it helps ALOT. So thank you.
  #6 (permalink)  
Old 12-21-2005
sreedharange sreedharange is offline
Registered User
  
 

Join Date: Dec 2005
Posts: 2
Hi Guys,

Thanks for your input...

From what I hear here, I guess that the way I've taken is the best. The C program is pretty fast, but then, with the hundreds of thousands of files, I'm looking for greased lightning. Every day about 20,000 files get added. I'm supposed to archive the old files (those older than 60 days). Sixty days means about 1.2 million files, with the rate growing as time goes by.

I guess one way to beat the large number of files is to go deeper down the file structure. But then, I'll have to run the application multiple times, one for each folder path that I select.

Just to complete the picture, the list of files older than 60 days is then fed to NetBackup (that archiving application from Symantec). NetBackup then moves the files to tape. I use Netbackup because it becomes easy to restore particular files whenever I need them.
Closed Thread

Bookmarks

Tags
mtime

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT -4. The time now is 03:08 PM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0