Need help with listing file name and modified date on a huge directory


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Need help with listing file name and modified date on a huge directory
# 1  
Need help with listing file name and modified date on a huge directory

hi,

We have a huge directory that ha 5.1 Million files in it. We are trying to get the file name and modified timestamp of the most recent 3 years from this huge directory for a migration project.

However, the ls command (background process) to list the file names and timestamp is running for more than 24 hrs and it aborted after sometime.

Is there an efficient way to get the file names and modified timestamp in such huge directory in a relatively quicker time?

Appreciate any options/feedback. Thanks.
# 2  
You don't mention the OS you are using. If you have a GNU find the -printf option will assist with this task.

You could try:

Code:
find . -printf "%TY-%Tm-%Td %TH:%TM %f\n"

This will give modification time and files name in this format:
Code:
2019-08-21 13:37 .bashrc
2019-10-24 13:09 .bash_history
2019-08-21 13:44 .bash_profile

Other info like filesize, permissions, etc are also available using the -printf option.

As far as feedback goes if you've been managing this huge directory on a system for a while you probably realize it's not a good idea. Many simple commands, shell auto completes etc. will take a long time and usually require processes to be killed. If your migrating this I would consider a sub folder layout, this could be derived from the filename created date etc.

Last edited by Chubler_XL; 10-25-2019 at 04:43 PM..
This User Gave Thanks to Chubler_XL For This Post:
# 3  
Hi Chuber,

thanks for the quick response. Its a SUSE Linux.

The command used is:
find -newermt "oct 1, 2016" -ls
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #873
Difficulty: Medium
Self-modifying code is easier to read and maintain because the instructions in the source program listing are always the instructions that will be executed.
True or False?

10 More Discussions You Might Find Interesting

1. OS X (Apple)

Arrange file by modified date

Hi, Am performing a find based on filename and result can contain multiple files being found Let's say my find command is find /Archive -f -name 12345.pdf and result of find command is /Archive/Folder A/12345.pdf /Archive/Folder B/12345.pdf please note white space in folder names I... (2 Replies)
Discussion started by: gigagigosu
2 Replies

2. AIX

How do I display a file's last modified date?

I'm using a script that I need to get a file's "last modified date" in a format like 01:51:14 PM. We are running on AIX 6.1.0.0. I can't seem to find the right command parameters. Help! (4 Replies)
Discussion started by: mattadams1983
4 Replies

3. UNIX for Dummies Questions & Answers

To find the latest modified file in a directory

I am trying to fetch the latest modified file from a directory using the command find . -type f -exec ls -lt \{\} \+ | head | awk '{print $9}' After the O/P, I get the below mentioned error and the command doesnt terminate at all. find: ls terminated by signal 13 find: ls terminated by... (2 Replies)
Discussion started by: Sree10
2 Replies

4. Shell Programming and Scripting

current date modified file

Hi , In my directory , i have many days file but i want to see all those which are of todays date. i tried this but it gives all the files mtime -0 |ls -ltr I tried the below option as well. 19635 find -iname "*.LOG" -mtime 19636 ls -ltr *.LOG -mtime -1 19637 ls -ltr *.LOG... (7 Replies)
Discussion started by: guddu_12
7 Replies

5. Shell Programming and Scripting

Test if a file has a last modified date of within the last 24 hours

Hi there Im trying to find a way to test whether the last modified time is older than 1 day or not so #!/bin/bash if ; then $TOUCHED = "recently" else $TOUCHED = "not so recently" fi ive seen loads of posts where people are using find and the -mtime property but i... (2 Replies)
Discussion started by: rethink
2 Replies

6. UNIX for Advanced & Expert Users

Finding the modified date time of a file

Hi, I am new bie to Unix. Might be a simple question I am asking. I want to find the last modified time of a file and find the difference between the currrent time and the last modified time. Appreciate, if someone can throw some light on what commands can be used. Cheers, James (2 Replies)
Discussion started by: JamesJoe
2 Replies

7. UNIX for Dummies Questions & Answers

Number of files in a directory modified on a date

Hi How to list all the files in a directory that are modified on a particular date? Also need to know the count,i.e number of files modified on a particular date. Thanks Ashok (1 Reply)
Discussion started by: ashok.k
1 Replies

8. UNIX for Dummies Questions & Answers

How to get the latest modified file name in /home directory?

I only know how to list all sub-directories or files in specified directory. I don't know how to order them by modified date, furthermore, I don't know how to get the top one file in the sorted list. Wish you can do me a favor. Thanks in advance! (3 Replies)
Discussion started by: crest.boy
3 Replies

9. UNIX for Advanced & Expert Users

how to open a last modified file in a directory

how to directly open a last modified file in a directory? Please help (5 Replies)
Discussion started by: apsprabhu
5 Replies

10. UNIX for Dummies Questions & Answers

How do I get the last modified date of a file?

I am trying to load a group of files and their last dates modified into a text file that will in turn be used with SQL*Loader to load these files into Oracle. I am using a *.ksh script. I am getting the name of the file in by using the following: for file_ext in 'cat loaddir.ext'; do find... (2 Replies)
Discussion started by: akpopa
2 Replies

Featured Tech Videos