Delete all files created in specific year


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Delete all files created in specific year
# 1  
Old 07-03-2019
Delete all files created in specific year

I have more than 200K files created in year 2017 under directory having size of 50GB.
I want to all these files in one shot.
Is there any faster option available with find command to delete all these file ?
# 2  
Old 07-03-2019
Hi,

In theory, this should be fairly straightforward. If you want to delete all files created from 2017 and beyond (i.e. also files from 2016, 2015, etc - anything older than 2017), then you could do something like this:

Code:
find /path/to/files -type f -mtime +549 -exec rm -fv \{\} \;

That would find all files older than 549 days (at the time of writing, the number of days since December 31st 2017) and remove them.

Now, if you really want to only remove files from 2017, and only 2017, then it's slightly more complicated, but not much so. Assuming your version of find supports it, this would do the trick:

Code:
find /path/to/files -type f -newermt 2017-01-01 \! -newermt 2018-01-01 -exec rm -fv \{\} \;

This will very specifically remove files whose modification time is newer than 1st January 2017, but not newer than 1st January 2018 (in other words - files from 2017).

Hope this helps. Please test carefully before running this with a live rm command - swap it out for an ls or switch out the -exec for a -print first just to be sure it really is going to catch what you want, and only what you want, before turning this loose on your filesystem for real.
These 2 Users Gave Thanks to drysdalk For This Post:
# 3  
Old 07-04-2019
Note that using:
Code:
find /path/to/files -type f -newermt 2017-01-01 \! -newermt 2018-01-01 -exec rm -fv \{\} \;

will invoke the rm utility once for each of the 200K files you want to remove. That is always going to be relatively slow. If you change that command to:
Code:
find /path/to/files -type f -newermt 2017-01-01 \! -newermt 2018-01-01 -exec rm -fv '{}' +

then the find utility will group together a large number of files to be processed by each invocation of the rm utility and run considerably faster.

And, as stated before but not told how, don't use rm live until you have verified that find is correctly selecting the files you want to process. I would suggest using:
Code:
find /path/to/files -type f -newermt 2017-01-01 \! -newermt 2018-01-01 -exec echo rm -fv '{}' +

first, and, if that produces commands that have correctly identified the files you want to remove, rerun the command without the echo.
These 2 Users Gave Thanks to Don Cragun For This Post:
# 4  
Old 07-04-2019
@drysdalk @Don Cragun

Thank you both for your solutions. I have tested below command in one test directory having few files and it worked fine.
Code:
find /path/to/files -type f -newermt 2017-01-01 \! -newermt 2018-01-01 -exec rm -fv '{}' +

I was wondering if I can use -delete option instead of rm to fasten the process. Not really sure how much difference will it make.
Code:
find /path/to/files -type f -newermt 2017-01-01 \! -newermt 2018-01-01 -delete;

Moderator's Comments:
Mod Comment Please wrap your code in CODE tags. There is a tutorial here:

Last edited by rbatte1; 07-08-2019 at 07:23 AM..
# 5  
Old 07-04-2019
Quote:
Originally Posted by sp23029
find /path/to/files -type f -newermt 2017-01-01 \! -newermt 2018-01-01 -delete;
You can use that but be aware that it is NOT supported by standard-conforming versions of find. If you use it you limit the usability of what you do to systems where the installed find command is the same as in your local system.

In general experience shows that it is better to sacrifice simplicity for portability, so my suggestion would be to use what Don Cragun and drysdalk suggested, even if what you found would work on your system. It is never too early to start healthy habits.

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
# 6  
Old 07-04-2019
While I agree with what bakunin said in post #5 in this thread, note that the find -newermt primary is not in the standards either.

This is why you are always asked to tell us the operating system and shell you're using when posting a thread in the UNIX for Beginners Questions and Answers forum. If we don't know what environment you're using, you're likely to get suggestions that won't work in your environment. Since what drysdalk suggested works in your environment, we will assume that you're currently using a BSD-based or GNU-based find utility. If you had been using a traditional UNIX system, you'd have gotten an error about the unknown primary -newermt.

Since -newermt worked, -delete will also probably work and will probably be slightly faster than -exec rm -fv '{}' +. (The difference between the speed of -delete and -exec rm -fv '{}' + will be MUCH smaller than the difference between -exec rm -fv '{}' + and -exec rm -fv '{}' \;.)

Doing this in a manner just using standard features (and a little more accurately setting the start and end points) would require something more like:
Code:
trap 'rm -rf $$.start $$.end' EXIT
touch -d 2016-12-31T23:59:59.999999999 $$.start
touch -d 2017-12-31T23:59:59.999999999 $$.end
find /path/to/files -type f -newer $$.start ! -newer $$.end -exec rm -rf '{}' +

Note, however, that the timestamps being used here accurately reflect the end points you want to the nearest nanosecond (which is the timestamp precision stored in a standard stat structure), but there is a reasonable likelihood that the last record entered into a logfile at the end of a year will not actually be written to that logfile until a few milliseconds (or even seconds) into the next year. And, that last write into the logfile will determine the last modified timestamp of that file; not the string specifying the date in the name of the logfile. (And this problem affects all of the methods we have talked about when using find -newer and -newermt. So beware that find may be doing exactly what you asked it to do, but it might not be exactly what you wanted it to do.

And, if anyone edits a logfile, the modification time of that file may be years later than the name of the file would indicate!

I hope this further clarification helps you understand the limitations on what you're doing.
This User Gave Thanks to Don Cragun For This Post:
# 7  
Old 07-05-2019
Thanks Don Cragun for clarification. This is really helpful .Cheers !!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Files created on specific time

Hello Gurus, I am facing one issue to get a file for a specific time. There are about 300 files created between 6.30 pm to 7.15 pm everyday. Now I wanted only the file which is created on 6.45pm. No other files required. I have used "find" command to get the files, but not getting the expected... (3 Replies)
Discussion started by: pokhraj_d
3 Replies

2. Shell Programming and Scripting

Script needed to delete to the list of files in a directory based on last created & delete them

Hi My directory structure is as below. dir1, dir2, dir3 I have the list of files to be deleted in the below path as below. /staging/retain_for_2years/Cleanup/log $ ls -lrt total 0 drwxr-xr-x 2 nobody nobody 256 Mar 01 16:15 01-MAR-2015_SPDBS2 drwxr-xr-x 2 root ... (2 Replies)
Discussion started by: prasadn
2 Replies

3. Shell Programming and Scripting

Delete files using year

Hi All, how can i delete files from my Unix directory on the basis of year, i have files from 2001 to till 2014, but from their, i have to delete only 2013 file.Below is my file name rwxrwxrwx 1 guopt users 5169 Jul 12 00:30 grt592_20130712003000.SAP Thanks Kki (2 Replies)
Discussion started by: kki
2 Replies

4. Shell Programming and Scripting

Listing the file name and no of records in each files for the files created on a specific day

Hi, I want to display the file names and the record count for the files in the 2nd column for the files created today. i have written the below command which is listing the file names. but while piping the above command to the wc -l command its not working for me. ls -l... (5 Replies)
Discussion started by: Showdown
5 Replies

5. Shell Programming and Scripting

Delete files older than 1 year through FTP

Hi All, I want to login to a remote server using FTP command and then check for files older than 1 year and delete those files. Please let me know how can i achieve this using Unix Commands. Thanks in Advance, (10 Replies)
Discussion started by: HemaV
10 Replies

6. Shell Programming and Scripting

Delete all the files and subdirectories for the year 2006

Hi I have lot of files and subdirectories inside a directory which are created in the years 2006, 2007, 2008, 2009 and 2010. I want to delete all the files and subdirectories belonging to the year 2006 alone. How can I do that ? (5 Replies)
Discussion started by: samsungsamsung
5 Replies

7. UNIX for Dummies Questions & Answers

Delete files by year

can someone provide a command to delete files by year? I have several files created last year 2009. Im trying to list first ls -lrt | grep '/2009/ {print $10}' by it returns no result. Pls advise (2 Replies)
Discussion started by: lhareigh890
2 Replies

8. Shell Programming and Scripting

Delete files created before specific date.

There is a system logging a huge amount of data and we need to delete some of the older logs .I mean the files that are created before one week from today. Here is a listing of files that are sitting there: /usr/WebSphere/AppServer/logs # ls -l -rw-r--r-- 1 root system 3740694 May... (5 Replies)
Discussion started by: moustafashawky
5 Replies

9. Filesystems, Disks and Memory

How to list files with specific created date

Hi, Would like to ask, which command is used to list all the files for specific date (says 1st May) and its size, for all files (including its subdirectory), in a mounted NFS disk to HP-UX. I would like to check for the total files came into my disk on 1st May. Very much appreciating your... (2 Replies)
Discussion started by: Draculla
2 Replies

10. UNIX for Dummies Questions & Answers

delete compressed files from year 2005

I'm trying to delete files that were created/modified in the year 2005 that we compressed and have the .Z extension on them. I tried using the awk utility but the syntax is incorrect. I don't know how to use a wildcard to capture all the compressed files. Here's the code I used ( ls -lR |... (5 Replies)
Discussion started by: igidttam
5 Replies
Login or Register to Ask a Question