I have more than 200K files created in year 2017 under directory having size of 50GB.
I want to all these files in one shot.
Is there any faster option available with find command to delete all these file ?
In theory, this should be fairly straightforward. If you want to delete all files created from 2017 and beyond (i.e. also files from 2016, 2015, etc - anything older than 2017), then you could do something like this:
That would find all files older than 549 days (at the time of writing, the number of days since December 31st 2017) and remove them.
Now, if you really want to only remove files from 2017, and only 2017, then it's slightly more complicated, but not much so. Assuming your version of find supports it, this would do the trick:
This will very specifically remove files whose modification time is newer than 1st January 2017, but not newer than 1st January 2018 (in other words - files from 2017).
Hope this helps. Please test carefully before running this with a live rm command - swap it out for an ls or switch out the -exec for a -print first just to be sure it really is going to catch what you want, and only what you want, before turning this loose on your filesystem for real.
These 2 Users Gave Thanks to drysdalk For This Post:
Note that using:
will invoke the rm utility once for each of the 200K files you want to remove. That is always going to be relatively slow. If you change that command to:
then the find utility will group together a large number of files to be processed by each invocation of the rm utility and run considerably faster.
And, as stated before but not told how, don't use rm live until you have verified that find is correctly selecting the files you want to process. I would suggest using:
first, and, if that produces commands that have correctly identified the files you want to remove, rerun the command without the echo.
These 2 Users Gave Thanks to Don Cragun For This Post:
Thank you both for your solutions. I have tested below command in one test directory having few files and it worked fine.
I was wondering if I can use -delete option instead of rm to fasten the process. Not really sure how much difference will it make.
Moderator's Comments:
Please wrap your code in CODE tags. There is a tutorial here:
find /path/to/files -type f -newermt 2017-01-01 \! -newermt 2018-01-01 -delete;
You can use that but be aware that it is NOT supported by standard-conforming versions of find. If you use it you limit the usability of what you do to systems where the installed find command is the same as in your local system.
In general experience shows that it is better to sacrifice simplicity for portability, so my suggestion would be to use what Don Cragun and drysdalk suggested, even if what you found would work on your system. It is never too early to start healthy habits.
While I agree with what bakunin said in post #5 in this thread, note that the find -newermt primary is not in the standards either.
This is why you are always asked to tell us the operating system and shell you're using when posting a thread in the UNIX for Beginners Questions and Answers forum. If we don't know what environment you're using, you're likely to get suggestions that won't work in your environment. Since what drysdalk suggested works in your environment, we will assume that you're currently using a BSD-based or GNU-based find utility. If you had been using a traditional UNIX system, you'd have gotten an error about the unknown primary -newermt.
Since -newermt worked, -delete will also probably work and will probably be slightly faster than -exec rm -fv '{}' +. (The difference between the speed of -delete and -exec rm -fv '{}' + will be MUCH smaller than the difference between -exec rm -fv '{}' + and -exec rm -fv '{}' \;.)
Doing this in a manner just using standard features (and a little more accurately setting the start and end points) would require something more like:
Note, however, that the timestamps being used here accurately reflect the end points you want to the nearest nanosecond (which is the timestamp precision stored in a standard stat structure), but there is a reasonable likelihood that the last record entered into a logfile at the end of a year will not actually be written to that logfile until a few milliseconds (or even seconds) into the next year. And, that last write into the logfile will determine the last modified timestamp of that file; not the string specifying the date in the name of the logfile. (And this problem affects all of the methods we have talked about when using find -newer and -newermt. So beware that find may be doing exactly what you asked it to do, but it might not be exactly what you wanted it to do.
And, if anyone edits a logfile, the modification time of that file may be years later than the name of the file would indicate!
I hope this further clarification helps you understand the limitations on what you're doing.
This User Gave Thanks to Don Cragun For This Post:
Hello Gurus,
I am facing one issue to get a file for a specific time. There are about 300 files created between 6.30 pm to 7.15 pm everyday.
Now I wanted only the file which is created on 6.45pm. No other files required.
I have used "find" command to get the files, but not getting the expected... (3 Replies)
Hi
My directory structure is as below.
dir1, dir2, dir3
I have the list of files to be deleted in the below path as below.
/staging/retain_for_2years/Cleanup/log $ ls -lrt
total 0
drwxr-xr-x 2 nobody nobody 256 Mar 01 16:15 01-MAR-2015_SPDBS2
drwxr-xr-x 2 root ... (2 Replies)
Hi All,
how can i delete files from my Unix directory on the basis of year, i have files from 2001 to till 2014, but from their, i have to delete only 2013 file.Below is my file name
rwxrwxrwx 1 guopt users 5169 Jul 12 00:30 grt592_20130712003000.SAP
Thanks
Kki (2 Replies)
Hi,
I want to display the file names and the record count for the files in the 2nd column for the files created today.
i have written the below command which is listing the file names. but while piping the above command to the wc -l command
its not working for me.
ls -l... (5 Replies)
Hi All,
I want to login to a remote server using FTP command and then check for files older than 1 year and delete those files.
Please let me know how can i achieve this using Unix Commands.
Thanks in Advance, (10 Replies)
Hi
I have lot of files and subdirectories inside a directory which are created in the years 2006, 2007, 2008, 2009 and 2010.
I want to delete all the files and subdirectories belonging to the year 2006 alone.
How can I do that ? (5 Replies)
can someone provide a command to delete files by year? I have several files created last year 2009.
Im trying to list first ls -lrt | grep '/2009/ {print $10}' by it returns no result.
Pls advise (2 Replies)
There is a system logging a huge amount of data and we need to delete some of the older logs .I mean the files that are created before one week from today. Here is a listing of files that are sitting there:
/usr/WebSphere/AppServer/logs
# ls -l
-rw-r--r-- 1 root system 3740694 May... (5 Replies)
Hi,
Would like to ask, which command is used to list all the files for specific date (says 1st May) and its size, for all files (including its subdirectory), in a mounted NFS disk to HP-UX.
I would like to check for the total files came into my disk on 1st May.
Very much appreciating your... (2 Replies)
I'm trying to delete files that were created/modified in the year 2005 that we compressed and have the .Z extension on them. I tried using the awk utility but the syntax is incorrect. I don't know how to use a wildcard to capture all the compressed files. Here's the code I used
( ls -lR |... (5 Replies)