Need to delete large set of files (i.e) close to 100K from a directory based on the input file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need to delete large set of files (i.e) close to 100K from a directory based on the input file
# 36  
Old 08-24-2012
Thank you for the post and is there a way can we only get the output file and error_file but rm operation will be done later.
Code:
while read ID
do
        echo "${ID}.jpg"
        echo "thumbnail/${ID}-tn.jpg"
done < inputfile | xargs rm 2> errlog

So where ID is my input file and i dont want rm operation here and i want to capture the output file and then the files not found in the error_file. Belwo code works perfect to get the output file (i.e) which consists all the file names to be deleted, but the same way can i get the error_file (i.e) files not found.

Code:
while read ID
do
        echo "${ID}.jpg"
        echo "thumbnail/${ID}-tn.jpg"
done < ID >list_file

---------- Post updated 08-24-12 at 11:51 AM ---------- Previous update was 08-23-12 at 05:05 PM ----------

Is there a way that we can capture what files have been deleted from the below while loop?

Code:
while read ID
do
        echo "${ID}.jpg"
        echo "thumbnail/${ID}-tn.jpg"
done < inputfile | xargs rm 2> errlog

---------- Post updated at 11:52 AM ---------- Previous update was at 11:51 AM ----------

Code:
while read ID
do
        echo "${ID}.jpg"
        echo "files/${ID}-tn.jpg"
done < ID | xargs rm 2> errlog

Input file here is ID
# 37  
Old 08-25-2012
Quote:
Originally Posted by prash358
Thank you for the post and is there a way can we only get the output file and error_file but rm operation will be done later.
Not unless you have a psychic computer. How should it know which ones don't get deleted, before it's deleted them...?

Last edited by Corona688; 08-25-2012 at 04:57 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to compare files in 2 folders and delete the large file

Hello, my first thread here. I've been searching and fiddling around for about a week and I cannot find a solution.:confused: I have been converting all of my home videos to HEVC and sometimes the files end up smaller and sometimes they don't. I am currently comparing all the video files... (5 Replies)
Discussion started by: Josh52180
5 Replies

2. Shell Programming and Scripting

If no input then set directory to current

Hi GUys I have a script where i am counting the number of files that the user has read permissions to in a folder . They are asked for this folder at the start if they dont enter anything then i want to be able to do the search on the current directory, can you help me with that part please... (25 Replies)
Discussion started by: johnnybananas
25 Replies

3. Shell Programming and Scripting

Script needed to delete to the list of files in a directory based on last created & delete them

Hi My directory structure is as below. dir1, dir2, dir3 I have the list of files to be deleted in the below path as below. /staging/retain_for_2years/Cleanup/log $ ls -lrt total 0 drwxr-xr-x 2 nobody nobody 256 Mar 01 16:15 01-MAR-2015_SPDBS2 drwxr-xr-x 2 root ... (2 Replies)
Discussion started by: prasadn
2 Replies

4. Shell Programming and Scripting

Sed: Splitting A large File into smaller files based on recursive Regular Expression match

I will simplify the explaination a bit, I need to parse through a 87m file - I have a single text file in the form of : <NAME>house........ SOMETEXT SOMETEXT SOMETEXT . . . . </script> MORETEXT MORETEXT . . . (6 Replies)
Discussion started by: sumguy
6 Replies

5. Shell Programming and Scripting

Help needed - Split large file into smaller files based on pattern match

Help needed urgently please. I have a large file - a few hundred thousand lines. Sample CP START ACCOUNT 1234556 name 1 CP END ACCOUNT CP START ACCOUNT 2224444 name 1 CP END ACCOUNT CP START ACCOUNT 333344444 name 1 CP END ACCOUNT I need to split this file each time "CP START... (7 Replies)
Discussion started by: frustrated1
7 Replies

6. Shell Programming and Scripting

Script to delete files with an input for directories and an input for path/file

Hello, I'm trying to figure out how best to approach this script, and I have very little experience, so I could use all the help I can get. :wall: I regularly need to delete files from many directories. A file with the same name may exist any number of times in different subdirectories.... (3 Replies)
Discussion started by: *ShadowCat*
3 Replies

7. Shell Programming and Scripting

Splitting large file into multiple files in unix based on pattern

I need to write a shell script for below scenario My input file has data in format: qwerty0101TWE 12345 01022005 01022005 datainala alanfernanded 26 qwerty0101mXZ 12349 01022005 06022008 datainalb johngalilo 28 qwerty0101TWE 12342 01022005 07022009 datainalc hitalbert 43 qwerty0101CFG 12345... (19 Replies)
Discussion started by: jimmy12
19 Replies

8. Shell Programming and Scripting

Split a file into multiple files based on the input pattern

I have a file with lines something like. ...... 123_start ...... ....... 123_end .... ..... 456_start ...... ..... 456_end .... ..... 789_start .... .... 789_end (6 Replies)
Discussion started by: abinash
6 Replies

9. Shell Programming and Scripting

Sheel script to Delete a set of files from a given directory

I have a file <filestodelete> containing names of files to to be deleted from a directory <filesstore>. I want a script file which accptes the <filestodelete> and also the location of the directory(<filestore>) and deletes all files matching. Thanks in Advance.. (3 Replies)
Discussion started by: VardhiniVenkat
3 Replies

10. Shell Programming and Scripting

i want to delete a file based on existing file in a directory

hi i am having four files in a directory.like 1)sampleRej 2)exampleRej 3)samplemain 4)examplemain my requirement is i have to search for the rejected files (sampleRej,exampleRej) in a directory.if these files in that directory then i have to delete the main files... (3 Replies)
Discussion started by: srivsn
3 Replies
Login or Register to Ask a Question