Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Need to delete large set of files (i.e) close to 100K from a directory based on the input file
Top Forums Shell Programming and Scripting Need to delete large set of files (i.e) close to 100K from a directory based on the input file Post 302691033 by prash358 on Friday 24th of August 2012 12:52:45 PM
Old 08-24-2012
Thank you for the post and is there a way can we only get the output file and error_file but rm operation will be done later.
Code:
while read ID
do
        echo "${ID}.jpg"
        echo "thumbnail/${ID}-tn.jpg"
done < inputfile | xargs rm 2> errlog

So where ID is my input file and i dont want rm operation here and i want to capture the output file and then the files not found in the error_file. Belwo code works perfect to get the output file (i.e) which consists all the file names to be deleted, but the same way can i get the error_file (i.e) files not found.

Code:
while read ID
do
        echo "${ID}.jpg"
        echo "thumbnail/${ID}-tn.jpg"
done < ID >list_file

---------- Post updated 08-24-12 at 11:51 AM ---------- Previous update was 08-23-12 at 05:05 PM ----------

Is there a way that we can capture what files have been deleted from the below while loop?

Code:
while read ID
do
        echo "${ID}.jpg"
        echo "thumbnail/${ID}-tn.jpg"
done < inputfile | xargs rm 2> errlog

---------- Post updated at 11:52 AM ---------- Previous update was at 11:51 AM ----------

Code:
while read ID
do
        echo "${ID}.jpg"
        echo "files/${ID}-tn.jpg"
done < ID | xargs rm 2> errlog

Input file here is ID
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

i want to delete a file based on existing file in a directory

hi i am having four files in a directory.like 1)sampleRej 2)exampleRej 3)samplemain 4)examplemain my requirement is i have to search for the rejected files (sampleRej,exampleRej) in a directory.if these files in that directory then i have to delete the main files... (3 Replies)
Discussion started by: srivsn
3 Replies

2. Shell Programming and Scripting

Sheel script to Delete a set of files from a given directory

I have a file <filestodelete> containing names of files to to be deleted from a directory <filesstore>. I want a script file which accptes the <filestodelete> and also the location of the directory(<filestore>) and deletes all files matching. Thanks in Advance.. (3 Replies)
Discussion started by: VardhiniVenkat
3 Replies

3. Shell Programming and Scripting

Split a file into multiple files based on the input pattern

I have a file with lines something like. ...... 123_start ...... ....... 123_end .... ..... 456_start ...... ..... 456_end .... ..... 789_start .... .... 789_end (6 Replies)
Discussion started by: abinash
6 Replies

4. Shell Programming and Scripting

Splitting large file into multiple files in unix based on pattern

I need to write a shell script for below scenario My input file has data in format: qwerty0101TWE 12345 01022005 01022005 datainala alanfernanded 26 qwerty0101mXZ 12349 01022005 06022008 datainalb johngalilo 28 qwerty0101TWE 12342 01022005 07022009 datainalc hitalbert 43 qwerty0101CFG 12345... (19 Replies)
Discussion started by: jimmy12
19 Replies

5. Shell Programming and Scripting

Script to delete files with an input for directories and an input for path/file

Hello, I'm trying to figure out how best to approach this script, and I have very little experience, so I could use all the help I can get. :wall: I regularly need to delete files from many directories. A file with the same name may exist any number of times in different subdirectories.... (3 Replies)
Discussion started by: *ShadowCat*
3 Replies

6. Shell Programming and Scripting

Help needed - Split large file into smaller files based on pattern match

Help needed urgently please. I have a large file - a few hundred thousand lines. Sample CP START ACCOUNT 1234556 name 1 CP END ACCOUNT CP START ACCOUNT 2224444 name 1 CP END ACCOUNT CP START ACCOUNT 333344444 name 1 CP END ACCOUNT I need to split this file each time "CP START... (7 Replies)
Discussion started by: frustrated1
7 Replies

7. Shell Programming and Scripting

Sed: Splitting A large File into smaller files based on recursive Regular Expression match

I will simplify the explaination a bit, I need to parse through a 87m file - I have a single text file in the form of : <NAME>house........ SOMETEXT SOMETEXT SOMETEXT . . . . </script> MORETEXT MORETEXT . . . (6 Replies)
Discussion started by: sumguy
6 Replies

8. Shell Programming and Scripting

Script needed to delete to the list of files in a directory based on last created & delete them

Hi My directory structure is as below. dir1, dir2, dir3 I have the list of files to be deleted in the below path as below. /staging/retain_for_2years/Cleanup/log $ ls -lrt total 0 drwxr-xr-x 2 nobody nobody 256 Mar 01 16:15 01-MAR-2015_SPDBS2 drwxr-xr-x 2 root ... (2 Replies)
Discussion started by: prasadn
2 Replies

9. Shell Programming and Scripting

If no input then set directory to current

Hi GUys I have a script where i am counting the number of files that the user has read permissions to in a folder . They are asked for this folder at the start if they dont enter anything then i want to be able to do the search on the current directory, can you help me with that part please... (25 Replies)
Discussion started by: johnnybananas
25 Replies

10. Shell Programming and Scripting

Script to compare files in 2 folders and delete the large file

Hello, my first thread here. I've been searching and fiddling around for about a week and I cannot find a solution.:confused: I have been converting all of my home videos to HEVC and sometimes the files end up smaller and sometimes they don't. I am currently comparing all the video files... (5 Replies)
Discussion started by: Josh52180
5 Replies

LEARN ABOUT DEBIAN

mcpmv

MRENAME(1)						      General Commands Manual							MRENAME(1)

NAME

       mrename - program to rename files

SYNOPSIS

       mrename 'pattern' prefix [option]

DESCRIPTION

       This  manual  page  documents  briefly the mrename command.  This manual page was written for the Debian GNU/Linux distribution because the
       original program does not have a manual page.

       mrename is a tool for easy and automatic renaming of many files.  The 'pattern' is the pattern to search files to rename (quoted  to  avoid
       that  bash  resolve  it),  and prefix is the prefix that will be added to the name of each file. The two alternative options for copying or
       moving files in the new name are explained below.  All parameters are needed, and you have to stay and launch the script in the same direc-
       tory of the files to be renamed. The program should be able to write in this directory.

OPTIONS

       There are only the following three options.

       -c     The option -c will copy each file with the new filename.

       -m     The option -m will move each file in the new filename.

       -h     Display help.

EXAMPLE

       If  you	have  a  directory  with two jpeg images prof.jpg and forp.jpg and you want to add them a prefix like item0, item1 etc..  (that is
       item0prof.jpg, item1forp.jpg etc..) do this:

	  cd /path/to/the/images
	  mrename '*.jpg' item -c

       to copy each matching file into another with the new name

	  mrename '*.jpg' item -m

       to rename each file without keeping a copy with the previous name

Word-Wide-Web:
       http://alfalinux.sourceforge.net/mrename.php3

AUTHOR
:
       Giancarlo -rofus- Erra
       e-mail: rofus@mindless.com

       This manual page was written by Dr. Guenter Bechly <gbechly@debian.org>, for the Debian GNU/Linux system (but may be used by others). It is
       distributed under the GPL just like mrename itself.

								 October  22, 2000							MRENAME(1)
All times are GMT -4. The time now is 11:18 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy