Delete files whose file names are listed in a .txt file Post: 302751665

Sponsored Content

Top Forums UNIX for Dummies Questions & Answers Delete files whose file names are listed in a .txt file Post 302751665 by rbatte1 on Friday 4th of January 2013 09:39:31 AM

01-04-2013

Moderator

kamaldev,

itkamaraj has drawn you the skeleton of a solution. You will need to ftp one file at a time with the loop. If you test that the file has been delivered successfully, then you can remove it on the local server. If you try it how you have it at the moment, you will put every file every time you read a line from you input file, so one file will be put once (no worries there), but two will be put twice (hmmmm) or 10 files will be put ten times EACH. if the files are big then you could be thrashing things about repetitively quite a bit.

Do you really want to only put files with a dot in their name? From unix, an "mput *" is all that is required to state all files, but you are also sending every file with the mput so you may duplicate the FTP later on if you run the job again.

Capture the output from the FTP command and look for the success message in your script. The FTP command may still exit with return code zero (i.e. no error) even if the file fails to be written, because the error is remote and the FTP command itself works. If you schedule this with SFTP, then you would get a non-zero return code to test, but it depends on you having the SFTP client & server tools installed.

Something like this perhaps:-

Code:

while read filename
do
     ftp -i -n << EOF  > ftp_log.txt
        open 192.168.1.75
        user loguser PASSWORD
        prompt
        cd \logs_finacle
        put ${filename}
        EOF

     if [ `grep -c "^226 " ftp_log.txt` -eq 1 ]
     then
         rm ${filename}
     else
         echo "Failed to deliver file ${filename}"
         exit 99
     fi
done < /usr/ftprm.txt

Be careful to ensure that the EOF line is only indented with tab characters. Using a space will mean that the block of text passed to the FTP command will be incomplete and the FTP will fail with a strange message.

You will need to run an FTP manually to check what the success message you should get is. Mine is prefixed with the line starting "226", so that's what I've used. The ^ states that this is the beginning of the line.

I hope that this is helpful. It is a lot to take in at once, so ask if you need some clarification - or should if I've made an error. I would never profess to be perfect!

Robin
Liverpool/Blackburn
UK

rbatte1

View Public Profile for rbatte1

Visit rbatte1's homepage!

Find all posts by rbatte1

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How can I delete files using a file that containt path and names?

Recentily i receive virus ninda and my network was files *.eml. I find all *.eml with: find / -name *.eml -print > virus Virus has the path and name of the file,so, How can i delete all *.eml? Thanks

2. Shell Programming and Scripting

[Urgent]how to print the file names into a txt file???

HI, I have a folder with some 120 files...i just want to print all the file filenames(not the content or anything else) onto a file say .txt. please help me with this command Thanks a lot.

3. UNIX for Dummies Questions & Answers

recursive delete files from txt file or?

i have a txt file of image names that have to be deleted in pwd how can i use the txt file to delete the files in pwd and is it possible?--i might be able to import the txt files into a spreadsheet ahd same it as a csv file. i want it to be done recursive lly --what i mean is teh sysem goes thru...

4. Shell Programming and Scripting

Finding consecutive numbers in version names on a txt file

Hi all. I have a directory which contains files that can be versioned. All the files are named according to a pattern like this: TEXTSTRING1-001.EXTENSION TEXTSTRING2-001.EXTENSION TEXTSTRING3-001.EXTENSION ... TEXTSTRINGn-001.EXTENSION If a file is versioned, a file called ...

5. Shell Programming and Scripting

Copy files listed in text file to new directory

I am trying to write a script that will copy all file listed in a text file (100s of file names) to a new directory Assume script will run with main as current working directory and I know how many files/lines will be in List.txt Im trying to work up a test script using this model Contents of...

6. Shell Programming and Scripting

Exclude certain file names while selectingData files coming in different names in a file name called

Data files coming in different names in a file name called process.txt. 1. shipments_yyyymmdd.gz 2 Order_yyyymmdd.gz 3. Invoice_yyyymmdd.gz 4. globalorder_yyyymmdd.gz The process needs to discard all the below files and only process two of the 4 file names available ...

7. Shell Programming and Scripting

Delete files listed in text file

Hi Team, Here's the scenario, I have a text file called "file_list.txt". Its content is as follows. 111.tmp 112.tmp 113.tmp 114.tmp These files will present in "workdir" directory. It has many files. But only the files present in file_list.txt has to be deleted from the workdir...

8. Shell Programming and Scripting

Want to delete the junk files from a directory which are not listed in a TEXT file

Hello Everyone, I want to delete the image files from a directory, which are not listed in a TEXT file. The directory contains large number of image files (in millions) required / not required. I want to delete the image files which are "not required". I have generated a Text file having...

9. UNIX for Dummies Questions & Answers

Delete files in a txt file

Hi, I have very old files in my server like from 2012 and i want to delete them, Please help. Thanks in advance..

10. Shell Programming and Scripting

Delete files except the file names available in the Except File

Hi, I need some help in the below scenario. I need to delete all the files from the directory except the file name available in the Except file. Like the folder ABC have files like A1.txt,A2.txt......A10.txt and also have a file named Except.txt with the content A3.txt,A4.txt Need a...

LEARN ABOUT DEBIAN

bup-margin

bup-margin(1)						      General Commands Manual						     bup-margin(1)

NAME

       bup-margin - figure out your deduplication safety margin

SYNOPSIS

       bup margin [options...]

DESCRIPTION

       bup margin  iterates  through  all  objects  in	your  bup repository, calculating the largest number of prefix bits shared between any two
       entries.  This number, n, identifies the longest subset of SHA-1 you could use and still encounter a collision between your object ids.

       For example, one system that was tested had a collection of 11 million objects (70 GB), and bup margin returned 45.  That  means  a  46-bit
       hash  would be sufficient to avoid all collisions among that set of objects; each object in that repository could be uniquely identified by
       its first 46 bits.

       The number of bits needed seems to increase by about 1 or 2 for every doubling of the number of objects.  Since SHA-1 hashes have 160 bits,
       that  leaves 115 bits of margin.  Of course, because SHA-1 hashes are essentially random, it's theoretically possible to use many more bits
       with far fewer objects.

       If you're paranoid about the possibility of SHA-1 collisions, you can monitor your repository by running bup margin occasionally to see	if
       you're getting dangerously close to 160 bits.

OPTIONS

       --predict
	      Guess  the offset into each index file where a particular object will appear, and report the maximum deviation of the correct answer
	      from the guess.  This is potentially useful for tuning an interpolation search algorithm.

       --ignore-midx
	      don't use .midx files, use only .idx files.  This is only really useful when used with --predict.

EXAMPLE

	      $ bup margin
	      Reading indexes: 100.00% (1612581/1612581), done.
	      40
	      40 matching prefix bits
	      1.94 bits per doubling
	      120 bits (61.86 doublings) remaining
	      4.19338e+18 times larger is possible

	      Everyone on earth could have 625878182 data sets
	      like yours, all in one repository, and we would
	      expect 1 object collision.

	      $ bup margin --predict
	      PackIdxList: using 1 index.
	      Reading indexes: 100.00% (1612581/1612581), done.
	      915 of 1612581 (0.057%)

SEE ALSO

       bup-midx(1), bup-save(1)

BUP

       Part of the bup(1) suite.

AUTHORS

       Avery Pennarun <apenwarr@gmail.com>.

Bup unknown-															     bup-margin(1)

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How can I delete files using a file that containt path and names?

Discussion started by: AlvaroD

2. Shell Programming and Scripting

[Urgent]how to print the file names into a txt file???

Discussion started by: kumarsaravana_s

3. UNIX for Dummies Questions & Answers

recursive delete files from txt file or?

Discussion started by: plener

4. Shell Programming and Scripting

Finding consecutive numbers in version names on a txt file

Discussion started by: fox1212