read list of filenames from text file, archive, and remove


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting read list of filenames from text file, archive, and remove
# 1  
Old 03-19-2007
read list of filenames from text file, archive, and remove

I posted a week ago regarding this scripting question, but I need to revisit and have a few more questions answered..

User cfajohnson was extremely helpful with the archive script, but clarification on my part is needed to help steer the answer in a direction that works in this particular production environment.

The basics:

I have a large list of filenames, running in the thousands, with varying extension types....these extensions and . were pruned from this list, strictly just filenames now, ex., "filename.psd" has been changed to "filename"

The need:

To search through our production server traversing through multiple directory trees and to archive these files, each file size ranging anywhere from 200 KB to 200 MB with different file extensions such as .psd, .eps, or nothing at all, keeping in mind that directory structure in the archive is not necessary, only the file itself as a backup in one archive directory. Once backed up on DVD, etc., this file list can then be read from again and used to delete all files on the server traversing through multiple directory trees while maintaining directory structure and any other files inside these directories that do not match the file list being used.

The catch:

Having some trouble executing scripts, even simple ones such as:

#!/bin/bash
# script to list files and permissions, name: listFiles
ls -l

The permissions were changed to execute the file, i.e.,

chmod +x listFiles

...however, after attempting to execute the file at the command line by simply typing listFiles (I also tried typing $listFiles), bash returns with:

-bash: listFiles: command not found

Perhaps my permissions as admin or user are incorrect? or my path to /bin/bash needs to include a full path back through my permissions tree as a user?

Also, thank you to users srikanthus2002 and jacoden with:

find / -type f -name "fil*" -exec rm {} \;

however, is "fil*" actually READING the list provided in the file and using that list to feed or pipe into -exec rm {}, or strictly looking for the filename of the list file and deleting the list file?

Thanks for your help!

Last edited by fxvisions; 03-19-2007 at 06:43 PM..
# 2  
Old 03-20-2007
....ok, how bout just a command to read from the list and delete the files throughout the directory tree?
# 3  
Old 03-20-2007
If the script called listfiles is in the current directory, use
./listfiles
to run it. Alternatively, listfiles could be placed in one of the directories on your PATH. Use "echo $PATH" to see the path.

If qwerty.qqq has been recorded as simply qwerty, what happens if there is both a qwerty.qqq and a qwerty.yyy? Since you are looking for thousands of files, you will need to do something like this:
find / -print > /some/directory/longlisting
and then repeatedly search longlisting to get the paths to the files to be archived.
# 4  
Old 03-20-2007
I've determined now that I no longer need to archive these files, merely delete them using this list.

You are correct, qwerty will find all combinations including qwerty.aaa, qwerty.bbb, qwerty.ccc, etc., this is ok and exactly what I want since these files are, for all intents and purposes, the same file, simply different resolutions and formats for our needs....however, we no longer need this particular list of files and need to be purged.

The find command seems like it would work in combination with -exec rm , but how do I get find to read from the list?
# 5  
Old 03-20-2007
Don't try to get find to read from your list. Instead do this:
find / -name > /tmp/every.last.file
grep -f /path/to/mylist /tmp/every.last.file
This should produce a list of pathnames to files. Look it over and be sure you want to delete all of this stuff. If you are sure you want to delete them all, do:
grep -f /path/to/mylist /tmp/every.last.file | xargs rm
# 6  
Old 03-20-2007
hmm, lemme check this out...

Last edited by fxvisions; 03-20-2007 at 11:03 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to remove the text between all curly brackets from text file?

Hello experts, I have a text file with lot of curly brackets (both opening { & closing } ). I need to delete them alongwith the text between opening & closing brackets' pair. For ex: Input:- 59. Rh1 Qe4 {(Qf5-e4 Qd8-g8+ Kg6-f5 Qg8-h7+ Kf5-e5 Qh7-e7+ Ke5-f5 Qe7-d7+ Qe4-e6 Qd7-h7+ Qe6-g6... (6 Replies)
Discussion started by: prvnrk
6 Replies

2. UNIX for Beginners Questions & Answers

How do I custom sort the files in a directory using the filenames in a text file.?

Hi all, (5 Replies)
Discussion started by: KMusunuru
5 Replies

3. Shell Programming and Scripting

Read n lines from a text files getting n from within the text file

I dont even have a sample script cause I dont know where to start from. My data lookes like this > sat#16 #data: 15 site:UNZA baseline: 205.9151 0.008 -165.2465 35.8109 40.6685 21.9148 121.1446 26.4629 -18.4976 33.8722 0.017 -165.2243 48.2201 40.6908 ... (8 Replies)
Discussion started by: malandisa
8 Replies

4. UNIX for Advanced & Expert Users

ls -ltr a list of filenames-with-spaces within a text file

OS: RHEL 5.8 shell: bash 3.2.25 Directory /home/guest/ contains these files: file a file b file c fileD fileE fileF testFile.txt I'm trying to find the syntax to run ls -ltr against this list of files that is contained within a text file, testFile.txt. The file testFile.txt has... (4 Replies)
Discussion started by: uschaafm
4 Replies

5. Shell Programming and Scripting

Remove files from subdirectories given a list of filenames

Dear all, I have a dir structure like main_dir At_nn Ag_js Nf_hc .... mcd32 mgd43... mcd32 mgd43... mcd32 mgd43... and each subdir (e.g. mcd32, mgd43) contains files. Now, i... (15 Replies)
Discussion started by: yogeshkumkar
15 Replies

6. Shell Programming and Scripting

Shell Script Needed to Read a text from a list files

Hi, Below is my issue which I desperately need and I want a shell script which can do this job. I need this script as I m planning to put this for a system health check. Please assist me. 1. There are 10 log files in a particular location. 2. open each log file. Goto to the end of the... (4 Replies)
Discussion started by: kashriram
4 Replies

7. Programming

read file from tar.gz archive

I want to write a c-program which reads a textfile from a tar.gz archive. How can I do it? (9 Replies)
Discussion started by: krylin
9 Replies

8. Shell Programming and Scripting

Read specific file from a zip archive without extracting

Hi All, I would like to extract specific file from a zip archive. I have a zip archive "sample.zip". sample.zip contains few text files and images... text1.txt, text2.txt, pic.jpg etc... I need to read specific file "text2.txt" from "sample.zip" WITHOUT EXTRACTING the zip file. ... (4 Replies)
Discussion started by: sridharg
4 Replies

9. Shell Programming and Scripting

read list of filenames from text file and remove these files in multiple directories

I have a large list of filenames from an Excel sheet, which I then translate into a simple text file. I'd like to use this list, which contains various file extensions , to archive these files and then remove them recursively through multiple directories and subdirectories. So far, it looks like... (5 Replies)
Discussion started by: fxvisions
5 Replies

10. Shell Programming and Scripting

Read from file then purge or archive.

Hi All, I have a root directory /tmp and I want to purge files or archive files in its subsequent subfolders.I listed the path of files I want to purge(archive) and the #of days. (purge) DAYS PATH 7 /tmp/arsenal/* 5 /tmp/chelsea/* (archive? the same as above but different folders... (15 Replies)
Discussion started by: kayarsenal
15 Replies
Login or Register to Ask a Question