Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Find a list of files in directory, move to new, allow duplicates Post 302911988 by bakunin on Tuesday 5th of August 2014 04:29:00 AM
Old 08-05-2014
Quote:
Originally Posted by jim mcnamara
Code:
find /path/to/files -type f  | grep -Ff text_file_with_file_names > outputfile

while read filename
do
  mv $filename   /path/to/somewhere/
done <outputfile

That said - DO NOT agglomerate zillions of user data files adhoc in one file tree.
It is completely possible to have parked those files into meaningful directory names - i.e., the text you are using as a key. Then you can simply look for a directory and go from there. Pre-planning beats kludge like this every time.
Concur! Especially for the last Sentence!

Quote:
Originally Posted by jim mcnamara
PS: because you want to run this ONE time there is a huge performance penalty:
grep -Ff filename has to scan the entire 300 lines of filename for every file it finds. And since you seem to have large numbers of files be prepared to wait.
True. Here is my take at it, based on yours. The rationale is that after the first scan the inode list is cached in memory and the next finds will go a lot faster:

Code:
while read FILEMASK ; do
     find /path/to/sourcedir -type f -name "*${FILEMASK}*" |\
          while read MOVEFILE ; do
               FNAME="${MOVEFILE##*/}"
               if [ -f "/path/to/targetdir/$FNAME" ] ; then
                    mv "$MOVEFILE" "/path/to/targetdir/${FNAME}.$$"
               else
                    mv "$MOVEFILE" "/path/to/targetdir"
               fi
          done
done < /path/to/list.of.filemasks

Note that no security provisions are put into place: checks for exhausted diskspace, successful move operation, etc.. are all missing and you should add them before putting this sketch into a fire-and-forget script.

I hope this helps.

bakunin
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find duplicates from multuple files with 2 diff types of files

I need to compare 2 diff type of files and find out the duplicate after comparing each types of files: Type 1 file name is like: file1.abc (the extension abc could any 3 characters but I can narrow it down or hardcode for 10/15 combinations). The other file is file1.bcd01abc (the extension... (2 Replies)
Discussion started by: ricky007
2 Replies

2. UNIX for Dummies Questions & Answers

Move all files in a directory tree to a signal directory?

Is this possible? Let me know If I need specify further on what I am trying to do- I just want to spare you the boring details of my personal file management. Thanks in advance- Brian- (2 Replies)
Discussion started by: briandanielz
2 Replies

3. UNIX for Dummies Questions & Answers

Find files and display only directory list containing those files

I have a directory (and many sub dirs beneath) on AIX system, containing thousands of file. I'm looking to get a list of all directory containing "*.pdf" file. I know basic syntax of find command, but it gives me list of all pdf files, which numbers in thousands. All I need to know is, which... (4 Replies)
Discussion started by: r7p
4 Replies

4. Shell Programming and Scripting

find list of files from a list and copy to a directory

I will be very grateful if someone can help me with bash shell script that does the following: I have a list of filenames: A01_155716 A05_155780 A07_155812 A09_155844 A11_155876 that are kept in different sub directories within my current directory. I want to find these files and copy... (3 Replies)
Discussion started by: manishabh
3 Replies

5. Shell Programming and Scripting

Move files in a list to another directory

I have a number of files in a directory that can be grouped with something like "ls | grep SH2". I would like to move each file in this list to another directory. Thanks (4 Replies)
Discussion started by: kg6iia
4 Replies

6. Shell Programming and Scripting

Please help list/find files greater 1G move to different directory

I have have 6 empty directory below. I would like write bash scipt if any files less "1000000000" bytes then move to "/export/home/mytmp/final" folder first and any files greater than "1000000000" bytes then move to final1, final2, final3, final4, final4, final5 and that depend see how many files,... (6 Replies)
Discussion started by: dotran
6 Replies

7. Shell Programming and Scripting

Copying files from one directory to another, renaming duplicates.

Below is the script i have but i would like simplified but still do the same job. I need a script to copy files not directories or sub-directories into a existing or new directory. The files, if have the same name but different extension; for example 01.doc 01.pdf then only copy the .doc file. ... (1 Reply)
Discussion started by: Gilljambo
1 Replies

8. Shell Programming and Scripting

List files with date, create directory, move to the created directory

Hi all, i have a folder, with tons of files containing as following, on /my/folder/jobs/ some_name_2016-01-17-22-38-58_some name_0_0.zip.done some_name_2016-01-17-22-40-30_some name_0_0.zip.done some_name_2016-01-17-22-48-50_some name_0_0.zip.done and these can be lots of similar files,... (6 Replies)
Discussion started by: charli1
6 Replies

9. UNIX for Dummies Questions & Answers

How to move gz files from one source directory to destination directory?

Hi All, Daily i am doing the house keeping in one of my server and manually moving the files which were older than 90 days and moving to destination folder. using the find command . Could you please assist me how to put the automation using the shell script . ... (11 Replies)
Discussion started by: venkat918
11 Replies
fileutil(n)							  file utilities						       fileutil(n)

NAME
fileutil - Procedures implementing some file utilities SYNOPSIS
package require Tcl 8 package require fileutil ?1.4? ::fileutil::cat filename ::fileutil::fileType filename ::fileutil::find ?basedir ?filtercmd?? ::fileutil::findByPattern basedir ?-regexp|-glob? ?--? patterns ::fileutil::foreachLine var filename cmd ::fileutil::grep pattern ?files? ::fileutil::stripN path n ::fileutil::stripPwd path ::fileutil::touch ?-a? ?-c? ?-m? ?-r ref_file? ?-t time? filename ?...? DESCRIPTION
This package provides implementations of standard unix utilities. ::fileutil::cat filename A tcl implementation of the UNIX cat command. Returns the contents of the specified file. The first argument is the name of the file to read. ::fileutil::fileType filename An implementation of the UNIX file command, which uses various heuristics to guess the type of a file. Returns a list specifying as much type information as can be determined about the file, from most general (eg, "binary" or "text") to most specific (eg, "gif"). For example, the return value for a GIF file would be "binary graphic gif". The command will detect the following types of files: directory, empty, binary, text, script (with interpreter), executable elf, graphic gif, graphic jpeg, html, xml (with doctype if available), message pgp, and link. ::fileutil::find ?basedir ?filtercmd?? An implementation of the unix command find. Adapted from the Tcler's Wiki. Takes at most two arguments, the path to the directory to start searching from and a command to use to evaluate interest in each file. The path defaults to ".", i.e. the current directory. The command defaults to the empty string, which means that all files are of interest. The command takes care not to loose itself in infinite loops upon encountering circular link structures. The result of the command is a list containing the paths to the inter- esting files. ::fileutil::findByPattern basedir ?-regexp|-glob? ?--? patterns This command is based upon the TclX command recursive_glob, except that it doesn't allow recursion over more than one directory at a time. It uses ::fileutil::find internally and is thus able to and does follow symbolic links, something the TclX command does not do. First argument is the directory to start the search in, second argument is a list of patterns. The command returns a list of all files reachable through basedir whose names match at least one of the patterns. The options before the pattern-list determine the style of matching, either regexp or glob. glob-style matching is the default if no options are given. Usage of the option -- stops option processing. This allows the use of a leading '-' in the patterns. ::fileutil::foreachLine var filename cmd The command reads the file filename and executes the script cmd for every line in the file. During the execution of the script the variable var is set to the contents of the current line. The return value of this command is the result of the last invocation of the script cmd or the empty string if the file was empty. ::fileutil::grep pattern ?files? Implementation of grep. Adapted from the Tcler's Wiki. The first argument defines the pattern to search for. This is followed by a list of files to search through. The list is optional and stdin will be used if it is missing. The result of the procedures is a list containing the matches. Each match is a single element of the list and contains filename, number and contents of the matching line, separated by a colons. ::fileutil::stripN path n Removes the first n elements from the specified path and returns the modified path. If n is greater than the number of components in path an empty string is returned. ::fileutil::stripPwd path If the path is inside of the directory returned by [pwd] it is made relative to that directory. In other words, the current working directory is stripped from the path. The possibly modified path is returned as the result of the command. ::fileutil::touch ?-a? ?-c? ?-m? ?-r ref_file? ?-t time? filename ?...? Implementation of touch. Alter the atime and mtime of the specified files. If -c, do not create files if they do not already exist. If -r, use the atime and mtime from ref_file. If -t, use the integer clock value time. It is illegal to specify both -r and -t. If -a, only change the atime. If -m, only change the mtime. KEYWORDS
file utilities fileutil 1.4 fileutil(n)
All times are GMT -4. The time now is 11:02 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy