Recursive search for files and copy to new directories


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Recursive search for files and copy to new directories
# 1  
Old 06-29-2012
Recursive search for files and copy to new directories

So I have extremely limited experience with shell scripting and I was hoping someone could point out a few commands I need to use in order to pull this off with a shell script like BASH or whatnot (this is on OS X).

I need to search out for filenames with account numbers in the name itself (potentially more than one file with an account number plus other characters in the name), and copy them to a temporary folder, either in their own subfolders (named after the account number), or all in the same place. Basically I'm just trying to extract an copy some PDF files out of one location and put them in another. The search of the source location should be a recursive search. The filenames are basically "textstring12345689textstring.pdf"

I had this working at one point a while back using PHP, though it would sometime crap out on me after a larger number of runs through the loop. Regardless, I'd ideally like to just do this in a shell script anyway as the PHP script is pretty slow (it's searching through like a million files in a bunch of folders). Here is basically what I did in PHP:

Code:
<?php

$handle = @fopen('/Users/me/www/nexus/media/encore1.csv', "r");
mkdir ("/Users/me/www/temp");

if ($handle) {
	while (!feof($handle)) {
		$lines[] = fgets($handle);
	}
	fclose($handle);
}

for ($a=0; $a<count($lines); $a++)
{
	$cardnumber = trim($lines[$a]);
	print "\"" . $cardnumber . "\"<br>";
	$searchStr = "find . -name \"*" . $cardnumber . "*\"";
	
	print $searchStr . "<br>";    // just watching the progress
	
	$tempRoot = "/Users/me/www/temp/";
	$tempFolder = $tempRoot . $cardnumber;
	
	if (strlen($cardnumber) >= 16) {
		exec($searchStr, $output, $return);
	
		mkdir($tempFolder);
		
		foreach ( $output as $file) {
	
			$destinationFile = $tempFolder . "/" . basename($file);
			copy($file, $destinationFile);
			print basename($file) . " copied to " . $destinationFile . "<br>";   // again progress check
			
		}
	} else {
		print "Invalid Account Number ";
	}
	unset($output); 
}
?>

Pardon the lack of indentation in that code, it doesn't pass through when pasting in it.

The CSV file is just a single column list of account numbers. I'm sure this can be easily done with some kind of search and copy and make directory commands in a shell script, but I'm not really familiar with any.

Would anyone one have some pointers on which commands that are counterparts to the PHP ones above?

Last edited by flyawaymike; 06-29-2012 at 09:28 PM..
# 2  
Old 06-29-2012
Quote:
Originally Posted by flyawaymike
Pardon the lack of indentation in that code, it doesn't pass through when pasting in it.

Put code inside [CODE] tags, not [QUOTE] tags.

(You can edit your post to fix it.)
This User Gave Thanks to cfajohnson For This Post:
# 3  
Old 06-29-2012
In POSIX shell, like bash or ksh you can do something like this:

Code:
 
for n in $(cat your_acct_list); do 
        echo "Processing $n"; 
        find . -name "*${n}*" -type f | xargs -i ... {} target_here;    
done

The for construct will loop through the lines in the list, I put echo for you to see.

Put your copy/mv command here in xargs. The {} construct will be substituted with the actual names find command did find
You can run it without piping to xargs and see what exactly will happen, which files are found by find command.
# 4  
Old 06-30-2012
Quote:
Originally Posted by migurus
In POSIX shell, like bash or ksh you can do something like this:

Code:
 
for n in $(cat your_acct_list); do 
        echo "Processing $n"; 
        find . -name "*${n}*" -type f | xargs -i ... {} target_here;    
done

The for construct will loop through the lines in the list, I put echo for you to see.

Put your copy/mv command here in xargs. The {} construct will be substituted with the actual names find command did find
You can run it without piping to xargs and see what exactly will happen, which files are found by find command.
What does the "-i" do with xargs?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Copy files in respective directories

Hi Guys, I need to copy the files to respective directories based on name of the file. My script is something like below con=$1 for file in `cat $con` do file_tmp=$(ls -t1 $path| grep -i $file | head -n 1) echo $file_tmp if then cp $path$file_tmp $DIR/ap if then... (16 Replies)
Discussion started by: Master_Mind
16 Replies

2. HP-UX

Recursive copy of Folders with files

Dear All, I will appreciate any help received. Our system is running on hpux v1 My problem is as follows: We have many customer folders with name fd000100, fd000101 and so on e.g. (Testrun)(testsqa):/>ll /TESTrun/fd000100 total 48 drwxrwx--- 2 fq000100 test 96 Jun 27 2004... (17 Replies)
Discussion started by: mhbd
17 Replies

3. Shell Programming and Scripting

recursive search and copy

Hello again. Well, I need help again sooner as I thought. Now I want to search for files with a known name within all subdirs, and copy the to differently named files in the same directory. For example if I had only one file to copy, I would just usecp fileName newFileNamebut to do this... (1 Reply)
Discussion started by: cabaciucia
1 Replies

4. Shell Programming and Scripting

Recursive looping through files and directories

hi; i need a script which will go to all directories and subdirectories and print the filenames as follow; here i m printing only files listing in current directory reason i m doing this is coz i want to perform some operations according to filename achieved so cant use find command;... (4 Replies)
Discussion started by: ajaypadvi
4 Replies

5. UNIX for Dummies Questions & Answers

how can i copy those files into other directories have the same name

how can i copy those files into other directories have the same name but different in the end i have files in directory called test: 10_10_asdadfsdfad.txt 10_10_11_asdawqefwkjasd.txt 10_10_11_12_asdafjjhoqwd.txt i want to put them in exist directory thart i have on my system i have... (1 Reply)
Discussion started by: t17
1 Replies

6. UNIX for Dummies Questions & Answers

Copy Directories excluding files

Hi guys, I want to copy folder and sub folders only. I don't want the files. If i use cp -r command it will copy entirely with files. Could any one suggest me. Thanks in advance (1 Reply)
Discussion started by: karthik82
1 Replies

7. Shell Programming and Scripting

Recursive call to find files and directories in Shell script from current path.

################################################################ Copy this script to your path from where you want to search for all the files and directories in subdirectories recursively. ################################################################# code starts here... (2 Replies)
Discussion started by: Ramit_Gupta
2 Replies

8. Cybersecurity

Recursive search for group or other writeable 'dot' files

I'm looking for a easy way to do a recursive search through a directory structure for all files that begin with '.' that are group or other writable. Any suggestions? (5 Replies)
Discussion started by: maficdan
5 Replies

9. UNIX for Dummies Questions & Answers

copy multiple files in different directories

I have a report file that is generated every day by a scheduled process. Each day the file is written to a directory named .../blah_blah/Y07/MM-DD-YY/reportmmddyy.tab I want to copy all of this reports to a separate directory without having to do it one by one. However, if I try cp... (3 Replies)
Discussion started by: ken2834
3 Replies

10. UNIX for Advanced & Expert Users

recursive copy of hidden files

I need to know how to copy hidden files recursively? cp -r sourceDir/* targetDir/. ignores the hidden files. Thank you!! (2 Replies)
Discussion started by: usfrog
2 Replies
Login or Register to Ask a Question