Shell Programming and Scripting

Hi,

I discovered the following single-line script works very well to cp a large number of files from a source directory to a destination directory while avoiding the "argument list too large" error:

Code:

# cpmany - copy large number of files
# Takes two parameters - source dir, destination dir
# Copies ALL regular files from source dir to destination dir
# Does not traverse subdirectories in source
# Does not cp hidden files
#
find $1 -maxdepth 1 -type f -name '*' -exec cp -v {} $2 \;

I'm not fluent in the ksh, so I'm hoping someone who is can explain in detail the operation of this script.

1. Since the result of the find cmd is a list of filenames, does the "-exec" command actually spawn a new process for every filename in the list in order to do the cp? Is it a case of spawning a new ksh and then overlaying that new ksh with cp to do the copy of a file in the list?

2. If you remove the single quotes from the *, you get the "argument list too large" error. Exactly why is this?

3. Obviously, the "{}" that follows the cp command means 'use a name from the list' produced by "find". Where can I find more on this construct? It's pretty cool!

4. Why is a final "\" needed at the end of the line, just before the ";"?

5. I've tried running this thing dirrectly from the ksh command prompt (rather than as a script) but I get "missing argument to exec". Why?

6. I've looked and looked for detailed information on the exec cmd in ksh, and what I've found is so limited that I came here to get answers. Is there a difference between "exec" and "-exec" in this context?

Thanks in advance for your help. I hate 'black boxes' even if they work very well.

Follow the answers:
1. Each file returned from the find is copied with the cp command. I mean, it "spawns" a cp command;
2. Do you really need the -name argument? Try:

Code:

find $1 -maxdepth 1 -type f -exec cp -v {} $2 \ ;

3. I think find man pages have information about, it is just a placeholder. You could also try the "xargs" command.
4. It is part of the find command structure, but it means that the -exec argument finishes.
5. I am not sure if it is a typo, but in the command you post there is a missing space between the backslash and the semi-colon.
6. exec is a shell builtin, try man exec.

I hope it helps!

Regards!

Yes No. -exec is directly launching the "cp" command with its arguments. There is no ksh involved here.

Because the shell expands * and you have too many of them in the current directory.

In the "find" manual page and in the various web pages about it.

Because ';' is required to delimit the end of the -exec clause but ';' alone would be picked by the current shell as a command separator if not escaped by '\'.

Not sure, did you replace $1 and $2 by something useful ?

Absolutely. exec is a ksh builtin command while "-exec" is an unrelated find option.

---------- Post updated at 18:38 ---------- Previous update was at 18:36 ----------

It is needed to fulfill this requirement:

Code:

 # Does not cp hidden files

---------- Post updated at 18:41 ---------- Previous update was at 18:38 ----------

That's the other way around. There is an extra space between the backslash and the semicolon in your suggestion. That space is defeating the backslash requirement.

jlliagre,

Your answers make perfect sense! This is very much appreciated. I'll study the "find" command.

This was sort of 'wigging me out' until you explained matters. Here in this script we have a seemingly bizarre example of connecting the output of one command to the input of another, and all without the use of a pipe! But of course "find" with its "-exec" option is doing much for us in this regard - so things make sense after all.

Thanks again!

---------- Post updated at 01:57 PM ---------- Previous update was at 10:56 AM ----------

One final question:

Given the script:

Code:

    find $1 -maxdepth 1 -type f -name '*' -exec cp -v {} $2 \;

which copies ALL regular files from $1 (source dir) to $2 (dest. dir)

How can I make this script take a file-pattern for -name instead of using '*', so that I can use the script to cp only the files I want?

Code:

          find $1 -maxdepth 1 -type f -name $2 -exec cp -v {} $3 \;

does not work if you use a wildcard like * in your 2nd parameter. What needs to be done to allow the use of a special char like * ?

Put it between double-quotes!

Code:

find $1 -maxdepth 1 -type f -name "$2" -exec cp -v {} $3 \;

Hi Filipe,

When I use the double quotes around the positional paramter like this ("$2") I get wierd results when I run the script.

For example, if I have a directory with the following files in it:

Code:

    cpmany
    file1
    file2
    file3
    file4

and I'm cd'ed into this directory and I invoke cpmany like this:

Code:

./cpmany  .  f*  /joe/stuff/

what happens is file1 is copied to file2, with no files copied to the dest dir I specified on the command line, and after doing the one copy operation the script finishes, with no errors.

It should have copied all four files to the dest dir - but it does not do so. What is wrong with my syntax?

That should be:

Code:

./cpmany . "f*" /joe/stuff/