Shell Programming and Scripting

I normally download a directory recursively using:

scp -r <name>@host:<path> .

This has worked fine. As everyone knows this will download all of the directory named in <path> and all of the sub directories.

I would like to know if it is possible to not download a particular file if it is in any part of the directory tree.

I would like to do this because there is a file in every subdirectory that contains a file (always the same name) which I do not need because it is very large, binary, and not very useful for me. So I would like to essentially skip this file and not waste time downloading it because there may be 100's of directories.

If this is not possible with scp, does anyone know how I might accomplish this requirement?

Thank you!

With scp alone, probably not. But you can try this ugly little thing

Code:

$ ssh <name>@host 'tar -cf - `find <path> ! -name exclude_me -print`' | tar -xf -

It's even possible to pipe it through compress/gzip/bzip on both sides as to save on bandwith.

Thanks for the input. I have tested the command and this it what it does.

It downloads all directories (e.g.,/dir1/dir2/dir3/dirofinterest), it does not download any of the files in dir1 dir2 or dir3 which is good.

However, in dirofinterest there are more (around 40) directories which contain a commonly named file Ill call (LEAVE) which I would like to exclude because I do not need them and they are usually at least 20MB.

This is what I typed:

Code:

$ ssh <name>@host 'tar -cf - `find /dir1/dir2/dir3/dirofinterest ! -name LEAVE -print`' | tar -xf -

Since I am new it is possible that I misinterpreted your instructions. let me know what you think.

This solution works for me:

Code:

tar czvf  <tar file name>  <root folder or list of each folder> --exclude <filetoexclude>

use scp to download the file compressed files.

Then unpack...

Code:

tar xzvf <tar filename>

also you could use rsync with exclude pattern :-)

I use this all the time, this is even gzipped (compressed for faster transfer).

Code:

tar czf - </path/to/file_or_folder> --exclude LEAVE | ssh <user@host> tar xzf - -C </path/to/copy_to>

Note:
If you are familiar with the tar command then you know that if you start at say "/" and tar the path "/var/www/html" then when you un-tar you will have the full path in that tar file (/var/www/html). If you wish to avoid this, the cd to "/var/www/html" and "tar ./*" so that you do NOT have full path recursion.