Shell Programming and Scripting

I'm having trouble understanding the exclude option in tar. From some web sites, it seems one is able to exclude several strings by enclosing them in curly brackets. However it seems to be "random" what gets excluded when using the curlies.

I've been using the exclude-from=myfile option in a script but would like to inline myfile into a variable and use $exclude={this,that,the_other} with the --exclude=$exclude option to keep the script self-contained.

Simple examples to illustrate:

Say I have directories C and D and want to make them into a tar file.

Code:

tar -cvf mytar.tar C D

This works.

Now for the sake of argument, I want to exclude C.

Code:

tar -cvf mytar.tar C D --exclude=C

This works as expected (C is excluded).

Now I bring in the curlies...

Code:

tar -cvf mytar.tar C D --exclude={C}

This does not exclude C, which is not what I expected.

If I try to exclude a subdirectory of C, it also doesn't work.

Code:

tar -cvf mytar.tar C D --exclude={C/subdir}

Nothing is excluded.

If I exclude both C and D, then it does work (both are excluded).

Code:

tar -cvf mytar.tar C D --exclude={C,D}

Any idea how I can get the same functionality as --exclude-from with --exclude?

Reading thru the gnutar (version 1.26) source I can't see any intended support for the {} matching.

I'd say this is probably an unsupported "feature" you should probably stick with the documented method of using multiple --exclude arguments for each pattern to exclude.

---------- Post updated at 11:32 AM ---------- Previous update was at 11:17 AM ----------

Thought: Could the shell be expanding your parameters?

Code:

$ echo {C}
{C}
$ echo {C,D}
C D

Chubler_XL's thought is right on the mark. The shell running the command supports brace expansion and it is generating multiple --exclude options.

When there's no comma between the braces, there's no brace expansion. When there's no brace expansion, the pattern passed to tar includes the braces themselves. Unless there's a filename with literal braces, that exclude will not match anything.

To confirm: set -x; tar .... ; set +x

Regards,
Alister

Thanks, at least I'm not going crazy Searching around a little more, it seems there is a way to include multiple arguments. Basically you create a temp file with the patterns to exclude and point --exclude-from (or -X) to that file. See below.

Code:

#!/bin/bash

include="C D"
exclude="C xyz"

tar -cvf mytar.tar $include -X <(for i in ${exclude}; do echo $i; done)

---------- Post updated at 03:20 PM ---------- Previous update was at 02:23 PM ----------

My mistake, this script has problems with wildcard expansion. E.g. if you wanted to exclude *.tar then it would expand the asterisk prematurely rather than passing it to the tar command. Here is a workaround - not too pretty but could be worse.

Code:

include="C D"
exclude=("C" "*.tar") # note this is an array now

mycommand="tar -cvf mytar.tar $include"
for i in `seq 1 ${#exclude[*]}`; do mycommand=$mycommand\ --exclude="${exclude[$i-1]}"; done

eval $mycommand

What is the eval for?

Eval is to be avoided. If someone puts `rm -Rf ~/` in your filenames, eval will execute that.

Sure, just use mutliple --exclude options.

It doesn't make much sense to resort to a dynamic, command-building process when the "variable" is constant data hardcoded into the script. Why not just add one --exclude pattern per pattern? Your script will be simpler, more readable, more efficient, and safer.

Still, if you're determined to store exclusion patterns in a variable, the following is much less complicated:

Code:

include='C D'
exclude='C
*.tar'

tar -cvf mytar.tar $include -X <(cat <<EOX
$exclude
EOX
)

This does not work with the only version of GNU tar that I have (an elderly 1.16, circa 2006). strace shows tar closing all file descriptors in the range 3 to 1023, sabotaging the shell's (bash) attempt to pass the cat process substitution on fd 63.

However, although I have not tested it, it should work with newer versions of GNU tar. A pair of 2007 commits removed both the function call and the function definition implementing the fd closures.

tar.git - GNU Tar - src/tar.c - (main): Don't call closeopen

tar.git - GNU Tar - src/misc.c - Don't include <sys/time.h>, <sys/resource.h>; no longer needed.

Regards,
Alister

The reason for using eval $mycommand is so that other users can edit the top of the script where exclude and include are defined rather than having to read through the code. Your second construct is elegant - but does not work on OpenSuse 11.4.