tar --exclude with curly braces


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting tar --exclude with curly braces
# 8  
Old 08-08-2012
Quote:
Originally Posted by majest
The reason for using eval $mycommand is so that other users can edit the top of the script where exclude and include are defined rather than having to read through the code.
But why are you using it? What function does it perform that tar by itself couldn't? You don't even need eval to expand wildcards, the shell does that when evaluating any unquoted variable.

Try $mycommand without the eval.
# 9  
Old 08-09-2012
Quote:
Originally Posted by Corona688
But why are you using it? What function does it perform that tar by itself couldn't? You don't even need eval to expand wildcards, the shell does that when evaluating any unquoted variable.

Try $mycommand without the eval.
The problem I encountered was that wildcards *were* expanded when I didn't want them to be. If exclude="C *.txt" then "for i in ${exclude}" expands the asterisk in *.txt to a list of .txt files in the current directory, rather than passing the string "*.txt" to tar. I couldn't see an easy way around it, hence the eval. Happy now or still hate it? Smilie
# 10  
Old 08-09-2012
Quote:
Originally Posted by majest
The problem I encountered was that wildcards *were* expanded when I didn't want them to be. If exclude="C *.txt" then "for i in ${exclude}" expands the asterisk in *.txt to a list of .txt files in the current directory, rather than passing the string "*.txt" to tar. I couldn't see an easy way around it, hence the eval. Happy now or still hate it? Smilie
As it is "eval" won't help you avoid this problem, but in fact make it worse. Without "eval" your command line becomes interpreted by the shell once, with it twice. I don't think "eval" should be avoided generally (like you seem to believe Corona688 is thinking), but in this case it is simply the wrong tool - and probably very dangerous, as Corona688 pointed out correctly.

If you want to avoid interpretation of a certain string use quotation - that is, what it is for. For intance:

Code:
exclude="*.txt"

$ echo $exclude
a.txt b.txt c.txt
$ echo "$exclude"
*.txt
$ echo '$exclude'
$exclude

Now, let us apply this to your problem. I use ":" as a delimiter here, like in the PATH variable, because commas or something such could be used in filenames and i want to avoid that:

Code:
#! /bin/ksh

excludelist="foo:*.bar:baz.*"

exclude=""

while [ -n "$excludelist" ] ; do
     exclude="$exclude --exclude \"${excludelist%%:*}\""
     if [ "${excludelist%%:*}" = "${excludelist}" ] ; then
          excludelist=""
     else
          excludelist="${excludelist#*:}"
     fi
     # this is just to show how it works, can be removed:
     print - "excludelist: \"$excludelist\" args to tar: \"$exclude\""
done

tar <options> $exclude <more options>

I would like to say for the record, that i don't think the contents of the variable excludelist should be modified by modifying the scripts text at all. Instead a configuration file should be passed to the script, which contains all the exclusions. How to construct a syntax for such a configuration file and how to parse it i have described here, but the way of feeding the options to tar will still be as lain out here.

I hope this helps.

bakunin
# 11  
Old 08-09-2012
After giving the problem some afterthought, i think to know how the "eval" came to pass:

If you want to use wildcards with a special meaning and pass these to a utility without having them interpreted by the shell you have to use quotation:

Code:
opt="*"
utility $opt     # won't work unexpanded
utility "$opt"   # will work without expansion by shell

If you try to bind together several options into a single variable and put this variable into a quotation it will count as a single parameter:

Code:
opt="-a * -b ??"
utility $opt     # will become expanded, therefore useless
utility "$opt"  # will not become expanded, but will count as only *one* option

There is a typical gotcha, which i found in some IBM software not too long ago: mysteriously the IBM utility made run "/dev" out of diskspace, which is a very bad idea under AIX. How did that come? Easy: first there was some line like:

Code:
tgt="/dev/null"
something > "$tgt"

But "something" also produced output on stderr and someone - very economically - changed "$tgt" therefore:

Code:
tgt="/dev/null 2>&1"
something > "$tgt"

This will not work any more, because now the output goes to a file named "null 2>&1", which is a regular file in "/dev" - of course, it will eventually run out of space.

If you now use "eval", this will restart the evaluation process and hence make the argument count correct again. Alas, this also restarts the evaluation process by the shell again and therefore wildcards will be expanded again too.

The only solution is to create your own option line in an evaluation-proof way like i have done in my script sketch: i surround the delicate wildcards with quotation of my own to protect them when i send the commandline unquotedly to the shell (see the "\"", which will create literal quotation marks in the output).

I hope this helps.

bakunin
# 12  
Old 08-09-2012
Quote:
Originally Posted by majest
The problem I encountered was that wildcards *were* expanded when I didn't want them to be. If exclude="C *.txt" then "for i in ${exclude}" expands the asterisk in *.txt to a list of .txt files in the current directory, rather than passing the string "*.txt" to tar. I couldn't see an easy way around it, hence the eval. Happy now or still hate it? Smilie
You're using eval to avoid expansion? That's the oddest thing I've heard this week.

It's not that I "hate it". It's that it's "such a bad idea that you're going to kick yourself for it when something bad happens".

I believe tar has a --excludefile option or something like it which would be a much more elegant solution without putting self-modifying code into eval and hoping nothing bad happens.
# 13  
Old 08-09-2012
Quote:
Originally Posted by Corona688
I believe tar has a --excludefile option or something like it which would be a much more elegant solution without putting self-modifying code into eval and hoping nothing bad happens.
It does.

From post #1:
Quote:
Originally Posted by majest
I've been using the exclude-from=myfile option in a script but would like to inline myfile into a variable and use $exclude={this,that,the_other} with the --exclude=$exclude option to keep the script self-contained
Regards,
Alister
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Check string end with curly braces

file.txt apple apples{ applepicture apple9 apple cake{ abple apple_and_cake appleapple apple apple( and my script while read line; do if ]; then echo "$line" fi done <file.txt read (10 Replies)
Discussion started by: cmdcmd
10 Replies

2. Shell Programming and Scripting

When curly braces needed?

Hello, i was trying to find get a command to list duplicated files so i tried ls dir1 dir2 | awk '{x++}' and it didnt work. After a bit of searching online i found that it works without the curly braces ls dir1 dir2 | awk 'x++' I thought the curly braces were needed in awk so... (6 Replies)
Discussion started by: andy391791
6 Replies

3. Shell Programming and Scripting

Curly braces in sed

Hi, I have below command in one of the script. Can you please let me know what does the curly braces do over here \{1,\}. The remaining part of the code atleast I am able to understand. sed -n 's/.*\-\()\{1,\}\)\-.*/\1/p' (13 Replies)
Discussion started by: tostay2003
13 Replies

4. Shell Programming and Scripting

How to remove newline, tab, spaces in curly braces.. :( Pls Help?

Hi Everyone, in the below "xyz (Exception e)" part... after the curly braces, there is a new line and immediately few tabs are present before closing curly brace. xyz (Exception e) { } note: there can be one or more newlines between the curly braces. My desired output should be ... (6 Replies)
Discussion started by: NY_777
6 Replies

5. Shell Programming and Scripting

** EMERGENCY ** Having trouble with curly braces.. :( Pls Help

Hi Everyone, in the below "xyz (Exception e)" part... after the curly braces, there is a new line and immediately few tabs are present before closing curly brace. xyz (Exception e) { } note: there can be one or... (1 Reply)
Discussion started by: NY_777
1 Replies

6. UNIX for Dummies Questions & Answers

How do I pull the value between curly braces?

Hi everyone, I've got a file that looks like this: uid{508}pid{22224}pname{/PPROGRAM/pprgramx -profile:LIVE -serv:as ... I want to pull the value of pid between the curly braces, or 22224 in this example. pid is always the second pair of curly braces, but the length of the number is... (7 Replies)
Discussion started by: Scottie1954
7 Replies

7. Shell Programming and Scripting

sed in windows does not parse curly braces

Hi everyone: I'm stuck at this point, could you guys please give me some hints about what I am doing wrong in the following script, I'm using sed for windows: sed ^"$ {^ a^ STRINGTABLE DISCARDABLE^ BEGIN^ #define CLIENT_MODULE, "%CLIENT_MODULE%"^ #define CLIENT_ID, "%CLIENT_ID%"^... (1 Reply)
Discussion started by: edgarvm
1 Replies

8. Shell Programming and Scripting

find -regex not recognizing curly braces

Must be a bug or something. Whether I escape them or not, it will not work. No matter what I set the minimum and maximum to nothing gets caught. For instance: find / -regex "/.{0, 50}.*" -maxdepth 1 or find / -regex "/.\{0, 50\}.*" -maxdepth 1 should pretty much catch everything residing within... (4 Replies)
Discussion started by: stevensw
4 Replies

9. Shell Programming and Scripting

Curly braces assigned to variables

Hi, Im pretty new to Unix. I came across a script which was using PLSQL inside a script and there was an unusual thing mentioned. there was a variable assigned as P_CUR=${1} and one more as V_TAGFILE="$1" Couldnt find the difference. Also the variables were used in PLSQL... (1 Reply)
Discussion started by: njks68
1 Replies

10. Shell Programming and Scripting

Use of curly braces with variables

Hi, I am new to shell scripting.I have worked somewhat with Perl though. I am not able to find what the second line does and how does it do. <code> FP_RUNNING=`service filepool status` FP_RUNNING=${FP_RUNNING%% *} <\code> After the first line,the variable FP_RUNNING stores '1 FilePool... (2 Replies)
Discussion started by: abhinavsinha
2 Replies
Login or Register to Ask a Question