Bash expansion


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Bash expansion
# 1  
Old 02-09-2017
Bash expansion

Hello.
I cannot write a command without using eval.
Any help is welcome

Note 1 : What does the function SOMETHING has no importance.
Note 2 : What does the command find has no importance.

It is an expansion variable problem : where to put ["] or [\"] or [ '] or anythings else

What works (FILTRE_1 and FILTRE_2 are taken into account) :

Code:
function CHERCHE () {

    MY_PATH1="$1"
    if [[  ! -d "$MY_PATH1" ]] ; then
        echo "doing : $MY_PATH1"
        /usr/bin/kate "$MY_PATH1"
    fi
}
export -f CHERCHE

MY_PATH="/home/bidon"

FILTRE_1=" -path  $MY_PATH/.local/share/kscreen  -o -path  $MY_PATH/.local/share/baloo   -o -path  $MY_PATH/.dbus"

FILTRE_2=" ! -name \".direct*\"  ! -name \".Xauthority\"  ! -name \".xsession*\"  ! -name \".bash_history\" "

CMD="find \"$MY_PATH\"  -type d \( $FILTRE_1  \) -prune -o  \( $FILTRE_2 -exec bash -c 'CHERCHE \$0 '  {} \; \)  "
echo "COMMAND : $CMD"
eval "$CMD"

# The command return 9 files which is the good result

What does not works (FILTRE_1 is taken into account, FILTRE_2 or FILTRE_3 or FILTRE_2 is ignored ) :
Filter_2 is same as above

Code:
FILTRE_2=" ! -name \".direct*\"  ! -name \".Xauthority\"  ! -name \".xsession*\"  ! -name \".bash_history\" "

FILTRE_3=' ! -name ".direct*"  ! -name ".Xauthority"  ! -name ".xsession*"  ! -name ".bash_history" '

FILTRE_4=" ! -name '.direct*'  ! -name '.Xauthority' ! -name '.xsession*'  ! -name '.bash_history' "

find "$MY_PATH"  -type d \( $FILTRE_1  \) -prune -o  \( $FILTRE_2 -exec bash -c 'CHERCHE "$0" ' {} \;  \)
#FILTER_2 does nothing

find "$MY_PATH"  -type d \( $FILTRE_1  \) -prune -o  \( $FILTRE_3 -exec bash -c 'CHERCHE "$0" ' {} \;  \)
#FILTER_3 does nothing

find "$MY_PATH"  -type d \( $FILTRE_1  \) -prune -o  \( $FILTRE_4 -exec bash -c 'CHERCHE "$0" ' {} \;  \)
#FILTER_4 does nothing

#Each command return 15 files because filter 3,4,5 , does nothing.

# 2  
Old 02-09-2017
Not clear. What is the problem?
- Do you want to eliminate eval?
- Does the overall logics not deliver what you need? Please be aware that a chain -o (OR operators) stops after the first to evaluate to TRUE.
- How are the two code snippets in above related? What and/or where is filter 5?
- Why does filter 2 work in the first snippet but doesn't in the second.
# 3  
Old 02-09-2017
I believe you don't need any quotes around the -name parameters as shell will expand the variables into a single argument anyway for example:

Code:
$ MY_PATH=/tmp/jcdole
$ mkdir -p $MY_PATH/{test,.local/share/{kscreen,ballo},.dbus}/{test{1,2},.direct{A,B,C},.Xauthority,.xsession1,.bash_history}
$ F1="  -path  $MY_PATH/.local/share/kscreen  -o -path  $MY_PATH/.local/share/ballo   -o -path  $MY_PATH/.dbus "
$ F2=" ! -name .direct*  ! -name .Xauthority  ! -name .xsession*  ! -name .bash_history "
$ find "$MY_PATH" -type d \( $F1 \)  -prune -o \( $F2 -print \)
/tmp/jcdole
/tmp/jcdole/.local
/tmp/jcdole/.local/share
/tmp/jcdole/test
/tmp/jcdole/test/test1
/tmp/jcdole/test/test2


Without 2nd Filter you get:

Code:
$ $ find "$MY_PATH" -type d \( $F1 \)  -prune -o -print
/tmp/jcdole
/tmp/jcdole/.local
/tmp/jcdole/.local/share
/tmp/jcdole/test
/tmp/jcdole/test/.bash_history
/tmp/jcdole/test/.directA
/tmp/jcdole/test/.directB
/tmp/jcdole/test/.directC
/tmp/jcdole/test/.Xauthority
/tmp/jcdole/test/.xsession1
/tmp/jcdole/test/test1
/tmp/jcdole/test/test2


Last edited by Chubler_XL; 02-09-2017 at 04:59 PM.. Reason: Added output without the 2nd filter
# 4  
Old 02-10-2017
Additional: if $MY_PATH has the potential to contain whitespace it gets a little more tricky. After some mucking around I did find this solution in bash using arrays:

Code:
$ MY_PATH=/tmp/jc\ dole
$ mkdir -p "$MY_PATH"/{test,.local/share/{kscreen,ballo},.dbus}/{test{1,2},.direct{A,B,C},.Xauthority,.xsession1,.bash_history}
$ F1=( -path "$MY_PATH"/.local/share/kscreen -o -path "$MY_PATH"/.local/share/ballo -o -path "$MY_PATH"/.dbus )
$ set -f
$ F2=(! -name .direct*  ! -name .Xauthority  ! -name .xsession*  ! -name .bash_history)
$ OLDIFS=$IFS
$ IFS=
$ find "$MY_PATH"  -type d \( ${F1[*]} \)  -prune -o \( ${F2[*]} -print \)
/tmp/jc dole
/tmp/jc dole/.local
/tmp/jc dole/.local/share
/tmp/jc dole/test
/tmp/jc dole/test/test1
/tmp/jc dole/test/test2
$ IFS=$OLDIFS
$ set +f

Note: set -f is required to stop glob expansion of wildcards within the F2 array as we want to pass the wildcards onto find

Last edited by Chubler_XL; 02-10-2017 at 04:48 AM..
# 5  
Old 02-10-2017
Given that there is no way to know what files might be present in the directory in which this script might be run, to guarantee that you won't get a syntax error in find if .direct* or .xsession* expand to more than one filename, and to guarantee that all files with the desired names will be matched if either of those patterns match a single filename in the current directory, I think it would be safer to change:
Code:
F2=(! -name .direct*  ! -name .Xauthority  ! -name .xsession*  ! -name .bash_history -print)

in the code Chubler_XL suggested in post #4 to:
Code:
F2=(! -name '.direct*'  ! -name .Xauthority  ! -name '.xsession*'  ! -name .bash_history -print)

(quoting those two filename matching patterns).
# 6  
Old 02-10-2017
Quote:
Originally Posted by RudiC
Not clear. What is the problem?
>- Do you want to eliminate eval?

yes.
I use { CMD="..." /// echo "$CMD" /// eval "$CMD" } when debuging scripts.

>- Does the overall logics not deliver what you need? Please be aware that a chain >-o (OR operators) stops after the first to evaluate to TRUE.
In brief I need this command :
find "path" "list of directories to ignore" and "list files to ignore" "pass result to function"
find "path" "FILTER_1" some_operator "FILTER_2" "pass result to function"
I want to ignore all directories enumerated in filter 1 (this work)
FILTER_1 is a list of directories
I want to ignore all files enumerated in filter 2 (this does not work together with FILTER_1)
FILTER_2 is a list of files

I have succeed with this command :
Code:
find "$MY_PATH"  -type d \( $FILTRE_1  \) -prune -o -exec bash -c 'CHERCHE "$0" '  {} \;

or
Code:
find "$MY_PATH"  \( $FILTRE_2 -exec bash -c 'CHERCHE "$0" '  {} \;

but not with
Code:
find "$MY_PATH"  -type d \( $FILTRE_1  \) -prune -o  \( $FILTRE_2 -exec bash -c 'CHERCHE \$0 '  {} \; \)

>- How are the two code snippets in above related? What and/or where is filter 5?
It is a typo error. you may read 2,3,4

>- Why does filter 2 work in the first snippet but doesn't in the second.
That the subject of my question.
How to convert this
Code:
CMD="find \"$MY_PATH\"  -type d \( $FILTRE_1  \) -prune -o  \( $FILTRE_2 -exec bash -c 'CHERCHE \$0 '  {} \; \)  " 
echo "COMMAND : $CMD" 
eval "$CMD"

by
Code:
find "$MY_PATH"  -type d \( $FILTRE_1  \) -prune -o  \( $FILTRE_2 -exec bash -c 'CHERCHE "$0" '  {} \; \)

FILTER_3 and FILTER_4 are the same filter but I try different manner to quote variable string.

---------- Post updated at 12:02 ---------- Previous update was at 11:36 ----------

Quote:
Originally Posted by Chubler_XL
I believe you don't need any quotes around the -name parameters as shell will expand the variables into a single argument anyway
You need quote if * is present
Code:
@ASUS-JC-NEW:~> find . -maxdepth 1 -name ".*"
.
./.config
./.fonts
./.local
./.bash_history

Code:
@ASUS-JC-NEW:~> find . -maxdepth 1 -name .*
find: paths must precede expression: ..
Usage: find [-H] [-L] [-P] [-Olevel] [-D help|tree|search|stat|rates|opt|exec] [path...] [expression]

and from manpage
Quote:
-name pattern
Base of file name (the path with the leading directories removed) matches shell
pattern pattern. Because the leading directories are removed, the file names
considered for a match with -name will never include a slash, so `-name a/b'
will never match anything (you probably need to use -path instead). The
metacharacters (`*', `?', and `[]') match a `.' at the start of the base name
(this is a change in findutils-4.2.2; see section STANDARDS CONFORMANCE below).
To ignore a directory and the files under it, use -prune; see an example in the
description of -path. Braces are not recognised as being special, despite the
fact that some shells including Bash imbue braces with a special meaning in
shell patterns. The filename matching is performed with the use of the
fnmatch(3) library function. Don't forget to enclose the pattern in quotes in
order to protect it from expansion by the shell
.
---------- Post updated at 12:12 ---------- Previous update was at 12:02 ----------

Thank you very much for your effort.
But I have to draw your attention on two things :
1°) FILTER_2 is a list of files and then ".Xauthority,.xsession1,.bash_history" are files.
2°) My real problem is when I try to add a second predicate (list of files in FILTER_2) after the first predicate ( FILTER_1 )

---------- Post updated at 12:24 ---------- Previous update was at 12:12 ----------

Quote:
Originally Posted by Don Cragun
Given that there is no way to know what files might be present in the directory in which this script might be run, to guarantee that you won't get a syntax error in find if .direct* or .xsession* expand to more than one filename, and to guarantee that all files with the desired names will be matched if either of those patterns match a single filename in the current directory ...............
If I run my command with FILTER_2 alone like this :
Code:
find "$MY_PATH"  \( $FILTRE_2 -exec bash -c 'CHERCHE "$0" '  {} \;

I did not get error, and the files
.xsession-errors
.xsession-errors-:0
are exclude as expected.

FILTER_2, FILTER_3 and FILTER_4 are the same filters but quoting is different.
But none give me the expected result.
FILTER_2 works only when I use
Code:
eval "$CMD"


Last edited by rbatte1; 02-10-2017 at 08:32 AM.. Reason: Added some ICODE tags
# 7  
Old 02-10-2017
Quote:
Originally Posted by Don Cragun
Given that there is no way to know what files might be present in the directory in which this script might be run, to guarantee that you won't get a syntax error in find if .direct* or .xsession* expand to more than one filename, and to guarantee that all files with the desired names will be matched if either of those patterns match a single filename in the current directory, I think it would be safer to change:
Code:
F2=(! -name .direct*  ! -name .Xauthority  ! -name .xsession*  ! -name .bash_history -print)

in the code Chubler_XL suggested in post #4 to:
Code:
F2=(! -name '.direct*'  ! -name .Xauthority  ! -name '.xsession*'  ! -name .bash_history -print)

(quoting those two filename matching patterns).
It still seems to match files in the current directory when expanding ${F2[*]}, only thing that I could find to avoid it was set -f:


Code:
$ MY_PATH=/tmp/jc\ dole
$ mkdir -p "$MY_PATH"/{test,.local/share/{kscreen,ballo},.dbus}/{test{1,2},.direct{A,B,C},.Xauthority,.xsession1,.bash_history}
$ F1=( -path "$MY_PATH"/.local/share/kscreen -o -path "$MY_PATH"/.local/share/ballo -o -path "$MY_PATH"/.dbus )
$ F2=(! -name '.direct*'  ! -name .Xauthority  ! -name '.xsession*'  ! -name .bash_history)
$ OLDIFS=$IFS
$ IFS=
$ touch .xsession_test
$ find "$MY_PATH"  -type d \( ${F1[*]} \)  -prune -o \( ${F2[*]} -print \)
/tmp/jc dole
/tmp/jc dole/.local
/tmp/jc dole/.local/share
/tmp/jc dole/test
/tmp/jc dole/test/.xsession1
/tmp/jc dole/test/test1
/tmp/jc dole/test/test2
$ IFS=$OLDIFS


Last edited by Chubler_XL; 02-10-2017 at 08:39 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Use parameter expansion over a parameter expansion in bash.

Hello All, Could you please do help me here as I would like to perform parameter expansion in shell over a parameter expansion. Let's say I have following variable. path="/var/talend/nat/cdc" Now to get only nat I could do following. path1="${path%/*}" path1="${path1##*/}" Here... (8 Replies)
Discussion started by: RavinderSingh13
8 Replies

2. Shell Programming and Scripting

Bash : More parameter expansion and IFS

I am trying to become more fluent with the interworking of bash and minimize the number of external calls. Sample Data. This will be the response of the snmp query. SNMPv2-MIB::sysName.0 = STRING: SomeHostName SNMPv2-MIB::sysObjectID.0 = OID: SNMPv2-SMI::enterprises.9.1.1745... (5 Replies)
Discussion started by: sumguy
5 Replies

3. Shell Programming and Scripting

Bash Parameter Expansion

#!/bin/bash SNMPW='/usr/bin/snmpwalk' while read h i do loc=$($SNMPW -v3 -u 'Myusername' -l authPriv -a SHA -A 'Password1' -x AES -X 'Password2' $i sysLocation.0 2>/dev/null) loc=${loc:-" is not snmpable."} loc=${loc##*: } loc=${loc//,/} echo "$i,$h,$loc" done < $1 My question is ... ... (1 Reply)
Discussion started by: sumguy
1 Replies

4. Shell Programming and Scripting

Bash Parameter Expansion

I have made the following examples that print various parameter expansions text: iv-hhz-sac/hpac/hhz.d/iv.hpac..hhz.d.2016.250.070018.sac (text%.*): iv-hhz-sac/hpac/hhz.d/iv.hpac..hhz.d.2016.250.070018 (text%%.*): iv-hhz-sac/hpac/hhz (text#*.): d/iv.hpac..hhz.d.2016.250.070018.sac... (2 Replies)
Discussion started by: kristinu
2 Replies

5. Shell Programming and Scripting

Bash variable expansion

Hello. The file /etc/fstab contains UUID=957c3295-9944-1593-82e2-2b90dede4312 / ext4 noatime,discard,acl,user_xattr 1 1 I fill a variable SOME_LINE=$( cat /etc/fstab | grep \/\..*ext4 | grep noatime,discard )echo $SOME_LINE... (3 Replies)
Discussion started by: jcdole
3 Replies

6. Shell Programming and Scripting

Bash shell expansion help

This is what I have in my directory. $ ls test1.txt test2.txt test3.txt test4.txt test5.txt test_script.sh This is what my shellscript looks like. #!/bin/bash for filename in /shell_expansion/*.txt; do for ((i=0; i<=3; i++)); do echo "$filename" ... (5 Replies)
Discussion started by: cokedude
5 Replies

7. Shell Programming and Scripting

Does SH support indirect expansion like BASH?

Hello, is there a kind soul who can answer me, does the SH support double substitution known as indirect expansion similar to BASH? The syntax for bash is ${!var}. For instance in bash I can write something like this: VAR="value" REF_VAR="VAR" echo ${!REF_VAR} and get the "value"... (1 Reply)
Discussion started by: dimentiy
1 Replies

8. Shell Programming and Scripting

Bash variable delayed expansion?

i write a batch file , here is the content. dirname='date +%Y-%m-%d' mkdir dirname but it doen's work, it just create a folder named date and +%Y-%m-%d. i have tried run the command seperately in the bash prompt. after the first sentence executed , i use $dirname to watch the value of... (4 Replies)
Discussion started by: premotheus
4 Replies

9. Shell Programming and Scripting

Brace expansion problem in Bash

I have a script that takes an option for server pools to run the script against. The option is given as a comma separated list (ie, -p 201,204,301). I'm using eval and brace expansion to get those pool numbers into an array. It works fine unless only 1 pool number is given. Here's the code: ... (5 Replies)
Discussion started by: mglenney
5 Replies

10. Shell Programming and Scripting

bash - delay expansion of variable

Hello - I have a bash script which does some logging, and I'd like to include the line number of the echo statement that pipes into $LOGGER: MYPID=$$ MYNAME=`basename $0` LOGGER="/usr/bin/logger -t $MYNAME($LINENO) -p daemon.error" ... echo 'this is an entry into the log file' | $LOGGER ... (3 Replies)
Discussion started by: scandora
3 Replies
Login or Register to Ask a Question