Shell Programming and Scripting

I think I must be missing something obvious but I have a file containing a list of files and paths, some with wildcard, others with spaces. e.g.

Code:

/this/is/a/file
/this/is/a/directory/
/this/is/a/collection/*
/this/has spaces/in/it
/this/had spaces/and/list/of/files*
~/and/a/relative/file.txt

I need a script in sh or bash to expand this list of all matching files, something like:

Code:

while IFS= read -r  FILENAME; do ls -d1 $FILENAME; done <  ~/files.lst

The above example though will treat a space in any filename as separating the parameters, causing ls to see /this/had and space/in/it as separate files to list.

Escaping or quoting the entries with spaces within files.lst simply sees the quote or slash treated as being part of the file name, and so the space still separates the parameter.

And quoting $FILENAME in the ls command obviously solves the problem with spaces, but it then treats the wildcard as a filename character instead.

What am I missing to be able to take a variable containing a path with a space and a wildcard and expand it?

Thanks.

That's two problems that you can't solve with one generic approach. For your special setup above, try

Code:

while IFS= read -r  FILENAME; do echo "$FILENAME"; ls -d1 "${FILENAME%\*}"${FILENAME#${FILENAME%\*}}; done < file

You can set IFS="" for the entire loop, then it works for the read and the ls

Code:

(
IFS=
while read -r  FILENAME; do ls -d1 $FILENAME; done <  ~/files.lst
)
# subshell ended. The original IFS is back.

I knew it had to be something simple, I did not realize it was just a case of ls also needing the IFS to be null too. For my needs both solutions work perfectly but that version is the most flexible.

Thank you so much to you both.

The read command really uses the IFS.
The ls command does not use it, but the IFS lets the shell process the command arguments differently.
Thank you for this interesting question!

To be a bit more more specific, since IFS is set to the empty string, the expansion of the variable FILENAME, $FILENAME is not being split into fields by the shell and will be passed as a single field to the ls command, while wildcard expansion (globbing) is still being performed.

In a way it is only the shell using the IFS as read is a built-in command, at least for the shells I use.

Although I take your point, MadeInGermany. It affects how read interprets its input but that ls has no awareness of it as, as Scrutinizer says, it instead affects what the shell passes to it.

Thanks for the explanations, always more helpful to understand the hows and whys. (And thanks for showing me the icode tag!)