Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Tricky BASH quoting question

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Tricky BASH quoting question
# 1  
Old 04-05-2013
Tricky BASH quoting question
I have some data files that I can identify by a certain pattern in the names using find.

Every one of those data files has an XML file associated with it (can be multiple data files per XML file).

The XML file is always up one directory from the data file(s) in a folder called
Code:
Run##### - RunName

##### can be taken from the data file name but RunName can only be found within the XML file itself. Since I am trying to locate the XML file to read several parameters from it in addition to RunName, this does not help me. However, there is only one folder per run number, so I can use a wildcard '*'.

Using directory traversal, I am trying to find the XML file at
Code:
data_file_path/../Run####*/xml_filename.xml

The problem is that the path and run name both contain spaces and I need a way to expand the wild card while at the same time not parse the spaces.

Code:
XML=$(<$"xmlLocation")

gives No such file or directory because is is not expanding the '*'
Code:
XML=$(<$xmlLocation)

gives ambiguous redirect because the spaces are not quoted.

Because I am parsing several thousand of these files in a loop and in a Cygwin/windows environment where process/subshell creation is very expensive and slow, I am looking for a solution using built-in commands only but am interested in general solutions as well for my education.

Mike
Last edited by Michael Stora; 04-07-2013 at 01:35 AM.. Reason: spelling
# 2  
Old 04-06-2013
I can't get the problem explanation. Maybe I'm just dense. Or maybe would help to have more (or better) examples. Especially given the "tricky" nature of the problem.

Too bad about those space in the paths. Any chance that could get fixed up, so no spaces? It seems such a bother to have spaces in paths.
# 3  
Old 04-06-2013
You can't have it both ways, unfortunately.
# 4  
Old 04-06-2013
The terminal IS interactive, so why not experiment with it first!

Do you mean something like this:-

This lists all the files in my "Others" folder, shown here as "Oth*"

Note the inverted commas ARE important!

Note "Health and safety.txt" has spaces in it...

Code:
Last login: Sat Apr  6 11:04:17 on ttys000
Barrys-MacBook-Pro:~ barrywalker$ text="/Users/barrywalker/Desktop/Oth*"
Barrys-MacBook-Pro:~ barrywalker$ ls $text
1-8504-140043.pdf		Atheism.txt~			KEVIN.TXT			PyAudio.py.txt			Tesco-Net-Dump.txt
8outputDTMF.pdf			Atheism1.txt			KEVIN1.TXT			Python14.readme			The_Day.txt
AMIGA.TXT			Atheist-Quotes.txt		Kung-Fu.txt			Python3.html			The_Day.txt~
AMIGA_Peek_Mem.txt		BT-20-05-2012.html		LXF_POEM.TXT			Python3_files			URLs.txt
AMINET_INDEX			BT-20-05-2012_files		Limerick.txt			RF_Attenuator.txt		Upside-Down.txt
ANSI_Codes.txt			BWBASIC.TXT			Locate_Demo.py.txt		RF_Attenuator.txt~		Useful_commands.txt
AQOTD.txt			Backwards-compatibility.TXT	MMU-INFO.TXT			RF_PROBE_CALIB.pdf		Useful_commands.txt~
AQOTD_hymn.txt			DE737HJ.txt			MYAUDIO.TXT			Republican_Scandals.txt		V3.3Builtins.txt
ATHEIST.TXT			DEVICES.TXT			Mac_Forum_Insult.txt		Republicans.txt			Vertex_Sp_Mic.pdf
ATHEIST.TXT~			DSC_0534.jpg			Maths_Fuctions.txt		Rhetoric.rtf			Victims_of_Christianity.txt
ATHEIST_DONE.TXT		DV2036EA.PDF			Mullard-ValveData-1951.pdf	Root.txt			Web-Address.txt
ATHEIST_DONE.TXT~		DataLogger.txt			Newspaper-bias.txt		Root.txt~			Web-Host-Ad.txt
Agostino_reply.txt		Discussion.txt			Nursery_Rhymes.txt		SETH.txt			amigados_manual.pdf
AppleScript_For_Beginners.pdf	E-UAE_Shortcut_Keys.txt		Ode_To_The_AMIGA.TXT		SETH2.txt			hexdump
Apple_Licence_Xcode.rtf		EVENTS.TXT			PCI.TXT				SUDO.TXT			my_monthly_fix.txt
Atheism-02-02-2013		Enter_UEFI.txt			PEEKMEM.BAS			Scope_Info.txt			python1.pdf
Atheism-02-02-2013~		Foward_Compatibility_1.txt	PYGAME.TXT			Scope_Info.txt~			seth1.txt
Atheism-03-02-2013		God_Rap.txt			Park-Resorts-HQ.txt		Skype.pdf			seth4.txt
Atheism-03-02-2013~		HW.TXT				Patents.txt			Store				tkinter.pdf
Atheism.txt			Health and safety.txt		Pauls-number.txt		Tams_Work.txt			water.txt
Barrys-MacBook-Pro:~ barrywalker$

EDIT:

Code:
Last login: Sat Apr  6 12:02:55 on ttys000
Barrys-MacBook-Pro:~ barrywalker$ textone="/Users/barrywalker/"
Barrys-MacBook-Pro:~ barrywalker$ texttwo="Desk*/"
Barrys-MacBook-Pro:~ barrywalker$ ls $textone$texttwo
AppleScript Language	Code			Others
Barrys-MacBook-Pro:~ barrywalker$ textthree="*"
Barrys-MacBook-Pro:~ barrywalker$ ls $textone$texttwo$textthree
/Users/barrywalker/Desktop/AppleScript Language

/Users/barrywalker/Desktop/Code:
1KHz_Function_Generator_OSX.py		MCG2x.py
1KHz_Function_Generator_OSX.py~		MCG3x.py
1KHz_Sine_Generator_OSX.py		MPO.b
1KHz_Sine_Generator_OSX.py~		MPO2x.py
1KHz_Sine_Wave.py			MPO3x.py
1KHz_Sine_Wave.py~			MagicEye1.py
4BitVerticalBargraph2x.py		MagicEye3x.py
4BitVerticalBargraph3x.py		MagicEye_AMIGA.py
4BitVerticalBargraph3x_1.py		MagicEye_AMIGA.py~
8bitaccess.py				Magic_Eye.py
8bitaccess.py~				Max_Freq.py
AC2DC.py				Max_Freq.py~
AC2DC_OSX.py				Metronome2x.py
AC2DC_OSX.py~				Metronome3x.py

Last edited by wisecracker; 04-06-2013 at 08:09 AM.. Reason: Adding wildcards in drawers and files!
# 5  
Old 04-06-2013
Quote:
Originally Posted by wisecracker
The terminal IS interactive, so why not experiment with it first!
Your sample session is unrelated to the OP's problem because it does not utilize the same type of command substitution redirection and (more importantly) none of your variables contain both a space and a pattern matching wildcard.

A space causes field splitting to generate multiple tokens from a single token (this is the source of the "ambiguous redirect" error). Pathname wildcards are expanded after field splitting. Double quoting disables both of these expansions. So ,while it protects against field splitting, it prevents pathname expansion wildcards from doing their job.

Because field splitting occurs before pathname expansion, spaces that appear as a result of wildcard expansion (as in your example) are never problem (unless eval is used to run through the parser a second time).

Regards,
Alister
Last edited by alister; 04-06-2013 at 08:45 AM..
# 6  
Old 04-06-2013
Hi alister...

OK, then how about the back door?
Ignore the pattern match and just use the wildcards first...
I've used /tmp here just for convenience...

Code:
ls "/full path to/../Run*/xml_filename.xml" > /tmp/xml_files.txt

Then obtain the files with the same name "xml_filename.xml" by searching a contiguous plain text file for the paths and files...

Get something working first, (code to work), then see if it can be bettered, (work to code).

I must be missing something.

Anyhow apologies if I misread the OPs question...
# 7  
Old 04-06-2013
Quote:
Originally Posted by wisecracker
Code:
ls "/full path to/../Run*/xml_filename.xml" > /tmp/xml_files.txt

/tmp/xml_files.txt will be empty unless there's a directory whose name is literally "Run*".

Regards,
Alister
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with Bash quoting

I am trying to write a BASH script that will prompt a user to enter a number of days, then calculate the date. My problem is the date command uses single or double quotes. For Example.. date -d "7 days" Here is an example of some same code I am trying to work through. echo "when do you... (4 Replies)
Discussion started by: javajockey
4 Replies

2. Shell Programming and Scripting

Quoting / interface question

Hi, My first shell script is one that prints the five largest directories in a given directory. My current effort is as follows, it gives me the output I'd like, but I have to quote a globbed pathname, which seems wrong: #!/bin/sh du -hs $1 | sort -rn | head -n 5 And I must invoke... (2 Replies)
Discussion started by: aardymir
2 Replies

3. UNIX for Dummies Questions & Answers

Tricky GREP question..

I have some large login files that I need to extract (user)@(server) from. Where it gets tricky is that there is usually more than one '@' sign on each line(although it does have a leading space if it's not part of the (user)@(server) string), I need only the (user)@(server) section, I need only... (6 Replies)
Discussion started by: Mordaris
6 Replies

4. Shell Programming and Scripting

bash: correct quoting with find and exiv2?

Hi, I need to embed a metatag to image files which contain qrcodes, i usually do this with exiv -M "set Exif.Image.DocumentName `zbarimg -q -Sdisable -Sqrcode.enable --raw image.tif`" image.tif which works fine. However I need to do this recursivly for whole directory and subdiretory... (4 Replies)
Discussion started by: mcframe
4 Replies

5. Shell Programming and Scripting

Another tricky sed or awk question

This post is in reference to https://www.unix.com/shell-programming-scripting/137977-tricky-sed-awk-question-post302428154.html#post302428154 I am trying to go the opposite direction now: I have the following file: a,b,C,f,g a,b,D,f,g a,b,E,f,g h,i,J,k,l m,n,O,t,u m,n,P,t,u m,n,Q,t,u... (3 Replies)
Discussion started by: awayand
3 Replies

6. Shell Programming and Scripting

Quoting issue: Trouble with bash strings in script

I can do this on the command line: sqsh -S 192.168.x.x -o tmp -U user -P fakepass -D horizon -C "\ select second_id from borrower where btype like '%wsd%' " I can also just leave the SQL at the end intact on one line .... ... However, when I throw this in a script like: $SQSH -o... (4 Replies)
Discussion started by: Bubnoff
4 Replies

7. Shell Programming and Scripting

quoting question

hi guys, i have a question related to quoting but i am not sure how to formulate it... lets say we want to simulate the following shell actions cd ~/project-dir ctags /home/work/folder1/*.sh /home/work/folder2/*.sh /home/work/folder3/*.sh so i make the following script buidtags.sh ... (2 Replies)
Discussion started by: aegis
2 Replies

8. Shell Programming and Scripting

BASH quoting behavior

The block below isn't a surprise:$ ls file1 file2 file3 $ x=* $ echo $x file1 file2 file3 $ echo '$x' $x $ echo "$x" * $But I found this block a bit bewildering:$ echo $x' >' * $I'm wondering why substitution wasn't performed on the $x, since it was unquoted (as far as I can tell).... (5 Replies)
Discussion started by: na5m
5 Replies

9. UNIX for Dummies Questions & Answers

Tricky Quotation Question

Hi, I am at a point in my script where I defined the number of the command line parameter I would like to set a variable equal to: parameter_number=14 I would then like to set a variable equal to the correct parameter: variable=$parameter_number The issue here is that {} is required... (2 Replies)
Discussion started by: msb65
2 Replies

10. Shell Programming and Scripting

Tricky script question

Hi, I'm in the midst of writing a UNIX script that sftp's files to an external host and am stuck with a problem. The problem is that the files created on my server as a order number that correlates to a sequence of directories on the remote host which is where the file should be ftp'ed. ... (3 Replies)
Discussion started by: budrito
3 Replies
Login or Register to Ask a Question