Shell quoting rules for other languages programmers


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell quoting rules for other languages programmers
# 1  
Old 10-15-2012
Shell quoting rules for other languages programmers

I've seen so many times that programmers were confused about shell quoting and white spaces interpretation, that I decided to investigate that problem deeper. And my conclusion is that quoting in shells is very different from other programming languages. Programmers who have bigger experience in C/Python/Java/PHP than in Shell should remember about two unintuitive rules:
  • everything in shell is a string, and one need to quote everything properly. Also when reading variables: "$X", not only storing X="hi there"
  • shell will parse lines of code just one time, leaving unresolved nested variables and commands. In this case programmer need to explicitly use "eval" function.

I've written short article about this if anyone will be more curious you can find it under "white shell cofoh"
This User Gave Thanks to wyderkat For This Post:
# 2  
Old 10-17-2012
I favor the use of the single quote for fastest and most certain processing, as it expands nothing. You can switch to double wuotes where you want expansion or a literal single quote, and then go back to single quoting. Since there is no expansion, there is less processing. For file wild card expansion, or globbing, you need to be outside quotes. I rather like a pipe: "echo '....' | command" over a here file: "command <<EOD
....
EOD" as I can predict and view expansions easily.

The eval I save for odd situations, like constructing a variable name from a varible and then fetching it. It is a shell built-in, so probably faster than `sh -c 'echo $x'"$m"`
# 3  
Old 10-18-2012
You are right, globbing is not expanded in any quoting. And single quotes
suppose to be faster than double quotes. But is it big difference
in performance?

Before I've investigated that problem I was thinking that eval is for odd
situation, like you. But now it looks to me like we should use it quite often!
For example, can anybody tell me how to do last line without eval:

USER_A=tom
USER_B=geoff
USER_C=anna
CURRENT='$USER_B' # without single quotes it would be just value copy
eval echo $CURRENT
# 4  
Old 10-18-2012
The first rule about using eval is "don't do that". If you find yourself using eval, you've probably taken a wrong turn somewhere -- your program likely could've been solved in ways which don't need it, usually by representing data in a different way. eval just seems the most obvious since it's a big enough hammer to drive the most stubborn nails, even those ones with strange sharp spirals down their sides (screws, better driven with a screwdriver).

So, how would I do that in shell? I wouldn't. I'd use double-quotes and store the variable contents in the first place -- eval is almost certainly slower than double quotes!

Last edited by Corona688; 10-18-2012 at 12:32 PM..
# 5  
Old 10-18-2012
When I have just few screws (few lines of code) and my bed is falling apart without it (software crushes) and those screws have that strange head shape that non of my screwdriver matches (pure shell syntax) I will use hammer if I want to go sleep Smilie

But seriously using eval cannot be so harmful...It's just re-evaluation of strings.
# 6  
Old 10-18-2012
Yes, I know what you mean. I remember when I was starting with Linux, I didn't understand shell very well, and tried to write most of everything in Perl. This didn't make bash stupid, or eval necessary, I simply didn't know what I was doing.

Quote:
But seriously using eval cannot be so harmful...It's just re-evaluation of strings.
Yes. Yes, it can.

Imagine that you're feeding variables containing filenames into eval. Did you know that `rm -Rf ~/` is a valid filename?

Using eval lets external things inject whatever code they want into your program unless done very very carefully. If you don't know enough to avoid eval, you probably don't know how to use it safely either.

Last edited by Corona688; 10-18-2012 at 02:15 PM..
# 7  
Old 10-18-2012
Well, avoiding injection attacks and massive mistakes, with or without eval, often revolves around using quotes corrrectly and expecting all sorts of poorly behaved input data.

Rather than storing data in multiple variables with intelligence in the name, use the array. I read that in bash, simple arrays are actually implemented in a linked list, so adjust your performance expectation for big sets downward accordingly. If numeric keys and simple arrays are inconvenient, use the array-like hash container types that take string keys, in bash called associtive arrays. Hash (random) access scales much better. You have to go to PERL and up to get real arrays, other sorts of linked lists and trees as well as hash mapped containters.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. What is on Your Mind?

The poorest of computer users are programmers...

Hi guys and gals... A mildly humourous blog from 2013, but I come into this category... ;oDD Languager: The Poorest Computer Users are Programmers (0 Replies)
Discussion started by: wisecracker
0 Replies

2. What is on Your Mind?

Multics Systems Programmers Manual as of 1969-04-01, comprising 996 PDF Files

While working on my current "UNIX history project" I ran across this: Jerry Saltzer created an online scanned copy of the Multics Systems Programmers' Manual (MSPM) in this directory. Based on the 1969-04-01 MSPM, the repo comprising 996 PDF files. In order to help preserver the MSPM, ... (1 Reply)
Discussion started by: Neo
1 Replies

3. Forum Support Area for Unregistered Users & Account Problems

Trojan is detected when I visit Programmers forum

When I visit Programmers forum my ESET Nod32 detects TrojanDownloader.Pegel.BH.trojan I don't know when it started, as I visited the site today after staying off-line a few days. (2 Replies)
Discussion started by: migurus
2 Replies

4. Programming

Hello UNIX programmers

i have MOTIF installed X11 a easy program is saved as hello.c there is the following message where can i get the X11/intrinsic.h , file ??? need help to compile my system : MX-16 Linux Debian Jessie / i386 hans@mx1:~/Documents $ cc push.c -o push -lXm -lXt -lX11 In file included from... (0 Replies)
Discussion started by: Zabo
0 Replies

5. Shell Programming and Scripting

Shell quoting problem while editing a remote file using sed

value of i = solarisbox ssh $i "cat /etc/hosts | sed "s/$i\.local\.//" | sed "s/$i\./$i/" | sed "s/$i/$i.sol.com/" > /usr/users/chidori/edit_hosts"While running the above one liner its i am not able to make the changes and write it to the file /usr/users/chidori/edit_hosts . I know there is a... (2 Replies)
Discussion started by: chidori
2 Replies

6. What is on Your Mind?

Programming languages polyglots: how many languages you know?

Post what languages (including scripting) you know, why and where you think that language is most usable. Also include libraries in which you're really good at (libusb, gtk, qt, etc). assembly? C or C++? perl or python? pascal? bash or csh/tcsh? opengl? gtk or qt? mono? (27 Replies)
Discussion started by: redoubtable
27 Replies

7. What is on Your Mind?

Suggested venues to look for advanced C programmers

Can someone suggest any online venues to assist in recruiting a senior C programmer (looking for someone interested in working on kerberos code). I've tried a bunch of the open source and higher ed lists (this is for Univ. of Michigan). The commercial services such as Dice or monster yield a... (7 Replies)
Discussion started by: painman
7 Replies

8. News, Links, Events and Announcements

Good Essays For Programmers

Check the essays out. http://www.paulgraham.com (0 Replies)
Discussion started by: photon
0 Replies

9. UNIX for Advanced & Expert Users

Perl - Programmers manual online?

I have never programmed in Perl (insert laughter, mock, etc. here ____) - so I need a sort of "Programming in Perl" covering the basics. I now have two Perl books, one is a 5 volume Unix resource kit, the other is "Mastering algorithms with Perl" - none of them explains how to produce "hello,... (11 Replies)
Discussion started by: AtleRamsli
11 Replies

10. UNIX Desktop Questions & Answers

Shell scripting & programming languages

If I want to do high-end 3d animation, what skell scripting languages, and programming languages shoul I learn? If you know any good resources for learning these languages they would be appreciated. (1 Reply)
Discussion started by: aloysius1001
1 Replies
Login or Register to Ask a Question