Home Man
Search
Today's Posts
Register

BSD, Linux, and UNIX shell scripting — Post awk, bash, csh, ksh, perl, php, python, sed, sh, shell scripts, and other shell scripting languages questions here.

ksh hidden characters in variables

Tags
ksh, rhel 6, shell scripts

Login to Reply

 
Thread Tools Search this Thread
# 1  
Old 11-16-2017
ksh hidden characters in variables

Hi. I'm getting the following hidden characters
Code:
\u[2013]

at the start of a string after I pass in variables from the command line. I only noticed this when I set -x in my KSH script. Can anybody tell me how this happens and how to remove them?
Many thanks.

Code:
+ STR=$'\u[2013]username testuser1'
+ print $'STR Variable: \u[2013]username testuser1'
STR Variable: username testuser1

# 2  
Old 11-17-2017
I think it may be an UTF-16 character.

What is the output of the following command on your command line:
Code:
locale

# 3  
Old 11-17-2017
Looks like it's UTF-8. Any idea how can I get rid of this hidden character?

Code:
+ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

# 4  
Old 11-17-2017
How did you call the script? What exactly is in the script?
Did you mayhap use a strange dash character?


Code:
U+2013 EN DASH

System	Representation
Nº	8211
UTF-8	E2 80 93
UTF-16	20 13                  
UTF-32	00 00 20 13
URL-Quoted 	%E2%80%93
HTML-Escape	–


Last edited by RudiC; 11-17-2017 at 05:19 AM..
# 5  
Old 11-17-2017
Yes indeed there is some old code preceding this that reads in the params from the command line:

Code:
while [ -n "$1" ]; do
        case $1 in
                -username)       if [ "`echo $2 | grep -e '^-[a-z]'`" ]; then { echo "missing value for '$1' (seen '$2')"; usage; exit 1; } else { shift; USERNAME=$1;   } fi ;;
                -surname)    if [ "`echo $2 | grep -e '^-[a-z]'`" ]; then { echo "missing value for '$1' (seen '$2')"; usage; exit 1; } else { shift; SURNAME=$1;     } fi ;;
                -address)    if [ "`echo $2 | grep -e '^-[a-z]'`" ]; then { ADDRESS=NO; }    else { shift; ADDRESS=$1;     } fi ;;
                -startdate)  if [ "`echo $2 | grep -e '^-[a-z]'`" ]; then { STARTDATE=NO; UNTIL=NO;}   else { shift; STARTDATE=$1; UNTIL=$1;  } fi ;;
                -req)        if [ "`echo $2 | grep -e '^-[a-z]'`" ]; then { REQ=Y; }    else { shift; REQ=$1;    } fi ;;
                -opt)
                        if [ -n "$2" ]; then
                                shift; OPT=$*
                                break
                        else
                                OPT=NOTSET
                        fi
                        ;;
                *)             echo "Invalid argument '$1'"; exit 1;;
        esac
        shift
done

I just want to parse the OPT variable after this.

---------- Post updated at 09:44 PM ---------- Previous update was at 09:53 AM ----------

These hidden characters are added on the line:
Code:
shift; OPT=$*


Last edited by user052009; 11-17-2017 at 10:30 PM..
# 6  
Old 11-17-2017
Quote:
Originally Posted by user052009
Yes indeed there is some old code preceding this that reads in the params from the command line:

Code:
while [ -n "$1" ]; do
        case $1 in
                -username)       if [ "`echo $2 | grep -e '^-[a-z]'`" ]; then { echo "missing value for '$1' (seen '$2')"; usage; exit 1; } else { shift; USERNAME=$1;   } fi ;;
                -surname)    if [ "`echo $2 | grep -e '^-[a-z]'`" ]; then { echo "missing value for '$1' (seen '$2')"; usage; exit 1; } else { shift; SURNAME=$1;     } fi ;;
                -address)    if [ "`echo $2 | grep -e '^-[a-z]'`" ]; then { ADDRESS=NO; }    else { shift; ADDRESS=$1;     } fi ;;
                -startdate)  if [ "`echo $2 | grep -e '^-[a-z]'`" ]; then { STARTDATE=NO; UNTIL=NO;}   else { shift; STARTDATE=$1; UNTIL=$1;  } fi ;;
                -req)        if [ "`echo $2 | grep -e '^-[a-z]'`" ]; then { REQ=Y; }    else { shift; REQ=$1;    } fi ;;
                -opt)
                        if [ -n "$2" ]; then
                                shift; OPT=$*
                                break
                        else
                                OPT=NOTSET
                        fi
                        ;;
                *)             echo "Invalid argument '$1'"; exit 1;;
        esac
        shift
done

I just want to parse the OPT variable after this.

---------- Post updated at 09:44 PM ---------- Previous update was at 09:53 AM ----------

These hidden characters are added on the line:
Code:
shift; OPT=$*

If the characters are stored in OPT by the command OPT=$*, then those characters were typed on the command line when your script was invoked.

One would not expect this if your script was invoked by typing commands into a shell.

One would not expect this if your script was invoked by a shell script edited with ed, ex, vi or another common UNIX system text editor.

One would expect this if your script was invoked by a shell script edited with a text editor designed to produce pretty-printed text (such as Microsoft word or Apple pages).

Last edited by Don Cragun; 11-17-2017 at 10:55 PM.. Reason: Add emphasis.
# 7  
Old 11-18-2017
I second Don Cragun in his third expectation; that's why I asked how you called the script (which unfortunately you did not answer). It looks like your parent script scrambles character (set)s as the case statement evaluates and recognizes -OPT correctly but then obviously fails when providing the option values themselves.
Login to Reply

« Previous Thread | Next Thread »
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Rsync - how to copy hidden folder or hidden files when using full path jcdole Shell Programming and Scripting 2 10-09-2017 07:12 AM
Hidden characters when pasting in vi/vim dampio UNIX for Beginners Questions & Answers 3 12-02-2016 12:59 PM
Hidden Characters in Regular Expression Matching Perl - Perl Newbie kittyluva2 Shell Programming and Scripting 4 07-28-2011 01:31 AM
How to see hidden characters..... Bashingaway Shell Programming and Scripting 8 11-03-2010 11:49 AM
hidden characters added to commands josebenitez Shell Programming and Scripting 2 09-20-2010 07:16 PM
Hidden control characters in a Unix Text File! kewl_guy Shell Programming and Scripting 6 09-30-2008 01:09 PM
hidden Characters Khoomfire UNIX for Advanced & Expert Users 5 07-23-2008 09:11 AM
Hidden Characters Khoomfire UNIX for Advanced & Expert Users 6 07-06-2007 04:40 AM
missing hidden characters SUSANR9999 UNIX for Dummies Questions & Answers 2 11-09-2006 03:30 PM


All times are GMT -4. The time now is 02:02 AM.

Unix & Linux Forums Content Copyright©1993-2018. All Rights Reserved.
UNIX.COM Login
Username:
Password:  
Show Password