Korn expr substr fails for non-numeric value


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Korn expr substr fails for non-numeric value
# 15  
Old 12-26-2009
In AIX (including 5.3) /bin/ksh is a ksh88, whereas /bin/ksh93 is a ksh93. The substr()-function in the form "${var:start:length}" is only implemented in ksh93.

cfajohnsons observation is correct in a very general way: using ksh93-functions always runs the risk of getting less portable. On the other hand: using an external tool (expr, sed, awk, whatever) to trim the variables content is expensive in terms of system calls. I would like to offer the following solution to this problem, which which work in every ksh.version:

Code:
sub1=${DDNAME%${DDNAME#?????}}
sub=${sub1#???}

The first line extracts the first 5 characters from the string $DDNAME, the second statement extracts the last 2 from these 5 characters, effectively giving the substring starting at pos 4, length 2. Because this is done completely in the shell and without calling an external program it shoulld be by far faster than any solution based on such a program.

cfajohnson already explained how this can be tested for being an integer or not.

I hope this helps.

bakunin
# 16  
Old 12-27-2009
Quote:
Originally Posted by bakunin
In AIX (including 5.3) /bin/ksh is a ksh88, whereas /bin/ksh93 is a ksh93. The substr()-function in the form "${var:start:length}" is only implemented in ksh93.

And in bash since version 2 (and probably zsh).

Quote:
cfajohnsons observation is correct in a very general way: using ksh93-functions always runs the risk of getting less portable. On the other hand: using an external tool (expr, sed, awk, whatever) to trim the variables content is expensive in terms of system calls. I would like to offer the following solution to this problem, which which work in every ksh.version:

Code:
sub1=${DDNAME%${DDNAME#?????}}
sub=${sub1#???}


That will work in any POSIX shell.

The following substr functions will work in any POSIX shell and don't use any external commands.

Code:
_substr()
{
  _SUBSTR=

  ## store the parameters
  ss_str=$1
  ss_first=$2
  ss_length=${3:-${#ss_str}}

  ## return an error if the first character wanted is beyond end of string
  if [ $ss_first -gt ${#ss_str} ]
  then
    return 1
  fi

  if [ $ss_first -gt 1 ]
  then

    ## build a string of question marks to use as a wildcard pattern
    _repeat "?" $(( $ss_first - 1 ))

    ## remove the beginning of string
    ss_str=${ss_str#$_REPEAT}
  elif [ ${ss_first} -lt 0 ] ## ${#ss_str} ]
  then
    ## count from end of string
    _repeat "?" ${ss_first#-}

    ## remove the beginning
    ss_str=${ss_str#${ss_str%$_REPEAT}}
  fi

  ## ss_str now begins at the point we want to start extracting
  ## print the desired number of characters
  if [ ${#ss_str} -gt $ss_length ]
  then
    _repeat "${ss_wild:-??}" $ss_length
    ss_wild=$_REPEAT
    _SUBSTR=${ss_str%${ss_str#$ss_wild}}
  else
    _SUBSTR=${ss_str}
  fi
}

substr()
{
  _substr "$@" && printf "%s\n" "$_SUBSTR"
}

# 17  
Old 12-27-2009
Code:
_repeat: command not found

# 18  
Old 12-27-2009
Quote:
Originally Posted by Scrutinizer
Code:
_repeat: command not found


Code:
_repeat()
{
  ## If the first argument is -n, repeat the string N times
  ## otherwise repeat it to a length of N characters
  case $1 in
    -n) shift
    r_num=$(( ${#1} * $2 ))
    ;;
    *) r_num=$2
    ;;
  esac
  r_str=$1
  _REPEAT=$1
  while [ ${#_REPEAT} -lt ${r_num} ]
  do
    if [ $(( ${#_REPEAT} * 2 )) -gt $r_num ]
    then
      while [ ${#_REPEAT} -lt $r_num ]
      do
        _REPEAT=$_REPEAT$r_str
      done
    elif [ $(( ${#_REPEAT} * 2 )) -eq $r_num ]
    then
      _REPEAT=$_REPEAT$_REPEAT
    else
      ## The length builds rapidly by concatenating 3 copies in each loop
      ## Your results may be different, but I have found that three is
      ## the optimum number; try it with more, if you like
      _REPEAT=$_REPEAT$_REPEAT$_REPEAT
    fi
  done
  while [ ${#_REPEAT} -gt $r_num ]
  do
    _REPEAT=${_REPEAT%?}
  done
}

# 19  
Old 12-27-2009
We could also make use of the printf statement:
Code:
substr()
{
  string=$1
  pos=$2
  len=$3
  while [ $pos -gt 0 ]            #- Remove characters before pos
  do
    string=${string#?}
    pos=$(( pos-1 ))
  done
  printf "%.${len}s\n" "$string"  #- print len characters
}


Last edited by Scrutinizer; 12-27-2009 at 11:41 AM..
# 20  
Old 12-28-2009
I have not tested this, but if your input data only contains letters and numbers then maybe something like this would work.

extractNumbers()
{
oldIFS="$IFS"
IFS="abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
set -- $1
IFS=${oldIFS}
unset oldIFS # Old Bourne Shells do not support typeset on variable definiions
# remove empty arguments
for a in $*
do
# or add logic to use arithmetic operators to extract your number portion or whatever else you want before
# echoing the result back
echo $1
break
done
}

Last edited by nabramovitz; 12-28-2009 at 12:06 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Expr: non-numeric argument syntax error on line 1, teletype

Hi, I tried to look up the issue i'm experiencing, but i'm confused what's wrong with my script. After executing the script I'm getting the following error expr: non-numeric argument syntax error on line 1, teletype After some research, it seems that the problem relates to bc. I have... (1 Reply)
Discussion started by: nms
1 Replies

2. UNIX for Dummies Questions & Answers

Substr

awk '/^>/{id=$0;next}length>=7 { print id, "\n"$0}' Test.txt Can I use substr to achieve the same task? Thanks! (8 Replies)
Discussion started by: Xterra
8 Replies

3. Shell Programming and Scripting

Error with expr - "expr: syntax error"

Hi All, I'm writing a shell script in KSH, where I want to store the filename, total record count and actual record count of all the source files. The source files reside in 4 different sub-folders under the same root folder. Below is code: #!/usr/bin/ksh... (6 Replies)
Discussion started by: jagari
6 Replies

4. Shell Programming and Scripting

awk substr fails

Hi all, I want to get each line of a data file from position 464 plus 8 characters. I tried in two different ways, and the results were different. I'd like to know why. First method, using awk: awk '{print substr($0,464,8)}' CONCIL_VUELTA_ALF_100112_0801.okSecond method, using scripting:... (5 Replies)
Discussion started by: AlbertGM
5 Replies

5. UNIX for Dummies Questions & Answers

Find and Replace random numeric value with non-numeric value

Can someone tell me how to change the first column in a very large 17k line file from a random 10 digit numeric value to a non numeric value. The format of lines in the file is: 1702938475,SNU022,201004 the first 10 numbers always begin with 170 (6 Replies)
Discussion started by: Bahf1s
6 Replies

6. Shell Programming and Scripting

expr: non-numeric argument

Hi all, i am facing the error "expr: non-numeric argument" when i use the expr command. Following is the expression which i want to execute HR=$(echo `date +%H`) MIN=$(echo `date +%M`) TOT_MIN=`expr "$HR" \* 60+$MIN` | bc echo $TOT_MIN Here I am being reported with the error expr:... (6 Replies)
Discussion started by: sparks
6 Replies

7. Shell Programming and Scripting

help required for 'expr substr' function

hi iam trying to extract a certain portion of the string whose value is stored below,but am getting syntax eror.The command is shown below for file in GMG_BASEL2*.txt do m1= cat reporting_date.txt year= expr substr $m1 1 2 echo $year done m1 has date 10/31/2009 but this vale... (6 Replies)
Discussion started by: jagadeeshn04
6 Replies

8. Shell Programming and Scripting

test expr VS [ expr ]

What is the difference between test expr VS . For example : if test 5 -eq 6 echo "Wrong" and if echo "Wrong" bot will give the same output as Wrong. Now, what is the difference between these two? though they are producing the same result why we need two? Any answer will be... (2 Replies)
Discussion started by: ashok.g
2 Replies

9. Shell Programming and Scripting

substr() thru awk Korn Shell Script

Hi, I am new stuff to learn substr() function through awk for writing the Korn shell script. Is there a way to copy from XXXX1234.ABCDEF to XXX1234 file names without changing both data files? I appreciate your time to response this email. Thanks, Steve (4 Replies)
Discussion started by: sbryant
4 Replies

10. Shell Programming and Scripting

Perl code to differentiate numeric and non-numeric input

Hi All, Is there any code in Perl which can differentiate between numeric and non-numeric input? (11 Replies)
Discussion started by: Raynon
11 Replies
Login or Register to Ask a Question