ksh pattern matching | Unix Linux Forums | Shell Programming and Scripting

  Go Back    


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

ksh pattern matching

Shell Programming and Scripting


Tags
linux, regex, regular expressions

Closed Thread    
 
Thread Tools Search this Thread Display Modes
    #1  
Old 02-10-2008
ripat ripat is offline Forum Advisor  
Registered User
 
Join Date: Oct 2006
Last Activity: 17 February 2014, 7:51 AM EST
Location: Belgium
Posts: 537
Thanks: 4
Thanked 38 Times in 28 Posts
ksh pattern matching

I try to use the pattern substitution operators as documented in the O'Reilley "Learning the Korn Shell" but it doesn't seem to work as advertised.

This works all right:

Code:
var='Regular expressions rules!'
$ echo ${var//e/#}
R#gular #xpr#ssions rul#s!

The docs says that using !(expr) matches anything that does't match expr but if I try to replace all but the "e" character, it does not seem to work:

Code:
var='Regular expressions rules!'
$ echo ${var//!(e)/#}
#

Any idea?

Last edited by ripat; 02-10-2008 at 07:05 AM..
Sponsored Links
    #2  
Old 02-10-2008
drl's Avatar
drl drl is online now Forum Advisor  
Registered Voter
 
Join Date: Apr 2007
Last Activity: 25 October 2014, 10:31 AM EDT
Location: Saint Paul, MN USA / BSD, CentOS, Debian, OS X, Solaris
Posts: 1,686
Thanks: 42
Thanked 197 Times in 179 Posts
Hi.

The newer shells, ksh and bash, have a lot of syntactical elements that are easily confused with one another.

The "Pattern Substitution Operators" syntax:

Code:
${variable_name}

can have a number of substitution operations with #, %, etc. They use the meta-characters, *, [], and ? -- page 123 ff, Learning the Korn Shell, 2nd Edition ("LTKS").

The "Patterns and Regular Expression" syntax uses:

Code:
*(exp), ?(exp), !(exp) ...

which correspond to the usual syntax we find in grep, etc:

Code:
grep "e*" ...

These patterns could be used within double brackets, for example:

Code:
if [[ $var == *!(e)* ]]

but not with string operator syntax (as far as I know) -- page 113 ff, 144 ff.

The ksh I use (pdksh, even on Solaris) notes a bad substitution for what I think is the right thing, but bash does it correctly in my opinion. Here's an example:

Code:
#!/bin/bash -
#!/bin/ksh -

# @(#) s1       Demonstrate string operators.

echo "(Versions displayed with local utility \"version\")"
version >/dev/null 2>&1 && version =o $(_eat $0 $1)

var='String operators rule!'
echo
echo " Replace e with _:"
echo ${var//e/_}

echo
echo " Replace everything except e with _:"
echo ${var//[^e]/_}

exit 0

Producing:

Code:
% ./s1
(Versions displayed with local utility "version")
Linux 2.6.11-x1
GNU bash 2.05b.0

 Replace e with _:
String op_rators rul_!

 Replace everything except e with _:
_________e__________e_

Perhaps someone will stop by with a better explanation or a better suggestion ... cheers, drl
Sponsored Links
    #3  
Old 02-10-2008
ripat ripat is offline Forum Advisor  
Registered User
 
Join Date: Oct 2006
Last Activity: 17 February 2014, 7:51 AM EST
Location: Belgium
Posts: 537
Thanks: 4
Thanked 38 Times in 28 Posts
Quote:
Originally Posted by drl View Post
The "Patterns and Regular Expression" syntax uses:

Code:
*(exp), ?(exp), !(exp) ...

(...) These patterns could be used within double brackets, for example (...) but not with string operator syntax (as far as I know) -- page 113 ff, 144 ff.
They do partially work in my ksh version (1993-12-28 r):


Code:
var='jo mike and dave are good friends'

$ echo ${var//a?(re)/_}
# returns > jo mike _nd d_ve _ good friends

$ echo ${var//g*(o)/_}
#returns > jo mike and dave are _d friends

$ echo ${var//+(o)/_}
#returns > j_ mike and dave are g_d friends

$ echo ${var//@(jo|dave)/_}
#returns > _ mike and _ are good friends

All returns as expected but I try to use the !(exp) like the PCRE look behind assertions (?<=exp). Still trying...
    #4  
Old 02-10-2008
Perderabo's Avatar
Perderabo Perderabo is offline Forum Staff  
Unix Daemon (Administrator Emeritus)
 
Join Date: Aug 2001
Last Activity: 24 October 2014, 3:06 PM EDT
Location: Ashburn, Virginia
Posts: 9,930
Thanks: 64
Thanked 437 Times in 261 Posts
It matters which version of ksh you are using. ksh93 has the // syntax while ksh88 does not. I am not sure about pdksh. On Solaris, dtksh is a souped vesion of ksh93. With dtksh...
Code:
$ /usr/dt/bin/dtksh
$ set -o emacs
$
$ x=hello
$ echo ${x//l/X} ${x//[!l]/X}
heXXo XXllX
$

Sponsored Links
    #5  
Old 02-10-2008
fpmurphy's Avatar
fpmurphy fpmurphy is offline Forum Staff  
who?
 
Join Date: Dec 2003
Last Activity: 25 October 2014, 1:14 AM EDT
Location: /dev/ph
Posts: 4,823
Thanks: 64
Thanked 430 Times in 399 Posts
I assume you are talking about section 4.3 (String Operators) of LTKS


Code:
#!/usr/bin/ksh93

echo ${.sh.version}
var='A regular expressions test'

echo "1>  //e/#"
echo ${var//e/#}
echo "2>  //[^e]/#"
echo ${var//[^e]/#}
echo "3>  //+(e)/#"
echo ${var//+(e)/#}
echo "4>  //-(e)/#"
echo ${var//-(e)/#}
echo "5>  //?(e)/#"
echo ${var//?(e)/#}
echo "6>  //*(e)/#"
echo ${var//*(e)/#}
echo "7>  //!(e)/#"
echo ${var//!(e)/#}

Gives the following output


Code:
Version M 1993-12-28 s+
1>  //e/#
A r#gular #xpr#ssions t#st
2>  //[^e]/#
###e######e###e########e##
3>  //+(e)/#
A r#gular #xpr#ssions t#st
4>  //-(e)/#
A regular expressions test
5>  //?(e)/#
###########################
6>  //*(e)/#
###########################
7>  //!(e)/#
#

Interesting! I am not sure what is going on.

Last edited by fpmurphy; 02-10-2008 at 12:27 PM..
Sponsored Links
    #6  
Old 02-10-2008
drl's Avatar
drl drl is online now Forum Advisor  
Registered Voter
 
Join Date: Apr 2007
Last Activity: 25 October 2014, 10:31 AM EDT
Location: Saint Paul, MN USA / BSD, CentOS, Debian, OS X, Solaris
Posts: 1,686
Thanks: 42
Thanked 197 Times in 179 Posts
Hi.

That's quite an array of very different results. If I were aiming for portability (which I usually am), I'd probably use the old standby sed:

Code:
#!/usr/dt/bin/dtksh
#!/usr/bin/ksh -
#!/usr/xpg4/bin/sh -
#!/bin/ksh -
#!/bin/bash -

# @(#) s2       Demonstrate pattern matching in dtksh and sed.

echo "(Versions displayed with local utility \"version\")"
version >/dev/null 2>&1 && version =o $(_eat $0 $1)

var='Regular expressions rules!'
echo
echo " Replace e with _:"
echo ${var//e/_}

echo
echo " Replace everything except e with _:"
echo re ${var//!(e)/_}
echo fe ${var//[!e]/_}

echo
echo " Replace everything except e with _ using sed:"
echo "$var" | sed -e 's|[^e]|_|g'

exit 0

Producing:

Code:
$ ./s2
(Versions displayed with local utility "version")
SunOS 5.10
dtksh M-12/28/93d

 Replace e with _:
R_gular _xpr_ssions rul_s!

 Replace everything except e with _:
re _
fe _e______e___e__________e__

 Replace everything except e with _ using sed:
_e______e___e__________e__

The re and fe above are regular expressions and filename expressions, intended to show the different syntax, and how one works and the other does not.

I got something out of this, namely dtksh, thanks to Perderabo. It's a bit tricky to find. I think it also carries a very large load of graphical baggage -- sort of like Tk (tcl/Tk). I dug through my very old pile of books, finding this:
Quote:
Title: Desktop KornShell Graphical Programming
Author: Pendergrast, Jr., J Stephen
Edition: 1st
Date: 1995
Publisher: Addison-Wesley Pub (Sd)
ISBN: 0-201-63375-2
Pages: 880
Categories: ksh, Korn Shell, scripting, unix, shell, programming, graphics
Comments: 4 stars ( Amazon, 4 reviews )
and at almost 1,000 pages, you can tell there is a lot. The size on Solaris X86 is 620144, even larger than bash.

You pays your money and you takes your chances (quoting either the cartoon character Popeye or one of my previous bosses ) ... cheers, drl
Sponsored Links
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
awk - pattern matching? citizencro UNIX for Dummies Questions & Answers 4 05-08-2011 08:32 PM
counting the lines matching a pattern, in between two pattern, and generate a tab d.chauliac Shell Programming and Scripting 4 03-19-2009 01:30 PM
comment/delete a particular pattern starting from second line of the matching pattern imas Shell Programming and Scripting 4 10-13-2008 02:37 AM
pattern matching mercuryshipzz Shell Programming and Scripting 4 01-14-2008 10:01 PM
AWK pattern matching, first and last smb_uk Shell Programming and Scripting 10 12-27-2007 08:03 PM



All times are GMT -4. The time now is 10:33 AM.