ksh pattern matching


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting ksh pattern matching
# 1  
Old 02-10-2008
ksh pattern matching

I try to use the pattern substitution operators as documented in the O'Reilley "Learning the Korn Shell" but it doesn't seem to work as advertised.

This works all right:
Code:
var='Regular expressions rules!'
$ echo ${var//e/#}
R#gular #xpr#ssions rul#s!

The docs says that using !(expr) matches anything that does't match expr but if I try to replace all but the "e" character, it does not seem to work:
Code:
var='Regular expressions rules!'
$ echo ${var//!(e)/#}
#

Any idea?

Last edited by ripat; 02-10-2008 at 08:05 AM..
# 2  
Old 02-10-2008
Hi.

The newer shells, ksh and bash, have a lot of syntactical elements that are easily confused with one another.

The "Pattern Substitution Operators" syntax:
Code:
${variable_name}

can have a number of substitution operations with #, %, etc. They use the meta-characters, *, [], and ? -- page 123 ff, Learning the Korn Shell, 2nd Edition ("LTKS").

The "Patterns and Regular Expression" syntax uses:
Code:
*(exp), ?(exp), !(exp) ...

which correspond to the usual syntax we find in grep, etc:
Code:
grep "e*" ...

These patterns could be used within double brackets, for example:
Code:
if [[ $var == *!(e)* ]]

but not with string operator syntax (as far as I know) -- page 113 ff, 144 ff.

The ksh I use (pdksh, even on Solaris) notes a bad substitution for what I think is the right thing, but bash does it correctly in my opinion. Here's an example:
Code:
#!/bin/bash -
#!/bin/ksh -

# @(#) s1       Demonstrate string operators.

echo "(Versions displayed with local utility \"version\")"
version >/dev/null 2>&1 && version =o $(_eat $0 $1)

var='String operators rule!'
echo
echo " Replace e with _:"
echo ${var//e/_}

echo
echo " Replace everything except e with _:"
echo ${var//[^e]/_}

exit 0

Producing:
Code:
% ./s1
(Versions displayed with local utility "version")
Linux 2.6.11-x1
GNU bash 2.05b.0

 Replace e with _:
String op_rators rul_!

 Replace everything except e with _:
_________e__________e_

Perhaps someone will stop by with a better explanation or a better suggestion ... cheers, drl
# 3  
Old 02-10-2008
Quote:
Originally Posted by drl
The "Patterns and Regular Expression" syntax uses:
Code:
*(exp), ?(exp), !(exp) ...

(...) These patterns could be used within double brackets, for example (...) but not with string operator syntax (as far as I know) -- page 113 ff, 144 ff.
They do partially work in my ksh version (1993-12-28 r):

Code:
var='jo mike and dave are good friends'

$ echo ${var//a?(re)/_}
# returns > jo mike _nd d_ve _ good friends

$ echo ${var//g*(o)/_}
#returns > jo mike and dave are _d friends

$ echo ${var//+(o)/_}
#returns > j_ mike and dave are g_d friends

$ echo ${var//@(jo|dave)/_}
#returns > _ mike and _ are good friends

All returns as expected but I try to use the !(exp) like the PCRE look behind assertions (?<=exp). Still trying...
# 4  
Old 02-10-2008
It matters which version of ksh you are using. ksh93 has the // syntax while ksh88 does not. I am not sure about pdksh. On Solaris, dtksh is a souped vesion of ksh93. With dtksh...
Code:
$ /usr/dt/bin/dtksh
$ set -o emacs
$
$ x=hello
$ echo ${x//l/X} ${x//[!l]/X}
heXXo XXllX
$

# 5  
Old 02-10-2008
I assume you are talking about section 4.3 (String Operators) of LTKS

Code:
#!/usr/bin/ksh93

echo ${.sh.version}
var='A regular expressions test'

echo "1>  //e/#"
echo ${var//e/#}
echo "2>  //[^e]/#"
echo ${var//[^e]/#}
echo "3>  //+(e)/#"
echo ${var//+(e)/#}
echo "4>  //-(e)/#"
echo ${var//-(e)/#}
echo "5>  //?(e)/#"
echo ${var//?(e)/#}
echo "6>  //*(e)/#"
echo ${var//*(e)/#}
echo "7>  //!(e)/#"
echo ${var//!(e)/#}

Gives the following output

Code:
Version M 1993-12-28 s+
1>  //e/#
A r#gular #xpr#ssions t#st
2>  //[^e]/#
###e######e###e########e##
3>  //+(e)/#
A r#gular #xpr#ssions t#st
4>  //-(e)/#
A regular expressions test
5>  //?(e)/#
###########################
6>  //*(e)/#
###########################
7>  //!(e)/#
#

Interesting! I am not sure what is going on.

Last edited by fpmurphy; 02-10-2008 at 01:27 PM..
# 6  
Old 02-10-2008
Hi.

That's quite an array of very different results. If I were aiming for portability (which I usually am), I'd probably use the old standby sed:
Code:
#!/usr/dt/bin/dtksh
#!/usr/bin/ksh -
#!/usr/xpg4/bin/sh -
#!/bin/ksh -
#!/bin/bash -

# @(#) s2       Demonstrate pattern matching in dtksh and sed.

echo "(Versions displayed with local utility \"version\")"
version >/dev/null 2>&1 && version =o $(_eat $0 $1)

var='Regular expressions rules!'
echo
echo " Replace e with _:"
echo ${var//e/_}

echo
echo " Replace everything except e with _:"
echo re ${var//!(e)/_}
echo fe ${var//[!e]/_}

echo
echo " Replace everything except e with _ using sed:"
echo "$var" | sed -e 's|[^e]|_|g'

exit 0

Producing:
Code:
$ ./s2
(Versions displayed with local utility "version")
SunOS 5.10
dtksh M-12/28/93d

 Replace e with _:
R_gular _xpr_ssions rul_s!

 Replace everything except e with _:
re _
fe _e______e___e__________e__

 Replace everything except e with _ using sed:
_e______e___e__________e__

The re and fe above are regular expressions and filename expressions, intended to show the different syntax, and how one works and the other does not.

I got something out of this, namely dtksh, thanks to Perderabo. It's a bit tricky to find. I think it also carries a very large load of graphical baggage -- sort of like Tk (tcl/Tk). I dug through my very old pile of books, finding this:
Quote:
Title: Desktop KornShell Graphical Programming
Author: Pendergrast, Jr., J Stephen
Edition: 1st
Date: 1995
Publisher: Addison-Wesley Pub (Sd)
ISBN: 0-201-63375-2
Pages: 880
Categories: ksh, Korn Shell, scripting, unix, shell, programming, graphics
Comments: 4 stars ( Amazon, 4 reviews )
and at almost 1,000 pages, you can tell there is a lot. The size on Solaris X86 is 620144, even larger than bash.

You pays your money and you takes your chances (quoting either the cartoon character Popeye or one of my previous bosses Smilie ) ... cheers, drl
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Big pattern file matching within another pattern file in awk or shell

Hi I need to do a patten match between files . I am new to shell scripting and have come up with this so far. It take 50 seconds to process files of 2mb size . I need to tune this code as file size will be around 50mb and need to save time. Main issue is that I need to search the pattern from... (2 Replies)
Discussion started by: nitin_daharwal
2 Replies

2. UNIX for Dummies Questions & Answers

Grep -v lines starting with pattern 1 and not matching pattern 2

Hi all! Thanks for taking the time to view this! I want to grep out all lines of a file that starts with pattern 1 but also does not match with the second pattern. Example: Drink a soda Eat a banana Eat multiple bananas Drink an apple juice Eat an apple Eat multiple apples I... (8 Replies)
Discussion started by: demmel
8 Replies

3. Shell Programming and Scripting

PHP - Regex for matching string containing pattern but without pattern itself

The sample file: dept1: user1,user2,user3 dept2: user4,user5,user6 dept3: user7,user8,user9 I want to match by '/^dept2.*/' but don't want to have substring 'dept2:' in output. How to compose such regex? (8 Replies)
Discussion started by: urello
8 Replies

4. Shell Programming and Scripting

Sed: printing lines AFTER pattern matching EXCLUDING the line containing the pattern

'Hi I'm using the following code to extract the lines(and redirect them to a txt file) after the pattern match. But the output is inclusive of the line with pattern match. Which option is to be used to exclude the line containing the pattern? sed -n '/Conn.*User/,$p' > consumers.txt (11 Replies)
Discussion started by: essem
11 Replies

5. UNIX for Dummies Questions & Answers

Find pattern suffix matching pattern

Hi, I am trying to get a result out of this but fails please help. Have two files /tmp/1 & /tmp/hosts. /tmp/1 IP=123.456.789.01 WAS_HOSTNAME=abcdefgh.was.tb.dsdc /tmp/hosts 123.456.789.01 I want this result in /tmp/hosts if hostname is already there dont want duplicate entry. ... (5 Replies)
Discussion started by: rajeshwebspere
5 Replies

6. Shell Programming and Scripting

ksh Multiple Pattern Matching Operators

I figured this would be simple, but I am stuck. Variable longpath="/dir1/dir2/dir3/filename.stuff.morestuff.garbage" I want to end up with just "filename.extra.moreextra". So, I want to get rid of the path and .garbage I want to do this with just ksh internals. So, no sed,grep,awk,expr,... (4 Replies)
Discussion started by: Topaz
4 Replies

7. Shell Programming and Scripting

Pattern matching in ksh

In the Korn shell is it possible to truncate a pattern like this: - ) ]] So that it is not necessary to enter 8 times? (2 Replies)
Discussion started by: steadyonabix
2 Replies

8. Shell Programming and Scripting

sed - matching pattern one but not pattern two

All, I have the following file: -------------------------------------- # # /etc/pam.d/common-password - password-related modules common to all services # # This file is included from other service-specific PAM config files, # and should contain a list of modules that define the services... (2 Replies)
Discussion started by: RobertBerrie
2 Replies

9. Shell Programming and Scripting

counting the lines matching a pattern, in between two pattern, and generate a tab

Hi all, I'm looking for some help. I have a file (very long) that is organized like below: >Cluster 0 0 283nt, >01_FRYJ6ZM12HMXZS... at +/99% 1 279nt, >01_FRYJ6ZM12HN12A... at +/99% 2 281nt, >01_FRYJ6ZM12HM4TS... at +/99% 3 283nt, >01_FRYJ6ZM12HM946... at +/99% 4 279nt,... (4 Replies)
Discussion started by: d.chauliac
4 Replies

10. Shell Programming and Scripting

comment/delete a particular pattern starting from second line of the matching pattern

Hi, I have file 1.txt with following entries as shown: 0152364|134444|10.20.30.40|015236433 0233654|122555|10.20.30.50|023365433 ** ** ** In file 2.txt I have the following entries as shown: 0152364|134444|10.20.30.40|015236433 0233654|122555|10.20.30.50|023365433... (4 Replies)
Discussion started by: imas
4 Replies
Login or Register to Ask a Question