Regex to hunt for a string in the right hand column


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Regex to hunt for a string in the right hand column
# 1  
Old 04-02-2017
Regex to hunt for a string in the right hand column

I have a database which has the following structure
Code:
English word=IPA notation

as in the example below
Code:
huckleberry=ˈhʌkəl-bəriː
huddling=ˈhʌd-lɪŋ
huffish=ˈhʌ-fɪʃ
hugger-mugger=ˈhʌgər-mʌgər
hulling=ˈhʌ-lɪŋ
human=ˈhjuː-mən
humanitarian=hjuːˌ-mænɪˈteərɪyən
humanitarianism=hjuːˌ-mænɪˈteərɪənɪzəm
humanitarians=hjuːˌ-mænɪˈteərɪyənz
humbler=ˈhʌmb-lər
humblest=ˈhʌmb-lest
humbling=ˈhʌmb-lɪŋ
humid=ˈhjuː-mɪd
humming-top=ˈhʌmɪŋ-tɒp
humming-tops=ˈhʌmɪŋ-tɒps
humourless=ˈhjuːmər-læs
hunchback=ˈhʌnʧ-bæk
hunchbacks=ˈhʌnʧ-bæks
hundredfold=ˈhʌndred-fold
hunger-march=ˈhʌŋər-mɑːrʧ
hunger-marcher=ˈhʌŋər-ˌmɑːrʧər
hunger-marchers=ˈhʌŋər-ˌmɑːrʧərz

I want to identify all instances of a hyphen in the right-hand column i.e. IPA and which are missing in the left-hand column i.e. the English Words. A small example is given below
Code:
humourless=ˈhjuːmər-læs
hunchback=ˈhʌnʧ-bæk
hunchbacks=ˈhʌnʧ-bæks
hundredfold=ˈhʌndred-fold

I work under windows but a Perl or even Unix regex could help. Many thanks
# 2  
Old 04-03-2017
With your sample data contained in a file named database.txt, any of the following commands:
Code:
grep '^[^-]*=.*-' database.txt

Code:
awk '/^[^-]*=.*-' database.txt

Code:
sed -n '/^[^-]*=.*-/p' database.txt

produce the output:
Code:
huckleberry=ˈhʌkəl-bəriː
huddling=ˈhʌd-lɪŋ
huffish=ˈhʌ-fɪʃ
hulling=ˈhʌ-lɪŋ
human=ˈhjuː-mən
humanitarian=hjuːˌ-mænɪˈteərɪyən
humanitarianism=hjuːˌ-mænɪˈteərɪənɪzəm
humanitarians=hjuːˌ-mænɪˈteərɪyənz
humbler=ˈhʌmb-lər
humblest=ˈhʌmb-lest
humbling=ˈhʌmb-lɪŋ
humid=ˈhjuː-mɪd
humourless=ˈhjuːmər-læs
hunchback=ˈhʌnʧ-bæk
hunchbacks=ˈhʌnʧ-bæks
hundredfold=ˈhʌndred-fold

As always, without knowing what utility will be using a regular expression, we don't know the type of regular expression nor how that RE will need to be delimited to get the results you want.
# 3  
Old 04-03-2017
Thanks a lot for your kind help. I am sorry I should have been more explicit.
I use Ultraedit which allows for regexes both under PERl and UNIX environments
Basically I want to plug in the regex in the editor and write a macro which will remove all such unwanted hyphens.
The regexes you provided work with Grep or Awk but when I plug them into the editor as Unix regexes, they do not identify correctly although the syntax seems correct.
Do not find a hyphen in column one but find in column 2
Thanks once again for your help
# 4  
Old 04-03-2017
Quote:
Originally Posted by gimley
Thanks a lot for your kind help. I am sorry I should have been more explicit.
I use Ultraedit which allows for regexes both under PERl and UNIX environments
Basically I want to plug in the regex in the editor and write a macro which will remove all such unwanted hyphens.
The regexes you provided work with Grep or Awk but when I plug them into the editor as Unix regexes, they do not identify correctly although the syntax seems correct.
Do not find a hyphen in column one but find in column 2
Thanks once again for your help
Please, try the following:
Code:
perl -ne '/^\w+=.+-/ and print'

Or test with any regex engine that suport Perl regex.
Code:
/^\w+=.+-/

This User Gave Thanks to Aia For This Post:
# 5  
Old 04-03-2017
Many thanks. The second syntax
Code:
/^\w+=.+-/

worked just fine.
# 6  
Old 04-04-2017
I am carrying the query a bit further as a matter of curiosity. The regex string in Perl
Code:
^\w+=.+-

checks for a hyphen on the right hand side of the database delimited by an
Code:
=

sign
whereas no hyphen exists in the left hand side.

Out of curiosity I reversed the regex as under to find all instances in the database where a hyphen exists in the left hand side but not in the right hand side as in the examples below:
Code:
a-bomb=ˈeɪbɒm
a-bombs=ˈeɪbɒmz
a-level=ˈeɪlevəl
a-levels=ˈeɪlevəlz

I tried the regex
Code:
.+-/=^\w+

but it failed and did not detect anything. Why did this fail?
What modification is needed? Thanks a lot for satisfying my curiosity.
# 7  
Old 04-04-2017
Quote:
Originally Posted by gimley
...
...
I reversed the regex as under to find all instances in the database where a hyphen exists in the left hand side but not in the right hand side as in the examples below:
Code:
a-bomb=ˈeɪbɒm
a-bombs=ˈeɪbɒmz
a-level=ˈeɪlevəl
a-levels=ˈeɪlevəlz

I tried the regex
Code:
.+-/=^\w+

but it failed and did not detect anything. Why did this fail?
What modification is needed? ...
A couple of things to note first:

(1) The "/" before the "=" is usually incorrect. By default, Perl uses forward-slashes as "pattern terminators" to match a pattern, like so:
Code:
if (/abc/) { <do-something> }

So a forward-slash inside the pattern will not work by default.
You could make it work, however, if you use non-default pattern terminators with the "m" (match) operator, like so:

Code:
if (m|abc/def|) { <do-something> }

Many other pattern terminators and even "pairs" are allowed: "[]", "{}", "()", "!!" etc.

(2) Note that the forward-slash was not present inside the pattern in the original solution. So, if your intention was to escape the "=", then the correct character is the back-slash "\". And even then, the "=" does not need to be escaped in Perl. It does not hold any special meaning.
So, now the regex is reduced to:
Code:
.+-=^\w+

(3) Next, notice the caret ("^"). It is a start-of-line anchor. It does not match any character. It matches a position: the "start of pattern". Hence it is always used at the start of the pattern. If you use it within the pattern, it will not match anything.
So, now the regex is reduced to:
Code:
^.+-=\w+

(4) Now if you take a step back and look at the original regex:

Code:
/^\w+=.+-/
/              => left pattern terminator
^              => beginning of pattern
\w+            => a word (by definition, a "word" does not have hyphens in Perl)
=              => followed by the "=" character
.+             => 1 or more occurrences of any character
-              => followed by a hyphen character
/              => right pattern terminator

In the original regex, we stop the moment we reach a "-" on the right hand side. We found what we wanted, so we take the relevant action.

But when you switch the patterns around the "=" character, that may not be true. So you put something after the "-" character on the left hand side, like so:
Code:
^.+-\w+=\w+

(5) Finally, you want to ensure that the word on the right hand side does not have any hyphens. The regex above matches a "word" on the right hand side, but if that "word" is followed by a hypen, it will still match it.

To ensure that you have a word without any hyphens on the right hand side till the end of the string, you put a "end of pattern" anchor: "$".
It is the counterpart of the "start of pattern" anchor: "^".
So now the regex becomes:
Code:
^.+-\w+=\w+$

And it should work with your data:

Code:
$
$ echo "a-levels=eilevelz" | perl -lne 'if (/^.+-\w+=\w+$/){print "matched"} else {print "unmatched"}'
matched
$
$ echo "alevels=ei-levelz" | perl -lne 'if (/^.+-\w+=\w+$/){print "matched"} else {print "unmatched"}'
unmatched
$
$ echo "a-levels=ei-levelz" | perl -lne 'if (/^.+-\w+=\w+$/){print "matched"} else {print "unmatched"}'
unmatched
$
$ echo "alevels=eilevelz" | perl -lne 'if (/^.+-\w+=\w+$/){print "matched"} else {print "unmatched"}'
unmatched
$
$

You could, alternatively use the regex
Code:
/^\w+-\w+=\w+$/


Last edited by durden_tyler; 04-04-2017 at 02:39 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Regex matching column awk

Hi all, I want to extract rows with the pattern ALPHANUMERIC/ALPHANUMNERIC in the 2nd column. I dont wan rows with more than 1 slash or without any slash in 2nd column. a a/b b a/b/c c a/b//c d t/y e r f /f I came up with the regex grep '\/$' file a a/b b a/b/c d t/y (3 Replies)
Discussion started by: jianp83
3 Replies

2. Shell Programming and Scripting

Search Replace Specific Column using RegEx

Have Pipe Delimited File: > BRYAN BAKER|4/4/2015|518 VIRGINIA AVE|TEST > JOE BAXTER|3/30/2015|2233 MockingBird RD|ROW2On 3rd column where the address is located, I want to add a space after every numeric value - basically doing a "s//&\ / ": > BRYAN BAKER|4/4/2015|5 1 8 VIRGINIA AVE|TEST > JOE... (5 Replies)
Discussion started by: svn
5 Replies

3. Shell Programming and Scripting

Grep with regex containing one string but not the other

Hi to you all, I'm just struggling with a regex problem and I'm pretty sure that I'm missing sth obvious... :confused: I need a regex to feed my grep in order to find lines that contain one string but not the other. Here's the data example: 2015-04-08 19:04:55,926|xxxxxxxxxx| ... (11 Replies)
Discussion started by: stresing
11 Replies

4. Shell Programming and Scripting

How to use regex on particular column (Removing comma from particular column)?

Hi, I have pipe separated file which contains some data having comma(,) in it. I want to remove the comma(,) only from particular column without changing data in other columns. Below is the sample data file, I want to remove the comma(,) only from 5th column. $ cat file1 ABC | DEF, HIJ|... (6 Replies)
Discussion started by: Prathmesh
6 Replies

5. Shell Programming and Scripting

Merge left hand strings mapping to different right hand strings

Hello, I am working on an Urdu to Hindi dictionary which has the following structure: a=b a=c n=d n=q and so on. i.e. Headword separated from gloss by a = I am giving below a live sample بتا=बता بتا=बित्ता بتا=बुत्ता بتان=बतान بتان=बितान بتانا=बिताना I need the following... (3 Replies)
Discussion started by: gimley
3 Replies

6. Shell Programming and Scripting

filtering out duplicate substrings, regex string from a string

My input contains a single word lines. From each line data.txt prjtestBlaBlatestBlaBla prjthisBlaBlathisBlaBla prjthatBlaBladpthatBlaBla prjgoodBlaBladpgoodBlaBla prjgood1BlaBla123dpgood1BlaBla123 Desired output --> data_out.txt prjtestBlaBla prjthisBlaBla... (8 Replies)
Discussion started by: kchinnam
8 Replies

7. Shell Programming and Scripting

Replace if regex on specific column matches expression?

I am attempting to convert rewrite rules to Nginx, and since due to the mass amount of rewrites we must convert, I've been trying to write a script to help me on a specific part, easily. So far I have this: rewrite ^action/static/(+)/$ staticPage.php?pg=$1&%$query_string; What I want done... (5 Replies)
Discussion started by: EXT3FSCK
5 Replies

8. UNIX for Dummies Questions & Answers

regex on first string in a variable.

Hi all, this is driving me nuts. I need to evaluate if a variable in a shell script has a heading s or m character e.g. s92342394 or m9233489 if so then I need to get rid of them. I'm quite familiar with PERL and could do it there in 3 mins but I have not found a decent way to do this in a shell.... (1 Reply)
Discussion started by: Endo
1 Replies

9. Shell Programming and Scripting

How to get the most left hand string ??

Hi, I remember once seeing a way to get the left most string in a word. Let's say: a="First.Second.Third" (separated by dot) echo ${a#*.} shows --> Second.Third echo ${a##*.} shows --> Third How do I get the the left most string "First" Or "First.Second" ??? Tried to replace #... (2 Replies)
Discussion started by: jfortes
2 Replies
Login or Register to Ask a Question