Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Grep with regex containing one string but not the other

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grep with regex containing one string but not the other
# 8  
Old 04-09-2015
Greedy matching is doing you in. All "(?!foobuzz)..." needs to find is one spot before user1 which doesn't start in 'f' and its condition is satisfied. The .* after it swallows everything. So it scans the first character, goes 'yay its not foobuzz', searches for the username and accepts.

So, you need to force the regex to check for (?!foobuzz) in a relevant spot. Right after a | character perhaps. And you have to make it check all of them so it won't cheat by skipping ahead.

You can match entire fields with [^|]*\|, and force them to not start with foobuzz via (?!foobuzz)[^|]*\|, match several in a row by wrapping it in ()* and force it to start at the beginning of the line with ^.

So ^((?!foobuzz)[^|]*\|)*UserID1 will only accept a line containing zero or more | fields, none of which begin with foobuzz, after which it must immediately find 'userid1'.

Code:
$ printf "2015-04-08 19:04:56,157|yyyyyyyyyy|          |foobuzz|          |INFO |REQUEST|UserID1:23 | ohnooo\n" |
        grep -P '^((?!foobuzz)[^|]*\|)*UserID1'

$ printf "2015-04-08 19:04:56,157|yyyyyyyyyy|          |frobuzz|          |INFO |REQUEST|UserID1:23 | ohnooo\n" | grep -P '^((?!foobuzz)[^|]*\|)*UserID1'
2015-04-08 19:04:56,157|yyyyyyyyyy|          |frobuzz|          |INFO |REQUEST|UserID1:23 | ohnooo

I don't think rewriting it in awk or sed would necessarily make it slower, but if you have a big pile of regexes already, could be a painful amount of work.
Last edited by Corona688; 04-09-2015 at 01:29 PM..
These 2 Users Gave Thanks to Corona688 For This Post:
disedorgue stresing
# 9  
Old 04-10-2015
Thank you for your explanation! Now I've got it! Smilie

Quote:
Originally Posted by Corona688
... rewriting ... could be a painful amount of work.
Yeah, you could bet on it... Smilie

---------- Post updated 04-10-15 at 04:56 PM ---------- Previous update was 04-09-15 at 07:23 PM ----------

D'oh, I cheered to early. The one-liner did the job, but now grep surprised me by saying "grep: the -P option only supports a single pattern". Ok, I will find a different solution by means of postprocessing.
# 10  
Old 04-10-2015
Well, if you're rewriting from scratch anyway, you could have a file full of awk expressions, like:

Code:
# Do NOT print line if this regex matches
/regex1/ { next }

# print line if it contains this literal string.  Faster than regex
index($0, "literalstring") { print ; next }

# print line if one regex matches and another does not
/regex1/ && !/regex2/ { print ; next }

# print line if any of these regexes match
/regex1/ || /regex2/ || /regex3/ || /regex4/ { print ; next }

etc..

You can use that file like
Code:
awk -f expressions.awk

with a filename optionally specified afterwards.

There is also a variant of awk that's further optimized for speed, mawk.
This User Gave Thanks to Corona688 For This Post:
stresing
# 11  
Old 04-10-2015
Hi.

There is a code, peg, that may do what you desire: multiple patterns of PERLRE expressions. Using the pattern from disedorgue and an augmented data file, a demo is:
Code:
#!/usr/bin/env bash

# @(#) s2	Demonstrate complex matching expressions: peg
# peg (various versions):
# http://www.cpan.org/authors/id/A/AD/ADAVIES/

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
LC_ALL=C ; LANG=C ; export LC_ALL LANG
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C peg

FILE=${1-data2}

pl " Input data file $FILE:"
cat $FILE

pl " Results:"
peg -e garble -e '\|(?!foobuzz)[^|]*\|[^|]*\|[^|]*\|[^|]*\|UserID1' $FILE

exit 0

producing:
Code:
$ ./s2

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian 5.0.8 (lenny, workstation) 
bash GNU bash 3.2.39
peg (local) 3.10

-----
 Input data file data2:
2015-04-08 19:04:55,926|xxxxxxxxxx|          |foobar|          |INFO |REQUEST|UserID1:42 | yeah
2015-04-08 19:04:56,157|yyyyyyyyyy|          |foobuzz|          |INFO |REQUEST|UserID1:23 | ohnooo
2015-04-08 19:04:56,157|yyyyyyyyyy|          |foobuz|          |INFO |REQUEST|UserID1:23 | ohnooo
garble

-----
 Results:
2015-04-08 19:04:55,926|xxxxxxxxxx|          |foobar|          |INFO |REQUEST|UserID1:42 | yeah
2015-04-08 19:04:56,157|yyyyyyyyyy|          |foobuz|          |INFO |REQUEST|UserID1:23 | ohnooo
garble

Here is what -e is defined as:
Code:
       -e overloaded
           -e PERLEXPR
               Specify a PERLEXPR to match.

               If used more than once, then it is equivalent to using -o.  For
               example, "peg -e foo -e bar baz", "peg -o foo bar -- baz", and
               "peg "/foo/ or /bar/" baz" are all equivalent.

Best wishes ... cheers, drl
This User Gave Thanks to drl For This Post:
stresing
# 12  
Old 04-10-2015
maybe a way with grep with option "-v", example:
my regex file:
Code:
$ cat reg.gr 
!UserID1
foobuzz

my file to parse:
Code:
$ cat gr.txt 
2015-04-08 19:04:55,926|xxxxxxxxxx|          |foobar|          |INFO |REQUEST|UserID1:42 | yeah
2015-04-08 19:04:56,157|yyyyyyyyyy|          |foobuzz|          |INFO |REQUEST|UserID1:23 | ohnooo
2015-04-08 19:04:56,157|yyyyyyyyyy|          |foobuz|          |INFO |REQUEST|UserID1:23 | ohnooo

The command grep:
Code:
$ grep -vf reg.gr gr.txt 
2015-04-08 19:04:55,926|xxxxxxxxxx|          |foobar|          |INFO |REQUEST|UserID1:42 | yeah
2015-04-08 19:04:56,157|yyyyyyyyyy|          |foobuz|          |INFO |REQUEST|UserID1:23 | ohnooo

$

Edit: No, it's wrong... not work
Regards.
Last edited by disedorgue; 04-10-2015 at 03:29 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Grep string with regex numeric characters

Hi all, I have the following entries in a file: Cause Indicators=80 90 Cause Indicators=80 90 Cause Indicators=82 90 Cause Indicators=82 90 Cause Indicators=82 90 The first 2 digits might change so I am after a sort of grep which could find any first 2 digits + the second 2,... (3 Replies)
Discussion started by: nms
3 Replies

2. Shell Programming and Scripting

grep regex, match exact string which includes "/" anywhere on line.

I have a file that contains the 2 following lines (from /proc/mounts) /dev/sdc1 /mnt/backup2 xfs rw,relatime,attr2,noquota 0 0 /dev/sdb1 /mnt/backup xfs rw,relatime,attr2,noquota 0 0 I need to match the string in the second column exactly so that only one result is returned, e.g. > grep... (2 Replies)
Discussion started by: jelloir
2 Replies

3. Shell Programming and Scripting

grep fixed string with regex

Hello, all! Maybe the title is badly formulated, you can help me with that...! I'm using the GNU grep, and I need to make sure that grep will extract only what I tell it to. I have the following regular expression: *? Well, I need to make sure I grep only a word which may start with a... (11 Replies)
Discussion started by: teresaejunior
11 Replies

4. Shell Programming and Scripting

filtering out duplicate substrings, regex string from a string

My input contains a single word lines. From each line data.txt prjtestBlaBlatestBlaBla prjthisBlaBlathisBlaBla prjthatBlaBladpthatBlaBla prjgoodBlaBladpgoodBlaBla prjgood1BlaBla123dpgood1BlaBla123 Desired output --> data_out.txt prjtestBlaBla prjthisBlaBla... (8 Replies)
Discussion started by: kchinnam
8 Replies

5. UNIX for Dummies Questions & Answers

Regex to match when input is not a certain string (can't use grep -v)

Hey everyone, Basically, all I'm looking for is a way to regex for not a certain string. The regex I'm looking to avoid matching is: D222 i.e. an equivalent of: awk '!/D222/' The problem is that I use this in the following command in a Bash script: ls ${source_directory} | awk... (1 Reply)
Discussion started by: kdelok
1 Replies

6. UNIX for Dummies Questions & Answers

| help | unix | grep (GNU grep) 2.5.1 | advanced regex syntax

Hello, I'm working on unix with grep (GNU grep) 2.5.1. I'm going through some of the newer regex syntax using Regular Expression Reference - Advanced Syntax a guide. ls -aLl /bin | grep "\(x\)" Which works, just highlights 'x' where ever, when ever. I'm trying to to get (?:) to work but... (4 Replies)
Discussion started by: MykC
4 Replies

7. UNIX for Dummies Questions & Answers

Help with grep and regex

Hi all, I'm a beginner with linux, regex, grep, etc I am trying to get data out of a file that has about 13,000 lines in this format name - location I want to grep all the names out to one file and the locations to another so I can put them into a spreadsheet. Some have hyphenated... (14 Replies)
Discussion started by: raichlea
14 Replies

8. UNIX for Dummies Questions & Answers

grep with Regex help!

Hello everybody, I'd like to know how is it I should write a regex in unix to match a string not followed by another string (anywhere in the line). To be more specific, I want to find lines where "drop table" is found, but not followed anywhere in the line by the character "&". For... (3 Replies)
Discussion started by: mvalonso
3 Replies

9. Shell Programming and Scripting

sed, grep, awk, regex -- extracting a matched substring from a file/string

Ok, I'm stumped and can't seem to find relevant info. (I'm not even sure, I might have asked something similar before.): I'm trying to use shell scripting/UNIX commands to extract URLs from a fairly large web page, with a view to ultimately wrapping this in PHP with exec() and including the... (2 Replies)
Discussion started by: ropers
2 Replies

10. UNIX for Dummies Questions & Answers

use of regex on grep

having a look on the regex site I saw that characters can be search using hex values http://www.regular-expressions.info/characters.html So I try to use it whith grep to find a è on a string (octal Decimal Hexa : 350 232 E8) but it doesn't work E.g. /usr/bin/echo '\0350' | egrep '\xE8' ... (0 Replies)
Discussion started by: solea
0 Replies
Login or Register to Ask a Question