Sponsored Content
Top Forums Shell Programming and Scripting Parsing the longest match substring Post 302992875 by Don Cragun on Thursday 2nd of March 2017 09:42:42 PM
Old 03-02-2017
With your sample data, the following seems to do what you want fairly efficiently:
Code:
awk -F// -v OFS="\t\t" '
BEGIN {	M = 0
	m = 32000
}
FNR == NR {
	if((l = length($0)) in pat) {
		pat[l] = pat[l] "|" $0
		#printf("appending pattern: pat[%d]=\"%s\"\n", l, pat[l])
	} else {pat[l] = $0
		#printf("adding pattern: pat[%d]=\"%s\"\n", l, pat[l])
		if(l > M)
			M = l
		if(l < m)
			m = l
	}
	next
}
#FNR == 1 {
#	for(i = M; i >= m; i--)
#		if(i in pat)
#			printf("pat[%d]=\"%s\"\n", i, pat[i])
#}
{	short = length($1) < length($2) ? 1 : 2
	for(i = M; i >= m; i--)
		if(i in pat && match($short, pat[i])) {
			print $0, substr($short, RSTART, RLENGTH)
			next
		}
	print $0, "No match"
}' primary scrambled

but I don't have any good way to test how this might work if you have EREs with thousands of patterns to be matched as alternatives in a single ERE. With your sample inputs, it produces:
Code:
())[pt22dx]dfdfs//(dsgfspp22dx/pg22dx)-signal		pt22dx
(\\b-[pt22dx]dfdfs/(dsgfspp22dx//[(\pp22dx)-@@----B-@--		pp22dx
signal-ef##$@pp22//[[((dsgfspp22dx/pg22dx)-signal[(\pp22dx)-@@----B-@--		pp22

as its output (which I think meets your requirements).

PS: Depending on the distribution of the lengths of your patterns, you might want to try the code I used in post #2 in this thread to sort the lengths and avoid processing lengths that don't exist in your data. I assume you won't have any problem merging the code from these two suggestions to get what you need.

Last edited by Don Cragun; 03-02-2017 at 11:09 PM.. Reason: Add PS.
This User Gave Thanks to Don Cragun For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Finding longest common substring among filenames

I will be performing a task on several directories, each containing a large number of files (2500+) that follow a regular naming convention: YYYY_MM_DD_XX.foo_bar.A.B.some_different_stuff.EXT What I would like to do is automatically discover the part of the filenames that are common to all... (1 Reply)
Discussion started by: cmcnorgan
1 Replies

2. Shell Programming and Scripting

Parsing file to match strings

I have a file with the following format 12g data/datasets/cct 8g data/dataset/cct 10 g data/two 5g data/something_different 10g something_different 5g data/two is there a way to loop through this... (1 Reply)
Discussion started by: yawalias
1 Replies

3. Shell Programming and Scripting

Awk Array doesnt match for substring

Awk Array doesnt match for substring nawk -F"," 'FNR==NR{a=$2 OFS $3;next} a{print $1,$2,a}' OFS="," file1 file2 I want cluster3 in file1 to match with cluster3int in file2 output getting: Output required: Help is appreciated (8 Replies)
Discussion started by: pinnacle
8 Replies

4. Shell Programming and Scripting

shell script: longest match from right?

Return the position of matched string from right, awk match can do from left only. e.g return pos 7 for search string "service" from "AA-service" or return the matched string "service", then caculate the string length. Thanks!. (3 Replies)
Discussion started by: honglus
3 Replies

5. Shell Programming and Scripting

Substring match

Hi, I want to find a file / directory with the name xxxxCELLxxx in the given path. The CELL is can be either in a UPPER or lower case. Thanks (4 Replies)
Discussion started by: youknowme
4 Replies

6. Shell Programming and Scripting

Longest word in a file

I am trying to write a command on just one line, i.e seperated by ';' and '|' etc, that finds the number of characters in the longest word of a file, preferably using the 'tr' and 'wc' commands. i no that wc shows the number of lines words and characters in a file but im not sure how to use it... (5 Replies)
Discussion started by: scotty85
5 Replies

7. UNIX for Dummies Questions & Answers

Deleting files based on Substring match

In folder there are files (eg ABS_18APR2012_XYZ.csv DSE_17APR2012_ABE.csv) . My requirement is to delete all the files except today's timestamp I tried doing this to list all the files not having today's date timestamp #!/bin/ksh DATE=`date +"%d%h%Y"` DIR=/data/rfs/... (9 Replies)
Discussion started by: manushi88
9 Replies

8. Shell Programming and Scripting

Match substring from a column of the second file

I want to merge the lines by matching substring of the first file with first column of the second file. file1: S00739A_ACAGTG_L001_R1.fq.gz S00739A_ACAGTG_L001_R2.fq.gz S00739B_GCCAAT_L001_R1.fq.gz S00739B_GCCAAT_L001_R2.fq.gz S00739D_GTGAAA_L001_R1.fq.gz S00739D_GTGAAA_L001_R2.fq.gz... (14 Replies)
Discussion started by: yifangt
14 Replies

9. Shell Programming and Scripting

Match text to lines in a file, iterate backwards until text or text substring matches, print to file

hi all, trying this using shell/bash with sed/awk/grep I have two files, one containing one column, the other containing multiple columns (comma delimited). file1.txt abc12345 def12345 ghi54321 ... file2.txt abc1,text1,texta abc,text2,textb def123,text3,textc gh,text4,textd... (6 Replies)
Discussion started by: shogun1970
6 Replies

10. UNIX for Beginners Questions & Answers

Replace substring by longest string in common field (awk)

Hi, Let's say I have a pipe-separated input like so: name_10|A|BCCC|cat_1 name_11|B|DE|cat_2 name_10|A|BC|cat_3 name_11|B|DEEEEEE|cat_4 Using awk, for records with common field 2, I am trying to replace all the shortest substrings by the longest string in field 3. In order to get the... (5 Replies)
Discussion started by: beca123456
5 Replies
Regexp::Common::list(3) 				User Contributed Perl Documentation				   Regexp::Common::list(3)

NAME
Regexp::Common::list -- provide regexes for lists SYNOPSIS
use Regexp::Common qw /list/; while (<>) { /$RE{list}{-pat => 'w+'}/ and print "List of words"; /$RE{list}{-pat => $RE{num}{real}}/ and print "List of numbers"; } DESCRIPTION
Please consult the manual of Regexp::Common for a general description of the works of this interface. Do not use this module directly, but load it via Regexp::Common. $RE{list}{-pat}{-sep}{-lastsep} Returns a pattern matching a list of (at least two) substrings. If "-pat=P" is specified, it defines the pattern for each substring in the list. By default, P is "qr/.*?S/". In Regexp::Common 0.02 or earlier, the default pattern was "qr/.*?/". But that will match a single space, causing unintended parsing of "a, b, and c" as a list of four elements instead of 3 (with "-word" being "(?:and)"). One consequence is that a list of the form "a,,b" will no longer be parsed. Use the pattern "qr /.*?/" to be able to parse this, but see the previous remark. If "-sep=P" is specified, it defines the pattern P to be used as a separator between each pair of substrings in the list, except the final two. By default P is "qr/s*,s*/". If "-lastsep=P" is specified, it defines the pattern P to be used as a separator between the final two substrings in the list. By default P is the same as the pattern specified by the "-sep" flag. For example: $RE{list}{-pat=>'w+'} # match a list of word chars $RE{list}{-pat=>$RE{num}{real}} # match a list of numbers $RE{list}{-sep=>" "} # match a tab-separated list $RE{list}{-lastsep=>',s+ands+'} # match a proper English list Under "-keep": $1 captures the entire list $2 captures the last separator $RE{list}{conj}{-word=PATTERN} An alias for $RE{list}{-lastsep=>'s*,?s*PATTERNs*'} If "-word" is not specified, the default pattern is "qr/and|or/". For example: $RE{list}{conj}{-word=>'et'} # match Jean, Paul, et Satre $RE{list}{conj}{-word=>'oder'} # match Bonn, Koln oder Hamburg $RE{list}{and} An alias for $RE{list}{conj}{-word=>'and'} $RE{list}{or} An alias for $RE{list}{conj}{-word=>'or'} SEE ALSO
Regexp::Common for a general description of how to use this interface. AUTHOR
Damian Conway (damian@conway.org) MAINTAINANCE
This package is maintained by Abigail (regexp-common@abigail.be). BUGS AND IRRITATIONS
Bound to be plenty. For a start, there are many common regexes missing. Send them in to regexp-common@abigail.be. LICENSE and COPYRIGHT This software is Copyright (c) 2001 - 2009, Damian Conway and Abigail. This module is free software, and maybe used under any of the following licenses: 1) The Perl Artistic License. See the file COPYRIGHT.AL. 2) The Perl Artistic License 2.0. See the file COPYRIGHT.AL2. 3) The BSD Licence. See the file COPYRIGHT.BSD. 4) The MIT Licence. See the file COPYRIGHT.MIT. perl v5.18.2 2013-03-08 Regexp::Common::list(3)
All times are GMT -4. The time now is 01:44 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy