Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Replace substring by longest string in common field (awk)
Top Forums UNIX for Beginners Questions & Answers Replace substring by longest string in common field (awk) Post 303042958 by beca123456 on Tuesday 14th of January 2020 05:44:14 AM
Old 01-14-2020
Replace substring by longest string in common field (awk)
Hi,

Let's say I have a pipe-separated input like so:
Code:
name_10|A|BCCC|cat_1
name_11|B|DE|cat_2
name_10|A|BC|cat_3
name_11|B|DEEEEEE|cat_4

Using awk, for records with common field 2, I am trying to replace all the shortest substrings by the longest string in field 3.
In order to get the following (changes in bold):
Code:
name_10|A|BCCC|cat_1
name_11|B|DEEEEEE|cat_2
name_10|A|BCCC|cat_3
name_11|B|DEEEEEE|cat_4

A beginning of a code so far, but I am getting stuck:
Code:
echo -e "name_10|A|BCCC|cat_1\nname_11|B|DE|cat_2\nname_10|A|BC|cat_3\nname_11|B|DEEEEEE|cat_4" |
awk '
BEGIN{FS="|"}
{
    if(a[$2] < length($3)){
        a[$2]=$3
    }
}
END{
    for(i in a){
        print i FS a[i]
    }
}'

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Finding longest common substring among filenames

I will be performing a task on several directories, each containing a large number of files (2500+) that follow a regular naming convention: YYYY_MM_DD_XX.foo_bar.A.B.some_different_stuff.EXT What I would like to do is automatically discover the part of the filenames that are common to all... (1 Reply)
Discussion started by: cmcnorgan
1 Replies

2. Shell Programming and Scripting

Advanced AWK Regexp substring to int & Replace

Hi! I have a difficult problem, to step up a unknown version number in a text file, and save the file. It would be great to run script.sh and the version gets increased. Example the content of the textfile.txt hello version = x bye This include three steps 1. First find the char after... (2 Replies)
Discussion started by: Beachboy72
2 Replies

3. Shell Programming and Scripting

Awk Search text string in field, not all in field.

Hello, I am using awk to match text in a tab separated field and am able to do so when matching the exact word. My problem is that I would like to match any sequence of text in the tab-separated field without having to match it all. Any help will be appreciated. Please see the code below. awk... (3 Replies)
Discussion started by: rocket_dog
3 Replies

4. Shell Programming and Scripting

sed or awk command to replace a string pattern with another string based on position of this string

here is what i want to achieve... consider a file contains below contents. the file size is large about 60mb cat dump.sql INSERT INTO `table1` (`id`, `action`, `date`, `descrip`, `lastModified`) VALUES (1,'Change','2011-05-05 00:00:00','Account Updated','2012-02-10... (10 Replies)
Discussion started by: vivek d r
10 Replies

5. Shell Programming and Scripting

awk uniq and longest string of a column as index

I met a challenge to filter ~70 millions of sequence rows and I want using awk with conditions: 1) longest string of each pattern in column 2, ignore any sub-string, as the index; 2) all the unique patterns after 1); 3) print the whole row; input: 1 ABCDEFGHI longest_sequence1 2 ABCDEFGH... (12 Replies)
Discussion started by: yifangt
12 Replies

6. UNIX for Dummies Questions & Answers

Values with common field in same line with awk

Hi all ! I almost did it but got a small problem. input: cars red cars blue cars green truck black Wanted: cars red-blue-green truck black Attempt: gawk 'BEGIN{FS="\t"}{a = a (a?"-":"")$2; $2=a; print $1 FS $2}' input But I also got the intermediate records... (2 Replies)
Discussion started by: beca123456
2 Replies

7. Shell Programming and Scripting

Parsing the longest match substring

Hello gurus, I have a database of possible primary signal strings pp22 pt22dx pp22dx jty2234 Also I have a list of scrambled signals which has a shorter string and a longer string separated by // (double slash ). Always the shorter string of a scrambled signal will have the primary... (6 Replies)
Discussion started by: senhia83
6 Replies

8. Shell Programming and Scripting

awk to update field using matching value in file1 and substring in field in file2

In the awk below I am trying to set/update the value of $14 in file2 in bold, using the matching NM_ in $12 or $9 in file2 with the NM_ in $2 of file1. The lengths of $9 and $12 can be variable but what is consistent is the start pattern will always be NM_ and the end pattern is always ;... (2 Replies)
Discussion started by: cmccabe
2 Replies

9. UNIX for Beginners Questions & Answers

Awk: output lines with common field to separate files

Hi, A beginner one. my input.tab (tab-separated): h1 h2 h3 h4 h5 item1 grpA 2 3 customer1 item2 grpB 4 6 customer1 item3 grpA 5 9 customer1 item4 grpA 0 0 customer2 item5 grpA 9 1 customer2 objective: output a file for each customer ($5) with the item number ($1) only if $2 matches... (2 Replies)
Discussion started by: beca123456
2 Replies

10. Shell Programming and Scripting

Replace substring from a string variable

Hi, Wish to remove "DR-" from the string variable (var). var="DR-SERVER1" var=`echo $var | sed -e 's/DR-//g'` echo "$var" Expected Output: However, I get the below error: Can you please suggest. (4 Replies)
Discussion started by: mohtashims
4 Replies

LEARN ABOUT DEBIAN

locale::language

Locale::Language(3perl) 				 Perl Programmers Reference Guide				   Locale::Language(3perl)

NAME

       Locale::Language - standard codes for language identification

SYNOPSIS

	  use Locale::Language;

	  $lang = code2language('en');	      # $lang gets 'English'
	  $code = language2code('French');    # $code gets 'fr'

	  @codes   = all_language_codes();
	  @names   = all_language_names();

DESCRIPTION

       The "Locale::Language" module provides access to standard codes used for identifying languages, such as those as defined in ISO 639.

       Most of the routines take an optional additional argument which specifies the code set to use. If not specified, the default ISO 639 two-
       letter codes will be used.

SUPPORTED CODE SETS

       There are several different code sets you can use for identifying languages. The ones currently supported are:

       alpha-2
	   This is the set of two-letter (lowercase) codes from ISO 639, such as 'he' for Hebrew.

	   This code set is identified with the symbol "LOCALE_LANG_ALPHA_2".

	   This is the default code set.

       alpha-3
	   This is the set of three-letter (lowercase) bibliographic codes from ISO 639, such as 'heb' for Hebrew.

	   This code set is identified with the symbol "LOCALE_LANG_ALPHA_3".

       term
	   This is the set of three-letter (lowercase) terminologic codes from ISO 639.

	   This code set is identified with the symbol "LOCALE_LANG_TERM".

ROUTINES

       code2language ( CODE [,CODESET] )
       language2code ( NAME [,CODESET] )
       language_code2code ( CODE ,CODESET ,CODESET2 )
       all_language_codes ( [CODESET] )
       all_language_names ( [CODESET] )
       Locale::Language::rename_language  ( CODE ,NEW_NAME [,CODESET] )
       Locale::Language::add_language  ( CODE ,NAME [,CODESET] )
       Locale::Language::delete_language  ( CODE [,CODESET] )
       Locale::Language::add_language_alias  ( NAME ,NEW_NAME )
       Locale::Language::delete_language_alias	( NAME )
       Locale::Language::rename_language_code  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Language::add_language_code_alias  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Language::delete_language_code_alias  ( CODE [,CODESET] )
	   These routines are all documented in the Locale::Codes man page.

SEE ALSO

       Locale::Codes
       Locale::Constants
       http://www.loc.gov/standards/iso639-2/
	   Source of the ISO 639 codes.

AUTHOR

       See Locale::Codes for full author history.

       Currently maintained by Sullivan Beck (sbeck@cpan.org).

COPYRIGHT

	  Copyright (c) 1997-2001 Canon Research Centre Europe (CRE).
	  Copyright (c) 2001-2010 Neil Bowers
	  Copyright (c) 2010-2011 Sullivan Beck

       This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

perl v5.14.2							    2011-09-26						   Locale::Language(3perl)
All times are GMT -4. The time now is 05:33 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy