Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Replace substring by longest string in common field (awk)
Top Forums UNIX for Beginners Questions & Answers Replace substring by longest string in common field (awk) Post 303042964 by MadeInGermany on Tuesday 14th of January 2020 07:54:19 AM
Old 01-14-2020
An not inititalized variable (or array element) becomes 0 in number context, and "" in string context.
In this case, since 0 is the minimal possible string length, the 0 is perfect.

With two passes through the input file one only needs one array that holds the longest string:

Code:
awk '
BEGIN { FS=OFS="|" }
# NR == FNR when reading the 1st file
NR == FNR {
# 1st file
# a[$2] holds the longest $3
  if (length(a[$2]) < length($3)) a[$2]=$3
# jump to next input cycle, do not run the following code
  next
}
# 2nd file, here: pass 2
{
# always update $3
  $3=a[$2]
  print
}' Input_file Input_file

And, similar to post#4, with one pass through the input file, where everything is read into a line[] array, and in the END section this is printed in a loop.
Code:
awk '
BEGIN { FS=OFS="|" }
{
# store $0 in line[1..]
  line[NR]=$0
# a[$2] holds the longest $3
  if (length(a[$2]) < length($3)) a[$2]=$3
}
END {
  for (n=1; n<=NR; n++) {
# restore $0
    $0=line[n]
# always update $3
    $3=a[$2]
    print
  }
}
' Input_file

Last edited by MadeInGermany; 01-14-2020 at 09:23 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Finding longest common substring among filenames

I will be performing a task on several directories, each containing a large number of files (2500+) that follow a regular naming convention: YYYY_MM_DD_XX.foo_bar.A.B.some_different_stuff.EXT What I would like to do is automatically discover the part of the filenames that are common to all... (1 Reply)
Discussion started by: cmcnorgan
1 Replies

2. Shell Programming and Scripting

Advanced AWK Regexp substring to int & Replace

Hi! I have a difficult problem, to step up a unknown version number in a text file, and save the file. It would be great to run script.sh and the version gets increased. Example the content of the textfile.txt hello version = x bye This include three steps 1. First find the char after... (2 Replies)
Discussion started by: Beachboy72
2 Replies

3. Shell Programming and Scripting

Awk Search text string in field, not all in field.

Hello, I am using awk to match text in a tab separated field and am able to do so when matching the exact word. My problem is that I would like to match any sequence of text in the tab-separated field without having to match it all. Any help will be appreciated. Please see the code below. awk... (3 Replies)
Discussion started by: rocket_dog
3 Replies

4. Shell Programming and Scripting

sed or awk command to replace a string pattern with another string based on position of this string

here is what i want to achieve... consider a file contains below contents. the file size is large about 60mb cat dump.sql INSERT INTO `table1` (`id`, `action`, `date`, `descrip`, `lastModified`) VALUES (1,'Change','2011-05-05 00:00:00','Account Updated','2012-02-10... (10 Replies)
Discussion started by: vivek d r
10 Replies

5. Shell Programming and Scripting

awk uniq and longest string of a column as index

I met a challenge to filter ~70 millions of sequence rows and I want using awk with conditions: 1) longest string of each pattern in column 2, ignore any sub-string, as the index; 2) all the unique patterns after 1); 3) print the whole row; input: 1 ABCDEFGHI longest_sequence1 2 ABCDEFGH... (12 Replies)
Discussion started by: yifangt
12 Replies

6. UNIX for Dummies Questions & Answers

Values with common field in same line with awk

Hi all ! I almost did it but got a small problem. input: cars red cars blue cars green truck black Wanted: cars red-blue-green truck black Attempt: gawk 'BEGIN{FS="\t"}{a = a (a?"-":"")$2; $2=a; print $1 FS $2}' input But I also got the intermediate records... (2 Replies)
Discussion started by: beca123456
2 Replies

7. Shell Programming and Scripting

Parsing the longest match substring

Hello gurus, I have a database of possible primary signal strings pp22 pt22dx pp22dx jty2234 Also I have a list of scrambled signals which has a shorter string and a longer string separated by // (double slash ). Always the shorter string of a scrambled signal will have the primary... (6 Replies)
Discussion started by: senhia83
6 Replies

8. Shell Programming and Scripting

awk to update field using matching value in file1 and substring in field in file2

In the awk below I am trying to set/update the value of $14 in file2 in bold, using the matching NM_ in $12 or $9 in file2 with the NM_ in $2 of file1. The lengths of $9 and $12 can be variable but what is consistent is the start pattern will always be NM_ and the end pattern is always ;... (2 Replies)
Discussion started by: cmccabe
2 Replies

9. UNIX for Beginners Questions & Answers

Awk: output lines with common field to separate files

Hi, A beginner one. my input.tab (tab-separated): h1 h2 h3 h4 h5 item1 grpA 2 3 customer1 item2 grpB 4 6 customer1 item3 grpA 5 9 customer1 item4 grpA 0 0 customer2 item5 grpA 9 1 customer2 objective: output a file for each customer ($5) with the item number ($1) only if $2 matches... (2 Replies)
Discussion started by: beca123456
2 Replies

10. Shell Programming and Scripting

Replace substring from a string variable

Hi, Wish to remove "DR-" from the string variable (var). var="DR-SERVER1" var=`echo $var | sed -e 's/DR-//g'` echo "$var" Expected Output: However, I get the below error: Can you please suggest. (4 Replies)
Discussion started by: mohtashims
4 Replies

LEARN ABOUT DEBIAN

dpkg-ruby

DPKG-RUBY(1)						      General Commands Manual						      DPKG-RUBY(1)

NAME

       dpkg-ruby - Utility to read a dpkg style db file, dpkg-awk clone

SYNOPSIS

       dpkg-ruby  [(-f|--file) filename] [(-d|--debug) ##] [(-s|--sort) list] [(-n|--numeric) list] [(-rs|--rec_sep) ??] '<fieldname>:<regex>' ...
       -- <out_fieldname> ..

DESCRIPTION

       dpkg-ruby Parses a dpkg status file(or other similarly formated file) and outputs the resulting records.  It can use  regex  on	the  field
       values to limit the returned records, and it can also be told which fields to output.  As another option, it can sort the matched fields.

OPTIONS

       -f filename
       --file filename
	      The file to parse.  The default is /var/lib/dpkg/status.

       -d [#]
       --debug [#]
	      Each time this is specified, it increased the debug level.

       -s field(s)
       --sort field(s)
	      A space or comma separated list of fields to sort on.

       -n field(s)
       --numeric field(s)
	      A space or comma separated list of fields that should be interpreted as numeric in value.

       -rs ??
       --rec_sep ??
	      Output this string at the end of each output paragraph.

       -h
       --help Display some help.

       fieldname
	      The fields from the file, that are matched with the regex given.	The fieldnames are case insensitive.

       out_fieldname
	      The  fields  from the file, that are outputted for each record.  If the first field listed is begins with ^, then the list that fol-
	      lows are fields NOT to be outputted.

BUGS

       Be warned that the author has only a shallow understanding of the dpkg packaging system, so there are probably tons of bugs  in	this  pro-
       gram.

       This  program  comes with no warranties.  If running this program causes fire and brimstone to rain down upon the earth, you will be on our
       own.

       This program accesses the dpkg database directly in places, querying for data that cannot be gotten via dpkg.

AUTHOR

       Fumitoshi UKAI <ukai@debian.or.jp>.  This manual page are based on (or almost copy from :) dpkg-awk(1) manual written by Adam  Heath  <doo-
       gie@debian.org>

DEBIAN
								 Debian Utilities						      DPKG-RUBY(1)
All times are GMT -4. The time now is 12:16 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy