Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Awk/sed summation of one column based on some entry in first column Post 303041926 by RavinderSingh13 on Saturday 7th of December 2019 07:23:53 AM
Old 12-07-2019
Quote:
Originally Posted by RavinderSingh13
Hello kshitij,
Could you please try following.
Code:
awk '
FNR==NR{
  if($0~/^[0-9]+/){
     ++count
  }
  sum[count]+=$NF
  next
}
/^[0-9]+/{
  print $0,sum[++var]
  next
}
1
'  Input_file  Input_file

Output will be as follows.

Code:
6  ddk/djhdj/djhdj/Q  10 0.5  1.5
    dhd/jdjd.djd.nd/QB 01 0.5
    hdhd/jd/jd/jdj/Q  10 0.5
512 hd/hdh/gdh/Q 01 0.5 1.5
      jdjd/jd/ud/j/QB 10 0.5
      HD/jsj/djd/Q  01 0.5
71 hdh/jjd/dj/jd/Q  10 0.5 2
    jd/jdld/je/j/QB 01 0.5
    IDP/jd/jdd/Q 10 0.5
    1K/JDJ/JDJK/QL 01 0.5

Thanks,
R. Singh
Hello kshitij,

Adding a detailed level explanation for my code.

Code:
awk '                            ##Starting awk program from here.
FNR==NR{                         ##Checking condition FNR==NR which will be TRUE when Input_file is being read first time.
  if($0~/^[0-9]+/){              ##Checking condition if a line starts from digit then do following.
     ++count                     ##Increment variable count with 1, each time cursor comes here.
  }                              ##Closing BLOCK for if condition here.
  sum[count]+=$NF                ##Creating an array named sum with index of count and keep on adding value of $NF to its own value.
  next                           ##next will skip all further statements from here.
}                                ##Closing BLOCK for condition FNR==NR here.
/^[0-9]+/{                       ##Checking condition if a line starts from digit then do following.
  print $0,sum[++var]            ##Printing current line and array sum with index of variable var with its increment of 1 each time cursor comes here.
  next                           ##next will skip all further statements from here.
}                                ##Closing BLOCK for /^[0-9]+/ condition here.
1                                ##Mentioning 1 for printing edited/non-edited line here.
'  Input_file  Input_file        ##Mentioning Input_file 2 times here.

Thanks,
R. Singh
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk/sed column replace using column header - help

$ cat log.txt Name Age Sex Lcation nfld alias xsd CC 25 M XYZ asx KK Y BB 21 F XAS awe SS N SD 21 M AQW rty SD A How can I replace the column with header "Lcation" with the column with header "alias" and delete the "alias" column? so that the final output will become: Name Age Sex... (10 Replies)
Discussion started by: jkl_jkl
10 Replies

2. Shell Programming and Scripting

Awk or Sed, fubd match in column, then edit column.

FILE A: 9780743551526,(Abridged) 9780743551779,(Unabridged) 9780743582469,(Abridged) 9780743582483,(Unabridged) 9780743563468,(Abridged) 9780743563475,(Unabridged) FILE B: c3saCandyland 9780743518321 "CANDYLAND" "MCBAIN, ED" 2001 c3sbCandyland 9780743518321 ... (7 Replies)
Discussion started by: glev2005
7 Replies

3. Shell Programming and Scripting

for each different entry in column 1 extract maximum values from column 2 in unix/awk

Hello, I have 2 columns (1st column has multiple entries but the corresponding values in the column 2 may be the same or different.) however I want to extract unique values for each entry in column 1 by assigning the max value from column 2 SDF4 -0.211654 SDF4 0.978068 ... (1 Reply)
Discussion started by: Diya123
1 Replies

4. Shell Programming and Scripting

Rename a header column by adding another column entry to the header column name URGENT!!

Hi All, I have a file example.csv which looks like this GrpID,TargetID,Signal,Avg_Num CSCH74_1_1,2007,61,256 CSCH74_1_1,212007,647,679 CSCH74_1_1,12007,3,32 CSCH74_1_1,207,299,777 I want the output as GrpID,TragetID,Signal-CSCH74_1_1,Avg_Num CSCH74_1_1,2007,61,256... (4 Replies)
Discussion started by: Vavad
4 Replies

5. Shell Programming and Scripting

awk or sed: change the color of a column w/o screwing up column spacing

Hey folks. I wrote a little awk script that summarizes /proc/net/dev info and then pipes it to the nix column command to set up column spacing appropriately. Here's some example output: Iface RxMBytes RxPackets RxErrs RxDrop TxMBytes TxPackets TxErrs TxDrop bond0 9 83830... (3 Replies)
Discussion started by: ryran
3 Replies

6. Shell Programming and Scripting

Pick the column value based on another column using awk or CUT

My scenario is that I need to pick value from third column based on fourth column value, if fourth column value is 1 then first value of third column.Third column (2|3|4|6|1) values are cancatenated. Please someone help me to resolve this issue. Source column1 column2 column3 column4... (2 Replies)
Discussion started by: Ganesh L
2 Replies

7. Shell Programming and Scripting

awk to sum a column based on duplicate strings in another column and show split totals

Hi, I have a similar input format- A_1 2 B_0 4 A_1 1 B_2 5 A_4 1 and looking to print in this output format with headers. can you suggest in awk?awk because i am doing some pattern matching from parent file to print column 1 of my input using awk already.Thanks! letter number_of_letters... (5 Replies)
Discussion started by: prashob123
5 Replies

8. Shell Programming and Scripting

awk script to call another script based on second column entry

Hi I have a text file (Input.txt) with two column entries separated by tab as given below: aaa str1 bbb str2 cccccc str3 dddd str4 eee str3 ssss str2 sdf str3 hhh str1 fff str2 ccc str3 ..... ..... ..... (1 Reply)
Discussion started by: my_Perl
1 Replies

9. Shell Programming and Scripting

Solution for replacement of 4th column with 3rd column in a file using awk/sed preserving delimters

input "A","B","C,D","E","F" "S","T","U,V","W","X" "AA","BB","CC,DD","EEEE","FFF" required output: "A","B","C,D","C,D","F" "S", T","U,V","U,V","X" "AA","BB","CC,DD","CC,DD","FFF" tried using awk but double quotes not preserving for every field. any help to solve this is much... (5 Replies)
Discussion started by: khblts
5 Replies

10. Shell Programming and Scripting

sed command to add a new column entry

My input file looks like this 12 3 5.122.281.413 172.31.15.220 3421 4133 2 2 1454 3421 4133 2 2 0 12 44036 214.215.52.146 90.123.245.211 2312 3911 4 4 521 2312 3911 4 4 1 14 504 6.254.324.219 192.61.27.120 4444 5611 7 5 1415 4444 5611 7 5 1 ... (2 Replies)
Discussion started by: sampitosh
2 Replies
List::Util(3)						User Contributed Perl Documentation					     List::Util(3)

NAME
List::Util - A selection of general-utility list subroutines SYNOPSIS
use List::Util qw(first max maxstr min minstr reduce shuffle sum); DESCRIPTION
"List::Util" contains a selection of subroutines that people have expressed would be nice to have in the perl core, but the usage would not really be high enough to warrant the use of a keyword, and the size so small such that being individual extensions would be wasteful. By default "List::Util" does not export any subroutines. LIST-REDUCTION FUNCTIONS The following set of functions all reduce a list down to a single value. $result = reduce { BLOCK } @list Reduces @list by calling "BLOCK" in a scalar context multiple times, setting $a and $b each time. The first call will be with $a and $b set to the first two elements of the list, subsequent calls will be done by setting $a to the result of the previous call and $b to the next element in the list. Returns the result of the last call to the "BLOCK". If @list is empty then "undef" is returned. If @list only contains one element then that element is returned and "BLOCK" is not executed. The following examples all demonstrate how "reduce" could be used to implement the other list-reduction functions in this module. (They are not in fact implemented like this, but instead in a more efficient manner in individual C functions). $foo = reduce { defined($a) ? $a : $code->(local $_ = $b) ? $b : undef } undef, @list # first $foo = reduce { $a > $b ? $a : $b } 1..10 # max $foo = reduce { $a gt $b ? $a : $b } 'A'..'Z' # maxstr $foo = reduce { $a < $b ? $a : $b } 1..10 # min $foo = reduce { $a lt $b ? $a : $b } 'aa'..'zz' # minstr $foo = reduce { $a + $b } 1 .. 10 # sum $foo = reduce { $a . $b } @bar # concat $foo = reduce { $a || $code->(local $_ = $b) } 0, @bar # any $foo = reduce { $a && $code->(local $_ = $b) } 1, @bar # all $foo = reduce { $a && !$code->(local $_ = $b) } 1, @bar # none $foo = reduce { $a || !$code->(local $_ = $b) } 0, @bar # notall # Note that these implementations do not fully short-circuit If your algorithm requires that "reduce" produce an identity value, then make sure that you always pass that identity value as the first argument to prevent "undef" being returned $foo = reduce { $a + $b } 0, @values; # sum with 0 identity value The remaining list-reduction functions are all specialisations of this generic idea. $b = any { BLOCK } @list Similar to "grep" in that it evaluates "BLOCK" setting $_ to each element of @list in turn. "any" returns true if any element makes the "BLOCK" return a true value. If "BLOCK" never returns true or @list was empty then it returns false. Many cases of using "grep" in a conditional can be written using "any" instead, as it can short-circuit after the first true result. if( any { length > 10 } @strings ) { # at least one string has more than 10 characters } $b = all { BLOCK } @list Similar to "any", except that it requires all elements of the @list to make the "BLOCK" return true. If any element returns false, then it returns false. If the "BLOCK" never returns false or the @list was empty then it returns true. $b = none { BLOCK } @list $b = notall { BLOCK } @list Similar to "any" and "all", but with the return sense inverted. "none" returns true only if no value in the LIST causes the BLOCK to return true, and "notall" returns true only if not all of the values do. $val = first { BLOCK } @list Similar to "grep" in that it evaluates "BLOCK" setting $_ to each element of @list in turn. "first" returns the first element where the result from "BLOCK" is a true value. If "BLOCK" never returns true or @list was empty then "undef" is returned. $foo = first { defined($_) } @list # first defined value in @list $foo = first { $_ > $value } @list # first value in @list which # is greater than $value $num = max @list Returns the entry in the list with the highest numerical value. If the list is empty then "undef" is returned. $foo = max 1..10 # 10 $foo = max 3,9,12 # 12 $foo = max @bar, @baz # whatever $str = maxstr @list Similar to "max", but treats all the entries in the list as strings and returns the highest string as defined by the "gt" operator. If the list is empty then "undef" is returned. $foo = maxstr 'A'..'Z' # 'Z' $foo = maxstr "hello","world" # "world" $foo = maxstr @bar, @baz # whatever $num = min @list Similar to "max" but returns the entry in the list with the lowest numerical value. If the list is empty then "undef" is returned. $foo = min 1..10 # 1 $foo = min 3,9,12 # 3 $foo = min @bar, @baz # whatever $str = minstr @list Similar to "min", but treats all the entries in the list as strings and returns the lowest string as defined by the "lt" operator. If the list is empty then "undef" is returned. $foo = minstr 'A'..'Z' # 'A' $foo = minstr "hello","world" # "hello" $foo = minstr @bar, @baz # whatever $num = product @list Returns the numerical product of all the elements in @list. If @list is empty then 1 is returned. $foo = product 1..10 # 3628800 $foo = product 3,9,12 # 324 $num_or_undef = sum @list Returns the numerical sum of all the elements in @list. For backwards compatibility, if @list is empty then "undef" is returned. $foo = sum 1..10 # 55 $foo = sum 3,9,12 # 24 $foo = sum @bar, @baz # whatever $num = sum0 @list Similar to "sum", except this returns 0 when given an empty list, rather than "undef". KEY
/VALUE PAIR LIST FUNCTIONS The following set of functions, all inspired by List::Pairwise, consume an even-sized list of pairs. The pairs may be key/value associations from a hash, or just a list of values. The functions will all preserve the original ordering of the pairs, and will not be confused by multiple pairs having the same "key" value - nor even do they require that the first of each pair be a plain string. @kvlist = pairgrep { BLOCK } @kvlist $count = pairgrep { BLOCK } @kvlist Similar to perl's "grep" keyword, but interprets the given list as an even-sized list of pairs. It invokes the "BLOCK" multiple times, in scalar context, with $a and $b set to successive pairs of values from the @kvlist. Returns an even-sized list of those pairs for which the "BLOCK" returned true in list context, or the count of the number of pairs in scalar context. (Note, therefore, in scalar context that it returns a number half the size of the count of items it would have returned in list context). @subset = pairgrep { $a =~ m/^[[:upper:]]+$/ } @kvlist As with "grep" aliasing $_ to list elements, "pairgrep" aliases $a and $b to elements of the given list. Any modifications of it by the code block will be visible to the caller. ( $key, $val ) = pairfirst { BLOCK } @kvlist $found = pairfirst { BLOCK } @kvlist Similar to the "first" function, but interprets the given list as an even-sized list of pairs. It invokes the "BLOCK" multiple times, in scalar context, with $a and $b set to successive pairs of values from the @kvlist. Returns the first pair of values from the list for which the "BLOCK" returned true in list context, or an empty list of no such pair was found. In scalar context it returns a simple boolean value, rather than either the key or the value found. ( $key, $value ) = pairfirst { $a =~ m/^[[:upper:]]+$/ } @kvlist As with "grep" aliasing $_ to list elements, "pairfirst" aliases $a and $b to elements of the given list. Any modifications of it by the code block will be visible to the caller. @list = pairmap { BLOCK } @kvlist $count = pairmap { BLOCK } @kvlist Similar to perl's "map" keyword, but interprets the given list as an even-sized list of pairs. It invokes the "BLOCK" multiple times, in list context, with $a and $b set to successive pairs of values from the @kvlist. Returns the concatenation of all the values returned by the "BLOCK" in list context, or the count of the number of items that would have been returned in scalar context. @result = pairmap { "The key $a has value $b" } @kvlist As with "map" aliasing $_ to list elements, "pairmap" aliases $a and $b to elements of the given list. Any modifications of it by the code block will be visible to the caller. @pairs = pairs @kvlist A convenient shortcut to operating on even-sized lists of pairs, this function returns a list of ARRAY references, each containing two items from the given list. It is a more efficient version of @pairs = pairmap { [ $a, $b ] } @kvlist It is most convenient to use in a "foreach" loop, for example: foreach ( pairs @KVLIST ) { my ( $key, $value ) = @$_; ... } @keys = pairkeys @kvlist A convenient shortcut to operating on even-sized lists of pairs, this function returns a list of the the first values of each of the pairs in the given list. It is a more efficient version of @keys = pairmap { $a } @kvlist @values = pairvalues @kvlist A convenient shortcut to operating on even-sized lists of pairs, this function returns a list of the the second values of each of the pairs in the given list. It is a more efficient version of @values = pairmap { $b } @kvlist OTHER FUNCTIONS
@values = shuffle @values Returns the values of the input in a random order @cards = shuffle 0..51 # 0..51 in a random order KNOWN BUGS
With perl versions prior to 5.005 there are some cases where reduce will return an incorrect result. This will show up as test 7 of reduce.t failing. SUGGESTED ADDITIONS
The following are additions that have been requested, but I have been reluctant to add due to them being very simple to implement in perl # How many elements are true sub true { scalar grep { $_ } @_ } # How many elements are false sub false { scalar grep { !$_ } @_ } SEE ALSO
Scalar::Util, List::MoreUtils COPYRIGHT
Copyright (c) 1997-2007 Graham Barr <gbarr@pobox.com>. All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. Recent additions and current maintenance by Paul Evans, <leonerd@leonerd.org.uk>. perl v5.18.2 2014-01-22 List::Util(3)
All times are GMT -4. The time now is 07:12 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy