Sponsored Content
Top Forums Shell Programming and Scripting awk summing specific lines and fields Post 302725235 by nakaedu on Friday 2nd of November 2012 12:59:22 AM
Old 11-02-2012
I have to apologize, my original post is not clear enough.
I have removed my original requirement to sum fields $13 and $14 because it is adding complexity and I don't really require them for the moment.
I will try to clarify my requirement,
I have a very long list nearly 1100 lines,
Each line is very similar to these

Code:
x;x;x;x;x;x;x;x;467,390,611 Bytes;0.435291 GB;0.062247 GB;0.373045 GB;11,225;157
a;a;a;a;a;a;a;a;13,805,156,846 Bytes;12.857054 GB;1.838559 GB;11.018495 GB;151,063;18,933
b;b;b;b;b;b;b;b;232,797,478,723 Bytes;216.809547 GB;31.003765 GB;185.805782 GB;176,861;21,445;Parent: Line #545

And I need to sum all fields to obtain: the total number of bytes, and GB. (while removing those lines that contain wording "Parent: Line"

For my example the total number of bytes and gigabytes, would be something similar to:
Code:
14272547457 Bytes;13.292345 GB;1.900806 GB;11.39154 GB

I get those numbers by summing all the row of fields $9, $10, $11, $12

467,390,611 Bytes + 13,805,156,846 Bytes = 14272547457 Bytes
0.435291 GB + 12.857054 GB = 13.292345 GB
0.062247 GB + 1.838559 GB = 1.900806 GB
0.373045 GB + 11.018495 GB = 11.39154 GB

Last edited by nakaedu; 11-02-2012 at 02:04 AM..
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

summing up the fields in fixed width file

Hi, I have a fixed width file with some records as given below: " 1000Nalsdjflj243324jljlj" "-0300Njfowjljl309933fsf" " 0010Njsfsjklj342344fsl" I want to sum-up first field values(i.e from 2nd character to 6th character)of each record. so for the above file i want to add (1000 - 300+... (2 Replies)
Discussion started by: srilaxmi
2 Replies

2. UNIX for Dummies Questions & Answers

Updating specific fields with awk using conditions

Can someone help me again, I think with awk? I have a file with 4 columns (pipe-delimited): I just want to convert the last field so that e1 is now 'message 1', e2 is 'message 2', e0 is 'message 3', etc. I don't want to change any other columns because the e0-e10 code may appear as part of a... (4 Replies)
Discussion started by: giannicello
4 Replies

3. Shell Programming and Scripting

Summing over specific lines and replacing the lines with the sum using sed, awk

Hi friends, This is sed & awk type question. I have a text file which has numbers spread all over the file. I want to sum the series of numbers whenever i find it and produce an output file with the sum. For example ###start of input text file #### abc def ghi 1 2 3 4 kjld random... (3 Replies)
Discussion started by: kaaliakahn
3 Replies

4. Shell Programming and Scripting

Summing over specific lines and replacing the lines with the sum

Hi friends, This is sed & awk type question. It is slightly different from my previous question. I have a text file which has numbers spread all over the file. I want to sum the series of numbers (but no more than 10 numbers in series) whenever i find it and produce an output file with the... (4 Replies)
Discussion started by: kaaliakahn
4 Replies

5. Shell Programming and Scripting

Summing all fields in a file

I was playing around to see how stuff works, and was trying to sum all fields in a file. cat file 1 2 3 4 5 6 7 8 9 10 11 12 I made this script: awk 'BEGIN {OFS=RS}{$1=$1}{s+=$0} END {print "sum="s}' file This gives 15, why not 78? I test it like this awk 'BEGIN... (5 Replies)
Discussion started by: Jotne
5 Replies

6. UNIX for Dummies Questions & Answers

Read the file and generate specific fields by awk

Hi I need to generate these output file from the below input file. Output : customer_id as customer, zip as zip_cd, catg_cd as catg, Input: out.customer::in.customer_id; out.zip_cd::in.zip; out.catg::in.catg_cd; Could you please help me on this. Please use code tags next... (1 Reply)
Discussion started by: Murugesh
1 Replies

7. Shell Programming and Scripting

Getting max value of specific fields with awk

Hello All, Here is am trying to get maximum value of third field depending on first,second and fourth fields with awk command . delimeter is pipe(|) . input 0221|09|14.25|aaa 0221|09|44.27|aaa 0221|09|44.33|aaa 0221|09|44.53|bbb 0221|09|34.32|bbb 0221|09|37.13|bbb... (5 Replies)
Discussion started by: sayami00
5 Replies

8. Shell Programming and Scripting

awk to combine lines if fields match in lines

In the awk below, what I am attempting to do is check each line in the tab-delimeted input, which has ~20 lines in it, for a keyword SVTYPE=Fusion. If the keyword is found I am splitting $3 using the . (dot) and reading the portion before and after the dot in an array a. If it does have that... (12 Replies)
Discussion started by: cmccabe
12 Replies

9. UNIX for Beginners Questions & Answers

Grep or awk a unique and specific word across many fields

Hi there, I have data with similar structure as this: CHR START-SNP END-SNP REF ALT PATIENT1 PATIENT2 PATIENT3 PATIENT4 chr1 69511 69511 A G homo hetero homo hetero chr2 69513 69513 T C . hetero homo hetero chr3 69814 69814 G C . . homo homo chr4 69815 69815 C A hetero . . hetero is... (10 Replies)
Discussion started by: daashti
10 Replies
Human(3pm)						User Contributed Perl Documentation						Human(3pm)

NAME
Number::Bytes::Human - Convert byte count to human readable format SYNOPSIS
use Number::Bytes::Human qw(format_bytes); $size = format_bytes(0); # '0' $size = format_bytes(2*1024); # '2.0K' $size = format_bytes(1_234_890, bs => 1000); # '1.3M' $size = format_bytes(1E9, bs => 1000); # '1.0G' # the OO way $human = Number::Bytes::Human->new(bs => 1000, si => 1); $size = $human->format(1E7); # '10MB' $human->set_options(zero => '-'); $size = $human->format(0); # '-' DESCRIPTION
THIS IS ALPHA SOFTWARE: THE DOCUMENTATION AND THE CODE WILL SUFFER CHANGES SOME DAY (THANKS, GOD!). This module provides a formatter which turns byte counts to usual readable format, like '2.0K', '3.1G', '100B'. It was inspired in the "-h" option of Unix utilities like "du", "df" and "ls" for "human-readable" output. From the FreeBSD man page of "df": http://www.freebsd.org/cgi/man.cgi?query=df "Human-readable" output. Use unit suffixes: Byte, Kilobyte, Megabyte, Gigabyte, Terabyte and Petabyte in order to reduce the number of digits to four or fewer using base 2 for sizes. byte B kilobyte K = 2**10 B = 1024 B megabyte M = 2**20 B = 1024 * 1024 B gigabyte G = 2**30 B = 1024 * 1024 * 1024 B terabyte T = 2**40 B = 1024 * 1024 * 1024 * 1024 B petabyte P = 2**50 B = 1024 * 1024 * 1024 * 1024 * 1024 B exabyte E = 2**60 B = 1024 * 1024 * 1024 * 1024 * 1024 * 1024 B zettabyte Z = 2**70 B = 1024 * 1024 * 1024 * 1024 * 1024 * 1024 * 1024 B yottabyte Y = 2**80 B = 1024 * 1024 * 1024 * 1024 * 1024 * 1024 * 1024 * 1024 B I have found this link to be quite useful: http://www.t1shopper.com/tools/calculate/ If you feel like a hard-drive manufacturer, you can start counting bytes by powers of 1000 (instead of the generous 1024). Just use "bs => 1000". But if you are a floppy disk manufacturer and want to start counting in units of 1024000 (for your "1.44 MB" disks)? Then use "bs => 1_024_000". If you feel like a purist academic, you can force the use of metric prefixes according to the Dec 1998 standard by the IEC. Never mind the units for base 1000 are "('B', 'kB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB')" and, even worse, the ones for base 1024 are "('B', 'KiB', 'MiB', 'GiB', 'TiB', 'PiB', 'EiB', 'ZiB', 'YiB')" with the horrible names: bytes, kibibytes, mebibytes, etc. All you have to do is to use "si => 1". Ain't that beautiful the SI system? Read about it: http://physics.nist.gov/cuu/Units/binary.html You can try a pure Perl "ls -lh"-inspired command with the one-liner, er, two-liner: $ perl -MNumber::Bytes::Human=format_bytes -e 'printf "%5s %s ", format_bytes(-s), $_ for @ARGV' * Why to write such a module? Because if people can write such things in C, it can be written much easier in Perl and then reused, refactored, abused. And then, when it is much improved, some brave soul can port it back to C (if only for the warm feeling of painful programming). OBJECTS An alternative to the functional style of this module is the OO fashion. This is useful for avoiding the unnecessary parsing of the arguments over and over if you have to format lots of numbers for (@sizes) { my $fmt_size = format_bytes($_, @args); ... } versus my $human = Number::Format::Bytes->new(@args); for (@sizes) { my $fmt_size = $human->format($_); ... } for TODO [TODO] MAKE IT JUST A MATTER OF STYLE: memoize _parse_args() $seed == undef FUNCTIONS format_bytes $h_size = format_bytes($size, @options); Turns a byte count (like 1230) to a readable format like '1.3K'. You have a bunch of options to play with. See the section "OPTIONS" to know the details. METHODS new $h = Number::Bytes::Human->new(@options); The constructor. For details on the arguments, see the section "OPTIONS". format $h_size = $h->format($size); Turns a byte count (like 1230) to a readable format like '1.3K'. The statements $h = Number::Bytes::Human->new(@options); $h_size = $h->format($size); are equivalent to "$h_size = format_bytes($size, @options)", with only one pass for the option arguments. set_options $h->set_options(@options); To alter the options of a "Number::Bytes::Human" object. See "OPTIONS". OPTIONS BASE block | base | block_size | bs => 1000 | 1024 | 1024000 base_1024 | block_1024 | 1024 => 1 base_1000 | block_1000 | 1000 => 1 The base to be used: 1024 (default), 1000 or 1024000. Any other value throws an exception. SUFFIXES suffixes => 1000 | 1024 | 1024000 | si_1000 | si_1024 | $arrayref By default, the used suffixes stand for '', 'K', 'M', ... for base 1024 and '', 'k', 'M', ... for base 1000 (which are indeed the usual metric prefixes with implied unit as bytes, 'B'). For the weird 1024000 base, suffixes are '', 'M', 'T', etc. ZERO zero => string | undef The string 0 maps to ('0' by default). If "undef", the general case is used. The string may contain '%S' in which case the suffix for byte is used. format_bytes(0, zero => '-') => '-' METRIC SYSTEM si => 1 ROUND round_function => $coderef round_style => 'ceil' | 'floor' TO_S QUIET quiet => 1 Suppresses the warnings emitted. Currently, the only case is when the number is large than "$base**(@suffixes+1)". EXPORT It is alright to import "format_bytes", but nothing is exported by default. DIAGNOSTICS
"unknown round style '$style'"; "invalid base: $block (should be 1024, 1000 or 1024000)"; "round function ($args{round_function}) should be a code ref"; "suffixes ($args{suffixes}) should be 1000, 1024, 1024000 or an array ref"; "negative numbers are not allowed" (??) TO DO
A function "parse_bytes" parse_bytes($str, $options) which transforms '1k' to 1000, '1K' to 1024, '1MB' to 1E6, '1M' to 1024*1024, etc. (like gnu du). $str =~ /^s*(d*.?d*)s*(S+)/ # $num $suffix SEE ALSO
lib/human.c and lib/human.h in GNU coreutils. The "_convert()" solution by COG in Filesys::DiskUsage. BUGS
Please report bugs via CPAN RT http://rt.cpan.org/NoAuth/Bugs.html?Dist=Number-Bytes-Human <http://rt.cpan.org/NoAuth/Bugs.html?Dist=Number-Bytes-Human> or mailto://bug-Number-Bytes-Human@rt.cpan.org <mailto://bug-Number-Bytes- Human@rt.cpan.org>. I will not be able to close the bug as BestPractical ignore my claims that I cannot log in, but I will answer anyway. AUTHOR
Adriano R. Ferreira, <ferreira@cpan.org> COPYRIGHT AND LICENSE
Copyright (C) 2005-2007 by Adriano R. Ferreira This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.14.2 2007-04-05 Human(3pm)
All times are GMT -4. The time now is 08:12 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy