Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Aggregation of huge data
Top Forums Shell Programming and Scripting Aggregation of huge data Post 302896110 by Corona688 on Friday 4th of April 2014 11:44:57 AM
Old 04-04-2014
The other thing to keep in mind is that awk uses floating point numbers. It's not infinite precision. If you print that number to 13 extra decimal places, most of those decimals will be meaningless garbage.

If you want a perfect sum, bc should do the job, if you convert the output into something it can use.

Code:
awk 'BEGIN { print "Z = 0;" } { sub(/-/, ""); print "Z += ",$1,";" } END { print "Z;" }' inputfile | bc

This prints "z = 0;" as the first line, then all lines afterwards as "z += number;" And the final line as "z;" to print the final sum.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

search and grab data from a huge file

folks, In my working directory, there a multiple large files which only contain one line in the file. The line is too long to use "grep", so any help? For example, if I want to find if these files contain a string like "93849", what command I should use? Also, there is oder_id number... (1 Reply)
Discussion started by: ting123
1 Replies

2. Shell Programming and Scripting

How to extract data from a huge file?

Hi, I have a huge file of bibliographic records in some standard format.I need a script to do some repeatable task as follows: 1. Needs to create folders as the strings starts with "item_*" from the input file 2. Create a file "contents" in each folders having "license.txt(tab... (5 Replies)
Discussion started by: srsahu75
5 Replies

3. UNIX for Advanced & Expert Users

A variable and sum of its value in a huge data.

Hi Experts, I got a question.. In the following output of `ps -elf | grep DataFlow` I get:- 242001 A mqsiadm 2076676 1691742 0 60 20 26ad4f400 130164 * May 09 - 3:02 DataFlowEngine EAIDVBR1_BROKER 5e453de8-2001-0000-0080-fd142b9ce8cb VIPS_INQ1 0 242001 A mqsiadm... (5 Replies)
Discussion started by: varungupta
5 Replies

4. Shell Programming and Scripting

Split a huge data into few different files?!

Input file data contents: >seq_1 MSNQSPPQSQRPGHSHSHSHSHAGLASSTSSHSNPSANASYNLNGPRTGGDQRYRASVDA >seq_2 AGAAGRGWGRDVTAAASPNPRNGGGRPASDLLSVGNAGGQASFASPETIDRWFEDLQHYE >seq_3 ATLEEMAAASLDANFKEELSAIEQWFRVLSEAERTAALYSLLQSSTQVQMRFFVTVLQQM ARADPITALLSPANPGQASMEAQMDAKLAAMGLKSPASPAVRQYARQSLSGDTYLSPHSA... (7 Replies)
Discussion started by: patrick87
7 Replies

5. UNIX for Dummies Questions & Answers

Copy huge data into vi editor

Hi All, HP-UX dev4 B.11.11 U 9000/800 3251073457 I need to copy huge data from windows text file to vi editor. when I tried copy huge data, the format of data is not preserverd and appered to scatterd through the vi, something like give below. Please let me know, how can I correct this? ... (18 Replies)
Discussion started by: alok.behria
18 Replies

6. Red Hat

Disk is Full but really does not contain huge data

Hi All, My disk usage show 100 % . When I check “df –kh” it shows my root partition is full. But when I run the “du –skh /” shows only 7 GB is used. Filesystem Size Used Avail Use% Mounted on /dev/sda1 30G 28G 260MB 100% / How I can identify who is using the 20 GB of memory. Os: Centos... (10 Replies)
Discussion started by: kalpeer
10 Replies

7. Shell Programming and Scripting

Aggregation of Huge files

Hi Friends !! I am facing a hash total issue while performing over a set of files of huge volume: Command used: tail -n +2 <File_Name> |nawk -F"|" -v '%.2f' qq='"' '{gsub(qq,"");sa+=($156<0)?-$156:$156}END{print sa}' OFMT='%.5f' Pipe delimited file and 156 column is for hash totalling.... (14 Replies)
Discussion started by: Ravichander
14 Replies

8. Shell Programming and Scripting

awk does not work well with huge data?

Dear all , I found that if we work with thousands line of data, awk does not work perfectly. It will cut hundreds line (others are deleted) and works only on the remain data. I used this command : awk '$1==1{$1="Si"}{print>FILENAME}' coba.xyz to change value of first column whose value is 1... (4 Replies)
Discussion started by: ariesto
4 Replies

9. Solaris

The Fastest for copy huge data

Dear Experts, I would like to know what's the best method for copy data around 3 mio (spread in a hundred folders, size each file around 1kb) between 2 servers? I already tried using Rsync and tar command. But using these command is too long. Please advice. Thanks Edy (11 Replies)
Discussion started by: edydsuranta
11 Replies

10. Shell Programming and Scripting

Phrase XML with Huge Data

HI Guys, I have Big XML file with Below Format :- Input :- <pokl>MKL=1,FN=1,GBNo=B10C</pokl> <d>192</d> <d>315</d> <d>35</d> <d>0,7,8</d> <pokl>MKL=1,dFN=1,GBNo=B11C</pokl> <d>162</d> <d>315</d> <d>35</d> <d>0,5,6</d> <pokl>MKL=1,dFN=1,GBNo=B12C</pokl> <d>188</d> (4 Replies)
Discussion started by: pareshkp
4 Replies

LEARN ABOUT DEBIAN

canadian-english-huge

canadian-english-huge(5)					   Users' Manual					  canadian-english-huge(5)

NAME

       canadian-english-huge - a list of English words

DESCRIPTION

       /usr/share/dict/canadian-english-huge is an ASCII file which contains an alphabetic list of words, one per line.

FILES

       There  may  be  any  number  of	word lists in /usr/share/dict/.  /etc/dictionaries-common/words is a symbolic link to the currently-chosen
       /usr/share/dict/<language> file.  /usr/share/dict/words is a symbolic link to /etc/dictionaries-common/words, and  is  the  name  by  which
       other  software should refer to the system word list.  See select-default-wordlist(8) for more information, and/or to change the currently-
       chosen word list.

       The directory /usr/share/dict can contain word lists for many languages, with name of the language in English, e.g., /usr/share/dict/french
       and  /usr/share/dict/danish  contain respectively lists of French and Danish words if they exist.  Such lists should be coded using the ISO
       8859-1 character set encoding.

SEE ALSO

       ispell(1), select-default-wordlist(8), and the Filesystem Hierarchy Standard.

HISTORY

       The words lists are not specific, and may be generated from any number of sources.

       The system word list used to be /usr/dict/words.  For compatibility, software should check that location if /usr/share/dict/words does  not
       exist.

AUTHOR

       Word  lists  are  collected and maintained by various authors.  The Debian English word lists are built from the SCOWL (Spell- Checker Ori-
       ented Word Lists) package, whose upstream editor is Kevin Atkinson <kevina@users.sourceforge.net>.

Debian								   16 June 2003 					  canadian-english-huge(5)
All times are GMT -4. The time now is 01:21 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy