Sponsored Content
Full Discussion: Aggregation of huge data
Top Forums Shell Programming and Scripting Aggregation of huge data Post 302896024 by Ravichander on Friday 4th of April 2014 03:27:50 AM
Old 04-04-2014
Question Aggregation of huge data

Hi Friends,

I have a file with sample amount data as follows:

Code:
 
 -89990.3456
8788798.990000128
55109787.20
-12455558989.90876

I need to exclude the '-' symbol in order to treat all values as an absolute one and then I need to sum up.The record count is around 1 million.

How can I perform this ?

Regards,
Ravichander
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

search and grab data from a huge file

folks, In my working directory, there a multiple large files which only contain one line in the file. The line is too long to use "grep", so any help? For example, if I want to find if these files contain a string like "93849", what command I should use? Also, there is oder_id number... (1 Reply)
Discussion started by: ting123
1 Replies

2. Shell Programming and Scripting

How to extract data from a huge file?

Hi, I have a huge file of bibliographic records in some standard format.I need a script to do some repeatable task as follows: 1. Needs to create folders as the strings starts with "item_*" from the input file 2. Create a file "contents" in each folders having "license.txt(tab... (5 Replies)
Discussion started by: srsahu75
5 Replies

3. UNIX for Advanced & Expert Users

A variable and sum of its value in a huge data.

Hi Experts, I got a question.. In the following output of `ps -elf | grep DataFlow` I get:- 242001 A mqsiadm 2076676 1691742 0 60 20 26ad4f400 130164 * May 09 - 3:02 DataFlowEngine EAIDVBR1_BROKER 5e453de8-2001-0000-0080-fd142b9ce8cb VIPS_INQ1 0 242001 A mqsiadm... (5 Replies)
Discussion started by: varungupta
5 Replies

4. Shell Programming and Scripting

Split a huge data into few different files?!

Input file data contents: >seq_1 MSNQSPPQSQRPGHSHSHSHSHAGLASSTSSHSNPSANASYNLNGPRTGGDQRYRASVDA >seq_2 AGAAGRGWGRDVTAAASPNPRNGGGRPASDLLSVGNAGGQASFASPETIDRWFEDLQHYE >seq_3 ATLEEMAAASLDANFKEELSAIEQWFRVLSEAERTAALYSLLQSSTQVQMRFFVTVLQQM ARADPITALLSPANPGQASMEAQMDAKLAAMGLKSPASPAVRQYARQSLSGDTYLSPHSA... (7 Replies)
Discussion started by: patrick87
7 Replies

5. UNIX for Dummies Questions & Answers

Copy huge data into vi editor

Hi All, HP-UX dev4 B.11.11 U 9000/800 3251073457 I need to copy huge data from windows text file to vi editor. when I tried copy huge data, the format of data is not preserverd and appered to scatterd through the vi, something like give below. Please let me know, how can I correct this? ... (18 Replies)
Discussion started by: alok.behria
18 Replies

6. Red Hat

Disk is Full but really does not contain huge data

Hi All, My disk usage show 100 % . When I check “df –kh” it shows my root partition is full. But when I run the “du –skh /” shows only 7 GB is used. Filesystem Size Used Avail Use% Mounted on /dev/sda1 30G 28G 260MB 100% / How I can identify who is using the 20 GB of memory. Os: Centos... (10 Replies)
Discussion started by: kalpeer
10 Replies

7. Shell Programming and Scripting

Aggregation of Huge files

Hi Friends !! I am facing a hash total issue while performing over a set of files of huge volume: Command used: tail -n +2 <File_Name> |nawk -F"|" -v '%.2f' qq='"' '{gsub(qq,"");sa+=($156<0)?-$156:$156}END{print sa}' OFMT='%.5f' Pipe delimited file and 156 column is for hash totalling.... (14 Replies)
Discussion started by: Ravichander
14 Replies

8. Shell Programming and Scripting

awk does not work well with huge data?

Dear all , I found that if we work with thousands line of data, awk does not work perfectly. It will cut hundreds line (others are deleted) and works only on the remain data. I used this command : awk '$1==1{$1="Si"}{print>FILENAME}' coba.xyz to change value of first column whose value is 1... (4 Replies)
Discussion started by: ariesto
4 Replies

9. Solaris

The Fastest for copy huge data

Dear Experts, I would like to know what's the best method for copy data around 3 mio (spread in a hundred folders, size each file around 1kb) between 2 servers? I already tried using Rsync and tar command. But using these command is too long. Please advice. Thanks Edy (11 Replies)
Discussion started by: edydsuranta
11 Replies

10. Shell Programming and Scripting

Phrase XML with Huge Data

HI Guys, I have Big XML file with Below Format :- Input :- <pokl>MKL=1,FN=1,GBNo=B10C</pokl> <d>192</d> <d>315</d> <d>35</d> <d>0,7,8</d> <pokl>MKL=1,dFN=1,GBNo=B11C</pokl> <d>162</d> <d>315</d> <d>35</d> <d>0,5,6</d> <pokl>MKL=1,dFN=1,GBNo=B12C</pokl> <d>188</d> (4 Replies)
Discussion started by: pareshkp
4 Replies
Text::Wrap(3pm) 					 Perl Programmers Reference Guide					   Text::Wrap(3pm)

NAME
Text::Wrap - line wrapping to form simple paragraphs SYNOPSIS
Example 1 use Text::Wrap $initial_tab = " "; # Tab before first line $subsequent_tab = ""; # All other lines flush left print wrap($initial_tab, $subsequent_tab, @text); print fill($initial_tab, $subsequent_tab, @text); @lines = wrap($initial_tab, $subsequent_tab, @text); @paragraphs = fill($initial_tab, $subsequent_tab, @text); Example 2 use Text::Wrap qw(wrap $columns $huge); $columns = 132; # Wrap at 132 characters $huge = 'die'; $huge = 'wrap'; $huge = 'overflow'; Example 3 use Text::Wrap $Text::Wrap::columns = 72; print wrap('', '', @text); DESCRIPTION
"Text::Wrap::wrap()" is a very simple paragraph formatter. It formats a single paragraph at a time by breaking lines at word boundries. Indentation is controlled for the first line ($initial_tab) and all subsequent lines ($subsequent_tab) independently. Please note: $ini- tial_tab and $subsequent_tab are the literal strings that will be used: it is unlikley you would want to pass in a number. Text::Wrap::fill() is a simple multi-paragraph formatter. It formats each paragraph separately and then joins them together when it's done. It will destory any whitespace in the original text. It breaks text into paragraphs by looking for whitespace after a newline. In other respects it acts like wrap(). OVERRIDES
"Text::Wrap::wrap()" has a number of variables that control its behavior. Because other modules might be using "Text::Wrap::wrap()" it is suggested that you leave these variables alone! If you can't do that, then use "local($Text::Wrap::VARIABLE) = YOURVALUE" when you change the values so that the original value is restored. This "local()" trick will not work if you import the variable into your own namespace. Lines are wrapped at $Text::Wrap::columns columns. $Text::Wrap::columns should be set to the full width of your output device. In fact, every resulting line will have length of no more than "$columns - 1". It is possible to control which characters terminate words by modifying $Text::Wrap::break. Set this to a string such as '[s:]' (to break before spaces or colons) or a pre-compiled regexp such as "qr/[s']/" (to break before spaces or apostrophes). The default is simply 's'; that is, words are terminated by spaces. (This means, among other things, that trailing punctuation such as full stops or commas stay with the word they are "attached" to.) Beginner note: In example 2, above $columns is imported into the local namespace, and set locally. In example 3, $Text::Wrap::columns is set in its own namespace without importing it. "Text::Wrap::wrap()" starts its work by expanding all the tabs in its input into spaces. The last thing it does it to turn spaces back into tabs. If you do not want tabs in your results, set $Text::Wrap::unexapand to a false value. Likewise if you do not want to use 8-character tabstops, set $Text::Wrap::tabstop to the number of characters you do want for your tabstops. If you want to separate your lines with something other than " " then set $Text::Wrap::seporator to your preference. When words that are longer than $columns are encountered, they are broken up. "wrap()" adds a " " at column $columns. This behavior can be overridden by setting $huge to 'die' or to 'overflow'. When set to 'die', large words will cause "die()" to be called. When set to 'overflow', large words will be left intact. Historical notes: 'die' used to be the default value of $huge. Now, 'wrap' is the default value. EXAMPLE
print wrap(" ","","This is a bit of text that forms a normal book-style paragraph"); AUTHOR
David Muir Sharnoff <muir@idiom.com> with help from Tim Pierce and many many others. perl v5.8.0 2002-06-01 Text::Wrap(3pm)
All times are GMT -4. The time now is 08:43 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy