Sponsored Content
Top Forums UNIX for Advanced & Expert Users Need Optimization shell/awk script to aggreagte (sum) for all the columns of Huge data file Post 303025070 by vgersh99 on Tuesday 23rd of October 2018 04:27:15 PM
Old 10-23-2018
Please post a sample representative chunk of a file.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script to check the unique numbers in huge data

Friends, I have to write a shell script,the description is---- i Have to check the uniqueness of the numbers in a file. A file is containing 200thousand tickets and a ticket have 15 numbers in asecending order.And there is a strip that is having 6 tickets that means 90 numbers.I... (7 Replies)
Discussion started by: namishtiwari
7 Replies

2. Shell Programming and Scripting

Find percent between sum of 2 columns awk help

Hi I'm new to this forum and I'm a beginner when it comes to shell and awk programming. But I have the following problem: I have 5 csv files (data1.csv, data2.csv, etc.) and need to calculate the average between the total sum of the 1st and 7 column. csv example:... (3 Replies)
Discussion started by: sapo51
3 Replies

3. Shell Programming and Scripting

awk sum columns

can anyone help me how do i add the colums using awk seperated by character @. for eg i have 3@4 2@9 5@1 the result should be 10 14 i tried using { sum+= $1 } END { print sum } but it just gives the result 10. can anyone help me with this one thank you and best regards (7 Replies)
Discussion started by: phone_book
7 Replies

4. UNIX for Advanced & Expert Users

A variable and sum of its value in a huge data.

Hi Experts, I got a question.. In the following output of `ps -elf | grep DataFlow` I get:- 242001 A mqsiadm 2076676 1691742 0 60 20 26ad4f400 130164 * May 09 - 3:02 DataFlowEngine EAIDVBR1_BROKER 5e453de8-2001-0000-0080-fd142b9ce8cb VIPS_INQ1 0 242001 A mqsiadm... (5 Replies)
Discussion started by: varungupta
5 Replies

5. Shell Programming and Scripting

AWK/Shell script for formatting data in a file

Hi All, Need an urgent help to convert a unix file in to a particular format: **source file:** 1111111 2d2f2h2 3dfgsd3 ........... 1111111 <-- repeats in every nth line. remaining all lines will be different 123ss41 432ff45 ........... 1111111 <-- repetition qwe1234 123weq3... (1 Reply)
Discussion started by: rajivnairfis
1 Replies

6. Shell Programming and Scripting

Sum up values of columns in 4 files using shell script

I am new to shell script.I have records like below in 4 different files which have about 10000 records each, all records unique and sorted based on column 2. 1 2 3 4 5 6 --------------------------- SR|1010478|000044590|1|0|0| SR|1014759|000105790|1|0|0| SR|1016609|000108901|1|0|0|... (2 Replies)
Discussion started by: reach.sree@gmai
2 Replies

7. Shell Programming and Scripting

Awk based script to find the median of all individual columns in a data file

Hi All, I have some data like below. Step1,Param1,Param2,Param3 1,2,3,4 2,3,4,5 2,4,5,6 3,0,1,2 3,0,0,0 3,2,1,3 ........ so on Where I need to find the median(arithmetic) of each column from Param1...to..Param3 for each set of Step1 values. (Sort each specific column, if the... (5 Replies)
Discussion started by: ks_reddy
5 Replies

8. Shell Programming and Scripting

awk based script to find the average of all the columns in a data file

Hi All, I need the modification for the below mentioned code (found in one more post https://www.unix.com/shell-programming-scripting/27161-script-generate-average-values.html) to find the average values for all the columns(but for a specific rows) and print the averages side by side. I have... (4 Replies)
Discussion started by: ks_reddy
4 Replies

9. Shell Programming and Scripting

awk does not work well with huge data?

Dear all , I found that if we work with thousands line of data, awk does not work perfectly. It will cut hundreds line (others are deleted) and works only on the remain data. I used this command : awk '$1==1{$1="Si"}{print>FILENAME}' coba.xyz to change value of first column whose value is 1... (4 Replies)
Discussion started by: ariesto
4 Replies

10. Shell Programming and Scripting

Sum of columns using awk

Hello everyone I am a beginner in Shell scripting. Need your help to achieve desired result. I have a file (sample format below) 001g8aX0007jxLz xxxxxxxxxxxxxxx 9213974926411 CO-COMM-133 CO-L001-DLY 7769995578239 44938 1 1 ... (1 Reply)
Discussion started by: Rohit Mallah
1 Replies
pack_fopen_chunk(3alleg4)					  Allegro manual					 pack_fopen_chunk(3alleg4)

NAME
pack_fopen_chunk - Opens a sub-chunk of a file. Allegro game programming library. SYNOPSIS
#include <allegro.h> PACKFILE *pack_fopen_chunk(PACKFILE *f, int pack); DESCRIPTION
Opens a sub-chunk of a file. Chunks are primarily intended for use by the datafile code, but they may also be useful for your own file rou- tines. A chunk provides a logical view of part of a file, which can be compressed as an individual entity and will automatically insert and check length counts to prevent reading past the end of the chunk. The PACKFILE parameter is a previously opened file, and `pack' is a bool- ean parameter which will turn compression on for the sub-chunk if it is non-zero. Example: PACKFILE *output = pack_fopen("out.raw", "w!"); ... /* Create a sub-chunk with compression. */ output = pack_fopen_chunk(output, 1); if (!output) abort_on_error("Error saving data!"); /* Write some data to the sub-chunk. */ ... /* Close the sub-chunk, recovering parent file. */ output = pack_fclose_chunk(output); The data written to the chunk will be prefixed with two length counts (32-bit, a.k.a. big-endian). For uncompressed chunks these will both be set to the size of the data in the chunk. For compressed chunks (created by setting the `pack' flag), the first length will be the raw size of the chunk, and the second will be the negative size of the uncompressed data. To read the chunk, use the following code: PACKFILE *input = pack_fopen("out.raw", "rp"); ... input = pack_fopen_chunk(input, 1); /* Read data from the sub-chunk and close it. */ ... input = pack_fclose_chunk(input); This sequence will read the length counts created when the chunk was written, and automatically decompress the contents of the chunk if it was compressed. The length will also be used to prevent reading past the end of the chunk (Allegro will return EOF if you attempt this), and to automatically skip past any unread chunk data when you call pack_fclose_chunk(). Chunks can be nested inside each other by making repeated calls to pack_fopen_chunk(). When writing a file, the compression status is inherited from the parent file, so you only need to set the pack flag if the parent is not compressed but you want to pack the chunk data. If the parent file is already open in packed mode, setting the pack flag will result in data being compressed twice: once as it is written to the chunk, and again as the chunk passes it on to the parent file. RETURN VALUE
Returns a pointer to the sub-chunked PACKFILE, or NULL if there was some error (eg. you are using a custom PACKFILE vtable). SEE ALSO
pack_fclose_chunk(3alleg4), pack_fopen(3alleg4) Allegro version 4.4.2 pack_fopen_chunk(3alleg4)
All times are GMT -4. The time now is 03:48 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy