Sponsored Content
Full Discussion: How big is my awk array?
Top Forums Shell Programming and Scripting How big is my awk array? Post 302135534 by vgersh99 on Monday 10th of September 2007 09:46:28 AM
Old 09-10-2007
or altrnatively.. change
Code:
                           per[$10]++
                           sum += $10
                           if ( per[$10] == 1 )
                                c++

TO
Code:
                           if ( !($10 in per) ) c++;
                           per[$10]++
                           sum += $10

 

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk not working as expected with BIG files ...

I am facing some strange problem. I know, there is only one record in a file 'test.txt' which starts with 'X' I ensure that with following command, awk /^X/ test.txt | wc -l This gives me output = '1'. Now I take out this record out of the file, as follows : awk /^X/ test.txt >... (1 Reply)
Discussion started by: videsh77
1 Replies

2. Shell Programming and Scripting

Big data file - sed/grep/awk?

Morning guys. Another day another question. :rolleyes: I am knocking up a script to pull some data from a file. The problem is the file is very big (up to 1 gig in size), so this solution: for results in `grep "^\ ... works, but takes ages (we're talking minutes) to run. The data is held... (8 Replies)
Discussion started by: dlam
8 Replies

3. Shell Programming and Scripting

Perl or awk/egrep from big files??

Hi experts. In one thread i have asked you how to grep the string from the below sample file- Unfortunately the script did not gave proper output (it missed many strings). It happened may be i did gave you the proper contents of the file That was the script- "$ perl -00nle'print join... (13 Replies)
Discussion started by: thepurple
13 Replies

4. Shell Programming and Scripting

awk with really big files

Hi, I have a text file that is around 7Gb which is basically a matrix of numbers (FS is a space and RS is \n). I need the most efficient way of plucking out a number from a specified row and column in the file. For example, for the value at row 15983, col 26332, I'm currently I'm using: ... (1 Reply)
Discussion started by: Jonny2Vests
1 Replies

5. Shell Programming and Scripting

awk column comparison big file

Hi all, I would like to compare a column in one file to a column in another file and when there is a match it prints the first column and the corresponding second column. Example File1 ABA ABC ABE ABF File 2 ABA 123 ABB 124 ABD 125 ABC 126 So what I would like printed to a... (6 Replies)
Discussion started by: pcg
6 Replies

6. Shell Programming and Scripting

AWK - Parse a big file

INPUT SAMPLE Symmetrix ID : 000192601507 Masking View Name : TS00P22_13E_1 Last updated at : 05:10:18 AM on Tue Mar 22,2011 Initiator Group Name : 10000000c960b9cd Host Initiators { WWN : 10000000c960b9cd } Port Group Name :... (8 Replies)
Discussion started by: greycells
8 Replies

7. Shell Programming and Scripting

How to Assign an shell array to awk array?

Hello All, Can you please help me with the below. #!/bin/bash ARR="No Differences In Stage Between HASH_TOTALS & HASH_TOTALS_COMP For UNINUM:0722075 PROVIDER:5 EXTRACT_DT:30-SEP-12 VER_NUM:1" ARR="No Differences In Stage Between HASH_TOTALS & HASH_TOTALS_COMP For UNINUM:0722075 PROVIDER:5... (14 Replies)
Discussion started by: Ariean
14 Replies

8. Shell Programming and Scripting

Split a big file into multiple files using awk

this thread is a continuation from previous thread https://www.unix.com/shell-programming-and-scripting/223901-split-big-file-into-multiple-files-based-first-four-characters.html ..I am using awk to split file and I have a syntax error while executing the below code I am using AIX 7.2... (4 Replies)
Discussion started by: etldev
4 Replies
sum(1)							      General Commands Manual							    sum(1)

NAME
sum - print checksum and block or byte count of file(s) SYNOPSIS
[file ...] Remarks is obsolescent and should not be used in new applications that are intended to be portable between systems. Use instead (see cksum(1)). DESCRIPTION
calculates and prints to standard output a checksum for each named file, and also prints the size of the file in 512 byte blocks, rounded up. The default algorithm is a 16-bit sum of the bytes in which overflow is ignored. Alternate algorithms can be selected with the and options. Standard input is used if no file names are given. is typically used to verify data integrity when copying files between systems. Options recognizes the following options: Use an alternate algorithm in which the 16-bit sum is right rotated with each byte in computing the checksum. Use the 32-bit cyclical redundancy check (CRC) algorithm used by RETURN VALUE
returns the following values upon completion: All files were processed successfully. One or more files could not be read or some other error occurred. If an inaccessible file is encountered, continues processing any remaining files, but the final exit status is affected. DIAGNOSTICS
Read error conditions are indistinguishable from end of file on most devices; check the block or byte count. WARNINGS
This command is likely to be withdrawn from X/Open standards. Applications using this command might not be portable to other vendors' platforms. The usage of cksum(1) is recommended. SEE ALSO
cksum(1), wc(1). STANDARDS CONFORMANCE
sum(1)
All times are GMT -4. The time now is 07:45 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy