Sponsored Content
Top Forums Shell Programming and Scripting Getting Sum, Count and Distinct Count of a file Post 302292491 by avi.skynet on Saturday 28th of February 2009 06:16:06 AM
Old 02-28-2009
MySQL Partial Solution

Hi i can solve your first 2 requirement the third i hope some one will post the solution ... will the solution for first 2 options follows
------------------------------
NumOfColumn=0
SumOfColumn=0
while read line
do
var1=`echo $line | cut -d"|" -f1`
SumOfColumn=`expr $SumOfColumn + $var1`
NumOfColumn=`expr $NumOfColumn + 1`
done < larg_file.txt

echo -e "SumOfColumn=$SumOfColumn\nNumOfColumn=$NumOfColumn"
#This will work Smilie
---------------------------------

for third option you have to read sort or uniq command in book
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Count the distinct list of ids

Hello guys, I have a file in the following format(each line seperated by TAB): ========= Filename id Filename id1 Filename id Filename1 id7 Filename1 id7 Filename2 id1 Filename2 id1 Filename2 id3 Filename3 id2 Filename3 id4 Filename3 id4 Filename3 id6 ========= I would like to... (2 Replies)
Discussion started by: jingi1234
2 Replies

2. Shell Programming and Scripting

To count distinct fields in a row

I have . dat file which contains data in a specific format: 0 3 892 921 342 1 3 921 342 543 2 4 817 562 718 765 3 3 819 562 717 761 i need to compare each field in a row with another field of the same column but different row and cont the... (8 Replies)
Discussion started by: Abhik
8 Replies

3. Shell Programming and Scripting

sum divided by count

Dear friends, I'm stuck with the task below, I would be thankful for all your replies. INPUT : Date Price Volume 20110601 73052811.61 2845833 20110602 61489062.96 9909230 20110603 72790724.65 1108927 20110606 48299507.20 7435881 20110607 ... (5 Replies)
Discussion started by: hernand
5 Replies

4. Shell Programming and Scripting

awk and count sum ?

I have a input.txt file which have 3 fields separate by a comma place, os and timediff in seconds tampa,win7, 2575 tampa,win7, 157619 tampa,win7, 3352 dallas,vista,604799 greenbay,winxp, 14400 greenbay,win7 , 518400 san jose,winxp, 228121 san jose,winxp, 70853 san jose,winxp, 193514... (5 Replies)
Discussion started by: sabercats
5 Replies

5. UNIX for Dummies Questions & Answers

count number of distinct values in each column with awk

Hi ! input: A|B|C|D A|F|C|E A|B|I|C A|T|I|B As the title of the thread says, I would need to get: 1|3|2|4 I tried different variants of this command, but I don't manage to obtain what I need: gawk 'BEGIN{FS=OFS="|"}{for(i=1; i<=NF; i++) a++} END {for (b in a) print b}' input ... (2 Replies)
Discussion started by: beca123456
2 Replies

6. Shell Programming and Scripting

Script Shell: Count The sum of numbers in a file

Hi all; Here is my file: V1.3=4 V1.4=5 V1.1=3 V1.2=6 V1.3=6 Please, can you help me to write a script shell that counts the sum of values in my file (4+5+3+6+6) ? Thank you so much for help. Kind regards. (3 Replies)
Discussion started by: chercheur111
3 Replies

7. Shell Programming and Scripting

Help with Getting distinct record count from a .dat file using UNIX command

Hi, I have a .dat file with contents like the below: Input file ============SEQ NO-1: COLUMN1========== 9835619 7152815 ============SEQ NO-2: COLUMN2 ========== 7615348 7015548 9373086 ============SEQ NO-3: COLUMN3=========== 9373086 Expected Output: (I just... (1 Reply)
Discussion started by: MS06
1 Replies

8. Shell Programming and Scripting

Finding total distinct count from multiple csv files through UNIX script

Hi All , I have multiple pipe delimited csv files are present in a directory.I need to find out distinct count on a column on those files and need the total distinct count on all files. We can't merge all the files here as file size are huge in millions.I have tried in below way for each... (9 Replies)
Discussion started by: STCET22
9 Replies

9. UNIX for Beginners Questions & Answers

Awk: count unique elements in a field and sum their occurence across the entire file

Hi, Sure it's an easy one, but it drives me insane. input ("|" separated): 1|A,B,C,A 2|A,D,D 3|A,B,B I would like to count the occurence of each capital letters in $2 across the entire file, knowing that duplicates in each record count as 1. I am trying to get this output... (5 Replies)
Discussion started by: beca123456
5 Replies

10. UNIX for Beginners Questions & Answers

Copy columns from one file into another and get sum of column values and row count

I have a file abc.csv, from which I need column 24(PurchaseOrder_TotalCost) to get the sum_of_amounts with date and row count into another file say output.csv abc.csv- UTF-8,,,,,,,,,,,,,,,,,,,,,,,,, ... (6 Replies)
Discussion started by: Tahir_M
6 Replies
FP2HDF(1)						      General Commands Manual							 FP2HDF(1)

NAME
fp2hdf - convert floating point data to HDF SYNOPSIS
fp2hdf -h[elp] fp2hdf infile [infile...] -o[utfile outfile] [-r[aster] [ras_options...]] [-f[loat]] DESCRIPTION
fp2hdf converts floating point data to HDF Scientific Data Set (SDS) and/or 8-bit Raster Image Set (RIS8) format, storing the results in an HDF file. The image data can be scaled about a mean value. Input file(s) contain a single two-dimensional or three-dimensional floating point array in either ASCII text, native floating point, or HDF SDS format. If an HDF file is used for input, it must contain an SDS. The SDS need only contain a dimension record and the data, but if it also contains maximum and minimum values and/or scales for each axis, these will be used. If the input format is ASCII text or native floating point, see "Notes" below on how it must be organized. OPTIONS
-h[elp] Print a helpful summary of usage, and exit. -o[utfile] outfile Data from one or more input files are stored as one or more data sets and/or images in one HDF output file, outfile. -r[aster] Store output as a raster image set in the output file -f[loat] Store output as a scientific data set in the the output file. This is the default if the "-r" option is not specified. ras_opts: -e[xpand] horiz vert [depth] Expand float data via pixel replication to produce the image(s). horiz and vert give the horizontal and vertical resolution of the image(s) to be produced; and optionally, depth gives the number of images or depth planes (for 3D input data). -i[nterp] horiz vert [depth] Apply bilinear, or trilinear, interpolation to the float data to produce the image(s). horiz, vert, and depth must be greater than or equal to the dimensions of the original dataset. -p[alfile] palfile Store the palette with the image. Get the palette from palfile; which may be an HDF file containing a palette, or a file containing a raw palette. -m[ean] mean If a floating point mean value is given, the image will be scaled about the mean. The new extremes (newmax and newmin), as given by: newmax = mean + max(abs(max-mean), abs(mean-min)) newmin = mean - max(abs(max-mean), abs(mean-min)) will be equidistant from the mean value. If no mean value is given, then the mean will be: 0.5 * (max + min) INPUT
If the input file format is ASCII text or native floating point, it must have the following input fields: format nplanes nrows ncols max_value min_value [plane1 plane2 plane3 ...] row1 row2 row3 ... col1 col2 col3 ... data1 data2 data3 ... ... Where: format Format designator ("TEXT", "FP32" or "FP64"). nplanes Dimension of the depth axis ("1" for 2D input). nrows Dimension of the vertical axis. ncols Dimension of the horizontal axis. max_value Maximum data value. min_value Minimum data value. plane1, plane2, plane3, ... Scales for depth axis. row1, row2, row3, ... Scales for the vertical axis. col1, col2, col3, ... Scales for the horizontal axis. data1, data2, data3, ... The data ordered by rows, left to right and top to bottom; then optionally, ordered by planes, front to back. For FP32 and FP64 input format, format, nplanes, nrows, ncols, and nplanes are native integers; where format is the integer repre- sentation of the appropriate 4-character string (0x46503332 for "FP32" and 0x46503634 for "FP64"). The remaining input fields are composed of native 32-bit floating point values for FP32 input format, or native 64-bit floating point values for FP64 input format. EXAMPLE
Convert floating point data in "f1.txt" to SDS format, and store it as an SDS in HDF file "o1": fp2hdf f1.txt -o o1 Convert floating point data in "f2.hdf" to 8-bit raster format, and store it as an RIS8 in HDF file "o2": fp2hdf f2.hdf -o o2 -r Convert floating point data in "f3.bin" to 8-bit raster format and SDS format, and store both the RIS8 and the SDS in HDF file "o3": fp2hdf f3.bin -o o3 -r -f Convert floating point data in "f4" to a 500x600 raster image, and store the RIS8 in HDF file "o4". Also store a palette from "palfile" with the image: fp2hdf f4 -o o4 -r -e 500 600 -p palfile Convert floating point data in "f5" to 200 planes of 500x600 raster images, and store the RIS8 in HDF file "o5". Also scale the image data so that it is centered about a mean value of 10.0: fp2hdf f5 -o o5 -r -i 500 600 200 -m 10.0 SEE ALSO
hdf(5) October 30, 1999 FP2HDF(1)
All times are GMT -4. The time now is 09:34 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy