Sponsored Content
Top Forums Shell Programming and Scripting [Solved] Find Specific records from file and add totals into variables Post 302340988 by veeru on Tuesday 4th of August 2009 11:32:04 PM
Old 08-05-2009
[Solved] Find Specific records from file and add totals into variables

Hi Eveyone,

I am working on one shell script to find the specific records from data file and add the totals into variables and print them. you can find the sample data file below for more clarification.

Sample Data File:

PXSTYL00__20090803USA
CHCART00__20090803IND
SSTRAN00__20090803USA
Records Sent: 100 Sum1: 10 20090803 1234567890 1234567890 123456789 123456789 123456789
Records Sent: 200 Sum1: 20 20090803 1234567890 1234567890 123456789 123456789 123456789
Records Sent: 300 Sum1: 30 20090803 1234567890 1234567890 123456789 123456789 123456789

In the above sample data file, I want to read all Records with word Records Sent and then get the totals and sums for all records together.

Out Put Should be like this

Total Records Sent: 100+200+300 = 600
Total Sum : 10+20+30 = 60

I tried the following code:

total = 0
sum = 0
grep "Records Sent:" filename > filename.tmp
cat filename.tmp |\
while read line
do
tot = `$line | cut -f3 -d " "`
su = `$line | cut -f5 -d " "`
total ='expr $total + $total`
sum =`expr $sum +su`
done
echo $total
echo $sum

I am having some issues while running this code. Those issues are
when i am using while read line so it will try to goby line by line but basically it should go by record by record because the Record which has Records Sent: will be 2 lines.
I had another error saying cannot create. I have no idea why this error is coming.
Can anybody write above script in different way.Please advise me on this

Best Regards
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extarct specific records from wide file

I have a file which is 5 million records. And each records has 412 fields has delimited by "|". So that makes each records to be 2923 bytes long. I wanted to extract specific records like top 100 or 2500 - 5000, 50001 - 10000 etc. from this file. I tried using head command for top 100 records,... (1 Reply)
Discussion started by: acheepi
1 Replies

2. Shell Programming and Scripting

cut specific records from a file

I am trying to cut the first 10 characters from a file only if the file has 'xyz' in field 185-188. I tried this cat filename | cut -c1-10 but this gives me all the records regardless of what is in field 185-188. Is this doable ? Thanks in advance for responses. (2 Replies)
Discussion started by: jxh461
2 Replies

3. UNIX for Dummies Questions & Answers

Cut specific fields from a file containing multiline records

Hi, I am looking for a method to get column13 to column 50 data from the 1st line of a multiline reord. The records are stored in a large file and are separated by newline. sample format is (data in red is to be extracted) <header> A001dfhskhfkdsh hajfhksdhfjh... (3 Replies)
Discussion started by: sunayana3112
3 Replies

4. UNIX for Dummies Questions & Answers

Grep specific records from a file of records that are separated by an empty line

Hi everyone. I am a newbie to Linux stuff. I have this kind of problem which couldn't solve alone. I have a text file with records separated by empty lines like this: ID: 20 Name: X Age: 19 ID: 21 Name: Z ID: 22 Email: xxx@yahoo.com Name: Y Age: 19 I want to grep records that... (4 Replies)
Discussion started by: Atrisa
4 Replies

5. Shell Programming and Scripting

[Solved] Find duplicate and add pattern in sed/awk

<Update> I have the solution: sed 's/\{3\}/&;&;---;4/' The thread can be marked as solved! </Update> Hi There, I'm working on a script processing some data from a website into cvs format. There is only one final problem left I can't find a solution. I've processed my file... (0 Replies)
Discussion started by: lolworlds
0 Replies

6. Shell Programming and Scripting

Extract error records based on specific criteria from Unix file

Hi, I look for a awk one liner for below issue. input file ABC 1234 abc 12345 ABC 4567 678 XYZ xyz ght 678 ABC 787 yyuu ABC 789 7890 777 zxr hyip hyu mno uii 678 776 ABC ty7 888 All lines should be started with ABC as first field. If a record has another value for 1st... (7 Replies)
Discussion started by: ratheesh2011
7 Replies

7. Shell Programming and Scripting

[Solved] Working with date (add minutes using variables)

Dear all, today I'm scratching my head with a simple (I believe) issue. Working with date is quite simple, so if I Need to add some seconds to current time, I'll use: date --date='+30 seconds' +"%Y-%m-%d %H:%M:%S"But, how to pass the value to add from a variable? I tried the following without... (2 Replies)
Discussion started by: Lord Spectre
2 Replies

8. Shell Programming and Scripting

[SOLVED] nawk FS using pipe read variables from file

I have a file data_1.out which contains: 1|abc mail|mail subject|mail body 2|def mail|mail subject|def mail body I am trying to read the variables from data_1.out and use them to print to 2 different files based on the id (first_column) The problem is I am not able to read the file... (8 Replies)
Discussion started by: sol_nov
8 Replies

9. Shell Programming and Scripting

[Solved] How to print specific text from a file?

Hi All, I have the below text file from which I have to cut particular section starting from PTR_Security_Rpeorting.cpf to PTR_Security_Reporting_Env93_export. Report Model............: "D:\Cognos_Publishing\tmp.a2R94KLQec"\PTR_Security_Reporting.cpf Report Output Script....:... (4 Replies)
Discussion started by: Vikram_Tanwar12
4 Replies

10. UNIX for Beginners Questions & Answers

Find records with specific characters in 2 nd field

Hi , I have a requirement to read a file ( 5 fields , ~ delimited) and find the records which contain anything other than Alphabets, Numbers , comma ,space and dot . ie a-z and A-Z and 0-9 and . and " " and , in 2nd field. Once I do that i would want the result to have field1|<flag> flag can... (2 Replies)
Discussion started by: ashwin3086
2 Replies
TOTAL(1)						      General Commands Manual							  TOTAL(1)

NAME
total - sum up columns SYNOPSIS
total [ -m ][ -sE | -p | -u | -l ][ -i{f|d}[N] ][ -o{f|d} ][ -tC ][ -N [ -r ]] [ file .. ] DESCRIPTION
Total sums up columns of real numbers from one or more files and prints out the result on its standard output. By default, total computes the straigt sum of each input column, but multiplication can be specified instead with the -p option. Likewise, the -u option means find the upper limit (maximum), and -l means find the lower limit (minimum). Sums of powers can be computed by giving an exponent with the -s option. (Note that there is no space between the -s and the exponent.) This exponent can be any real number, positive or negative. The absolute value of the input is always taken before the power is computed in order to avoid complex results. Thus, -s1 will produce a sum of absolute values. The default power (zero) is interpreted as a straight sum without taking absolute values. The -m option can be used to compute the mean rather than the total. For sums, the arithmetic mean is computed. For products, the geomet- ric mean is computed. (A logarithmic sum of absolute values is used to avoid overflow, and zero values are silently ignored.) If the input data is binary, the -id or -if option may be given for 64-bit double or 32-bit float values, respectively. Either option may be followed immediately by an optional count, which defaults to 1, indicating the number of double or float binary values to read per record on the input file. (There can be no space between the option and this count.) Similarly, the -od and -of options specify binary double or float output, respectively. These options do not need a count, as this will be determined by the number of input channels. A count can be given as the number of lines to read before computing a result. Normally, total reads each file to its end before producing its result, but this behavior may be overridden by inserting blank lines in the input. For each blank input line, total produces a result as if the end-of-file had been reached. If two blank lines immediately follow each other, total closes the file and proceeds to the next one (after reporting the result). The -N option (where N is a decimal integer) tells total to produce a result and reset the calculation after every N input lines. In addition, the -r option can be specified to override reinitialization and thus give a running total every N lines (or every blank line). If the end of file is reached, the current total is printed and the calculation is reset before the next file (with or without the -r option). The -tC option can be used to specify the input and output tab character. The default tab character is TAB. If no files are given, the standard input is read. EXAMPLE
To compute the RMS value of colon-separated columns in a file: total -t: -m -s2 input To produce a running product of values from a file: total -p -1 -r input BUGS
If the input files have varying numbers of columns, mean values will certainly be off. Total will ignore missing column entries if the tab separator is a non-white character, but cannot tell where a missing column should have been if the tab character is white. AUTHOR
Greg Ward SEE ALSO
cnt(1), neaten(1), rcalc(1), rlam(1), tabfunc(1) RADIANCE
2/3/95 TOTAL(1)
All times are GMT -4. The time now is 08:34 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy