Sponsored Content
Full Discussion: Summing lines in a file
Top Forums UNIX for Dummies Questions & Answers Summing lines in a file Post 302713519 by rdrtx1 on Wednesday 10th of October 2012 11:21:15 PM
Old 10-11-2012
or:
Code:
awk -F, 'BEGIN{OFS=","}{for(i=4;i<=NF;i++)s+=$i;print $0,s}' infile > another_file


Last edited by rdrtx1; 10-11-2012 at 12:29 AM..
This User Gave Thanks to rdrtx1 For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Summing the columns of a file

Hi All, I have a file like - num.txt 12, 34, 65, line1 34, 65, 89, line2 43, 65, 77, line3 I want to do two things - 1. Add first three columns of each line and print the line with largest value. i.e. (12+34+65) for 1st line and so on. 2. Add middle column of each line i.e.... (3 Replies)
Discussion started by: asahlot
3 Replies

2. UNIX for Advanced & Expert Users

Summing file size and output

:rolleyes:Hi, I need to find the sum of size of specific files in my directory Say for ex, mydir$ ls -ltr permisssion links user group size date time filename I want to display the sum of size of filenames having pattern "TS55". Note file size in this directory is near 400 MB. mydir$... (1 Reply)
Discussion started by: ramkrix
1 Replies

3. Shell Programming and Scripting

summing up the fields in fixed width file

Hi, I have a fixed width file with some records as given below: " 1000Nalsdjflj243324jljlj" "-0300Njfowjljl309933fsf" " 0010Njsfsjklj342344fsl" I want to sum-up first field values(i.e from 2nd character to 6th character)of each record. so for the above file i want to add (1000 - 300+... (2 Replies)
Discussion started by: srilaxmi
2 Replies

4. Shell Programming and Scripting

Summing over specific lines and replacing the lines with the sum using sed, awk

Hi friends, This is sed & awk type question. I have a text file which has numbers spread all over the file. I want to sum the series of numbers whenever i find it and produce an output file with the sum. For example ###start of input text file #### abc def ghi 1 2 3 4 kjld random... (3 Replies)
Discussion started by: kaaliakahn
3 Replies

5. Shell Programming and Scripting

Summing over specific lines and replacing the lines with the sum

Hi friends, This is sed & awk type question. It is slightly different from my previous question. I have a text file which has numbers spread all over the file. I want to sum the series of numbers (but no more than 10 numbers in series) whenever i find it and produce an output file with the... (4 Replies)
Discussion started by: kaaliakahn
4 Replies

6. Shell Programming and Scripting

awk summing specific lines and fields

Hi I would like to know if it is possible to sum some specific fields. I have this x;x;x;x;x;x;x;x;467,390,611 Bytes;0.435291 GB;0.062247 GB;0.373045 GB;11,225;157 a;a;a;a;a;a;a;a;13,805,156,846 Bytes;12.857054 GB;1.838559 GB;11.018495 GB;151,063;18,933 b;b;b;b;b;b;b;b;232,797,478,723... (5 Replies)
Discussion started by: nakaedu
5 Replies

7. UNIX Desktop Questions & Answers

Summing file sizes

I was just curious about how to sum the total file size of a certain type of file. For instance: $find . -name "*.aif" will print out the paths to each .aif file. Instead of printing, how could one sum the total space used by all of the aif files? Thanks! Please use code tags (3 Replies)
Discussion started by: Alexander4444
3 Replies

8. Shell Programming and Scripting

Help summing a file using awk

I'm trying to sum a text file using AWK. Here is an example of the file: 600|3H68| 46 600|3H69| 46 600|3H6F| 290 600|3H6G| 24 600|3HDY| 1 600|3HDY| 3 600|3HE0| 1 600|3HE0| 3 I would like to sum the third field if the first... (7 Replies)
Discussion started by: Drenhead
7 Replies

9. Shell Programming and Scripting

Summing columns over group of lines

I have an input file that looks like: ID1 V1 ID2 V2 P1 P2 P3 P4 ..... n no. of columns 1 1 1 1 1.0000 1.0000 1.0000 1.0000 1 1 1 2 0.9999 0.8888 0.7777 0.6666 1 2 1 1 0.8888 0.7777 0.6666 0.5555 1 2 1 2 0.7777 0.6666 0.5555 0.4444 2 1 1 1 0.6666 0.5555 0.4444 0.3333 2 1 1 2 0.5555 0.4444... (4 Replies)
Discussion started by: sdp
4 Replies

10. Shell Programming and Scripting

Summing all fields in a file

I was playing around to see how stuff works, and was trying to sum all fields in a file. cat file 1 2 3 4 5 6 7 8 9 10 11 12 I made this script: awk 'BEGIN {OFS=RS}{$1=$1}{s+=$0} END {print "sum="s}' file This gives 15, why not 78? I test it like this awk 'BEGIN... (5 Replies)
Discussion started by: Jotne
5 Replies
srec_fpc(5)							File Formats Manual						       srec_fpc(5)

NAME
srec_fpc - four packed code file format SYNOPSIS
All ASCII based file formats have one disadvantage in common: they all need more than double the amount of characters as opposed to the number of bytes to be sent. Address fields and checksums will add even more characters. So the shorter the records, the more characters have to be sent to get the file across. The FPC format may be used to reduce the number of characters needed to send a file in ASCII format, although it still needs more charac- ters than the actual bytes it sends. FPC stands for "Four Packed Code". The reduction is accomplished by squeezing 4 real bytes into 5 ASCII characters. In fact every ASCII character will be a digit in the base 85 number system. There aren't enough letters, digits and punctuation marks available to get 85 different characters, but if we use both upper case and lower case letters we will manage. This implies that the FPC is case sensitive, as opposed to all other ASCII based file formats. Base 85 The numbering system is in base 85, and is somewhat hard to understand for us humans who are usually only familiar with base 10 numbers. Some of us understand base 2 and base 16 as well, but base 85 is for most people something new. Luckily we don't have to do any math with this number system. We just convert a 32 bit number into a 5 digit number in base 85. A 32 bit number has a range of 4,294,967,296, while a 5 digit number in base 85 has a range of 4,437,053,125, which is enough to do the trick. One drawback is that we always have to send multiples of 4 bytes, even if we actually want to send 1, 2 or 3 bytes. Unused bytes are padded with zeroes, and are discarded at the receiving end. The digits of the base 85 numbering system start at %, which represents the value of 0. The highest value of a digit in base 85 is 84, and is represented by the character 'z'. If you want to check this with a normal ASCII table you will notice that we have used one character too many! Why? I don't know, but for some reason we have to skip the '*' character in the row. This means that after the ')' character follows the '+' character. We can use normal number conversion algorithms to generate the FPC digits, with this tiny difference. We have to check whether the digit is going to be equal or larger than the ASCII value for '*'. If this is the case we have to increment the digit once to stay clear of the '*'. In base 85 MSD digits go first, like in all number systems! The benefit of this all is hopefully clear. For every 4 bytes we only have to send 5 ASCII characters, as opposed to 8 characters for all other formats. Records Now we take a look at the the formatting of the FPC records. We look at the record at byte level, not at the actual base 85 encoded level. Only after formatting the FPC record at byte level we convert 4 bytes at a time to a 5 digit base 85 number. If we don't have enough bytes in the record to fill the last group of 5 digits we will add bytes with the value of 0 behind the record. +--+----+----+------+----------+----------+ |$ | ss | cc | ffff | aaaaaaaa | dddddddd | The field are defined as: +--+----+----+------+----------+----------+ $ Every line starts with the character $, all other characters are digits of base 85. ss The checksum. A one byte 2's-complement checksum of all bytes of the record. cc The byte-count. A one byte value, counting all the bytes in the record minus 4. ffff Format code, a two byte value, defining the record type. aaaaaaaa The address field. A 4 byte number representing the first address of this record. dddddddd The actual data of this record. Record Begin Every record begins with the ASCII character "$". No spaces or tabs are allowed in a record. All other characters in the record are formed by groups of 5 digits of base 85. Checksum field This field is a one byte 2's-complement checksum of the entire record. To create the checksum make a one byte sum from all of the bytes from all of the fields of the record: Then take the 2's-complement of this sum to create the final checksum. The 2's-complement is simply inverting all bits and then increment by 1 (or using the negative operator). Checking the checksum at the receivers end is done by adding all bytes together including the checksum itself, discarding all carries, and the result must be $00. The padding bytes at the end of the line, should they exist, should not be included in checksum. But it doesn't really matter if they are, for their influence will be 0 anyway. Byte Count The byte count cc counts the number of bytes in the current record minus 4. So only the number of address bytes and the data bytes are counted and not the first 4 bytes of the record (checksum, byte count and format flags). The byte count can have any value from 0 to 255. Usually records have 32 data bytes. It is not recommended to send too many data bytes in a record for that may increase the transmission time in case of errors. Also avoid sending only a few data bytes per record, because the address overhead will be too heavy in comparison to the payload. Format Flags This is a 2 byte number, indicating what format is represented in this record. Only a few formats are available, so we actually waste 1 byte in each record for the sake of having multiples of 4 bytes. Format code 0 means that the address field in this record is to be treated as the absolute address where the first data byte of the record should be stored. Format code 1 means that the address field in this record is missing. Simply the last known address of the previous record +1 is used to store the first data byte. As if the FPC format wasn't fast enough already ;-) Format code 2 means that the address field in this record is to be treated as a relative address. Relative to what is not really clear. The relative address will remain in effect until an absolute address is received again. Address Field The first data byte of the record is stored in the address specified by the Address field aaaaaaaa. After storing that data byte, the address is incremented by 1 to point to the address for the next data byte of the record. And so on, until all data bytes are stored. The length of the address field is always 4 bytes, if present of course. So the address range for the FPC format is always 2**32. If only the address field is given, without any data bytes, the address will be set as starting address for records that have no address field. Addresses between records are non sequential. There may be gaps in the addressing or the address pointer may even point to lower addresses as before in the same file. But every time the sequence of addressing must be changed, a format 0 record must be used. Addressing within one single record is sequential of course. Data Field This field contains 0 or more data bytes. The actual number of data bytes is indicated by the byte count in the beginning of the record less the number of address bytes. The first data byte is stored in the location indicated by the address in the address field. After that the address is incremented by 1 and the next data byte is stored in that new location. This continues until all bytes are stored. If there are not enough data bytes to obtain a multiple of 4 we use 0x00 as padding bytes at the end of the record. These padding bytes are ignored on the receiving side. End of File End of file is recognized if the first four bytes of the record all contain 0x00. In base 85 this will be "$%%%%%". This is the only decent way to terminate the file. Size Multiplier In general, binary data will expand in sized by approximately 1.7 times when represented with this format. Example Now it's time for an example. In the first table you can see the byte representation of the file to be transferred. The 4th row of bytes is not a multiple of 4 bytes. But that does not matter, for we append $00 bytes at the end until we do have a multiple of 4 bytes. These padding bytes are not counted in the byte count however! D81400000000B000576F77212044696420796F7520726561 431400000000B0106C6C7920676F207468726F7567682061 361400000000B0206C6C20746861742074726F75626C6520 591100000000B030746F207265616420746869733F000000 00000000 Only after converting the bytes to base 85 we get the records of the FPC type file format presented in the next table. Note that there is always a multiple of 5 characters to represent a multiple of 4 bytes in each record. $kL&@h%%,:,B.?00EPuX0K3rO0JI)) $;UPR'%%,:<Hn&FCG:at<GVF(;G9wIw $7FD1p%%,:LHmy:>GTV%/KJ7@GE[kYz $B[6;%%,:KIn?GFWY/qKI1G5:;-_e $%%%%% As you can see the length of the lines is clearly shorter than the original ASCII lines. SEE ALSO
http://sbprojects.fol.nl/knowledge/fileformats/fpc.htm AUTHOR
This man page was taken from the above Web page. It was written by San Bergmans <sanmail@bigfoot.com> For extra points: Who invented this format? Where is it used? Reference Manual SRecord srec_fpc(5)
All times are GMT -4. The time now is 06:28 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy