Sponsored Content
Top Forums Shell Programming and Scripting Summing columns over group of lines Post 302798305 by sdp on Wednesday 24th of April 2013 07:38:16 AM
Old 04-24-2013
Linux Summing columns over group of lines

I have an input file that looks like:
Code:
ID1 V1 ID2 V2 P1 P2 P3 P4 ..... n no. of columns
1 1 1 1 1.0000 1.0000 1.0000 1.0000
1 1 1 2 0.9999 0.8888 0.7777 0.6666
1 2 1 1 0.8888 0.7777 0.6666 0.5555
1 2 1 2 0.7777 0.6666 0.5555 0.4444
2 1 1 1 0.6666 0.5555 0.4444 0.3333
2 1 1 2 0.5555 0.4444 0.3333 0.2222
2 2 1 1 0.4444 0.3333 0.2222 0.1111
2 2 1 2 0.3333 0.2222 0.1111 0.1234

I would like to pick each field from column 5 i.e. P1 over each group of four lines and add them. The output needs to look like
Code:
ID1 ID2 P1 P2 P3 P4 .....n columns
1 1 3.6664 3.3331 ...... so on
2 1 1.9998 1.5554 ....... so on

Is there a way to do this using awk scripts ???

Last edited by vbe; 04-24-2013 at 08:40 AM.. Reason: please use code for your code and data Thanks
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Summing the columns of a file

Hi All, I have a file like - num.txt 12, 34, 65, line1 34, 65, 89, line2 43, 65, 77, line3 I want to do two things - 1. Add first three columns of each line and print the line with largest value. i.e. (12+34+65) for 1st line and so on. 2. Add middle column of each line i.e.... (3 Replies)
Discussion started by: asahlot
3 Replies

2. Shell Programming and Scripting

Awk: Summing values with group criteria

Hi Guys, I have a text file with ";" like separator F1;F2;F3;F4;F5 444;100041;IT;GLOB;1800000000 444;100041;TM;GLOB;1000000000 444;10300264;IT;GLOB;2000000000 444;10300264;IT;GLOB;2500000000 I have to sum the cullums F5 for same F2 and F3 collums The result must be: ... (7 Replies)
Discussion started by: gianluca2
7 Replies

3. Shell Programming and Scripting

Summing values in columns

Basically I have to process a text file which has been sorted this way: John 12 John 13 John 10 John 900 Peter 20 Peter 30 Peter 32 The first column is a name, and the second an arbitrary value, both delimited by a space. How can I sum them up such that it would become: John 935... (2 Replies)
Discussion started by: Dwee
2 Replies

4. Shell Programming and Scripting

Summing over specific lines and replacing the lines with the sum using sed, awk

Hi friends, This is sed & awk type question. I have a text file which has numbers spread all over the file. I want to sum the series of numbers whenever i find it and produce an output file with the sum. For example ###start of input text file #### abc def ghi 1 2 3 4 kjld random... (3 Replies)
Discussion started by: kaaliakahn
3 Replies

5. Shell Programming and Scripting

Summing over specific lines and replacing the lines with the sum

Hi friends, This is sed & awk type question. It is slightly different from my previous question. I have a text file which has numbers spread all over the file. I want to sum the series of numbers (but no more than 10 numbers in series) whenever i find it and produce an output file with the... (4 Replies)
Discussion started by: kaaliakahn
4 Replies

6. Homework & Coursework Questions

HELP with Unix scripts in summing columns in a file.

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: Hi guys, i'm a new guy here, and it's my first time creating a unix script. can you guys help me out here? i'd... (3 Replies)
Discussion started by: ramneim
3 Replies

7. Homework & Coursework Questions

HELP with Unix scripts in summing columns in a file

1. The problem statement, all variables and given/known data: Hi guys, i'm a new guy here, and it's my first time creating a unix script. can you guys help me out here? i'd really appreciate it. :( Here's my problem: This is the file i'm using, it has 6 columns, the first three columns are... (12 Replies)
Discussion started by: ramneim
12 Replies

8. Shell Programming and Scripting

Please Help!!!! Awk for summing columns based on selected column value

a,b,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z,aa,bb,cc,dd,ee,ff,gg,hh,ii a thru ii are digits and strings.... The awk needed....if coloumn 9 == i (coloumn 9 is string ), output the sum of x's(coloumn 22 ) in all records and sum of y's (coloumn 23 ) in all records in a file (records.txt).... (6 Replies)
Discussion started by: BrownBob
6 Replies

9. Shell Programming and Scripting

Summing columns in line

I have a file with the following format AAAAA 1.34B 0.76B 0.00B 0.00B 0.00B 0.00B 0.00B 0.00B 0.00B 0.00B 0.00B 0.00B 0.00B 0.00B 0.90B 0.00B 0.00B 0.46B 0.00B 0.03B 0.00B ... (4 Replies)
Discussion started by: ncwxpanther
4 Replies

10. Shell Programming and Scripting

Summing per group in a loop

I want to sum and average all other columns by first column GR1 1 4 7 GR1 2 5 8 GR1 3 6 9 GR2 11 14 17 GR2 13 16 19 GR3 1 3 5 GR3 2 4 6 For a limited number of columns I can do... (2 Replies)
Discussion started by: jianp83
2 Replies
SHOREWALL-TUNNELS(5)						  [FIXME: manual]					      SHOREWALL-TUNNELS(5)

NAME
tunnels - Shorewall VPN definition file SYNOPSIS
/etc/shorewall/tunnels DESCRIPTION
The tunnels file is used to define rules for encapsulated (usually encrypted) traffic to pass between the Shorewall system and a remote gateway. Traffic flowing through the tunnel is handled using the normal zone/policy/rule mechanism. See http://www.shorewall.net/VPNBasics.html for details. The columns in the file are as follows. TYPE - {ipsec[:{noah|ah}]|ipsecnat|ipip|gre|l2tp|pptpclient|pptpserver|COMMENT|{openvpn|openvpnclient|openvpnserver}[:{tcp|udp}][:port]|generic:protocol[:port]} Types are as follows: 6to4 or 6in4 - 6to4 or 6in4 tunnel. The 6in4 synonym was added in 4.4.24. ipsec - IPv4 IPSEC ipsecnat - IPv4 IPSEC with NAT Traversal (UDP port 4500 encapsulation) ipip - IPv4 encapsulated in IPv4 (Protocol 4) gre - Generalized Routing Encapsulation (Protocol 47) l2tp - Layer 2 Tunneling Protocol (UDP port 1701) pptpclient - PPTP Client runs on the firewall pptpserver - PPTP Server runs on the firewall openvpn - OpenVPN in point-to-point mode openvpnclient - OpenVPN client runs on the firewall openvpnserver - OpenVPN server runs on the firewall generic - Other tunnel type If the type is ipsec, it may be followed by :ah to indicate that the Authentication Headers protocol (51) is used by the tunnel (the default is :noah which means that protocol 51 is not used). NAT traversal is only supported with ESP (protocol 50) so ipsecnat tunnels don't allow the ah option (ipsecnat:noah may be specified but is redundant). If type is openvpn, openvpnclient or openvpnserver it may optionally be followed by ":" and tcp or udp to specify the protocol to be used. If not specified, udp is assumed. If type is openvpn, openvpnclient or openvpnserver it may optionally be followed by ":" and the port number used by the tunnel. if no ":" and port number are included, then the default port of 1194 will be used. . Where both the protocol and port are specified, the protocol must be given first (e.g., openvpn:tcp:4444). If type is generic, it must be followed by ":" and a protocol name (from /etc/protocols) or a protocol number. If the protocol is tcp or udp (6 or 17), then it may optionally be followed by ":" and a port number. Comments may be attached to Netfilter rules generated from entries in this file through the use of COMMENT lines. These lines begin with the word COMMENT; the remainder of the line is treated as a comment which is attached to subsequent rules until another COMMENT line is found or until the end of the file is reached. To stop adding comments to rules, use a line with only the word COMMENT. ZONE - zone The zone of the physical interface through which tunnel traffic passes. This is normally your internet zone. GATEWAY(S) (gateway or gateways) - address-or-range [ , ... ] The IP address of the remote tunnel gateway. If the remote gateway has no fixed address (Road Warrior) then specify the gateway as 0.0.0.0/0. May be specified as a network address and if your kernel and iptables include iprange match support then IP address ranges are also allowed. Beginning with Shorewall 4.5.3, a list of addresses or ranges may be given. Exclusion (shorewall-exclusion[1] (5) ) is not supported. GATEWAY ZONES (gateway_zone or gateway_zones) - [zone[,zone]...] Optional. If the gateway system specified in the third column is a standalone host then this column should contain a comma-separated list of the names of the zones that the host might be in. This column only applies to IPSEC tunnels where it enables ISAKMP traffic to flow through the tunnel to the remote gateway(s). EXAMPLE
Example 1: IPSec tunnel. The remote gateway is 4.33.99.124 and the remote subnet is 192.168.9.0/24. The tunnel does not use the AH protocol #TYPE ZONE GATEWAY ipsec:noah net 4.33.99.124 Example 2: Road Warrior (LapTop that may connect from anywhere) where the "gw" zone is used to represent the remote LapTop #TYPE ZONE GATEWAY GATEWAY ZONES ipsec net 0.0.0.0/0 gw Example 3: Host 4.33.99.124 is a standalone system connected via an ipsec tunnel to the firewall system. The host is in zone gw. #TYPE ZONE GATEWAY GATEWAY ZONES ipsec net 4.33.99.124 gw Example 4: Road Warriors that may belong to zones vpn1, vpn2 or vpn3. The FreeS/Wan _updown script will add the host to the appropriate zone using the shorewall add command on connect and will remove the host from the zone at disconnect time. #TYPE ZONE GATEWAY GATEWAY ZONES ipsec net 0.0.0.0/0 vpn1,vpn2,vpn3 Example 5: You run the Linux PPTP client on your firewall and connect to server 192.0.2.221. #TYPE ZONE GATEWAY GATEWAY ZONES pptpclient net 192.0.2.221 Example 6: You run a PPTP server on your firewall. #TYPE ZONE GATEWAY GATEWAY ZONES pptpserver net 0.0.0.0/0 Example 7: OPENVPN tunnel. The remote gateway is 4.33.99.124 and openvpn uses port 7777. #TYPE ZONE GATEWAY GATEWAY ZONES openvpn:7777 net 4.33.99.124 Example 8: You have a tunnel that is not one of the supported types. Your tunnel uses UDP port 4444. The other end of the tunnel is 4.3.99.124. #TYPE ZONE GATEWAY GATEWAY ZONES generic:udp:4444 net 4.3.99.124 FILES
/etc/shorewall/tunnels SEE ALSO
http://shorewall.net/configuration_file_basics.htm#Pairs shorewall(8), shorewall-accounting(5), shorewall-actions(5), shorewall-blacklist(5), shorewall-hosts(5), shorewall_interfaces(5), shorewall-ipsets(5), shorewall-maclist(5), shorewall-masq(5), shorewall-nat(5), shorewall-netmap(5), shorewall-params(5), shorewall-policy(5), shorewall-providers(5), shorewall-proxyarp(5), shorewall-rtrules(5), shorewall-routestopped(5), shorewall-rules(5), shorewall.conf(5), shorewall-secmarks(5), shorewall-tcclasses(5), shorewall-tcdevices(5), shorewall-tcrules(5), shorewall-tos(5), shorewall-zones(5) NOTES
1. shorewall-exclusion http://www.shorewall.net/manpages/shorewall-exclusion.html [FIXME: source] 06/28/2012 SHOREWALL-TUNNELS(5)
All times are GMT -4. The time now is 08:41 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy