Sponsored Content
Top Forums Shell Programming and Scripting Sum from successive lines following date header to create data for graphing connections Post 302746195 by shog63 on Wednesday 19th of December 2012 02:07:54 AM
Old 12-19-2012
Sum from successive lines following date header to create data for graphing connections

Hi,
I have a log file created from a load balancer showing connections to each member of a two member pool with the following format (where first field is source IP, second field is load balanced IP address and third field is destination member. I need to plot a graph by date/time and number of connections to each pool member (third IP address). By about yesterday lunchtime...

Please would someone be able to point me at the best way of parsing this file to get output similar to:
date IP1 numconnections IP2 numconnections? I'm imagining awk, but just can't get my head round how to start at the moment.

Sorry to appear dumb and really, really grateful for any helpful replies! Smilie
Code:
Mon Dec 17 15:07:16 GMT 2012
10.96.244.4:4901 <-> 10.18.1.212:32150 <-> 10.18.1.229:32150 tcp
10.96.244.4:4902 <-> 10.18.1.212:32150 <-> 10.18.1.229:32150 tcp.
10.18.0.148:33257 <-> 10.18.1.212:32150 <-> 10.18.1.230:32150 tcp
Mon Dec 17 15:07:46 GMT 2012
10.96.244.4:4903 <-> 10.18.1.212:32150 <-> 10.18.1.229:32150 tcp
10.96.244.4:4904 <-> 10.18.1.212:32150 <-> 10.18.1.229:32150 tcp.
10.18.0.148:33258 <-> 10.18.1.212:32150 <-> 10.18.1.230:32150 tcp
10.18.0.148:33259 <-> 10.18.1.212:32150 <-> 10.18.1.230:32150 tcp
10.18.0.148:33260 <-> 10.18.1.212:32150 <-> 10.18.1.230:32150 tcp


Last edited by Franklin52; 12-19-2012 at 03:22 AM.. Reason: Please use code tags for data and code samples
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Extract specific lines for graphing

Hello, I have a very large text file with about 2 million lines. Each of the lines starts like.. SNP_12345678 A 1212, 121, 343, ... SNP_12345678 B 4567, 567, 454, ... and so on. I want to extract specific SNPs and plot them by GNUplot or excel. The file is too large to be opened by text... (1 Reply)
Discussion started by: genehunter
1 Replies

2. Shell Programming and Scripting

Extract date from file header and prefix it to all lines

Hello All, I have a file in the following format. I want to extract the date(020090930, 020090929) in the string "STPAGE020090930" and "STPAGE020090929" and prefix it to all lines below them. The output must be put into a new file. STPAGE020090930 xyzz aalc... (3 Replies)
Discussion started by: john2022
3 Replies

3. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

4. Shell Programming and Scripting

How to create DB2 Connections in While loop using shell script

Hi , I have to create a db2 connection for the while loop in shell scripting. Below is the connection for DB2 I have to establish connect to TABLESCHENMA user $USERID using $PASSWORD Below is the while loop. while read TABLE; do db2 LOAD CLIENT FROM $DIRECTORY/$TABLE.ixf OF ixf INSERT... (1 Reply)
Discussion started by: vikyalex4
1 Replies

5. Shell Programming and Scripting

Summing over specific lines and replacing the lines with the sum using sed, awk

Hi friends, This is sed & awk type question. I have a text file which has numbers spread all over the file. I want to sum the series of numbers whenever i find it and produce an output file with the sum. For example ###start of input text file #### abc def ghi 1 2 3 4 kjld random... (3 Replies)
Discussion started by: kaaliakahn
3 Replies

6. Shell Programming and Scripting

Summing over specific lines and replacing the lines with the sum

Hi friends, This is sed & awk type question. It is slightly different from my previous question. I have a text file which has numbers spread all over the file. I want to sum the series of numbers (but no more than 10 numbers in series) whenever i find it and produce an output file with the... (4 Replies)
Discussion started by: kaaliakahn
4 Replies

7. Shell Programming and Scripting

Graphing data with awk/while loop

I need to graph data that's in a file. however, i only know of a way to graph data as it comes (meaning, as it is spat out by whatever process or job). i dont know how to graph data in a file. im using rrdtool to graph data. and i was wondering if anyone know of a way to pass a file (the... (3 Replies)
Discussion started by: SkySmart
3 Replies

8. Shell Programming and Scripting

Please help, need to create script to remove lines by date in file

Please Help (novice to PERL and SHELL scripting)…. Need to create a script which removes all lines in $filename = "cycle_calendar_ftp_out" older than current date – a variable which will be a number of days passed to script. For Ex it will look at the end date which is the last field (4) and... (2 Replies)
Discussion started by: m3pwr
2 Replies

9. Shell Programming and Scripting

[sed] Finding and sticking the pattern to the beginning of successive lines up to the next pattern

I have a file like below. 2018.07.01, Sunday 09:27 some text 123456789 0 21 0.06 0.07 0.00 2018.07.02, Monday 09:31 some text 123456789 1 41 0.26 0.32 0.00 09:39 some text 456789012 1 0.07 0.09 0.09 09:45 some text 932469494 1 55 0.29 0.36 0.00 16:49 some text 123456789 0 48 0.12 0.15 0.00... (9 Replies)
Discussion started by: father_7
9 Replies

10. Shell Programming and Scripting

Find header in a text file and prepend it to all lines until another header is found

I've been struggling with this one for quite a while and cannot seem to find a solution for this find/replace scenario. Perhaps I'm getting rusty. I have a file that contains a number of metrics (exactly 3 fields per line) from a few appliances that are collected in parallel. To identify the... (3 Replies)
Discussion started by: verdepollo
3 Replies
ar(4)							     Kernel Interfaces Manual							     ar(4)

NAME
ar - common archive file format SYNOPSIS
DESCRIPTION
The command is used to concatenate several files into an archive file (see ar(1)). Archives are used mainly as libraries to be searched by the link editor (see ld(1)). Each archive begins with the archive magic string. Following the archive magic string are the archive file members. Each file member is preceded by a file member header which is of the fol- lowing format: All information in the file member headers is in printable ASCII. The numeric information contained in the headers is stored as decimal numbers (except for which is in octal). Thus, if the archive contains printable files, the archive itself is printable. The contents of the field are slash terminated and blank-padded. The field is the modification date of the file at the time of its inser- tion into the archive. Common format archives can be moved from system to system as long as the portable archive command is used. Note that older versions of did not use the common archive format, and those archives cannot be read or written by the common archiver. Each archive file member begins on an even byte boundary; a new-line character is inserted between files if necessary. Nevertheless, the size given reflects the actual size of the file exclusive of padding. Notice there is no provision for empty areas in an archive file. If the archive symbol table exists, the first file in the archive has a zero-length name (i.e., and The contents of this archive member are machine-dependent. Refer to the appropriate a.out(4) manual entry for more information. Each archive which contains object files (see a.out(4)) may include an archive symbol table. This symbol table is used by the link editor (see ld(1)) to determine which archive members must be loaded during the link edit process. The archive symbol table (if it exists) is always the first member in the archive (but is never listed) and is automatically created and/or updated by If a member with a file name greater than 15 bytes exists within the archive, then the archive will also contain an additional special mem- ber to store the long file name string table. The special string table member has a zero length name where and If a special string table exists, it will precede all non-special archive members. If both a symbol table member and a string table member exist then the symbol table member will always precede the string table member. Each entry in the string table is followed by a slash and a new-line character. The offset of the table begins at zero. If an archive mem- ber name exceeds 15 bytes, then the entry in the member's header does not contain a name, instead it contains the offset into the string table preceded by a slash. For example, the member name contains in the field. This value represents the offset into the string table. The member name contains in the field. The long name string table would have the following format: +0 +1 +2 +3 +4 +5 +6 +7 +8 +9 +---------------------------------------------------------- 0| t | h | i | s | i | s | a | v | e | r | +---------------------------------------------------------- 10| y | l | o | n | g | f | i | l | e | n | +---------------------------------------------------------- 20| a | m | e | . | o | / | | y | e | t | +---------------------------------------------------------- 30| a | n | o | t | h | e | r | l | o | n | +---------------------------------------------------------- 40| g | f | i | l | e | n | a | m | e | . | +---------------------------------------------------------- 50| o | / | | +---------------- SEE ALSO
System Tools: ar(1) create archived libraries ld(1) invoke the link editor Miscellaneous: a.out(4) assembler, compiler, and linker output magic(4) magic number for HP-UX implementations ranlib(1) regenerate an archive symbol table strip(1) strip symbol and line number information from an object file CAVEATS
removes the archive symbol table member from the archive (see strip(1)). The archive symbol table must be restored by using the option of the command or the ranlib(1) command before the archive can be used with the link editor. ar(4)
All times are GMT -4. The time now is 12:47 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy