Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Outputting data from log file to report

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Outputting data from log file to report
# 1  
Old 01-25-2019
Outputting data from log file to report
I have a log file that looks like this. the lines are grouped. 2 lines per entry.

Code:
M: 2019-01-25 13:02:31.698 P25, received network transmission from KI4EKI to TG 10282
M: 2019-01-25 13:02:35.694 P25, network end of transmission, 4.3 seconds, 1% packet loss
M: 2019-01-25 13:02:38.893 P25, received network transmission from AE4ML to TG 10282
M: 2019-01-25 13:02:39.595 P25, network end of transmission, 0.9 seconds, 0% packet loss
M: 2019-01-25 13:02:42.747 P25, received network transmission from KI4EKI to TG 10282
M: 2019-01-25 13:02:52.123 P25, network end of transmission, 9.7 seconds, 0% packet loss
M: 2019-01-25 13:03:00.388 P25, received network transmission from AE4ML to TG 10282
M: 2019-01-25 13:03:03.426 P25, network end of transmission, 3.2 seconds, 0% packet loss



I'm trying to pull data into one line , initial date, time and mode (P25) , past "transmission from" , To TG ", " end of transmission" and packet loss.

I have tried to grab everything from "Transmission from" to "loss"

using awk I can only pull what I want from the first line.

Code:
awk ' /transmission\ from/,/loss/ {for (I=1;I<=NF;I++) if ($I == "from") {print " CALL: "$(I+1) " TalkGroup "$(I+4)};} ' MMDVM.log




Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!
Last edited by RudiC; 01-25-2019 at 04:39 PM.. Reason: Added CODE tags.
# 2  
Old 01-25-2019
Like so?
Code:
awk -F, -vOFS=,  '/transmission from/ {getline X; $0 = $0 FS X; print $1, $2, $4, $6}' file
M: 2019-01-25 13:02:31.698 P25, received network transmission from KI4EKI to TG 10282, network end of transmission, 1% packet loss
M: 2019-01-25 13:02:38.893 P25, received network transmission from AE4ML to TG 10282, network end of transmission, 0% packet loss
M: 2019-01-25 13:02:42.747 P25, received network transmission from KI4EKI to TG 10282, network end of transmission, 0% packet loss
M: 2019-01-25 13:03:00.388 P25, received network transmission from AE4ML to TG 10282, network end of transmission, 0% packet loss

# 3  
Old 01-25-2019
I don't need "received network transmission from" or the "network end of transmission"


Code:
M: 2019-01-25 18:46:00.771 P25, AE4ML to TG 10282, 0.5 seconds, 0% packet loss



Thank you
Last edited by Don Cragun; 01-25-2019 at 08:27 PM..
# 4  
Old 01-25-2019
One could try:
Code:
awk '
NR % 2 {sub(/ received.*from/, "")
	printf("%s", $0)
}
!(NR % 2) {
	sub(/.*mission/, "")
	print
}' MMDVM.log

which produces the output:
Code:
M: 2019-01-25 13:02:31.698 P25, KI4EKI to TG 10282, 4.3 seconds, 1% packet loss
M: 2019-01-25 13:02:38.893 P25, AE4ML to TG 10282, 0.9 seconds, 0% packet loss
M: 2019-01-25 13:02:42.747 P25, KI4EKI to TG 10282, 9.7 seconds, 0% packet loss
M: 2019-01-25 13:03:00.388 P25, AE4ML to TG 10282, 3.2 seconds, 0% packet loss

from your sample input. But, since 0.5 seconds doesn't appear anywhere in your sample input data, I have no idea where the output you specified in post #3 in this thread came from???
This User Gave Thanks to Don Cragun For This Post:
Neo
# 5  
Old 01-26-2019
Quote:
Originally Posted by ae4ml
I don't need "received network transmission from" or the "network end of transmission"


Code:
M: 2019-01-25 18:46:00.771 P25, AE4ML to TG 10282, 0.5 seconds, 0% packet loss

Thank you

How about adapting the proposal to exactly fit the (fuzzy) spec (which e.g. lacks the seconds)?
# 6  
Old 01-26-2019
Don thanks but that didn't work. The output was mixed. It gave me more output than what I needed. it wasn't consistent. Using what you have shown me I haven't been able to do any better.



Code:
M: 2019-01-25 12:12:13.929 P25, AE4ML to TG 10282, 1.8 seconds, 0% packet loss
M: 2019-01-25 12:12:20.828 P25, AE4ML to TG 10282, 3.1 seconds, 0% packet loss
M: 2019-01-25 12:12:26.359 P25, AE4ML to TG 10282, 1.6 seconds, 0% packet loss
M: 2019-01-25 12:22:44.126 P25, network end of transmission, 1.6 seconds, 0% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:22:55.009 P25, network end of transmission, 4.7 seconds, 1% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:23:02.687 P25, network end of transmission, 2.7 seconds, 0% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:23:10.208 P25, network end of transmission, 5.0 seconds, 1% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:23:31.438 P25, network end of transmission, 17.6 seconds, 0% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:23:40.070 P25, network end of transmission, 6.1 seconds, 0% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:23:44.716 P25, network end of transmission, 2.0 seconds, 0% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:24:16.129 P25, network end of transmission, 28.1 seconds, 0% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:24:36.629 P25, network end of transmission, 13.3 seconds, 0% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:25:16.532 P25, network end of transmission, 36.2 seconds, 0% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:25:35.567 P25, network end of transmission, 12.6 seconds, 1% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:26:01.025 P25, network end of transmission, 21.8 seconds, 0% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:26:39.710 P25, network end of transmission, 33.5 seconds, 0% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:27:01.881 P25, network watchdog has expired, 15.7 seconds, 0% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:27:29.827 P25, network end of transmission, 21.6 seconds, 0% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:28:09.817 P25, network end of transmission, 1.8 seconds, 1% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:29:15.648 P25, network end of transmission, 5.0 seconds, 1% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:29:21.679 P25, network end of transmission, 2.3 seconds, 0% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:29:54.216 P25, network end of transmission, 29.3 seconds, 0% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:30:03.202 P25, network end of transmission, 3.2 seconds, 0% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:30:55.123 P25, network end of transmission, 43.9 seconds, 0% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:31:09.964 P25, network end of transmission, 6.1 seconds, 2% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:31:44.923 P25, network end of transmission, 31.1 seconds, 0% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:32:09.982 P25, network end of transmission, 13.1 seconds, 0% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:32:49.741 P25, network end of transmission, 35.3 seconds, 0% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:33:09.973 P25, network end of transmission, 11.0 seconds, 0% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:33:33.505 P25, network end of transmission, 19.4 seconds, 0% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:33:46.262 P25, network end of transmission, 6.8 seconds, 0% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:33:55.647 P25, network watchdog has expired, 2.7 seconds, 0% packet loss from 0 to TG 0
M: 2019-01-25 12:34:16.471 P25, network end of transmission, 19.3 seconds, 0% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:34:40.820 P25, network end of transmission, 15.5 seconds, 0% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:35:05.293 P25, network end of transmission, 20.9 seconds, 0% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:35:38.192 P25, network end of transmission, 24.7 seconds, 0% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:36:11.575 P25, network end of transmission, 28.1 seconds, 0% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:36:47.519 P25, network end of transmission, 23.4 seconds, 0% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:37:39.276 P25, network end of transmission, 46.6 seconds, 0% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:37:54.973 P25, network end of transmission, 8.8 seconds, 0% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:39:06.123 P25, network end of transmission, 64.8 seconds, 0% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:39:16.175 P25, network end of transmission, 4.9 seconds, 0% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:40:07.191 P25, network end of transmission, 46.3 seconds, 0% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:40:24.352 P25, network end of transmission, 13.3 seconds, 0% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:40:50.022 P25, network end of transmission, 22.3 seconds, 0% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:41:01.398 P25, network end of transmission, 3.4 seconds, 0% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:41:22.434 P25, network end of transmission, 18.0 seconds, 0% packet loss from AE4ML to TG 10282
M: 2019-01-25 12:41:29.433 P25, network end of transmission, 4.5 seconds, 0% packet loss from KI4EKI to TG 10282
M: 2019-01-25 12:41:34.259 P25, network end of transmission, 2.7 seconds, 2% packet loss, 2.7 seconds, 2% packet loss
M: 2019-01-25 12:54:00.882 P25, AE4ML to TG 10282, 0.4 seconds, 0% packet loss
M: 2019-01-25 12:54:06.856 P25, AE4ML to TG 10282, 0.5 seconds, 0% packet loss
M: 2019-01-25 13:02:23.852 P25, network end of transmission, 0.5 seconds, 0% packet loss, 0.5 seconds, 0% packet loss
M: 2019-01-25 13:02:31.698 P25, KI4EKI to TG 10282, 4.3 seconds, 1% packet loss
M: 2019-01-25 13:02:38.893 P25, AE4ML to TG 10282, 0.9 seconds, 0% packet loss
M: 2019-01-25 13:02:42.747 P25, KI4EKI to TG 10282, 9.7 seconds, 0% packet loss
M: 2019-01-25 13:03:00.388 P25, AE4ML to TG 10282, 3.2 seconds, 0% packet loss
M: 2019-01-25 13:03:06.106 P25, KI4EKI to TG 10282, 15.1 seconds, 0% packet loss
M: 2019-01-25 13:03:24.864 P25, AE4ML to TG 10282, 6.5 seconds, 0% packet loss
M: 2019-01-25 13:03:34.471 P25, KI4EKI to TG 10282, 6.5 seconds, 0% packet loss
M: 2019-01-25 13:03:49.833 P25, AE4ML to TG 10282, 3.8 seconds, 0% packet loss
M: 2019-01-25 13:08:08.229 P25, AE4ML to TG 10282, 0.4 seconds, 0% packet loss
M: 2019-01-25 13:08:44.319 P25, AE4ML to TG 10282, 3.1 seconds, 0% packet loss

# 7  
Old 01-26-2019
Does the sample given in post #1 represent the log file's structure one to one? I guess you don't have two alternating consecutive line types but quite some more ("watchdog"), and mayhap the pairs split, like some initial lines, and then some end lines.


Try this adapted - easily done by the interested person - version of my earlier proposal:

Code:
 awk -F, -vOFS=,  'match ($0, /, .*transmission from/) {sub (substr ($0, RSTART, RLENGTH), ","); getline X; $0 = $0 FS X; print $1, $2, $5, $6}' file

This would not help if the interesting lines are not adjacent. For that case, you'd need to identify a characteristic connecting the two lines.
Last edited by RudiC; 01-26-2019 at 10:33 AM..
These 2 Users Gave Thanks to RudiC For This Post:
ae4ml Neo
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Outputting Errors to a Log file

Good Morning, Every so often, I have copy scripts that to don't complete, but I don't immediately know why. It usually ends up being a permissions issue or a length issue. The scripts edit a log file, so I'd like to include any copy errors/issues in that file to check if the copies... (4 Replies)
Discussion started by: Stellaman1977
4 Replies

2. Shell Programming and Scripting

Create a report for client with a text data file

Hi, I am an amateur bash scriptwriter and I need to write a script which creates a report in a formatted, easy to read table-like that is displayed to standard output. The script has to export the followings: Process ID,User Name, Command Name,Priority..... Now I have a file that I can see all... (3 Replies)
Discussion started by: bashily
3 Replies

3. UNIX for Dummies Questions & Answers

outputting set -x to file

I need to enable set -x in my croned script as at times the script is not returning all data that it should be. This only happens intermittently and as such I would like a means of being able to check what goes wrong. My question is how to output the debug of set -x to file? (1 Reply)
Discussion started by: rob171171
1 Replies

4. Red Hat

Outputting clive --debug to file?

I am running clive on Linux version 2.6.24.5-85.fc8 (Red Hat 4.1.2-33) with the debug command as follows clive --debug -O <logfile> http://www.youtube.com/watch?v.... I need to get the contents of the debug written to file to extract data later. Using the -O option actually writes the... (0 Replies)
Discussion started by: rob171171
0 Replies

5. Shell Programming and Scripting

Outputting data from multiple lines

Hi guys, looking for a bit of advise, and as I am a complete novice, please excuse the daft questions!! I have a list of events and of which entry looks like this; # # Event 1 # NAME = Event 1 # 12345 : 123 : 1 : 1 : L,1,N : 1,0 : Event # # Event 2 # NAME = Event 2 # 12346... (8 Replies)
Discussion started by: JayC89
8 Replies

6. Shell Programming and Scripting

outputting data in table from grep results

Hi all. I'm a real unix newbie and looking for some help on a shell scripting problem. I'm going the longest ways around everything but I'm getting there. My next problem however has me stumped. I have set up a program that takes inputs from a user for a particular month and year (and stores them... (2 Replies)
Discussion started by: Chillspark
2 Replies

7. UNIX for Dummies Questions & Answers

create data file from report file

Dear Ones, kindly help me to create a data file from the report file as shown here under( i am new one to unix KNOW BASIC COMMANDS) file:rama.prt (ist record)(3 to 4 lines of text with different filed names) Name :M.LALITHA DOB:12/11/45 DESG :JA(P) STANO:300175 ... (3 Replies)
Discussion started by: cvvsnm
3 Replies

8. Shell Programming and Scripting

Outputting formatted Result log file from old 30000 lines result log<help required>

Well I have a 3000 lines result log file that contains all the machine data when it does the testing... It has 3 different section that i am intrsted in 1) starting with "20071126 11:11:11 Machine Header 1" 1000 lines... "End machine header 1" 2) starting with "20071126 12:12:12 Machine... (5 Replies)
Discussion started by: vikas.iet
5 Replies

9. Shell Programming and Scripting

Help in outputting the result in log files

This is the file am having: "40","1G1AL55 ",30482,9000 "40","1G1ZT58 ",29098,10600 "40","1G1AL15 ",29222,9400 "46","1G6KD57 ",3083,28400 "46","1G6KD57 ",27909,25200 "49","1G1ZU57 ",16391,13900 "49","1G2ZG58 ",28856,12400 I want to display the output in three files... (23 Replies)
Discussion started by: dave_nithis
23 Replies

10. Shell Programming and Scripting

Problem getting data to a report file.

Hi all, I'm trying in vain to workout how I can generate a report from a months worth of files that get created every day. There is one file per day and each daily file contain the output from a df -v command. With the following section of code ... for xdffile in $1$2/df?? do ... (4 Replies)
Discussion started by: Cameron
4 Replies
Login or Register to Ask a Question