Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Using awk to Summarize Log File in 5min Intervals

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Using awk to Summarize Log File in 5min Intervals
# 1  
Old 03-31-2017
Using awk to Summarize Log File in 5min Intervals
I have huge log file that taken every minute
and I need the total at 5min intervals.


Sample log:
Code:
#timestamp(yyyymmddhhmm);result;transaction
201703280000;120;6
201703280001;120;3
201703280002;105;3
201703280003;105;5
201703280004;105;5
201703280005;105;4
201703280006;120;2
201703280007;120;2
201703280008;105;5
201703280009;105;8


Then roughly be like this
Code:
201703280000;120;9
201703280000;105;13
201703280005;105;17
201703280005;120;4

Last edited by wwolfking; 03-31-2017 at 08:24 AM..
# 2  
Old 03-31-2017
That is sparse a specification! Try
Code:
awk -F";" 'NR > 1 {T[int($1/5)*5 FS $2]+=$3} END {for (t in T) print t FS T[t]}' CONVFMT="%.f" file
201703280000;120;9
201703280005;105;17
201703280005;120;4
201703280000;105;13

These 2 Users Gave Thanks to RudiC For This Post:
RavinderSingh13 wwolfking
# 3  
Old 04-01-2017
Quote:
Originally Posted by RudiC
That is sparse a specification! Try
Code:
awk -F";" 'NR > 1 {T[int($1/5)*5 FS $2]+=$3} END {for (t in T) print t FS T[t]}' CONVFMT="%.f" file
201703280000;120;9
201703280005;105;17
201703280005;120;4
201703280000;105;13




Hi RudiC
Awesome dude, Thanks for the answer!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Awk: time intervals based on epoch time

I have a list of epoch times delimited by "-" as follows: 1335078000 - 1335176700 1335340800 - 1335527400 1335771300 - 1335945600 1336201200 - 1336218000 The corresponding dates are: 20120422 1000 - 20120423 1325 20120425 1100 - 20120427 1450 20120430 1035 - 20120502 1100 ... (3 Replies)
Discussion started by: alex2005
3 Replies

2. Shell Programming and Scripting

Appending information from 2nd file into 1st based on intervals

Hi, I am trying to gather information from the second file and append it to the first file. input HWUSI-EAS000_29:1:100:10000:11479#0/1 + chr5 14458050 ATTGGCTGAGGTCCTACTAGTTGTGATGTGTAAGTGT HHHHHHGDGGEDGGGDGCGEDDEFFFAGE 0 second file:... (14 Replies)
Discussion started by: Diya123
14 Replies

3. UNIX for Advanced & Expert Users

Round up time to 5min

Hi, I have a data like Date Time Pulse 04/02/2012 00:01:05 2 04/02/2012 00:01:48 5 04/02/2012 00:02:09 3 04/02/2012 00:03:19 4 04/02/2012 00:04:40 1 04/02/2012 00:04:45 9 04/02/2012 00:05:15 15 04/02/2012 00:05:48 9 04/02/2012 00:06:49 12 04/02/2012 00:07:00 19 04/02/2012 00:07:45 1... (4 Replies)
Discussion started by: aniketdixit
4 Replies

4. Shell Programming and Scripting

Compare intervals (columns) from two files (awk, grep, Perl?)

Hi dear users, I need to compare numeric columns in two files. These files have the following structure. K.txt (4 columns) A001 chr21 9805831 9846011 A002 chr21 9806202 9846263 A003 chr21 9887188 9988593 A003 chr21 9887188 ... (2 Replies)
Discussion started by: jcvivar
2 Replies

5. Shell Programming and Scripting

Summarize file with column matching

Guys, Please help me with this code. I have 2GB file to process and shell seems to be the best option. I am a biologist and though I can think of the logic, the commands are beyond me. Any help is greatly appreciated. Please look at the attched file and the requirement will be very clear. I... (6 Replies)
Discussion started by: newbie83
6 Replies

6. Shell Programming and Scripting

Using SED/AWK to Summarize Log File in 10min Intervals

I have this huge log file on my linux box that gets generated every day. I'm able to extract the information I need; however I really would like it to be broken down every 10mins. Log File Snippet 01:23:45 MARYHADA Maryhadalittle.lamb(): fleece as white as snow 1394 for and everywhere that... (8 Replies)
Discussion started by: ravzter
8 Replies

7. Shell Programming and Scripting

Shell Script - Copy File at intervals

Hi, I want to copy some files from a Folder say, /usr/X at random intervals to another location. Basically, new files will be dumped at random intervals to location /usr/X and I have to copy those new files to some other location (after copying, I cannot delete those files from source... (2 Replies)
Discussion started by: angshuman_ag
2 Replies

8. Shell Programming and Scripting

Grouping data numbers in a text file into prescribed intervals and count

I have a text file that contains numbers (listed from the smallest to the largest). For ex. 34 817 1145 1645 1759 1761 3368 3529 4311 4681 5187 5193 5199 5417 5682 . . (5 Replies)
Discussion started by: Lucky Ali
5 Replies

9. HP-UX

Location Of SAM 5min Shutdown Warning

Posted this question in another segment by error, but here goes. Lost my notes on the location of the 5min shutdown warning when using SAM. I set my single user system to 0mins so I don't have to keep backspacing out the 5mins and changing it to 0mins. Thanks (Located my old notes by accident,... (4 Replies)
Discussion started by: MINICooperS
4 Replies

10. Windows & DOS: Issues & Discussions

Schedule a Batch file to delete files at particular intervals

Hi, I need to write a batch file/shell script that runs at specified intervals daily and deletes specified set of files. Can anyone pls help me with the code. Thanks, Indom. (6 Replies)
Discussion started by: Indom
6 Replies
Login or Register to Ask a Question