Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Grepping a timestamp range and assigning it to a constant

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grepping a timestamp range and assigning it to a constant
# 8  
Old 03-24-2017
sounds like you want to place your records in the "bins" 3 hours each based on the start time.
If so, then Corona688 guessed it right.
# 9  
Old 03-24-2017
Quote:
Originally Posted by dsid
.
.
.
hope I am clear
Sorry, no.
You say that "CUT 1" is assigned if the stamp falls into the two hour interval 15:00h - 17:00h, "CUT 2" for 4 hours between 18:00h and 22:00h, and "CUT 3" for the 3 hrs from 0:00h till 3:00h. So there's no chance to develop / apply an algorithm but an "artificial" categorisation is needed.
What if the duration (start to end time) spans two intervals? What if either falls into a gap?

How far did you get with your sed approach?
# 10  
Old 03-24-2017
Quote:
Originally Posted by dsid
@Corona688 i have not used much awk so don't know a lot about it, just here n there. I might give it a look now that you have suggested, but just that sed was something that came into mind as I needed a search and replace.
sed doesn't really have comparisons, variables, expressions, or even numbers... It doesn't have anything you're asking it to do. You'd have to tell sed how to comprehend numbers in the form of many big and ugly regular expressions like /0[0-2]:[0-9][0-9]:[0-9][0-9]/ one per range.

Or you can just use awk, which actually has numbers, variables, expressions, and comparisons, and use these numbers, variables, expressions, and comparisons to solve it in one operation instead of 8 plus further processing.

If the number of lines is intimidating, I bet I can simplify it further given certain assumptions, like column 1 never containing the : character.
Last edited by Corona688; 03-24-2017 at 01:57 PM..
This User Gave Thanks to Corona688 For This Post:
dsid
# 11  
Old 03-24-2017
@vgersh99, yes I think Corona688 has understood what I want

@RudiC: there can never been over lap in the times, as the CUT's are basically a script loading files. Only when the earlier loading finishes ,then the next loading starts. But that is not the point I think. Yes it requires an artificial categorization but that is only because I'm aware the script will finish running within that particular period of time/or my artificial cuts

sample code of what I am trying to do, though its not complete, but just to show you what I meant
Code:
$ cat int_log_file_details |awk -F, '{print $2}'|tr -d ' '|sed 's/^16*/CUT1/'
CUT1:17:10
CUT1:17:10
CUT1:17:11
CUT1:17:14
CUT1:17:18
CUT1:17:22
CUT1:18:30
CUT1:18:38
CUT1:18:53
CUT1:19:07
CUT1:19:11
CUT1:19:47
CUT1:19:54
CUT1:20:02
CUT1:20:04
CUT1:20:06
00:47:31
00:47:37
00:47:43
00:47:43
00:47:48
00:47:49
CUT1:20:08
CUT1:20:10
CUT1:20:12
CUT18:30:54
CUT18:31:56
CUT18:32:08
CUT18:32:17
01:55:32

hope im clear

---------- Post updated at 04:52 PM ---------- Previous update was at 04:50 PM ----------

Quote:
Originally Posted by Corona688
sed doesn't really have comparisons... variables... numbers... In short, it doesn't do anything you're asking for.

It's possible to do it with number ranges and such, but would be a pointlessly ugly hack.
Let me try to do some work with it.
# 12  
Old 03-24-2017
Lots of code here is less than ideal for purposes of brevity and using the tools available, but doing it in sed is just so pointlessly difficult, vs it being so laughably easy in awk, that I really don't understand keeping to the path of maximum resistance. One of these tools is literally meant for the job, having the tools you need to express the problem. The other... isn't. You already use awk for more than '{print $1}' so this is just one step more than what you know.
This User Gave Thanks to Corona688 For This Post:
dsid
# 13  
Old 03-24-2017
Quote:
Originally Posted by Corona688
Lots of code here is less than ideal for purposes of brevity and using the tools available, but doing it in sed is just so pointlessly difficult, vs it being so laughably easy in awk, that I really don't understand keeping to the path of maximum resistance. One of these tools is literally meant for the job, having the tools you need to express the problem. The other... isn't. You already use awk for more than '{print $1}' so this is just one step more than what you know.

going through your awk code now, let me see what I can come up with, but just to let you know I have in the mean time come up with the below somewhat ugly code


Code:
[dsiddiqui@lxserv01 scripts]$ cat int_log_file_details
pvis_POB_80byte_mccy_1303170025574.inc,16:17:10,16:24:48
PVIS_PPL_80byte_1303170010001.inc,     16:17:10,16:17:19
pvis_POU_80byte_mccy_1203172240000.inc,16:17:11,16:17:13
pvis_WRT_80byte_mccy_1303171335182.inc,16:17:14,16:23:13
pvis_WRT_80byte_mccy_1303170035595.inc,16:17:18,16:26:40
pvis_POB_80byte_mccy_1203170725442.inc,16:17:22,16:22:58
pvis_POB_80byte_mccy_1303171325241.inc,16:18:30,16:25:40
pvis_POB_80byte_mccy_1303170725547.inc,16:18:38,16:23:51
pvis_WRT_80byte_mccy_1203171335476.inc,16:18:53,16:25:19
pvis_WLB_80byte_mccy_1203171725359.inc,16:19:07,16:19:10
pvis_POB_80byte_mccy_1203171325525.inc,16:19:11,16:26:43
pvis_DSB_80byte_mccy_1203171735351.inc,16:19:47,16:19:51
pvis_POB_80byte_mccy_1203171926002.inc,16:19:54,16:29:43
pvis_MAC_80byte_mccy_1203171715358.inc,16:20:02,16:20:11
pvis_CAD_80byte_mccy_1303170745408.inc,16:20:04,16:20:19
pvis_CAD_80byte_mccy_1203171945363.inc,16:20:06,16:20:16
pvis_POU_80byte_mccy_1303172240000.inc,00:47:31,00:47:34
pvis_CAD_80byte_mccy_1303171945098.inc,00:47:37,00:48:05
pvis_POB_80byte_mccy_1303171925327.inc,00:47:43,00:57:26
PVIS_PPL_80byte_1403170030002.inc,     00:47:43,00:48:03
pvis_POB_80byte_mccy_1403170025289.inc,00:47:48,00:55:38
PVIS_PPL_80byte_1403170010001.inc,     00:47:49,00:47:57
pvis_CAD_80byte_mccy_1203170745323.inc,16:20:08,16:20:20
pvis_80byte_mccy_1203171705377.inc,    16:20:10,16:21:08
PVIS_PPL_80byte_1303170030002.inc,     16:20:12,16:20:31
pvis_80byte_mccy_1303171705113.inc,    18:30:54,18:32:57
pvis_MAC_80byte_mccy_1303171715074.inc,18:31:56,18:32:09
pvis_WLB_80byte_mccy_1303171725075.inc,18:32:08,18:32:14
pvis_DSB_80byte_mccy_1303171735086.inc,18:32:17,18:32:22
pvis_WRT_80byte_mccy_1403170035301.inc,01:55:32,02:04:02
[dsiddiqui@lxserv01 scripts]$ cat int_log_file_details |awk -F, '{print $2}'|tr -d ' '
16:17:10
16:17:10
16:17:11
16:17:14
16:17:18
16:17:22
16:18:30
16:18:38
16:18:53
16:19:07
16:19:11
16:19:47
16:19:54
16:20:02
16:20:04
16:20:06
00:47:31
00:47:37
00:47:43
00:47:43
00:47:48
00:47:49
16:20:08
16:20:10
16:20:12
18:30:54
18:31:56
18:32:08
18:32:17
01:55:32
[dsiddiqui@lxserv01 scripts]$ cat int_log_file_details |awk -F, '{print $2}'|tr -d ' '|sed 's/^\(16.*\)/CUT1/'|sed 's/^\(18.*\)/CUT2/'|sed 's/^\(00.*\)/CUT3/'|sed 's/^\(01.*\)/CUT3/'
CUT1
CUT1
CUT1
CUT1
CUT1
CUT1
CUT1
CUT1
CUT1
CUT1
CUT1
CUT1
CUT1
CUT1
CUT1
CUT1
CUT3
CUT3
CUT3
CUT3
CUT3
CUT3
CUT1
CUT1
CUT1
CUT2
CUT2
CUT2
CUT2
CUT3
[dsiddiqui@lxserv01 scripts]$

# 14  
Old 03-24-2017
Simpler version which doesn't need any arrays, just substitution and math:

Code:
awk -F"[,:]" -v OFS="," '{
        sub(/^0/, "", $2);      # Remove leading zero
        $2=sprintf("CUT %d", ($2/3)+1); # Set column 2 to "CUT N"
        NF=2                    # Set number of columns to 2
        print                   # print
}'
Login or Register to Ask a Question

Previous Thread | Next Thread

7 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Grep lines between last hour timestamp and current timestamp

So basically I have a log file and each line in this log file starts with a timestamp: MON DD HH:MM:SS SEP 15 07:30:01 I need to grep all the lines between last hour timestamp and current timestamp. Then these lines will be moved to a tmp file from which I will grep for particular strings. ... (1 Reply)
Discussion started by: nms
1 Replies

2. Homework & Coursework Questions

IP Range Assigning

THIS IS A SAMPLE PRACTICAL EXAM QUESTION, COMPLETE FILE HAS BEEN ATTACHED AS WELL. Hi All, I'm a bit confused about assigning IP address from IP Ranges. I am using this scenario below to understand. Scenario Adatum.com an international IT solutions company, is launching 12 new branches in a... (10 Replies)
Discussion started by: TryllZ
10 Replies

3. IP Networking

IP Range Assigning

Hi All, I'm a bit confused about assigning IP address from IP Ranges. I am using this scenario below to understand. Scenario Adatum.com an international IT solutions company, is launching 12 new branches in a new country where they currently have no existing branches. The sWin CIO has asked... (3 Replies)
Discussion started by: TryllZ
3 Replies

4. Shell Programming and Scripting

AIX : Need to convert UNIX Timestamp to normal timestamp

Hello , I am working on AIX. I have to convert Unix timestamp to normal timestamp. Below is the file. The Unix timestamp will always be preceded by EFFECTIVE_TIME as first field as shown and there could be multiple EFFECTIVE_TIME in the file : 3.txt Contents of... (6 Replies)
Discussion started by: rahul2662
6 Replies

5. Shell Programming and Scripting

To check timestamp in logfile and display lines upto 3 hours before current timestamp

Hi Friends, I have the following logfile. Currently time in india is 07/31/2014 12:33:34 and i have the following content in logfile. I want to display only those entries which contain string 'Exception' within last 3 hours. In this case, it would be the last line only I can get the... (12 Replies)
Discussion started by: srkmish
12 Replies

6. Programming

grepping a range of values

I need to return all records in a file starting with a row that says TABLE: <tabl name> lists of hexadecimal records TABLE: <some table> TABLe is a key word in the file. I know the name of the table I want to start with. I do not know the name of the table that I will end with. I just... (4 Replies)
Discussion started by: guessingo
4 Replies

7. UNIX for Dummies Questions & Answers

How to compare a file by its timestamp and store in a different location whenever timestamp changes?

Hi All, I am new to unix programming. I am trying for a requirement and the requirement goes like this..... I have a test folder. Which tracks log files. After certain time, the log file is getting overwritten by another file (randomly as the time interval is not periodic). I need to preserve... (2 Replies)
Discussion started by: mailsara
2 Replies
Login or Register to Ask a Question