Sponsored Content
Top Forums Shell Programming and Scripting Fetching timestamps from the logs. Post 303011488 by srikanth38 on Thursday 18th of January 2018 09:47:32 PM
Old 01-18-2018
Fetching timestamps from the logs.

Dear Experts,

I need some help to get the time stamps from the Unix logs. Basically I am want to analyze which step is taking more time. Please give me some inputs. Thanks a lot for your help.

I have got different codes for each for phase as we can see in the logs
Code:
00002 – UPDT Starting
00008 – UPDT ending
00199 – TPLD Starting
00907 – TPLD Ending


Input:

Code:
01182018 095440 Prod TPLD test_system-00199-Information (Z0000001) Begin FastLoad for table Z0000001 on Prod XXXXX5.
01182018 100009 Prod TPLD test_system-00199-Information (Z0000001) FastLoad for table Z0000001 on Prod XXXXX5 completed.
01182018 100010 Prod TPLD test_system-00907-Information (Z0000001) 17403078 records loaded to Z0000001 on Prod XXXXX5.
01182018 101127 Prod TPLD test_system-00902-Information (Z0000001) Finished Temporary Load Phase (Z0000001)
01182018 101324 Prod UPDT test_system-00901-Information (Z0000001) Started Update Phase (Z0000001)
01182018 101332 Prod UPDT test_system-00002-Information (Z0000001) Started OTHER (01) on table sample_table (Prod XXXXX5)
01182018 101336 Prod UPDT test_system-00008-Information (Z0000001) CIH/other_IH92_1.sql script successfull on sample_table table on
Prod XXXXX5
01182018 101340 Prod UPDT test_system-00002-Information (Z0000001) Started OTHER (02) on table sample_table (Prod XXXXX5)
01182018 102530 Prod UPDT test_system-00008-Information (Z0000001) CIH/other_IH92_2.sql script successfull on sample_table table on
Prod XXXXX5
01182018 102626 Prod UPDT test_system-00002-Information (Z0000001) Started INSERT (03) on table sample_table (Prod XXXXX5)
01182018 102959 Prod UPDT test_system-00008-Information (Z0000001) 17403694 record(s) INSERTed (03) into sample_table table on Prod
XXXXX5
01182018 112646 Prod UPDT test_system-00902-Information (Z0000001) Finished Update Phase (Z0000001)


Desired output:

Code:
FILE_NBR|PHASE|START_TIME|END_TIME|STEP
Z0000001|TPLD|01182018 095440|01182018 100010|NA
Z0000001|UPDT|01182018 101332|01182018 101336|01
Z0000001|UPDT|01182018 101340|01182018 102530|02
Z0000001|UPDT|01182018 102626|01182018 102959|03


Last edited by srikanth38; 01-19-2018 at 11:04 AM.. Reason: Add missing CODE tags.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Unix timestamps

Can someone help me with a Unix or perl script to convert the unix timestamps to human readable format? Any help will be highly appreciated... (3 Replies)
Discussion started by: hamsasal
3 Replies

2. Solaris

Difference between two timestamps

I'm writting a script to find the difference between two timestamp. One field i get on delivery time of the file like 07:17 AM and other is my SLA time 06:30 AM I need to find the difference between these two time (time exceeded to meet SLA). Need some suggestions. (8 Replies)
Discussion started by: raman1605
8 Replies

3. Shell Programming and Scripting

timestamps

Hello! I have the following problem. I read a file using perl, each line of this file has the fllowing format. 14/4/2008 8:42:03 πμ|10800|306973223399|4917622951117|1||1259|1|126|492|433||19774859454$ Th first field is the timestamp and the second field is the offset in seconds. How can... (1 Reply)
Discussion started by: chriss_58
1 Replies

4. Shell Programming and Scripting

Grep yesterday logs from weblogic logs

Hi, I am trying to write a script which would go search and get the info from the logs based on yesterday timestamp and write yesterday logs in new file. The log file format is as follows: """"""""""""""""""""""""""... (3 Replies)
Discussion started by: harish.parker
3 Replies

5. Shell Programming and Scripting

Strip timestamps from logs using sed?

Hullo. I have a log file: (I) 02/03/2009 12:31:01 : Service Started (I) 02/03/2009 12:31:02 : XML: <xml> <stuff> ... </xml> (I) 02/03/2009 12:31:02 : Service Stoped ... ...and I would like to remove date/time/PID-stamps so I can create a baseline for future comparisson. Ideal... (5 Replies)
Discussion started by: cs03dmj
5 Replies

6. Shell Programming and Scripting

Arithmetic on timestamps

Hi Friends, please advise on shell script to add two time stamps for example : a=12:32 b=12:00 c=a+b=00:32 please help me to find shell script to add to two time stamps, as i need to convert time from EST to GMT or SST to prepare status of jobs in unix and to specify estimated time to... (3 Replies)
Discussion started by: balireddy_77
3 Replies

7. Shell Programming and Scripting

Logs access in windows fetching the data from a unix server

How I can get the logs that are getting stored in specific location in unix server through an Apache web server installed in unix server? Requirement is to access the logs through the URL in windows browser without any access. (1 Reply)
Discussion started by: alvida
1 Replies

8. Shell Programming and Scripting

Compare Timestamps

Hi! Long time reader first time registered user and poster. I've picked up some times and tricks and I'm at a dead end... I've parsed a log file for duplicates and printed only the two fields I need (duplicate entry and time stamp). My question is, with this output, how would I script... (2 Replies)
Discussion started by: rexpokinghorn
2 Replies

9. Shell Programming and Scripting

[Solved] Fetching logs of last few days from logfile

Hi All, I have a requirement to fetch logs of last 'N' days. I am trying the following command which is working fine if it finds the date of that day in logfile. START=`TZ="GMT+$((24*N))" date +"%Y %b %d"` this is being used to fetch 'N'th day's date and awk '/'"$START"'/{p=1}... (24 Replies)
Discussion started by: KDMishra
24 Replies

10. Shell Programming and Scripting

If I ran perl script again,old logs should move with today date and new logs should generate.

Appreciate help for the below issue. Im using below code.....I dont want to attach the logs when I ran the perl twice...I just want to take backup with today date and generate new logs...What I need to do for the below scirpt.............. 1)if logs exist it should move the logs with extention... (1 Reply)
Discussion started by: Sanjeev G
1 Replies
ROTATELOGS(8)							    rotatelogs							     ROTATELOGS(8)

NAME
rotatelogs - Piped logging program to rotate Apache logs SYNOPSIS
rotatelogs [ -l ] [ -L linkname ] [ -p program ] [ -f ] [ -v ] [ -e ] [ -c ] [ -n number-of-files ] logfile rotationtime|filesize(B|K|M|G) [ offset ] SUMMARY
rotatelogs is a simple program for use in conjunction with Apache's piped logfile feature. It supports rotation based on a time interval or maximum size of the log. OPTIONS
-l Causes the use of local time rather than GMT as the base for the interval or for strftime(3) formatting with size-based rotation. -L linkname Causes a hard link to be made from the current logfile to the specified link name. This can be used to watch the log continuously across rotations using a command like tail -F linkname. -p program If given, rotatelogs will execute the specified program every time a new log file is opened. The filename of the newly opened file is passed as the first argument to the program. If executing after a rotation, the old log file is passed as the second argument. rotatelogs does not wait for the specified program to terminate before continuing to operate, and will not log any error code returned on termination. The spawned program uses the same stdin, stdout, and stderr as rotatelogs itself, and also inherits the environment. -f Causes the logfile to be opened immediately, as soon as rotatelogs starts, instead of waiting for the first logfile entry to be read (for non-busy sites, there may be a substantial delay between when the server is started and when the first request is handled, meaning that the associated logfile does not "exist" until then, which causes problems from some automated logging tools) -t Causes the logfile to be truncated instead of rotated. This is useful when a log is processed in real time by a command like tail, and there is no need for archived data. No suffix will be added to the filename, however format strings containing '%' characters will be respected. -v Produce verbose output on STDERR. The output contains the result of the configuration parsing, and all file open and close actions. -e Echo logs through to stdout. Useful when logs need to be further processed in real time by a further tool in the chain. -c Create log file for each interval, even if empty. -n number-of-files Use a circular list of filenames without timestamps. With -n 3, the series of log files opened would be "logfile", "logfile.1", "logfile.2", then overwriting "logfile". logfile rotationtime The time between log file rotations in seconds. The rotation occurs at the beginning of this interval. For example, if the rotation time is 3600, the log file will be rotated at the beginning of every hour; if the rotation time is 86400, the log file will be rotated every night at midnight. (If no data is logged during an interval, no file will be created.) filesize(B|K|M|G) The maximum file size in followed by exactly one of the letters B (Bytes), K (KBytes), M (MBytes) or G (GBytes). .PP When time and size are specified, the size must be given after the time. Rotation will occur whenever either time or size limits are reached. offset The number of minutes offset from UTC. If omitted, zero is assumed and UTC is used. For example, to use local time in the zone UTC -5 hours, specify a value of -300 for this argument. In most cases, -l should be used instead of specifying an offset. EXAMPLES
CustomLog "|bin/rotatelogs /var/logs/logfile 86400" common This creates the files /var/logs/logfile.nnnn where nnnn is the system time at which the log nominally starts (this time will always be a multiple of the rotation time, so you can synchronize cron scripts with it). At the end of each rotation time (here after 24 hours) a new log is started. CustomLog "|bin/rotatelogs -l /var/logs/logfile.%Y.%m.%d 86400" common This creates the files /var/logs/logfile.yyyy.mm.dd where yyyy is the year, mm is the month, and dd is the day of the month. Logging will switch to a new file every day at midnight, local time. CustomLog "|bin/rotatelogs /var/logs/logfile 5M" common This configuration will rotate the logfile whenever it reaches a size of 5 megabytes. ErrorLog "|bin/rotatelogs /var/logs/errorlog.%Y-%m-%d-%H_%M_%S 5M" This configuration will rotate the error logfile whenever it reaches a size of 5 megabytes, and the suffix to the logfile name will be cre- ated of the form errorlog.YYYY-mm-dd-HH_MM_SS. CustomLog "|bin/rotatelogs -t /var/logs/logfile 86400" common This creates the file /var/logs/logfile, truncating the file at startup and then truncating the file once per day. It is expected in this scenario that a separate process (such as tail) would process the file in real time. PORTABILITY
The following logfile format string substitutions should be supported by all strftime(3) implementations, see the strftime(3) man page for library-specific extensions. o %A - full weekday name (localized) o %a - 3-character weekday name (localized) o %B - full month name (localized) o %b - 3-character month name (localized) o %c - date and time (localized) o %d - 2-digit day of month o %H - 2-digit hour (24 hour clock) o %I - 2-digit hour (12 hour clock) o %j - 3-digit day of year o %M - 2-digit minute o %m - 2-digit month o %p - am/pm of 12 hour clock (localized) o %S - 2-digit second o %U - 2-digit week of year (Sunday first day of week) o %W - 2-digit week of year (Monday first day of week) o %w - 1-digit weekday (Sunday first day of week) o %X - time (localized) o %x - date (localized) o %Y - 4-digit year o %y - 2-digit year o %Z - time zone name o %% - literal `%' Apache HTTP Server 2013-06-13 ROTATELOGS(8)
All times are GMT -4. The time now is 05:03 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy