Visit Our UNIX and Linux User Community


Extract data from log file from or after the specific date


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract data from log file from or after the specific date
# 1  
Old 04-20-2009
Extract data from log file from or after the specific date

Hi , I am having a script which will start a process and appends the process related logs to a log file. The log file writes logs with every line starting with date in the format of: date +"%Y %b %d %H:%M:%S".
So, in the script, before I start the process, I am storing the date as DATE=`date +"%Y %b %d %H:%M:%S"`
How do I only extract the logs from that log file only from that specific date till the end of the log file. If I can do that, I can grep for errors or exceptions and incorporate in the same script.

Note: the date I stored need not be in the log file, but I have to get the logs after that date from the file.

Thanks,
Chiru
# 2  
Old 04-20-2009
Try this:

Code:
awk -v var=$YourDate '!p && $0 ~ var{p=1}p' file

Regards
# 3  
Old 04-20-2009
I didn't read the note, if you have the date only at the start point:

Code:
awk -v var=$YourDate '$0 ~ var{p=1;next}p' file

# 4  
Old 04-20-2009
Thanks Franklin, it doesn't work if the date is not present in the log file, but if the date is in the log file, it works like a charm.

Any insight?

Thanks,
Chiru
# 5  
Old 04-21-2009
How do you want to recognize the start point without a date in the log file?

Regards
# 6  
Old 04-21-2009
hey Franklin, I'll have the date as a parameter to grep the log file. I am looking to get the logs after that date from the file even if that date is not available in the file.

Thanks..

Previous Thread | Next Thread
Test Your Knowledge in Computers #191
Difficulty: Easy
In Python, the tuple data type is immutable and cannot contain mixed types.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How-To Extract specific data from a file.

data.txt has several information like the below.. <SERVER>:WEB:MYDOM01:/tmp/cong/MYDOM01,/tmp/app/MYDOM01 <WEBER>:CANES:https-web01,https-web02:/web/apps/https-web01/config <SERVER>:WEB:MYDOM07:/tmp/cong/MYDOM07,/tmp/app/MYDOM07... (7 Replies)
Discussion started by: mohtashims
7 Replies

2. UNIX for Dummies Questions & Answers

Search for a specific String in a log file for a specific date range

Hi, I have log file which rolls out every second which is as this. HttpGenRequest - -<!--OXi dbPublish--> <created="2014-03-24 23:45:37" lastMsgId="" requestTime="0.0333"> <response request="getOutcomeDetails" code="114" message="Request found no matching data" debug="" provider="undefined"/>... (3 Replies)
Discussion started by: karthikprakash
3 Replies

3. UNIX for Dummies Questions & Answers

Extract date ranged data from log file

Hi, I am trying to extract lines of data within a log file on a Redhat 5 Linux system. eg I need all the lines with a particular username over the last 3 minutes. the log file may read like this, and I want a way to search all the lines extracting all the relevant lines over the last 3... (2 Replies)
Discussion started by: mantis100
2 Replies

4. Shell Programming and Scripting

Extract Specific pattern - log file

Hello everyone, I am on AIX (6.1). I can only use shell (ksh) script. I can't do this on my own, so will do my best to explain my needs.I also do not know what is the best idea to make it work, so here is what I am thinking, but I may wrong. I need help to extract info on... (3 Replies)
Discussion started by: Aswex
3 Replies

5. Shell Programming and Scripting

Search for a specific data in a file based on a date range

Hi, Currently I am working on a script to automate the process of converting the log file from binary into text format. To achieve this, partly I am depending on my application’s utility for this conversion and the rest I am relying on shell commands to search for directory, locate the file and... (5 Replies)
Discussion started by: svajhala
5 Replies

6. Shell Programming and Scripting

How to extract log data based on current date and month ?

Hi Gurus, I'm using HP-UX B.11.23 operating system. I've been trying to extract this log info based on the current date and month, but was having some issues as the date column which on the 4th column has a comma and the 5th column has a dot tied to it. Here is the output from my shut... (5 Replies)
Discussion started by: superHonda123
5 Replies

7. Shell Programming and Scripting

How to extract log data based on date

Hi Gurus, I've been having some problem in extracting the log data based on the current date and month. As shown in the sample data below, how to extract the log info for Aug 11? Sample data: root pts/ta userpc Wed Aug 11 09:46 - 20:21 (10:35) root pts/ta userpc... (13 Replies)
Discussion started by: superHonda123
13 Replies

8. Shell Programming and Scripting

How to extract specific data and count number containing sets from a file?

Hello everybody! I am quit new here and hope you can help me. Using an awk script I am trying to extract data from several files. The structure of the input files is as follows: TimeStep parameter1 parameter2 parameter3 parameter4 e.g. 1 X Y Z L 1 D H Z I 1 H Y E W 2 D H G F 2 R... (2 Replies)
Discussion started by: Daniel8472
2 Replies

9. Shell Programming and Scripting

Extract data into file with specific field specs

:confused: I have a tab delimited file that I need to extract data from and into a file with specific field specs. Each field has to be a certain amount of characters. So, the name field (from delimited file) might have only 15 characters but needs to be 25 (in new file) so I need to insert spaces... (5 Replies)
Discussion started by: criddel
5 Replies

10. Shell Programming and Scripting

extract specific data from xml format file.

Hi, I need to extract the start time value (bold, red font) under the '<LogEvent ID="Timer Start">' tag (black bold) from a file with the following pattern. There are other LogEventIDs listed in the file as well, making it harder for me to extract out the specific start time that I need. . .... (7 Replies)
Discussion started by: 60doses
7 Replies

Featured Tech Videos