Error while extracting data from log file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Error while extracting data from log file
# 1  
Old 03-04-2015
Error while extracting data from log file

I am running awk command to extract data from log file to calculate last 15 minutes log using below command and now i am getting bellow error:
Code:
awk '$0>=$from' from=$(`date -u +"####<%d-%b-%Y %H:%M:%S o'clock GMT>"-15min`) test.log


Error:
Code:
date: 0551-402 Invalid character in date/time specification. Usage: date [-u] [+"Field Descriptors"] awk: 0602-562 Field $() is not correct.  The input line number is 1. The file is test.log.  The source line number is 1.

Can anyone see what the problem is?

Last edited by Scrutinizer; 03-04-2015 at 06:38 AM..
# 2  
Old 03-04-2015
One problem is in the incorrect time format in the date command. Another problem is the incorrect dollar sign in the awk part (to refer to a variable in awk you do not use a dollar sign).
First try the date command and check if what it produces is what you were looking for.

Last edited by Scrutinizer; 03-04-2015 at 06:48 AM..
# 3  
Old 03-04-2015
yup.. it works fine

Code:
echo `date -u +"####<%d-%b-%Y %H:%M:%S o'clock GMT>`
####<04-Mar-2015 10:56:02 o'clock GMT>


Last edited by Scrutinizer; 03-04-2015 at 07:04 AM.. Reason: CODE tags
# 4  
Old 03-04-2015
Hold on, you left out the -15min part..

If it is OK then try (using your code as an example):
Code:
... from="$(date -u +"####<%d-%b-%Y %H:%M:%S o'clock GMT>"-15min)"

which also leaves out the backticks that are not needed here...



----
Please use CODE tags when you post code or data samples..
# 5  
Old 03-04-2015
Code:
awk: 0602-562 Field $(####<04-Mar-2015 11:10:45 o'clock GMT>-15min) is not correct.
 The input line number is 1. The file is test.log.
 The source line number is 1.

I am getting the pasted error also it is picking the system's date time not the log file given
# 6  
Old 03-04-2015
See the second sentence in post #2.

--
Also, with your date command I get this output using GNU date:
Code:
####<04-Mar-2015 11:16:07 o'clock GMT>-15min

Are you sure that is the output you are looking for?
# 7  
Old 03-04-2015
Many Thanks Scrutinizer.<br>I have a log file which consists of date in this format

Code:
##<01-Mar-2015 03:48:18 o'clock GMT> <info>  ##<01-Mar-2015 03:48:20 o'clock GMT> <info> ##<01-Mar-2015 03:48:30 o'clock GMT> <info> ##<01-Mar-2015 03:48:39 o'clock GMT> <info>


and I have to extract the date based on time interval of 15 minutes.This is what I am trying to accomplish. Since I am new to shell scripting, finding it bit tricky<br>

---------- Post updated at 06:32 AM ---------- Previous update was at 06:29 AM ----------

Quote:
Originally Posted by Scrutinizer
See the second sentence in post #2.

--
Also, with your date command I get this output using GNU date:
Code:
####<04-Mar-2015 11:16:07 o'clock GMT>-15min

Are you sure that is the output you are looking for?
no. I am getting error as pasted above
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extracting data from rsync log

Hi I have a daily rsync that runs and i am trying to find a easy way of extracting the start time and end time of the sync and extract the details of how much data was copied. I would like to use this information to see if i can increase the amount of rsyncs that run in a day. so the log... (3 Replies)
Discussion started by: treds
3 Replies

2. Shell Programming and Scripting

Extracting Delimiter 'TAG' Data From log files

Hi I am trying to extract data from within a log file and output format to a new file for further manipulation can someone provide script to do this? For example I have a file as below and just want to extract all delimited variances of tag 32=* up to the delimiter "|" and output to a new file... (2 Replies)
Discussion started by: Buddyluv
2 Replies

3. Shell Programming and Scripting

Extracting the last 10mins worth of data in a log file

Hi all, Hope someone here will be able to help me. Our system has some scripts that are run from a cron job every ten mins and is used to see how many error there are in that time frame. Problem is that in the scripts grep is used to find the data, but as the day goes on these log file grow to a... (7 Replies)
Discussion started by: Goldengreen
7 Replies

4. Shell Programming and Scripting

Extracting data blocks from file

Hi all, I want to extract blocks of data from a file depending on the contents of that block. The input file(table) has several blocks each starting with 'gene' in the first column. I want to extract only those blocks which do not have the expression '_T02' in the second column. Input file ... (3 Replies)
Discussion started by: newbie83
3 Replies

5. UNIX for Dummies Questions & Answers

Extracting data from file

I am trying to compare the data in lines 3 & 5 to see if they match up to the '-S570' (see first code set, all proprietary information has been removed from code set) spawn telnet Trying ... Connected to CA-LOS1234-ASE-S570.cl . Escape character is '^]'. CA-LOS1234-ASE-S570 Username: ... (1 Reply)
Discussion started by: slipshft
1 Replies

6. Shell Programming and Scripting

Need help in extracting data from xml file

Hello, This is my first post in here, so excuse me if I sound too noob here! I need to extract the path "/apps/mp/installedApps/V61/HRO/hrms_01698_A_qa.ear" from the below xml extract. The path will always appear with the key "binariesURL" <deployedObject... (6 Replies)
Discussion started by: abhishek2386
6 Replies

7. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

8. Shell Programming and Scripting

Extracting data from a log file with date formats

Hello, I have a log file for the year, which contains lines starting with the data in the format of YYYY-MM-DD. I need to get all the lines that contain the DD being 04, how would I do this? I tried using grep "*-*04" but it didn't work. Any quick one liners I should know about? Thank you. (2 Replies)
Discussion started by: cpickering
2 Replies

9. UNIX for Dummies Questions & Answers

Extracting Data from a File

Hi I need to calculate the number of occurrences of a item in a number of files using Perl. The item appears continually throughout the files but in each case I only want to calculate it in certain blocks of the file. Example - Calculalte the number of occurrences of a 'pass' in a block of... (0 Replies)
Discussion started by: oop
0 Replies

10. UNIX for Dummies Questions & Answers

extracting recursive data file

Hi Gurus, Can awk be able to do this source file: 1|SPFE2027G1|1PFE-7000|T34801188|5066-0844| 2|T34801188|5066-0844|T35002355|5066-0845| 3|T35002355|5066-0845|T35203409|QFBR-7798| 1|SPFE2027H1|1PFE-7000|T34801198|5066-0844| 2|T34801198|5066-0844|T35002365|5066-0845| formatted into:... (1 Reply)
Discussion started by: bbeugie
1 Replies
Login or Register to Ask a Question