Visit Our UNIX and Linux User Community


Get Data Between a specific Date Range from logs


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Get Data Between a specific Date Range from logs
# 1  
Old 02-04-2009
Get Data Between a specific Date Range from logs

I need to extract data from logs for a mentioned date range..Its quite urgent can anyone help me out with it..its to be written in unix..just thought its better to specify..
# 2  
Old 02-04-2009
Can you give more details; layout of your log file and the output you want for your data
# 3  
Old 02-04-2009
i actually have the data extracted from the log filesbut for the current date but unable to get the funda of running the loop for a given date range..
hope this helps..any example of getting data between a date range would do..
# 4  
Old 02-04-2009
here is the log pattern..

Debug eqweb/dbcrm/jsp [Feb 04 05:58:32:209]--><829> <59174E9FF10A03527514777B22CBA902> <TAYRIC> <COMPONENT> Request Parameters: request=page&page=PA100_MyAcc



out here i need to capture the request..
there are two cases here one is where the we get the data imm after the word request (highlighted in red)
for eg we have request=E10 so we capture E10 and for a given user like in this case is TAYRIC we calculate the number of hits..
the second kinda request which also needs to be considered is the above one where the request is actually "PA100_MyAcc"

Hope that explains..lemme know if u need any more details..as i really need this urgently..

Regards,
Sankasu
# 5  
Old 06-11-2009
Quote:
Originally Posted by sankasu
here is the log pattern..

Debug eqweb/dbcrm/jsp [Feb 04 05:58:32:209]--><829> <59174E9FF10A03527514777B22CBA902> <TAYRIC> <COMPONENT> Request Parameters: request=page&page=PA100_MyAcc



out here i need to capture the request..
there are two cases here one is where the we get the data imm after the word request (highlighted in red)
for eg we have request=E10 so we capture E10 and for a given user like in this case is TAYRIC we calculate the number of hits..
the second kinda request which also needs to be considered is the above one where the request is actually "PA100_MyAcc"

Hope that explains..lemme know if u need any more details..as i really need this urgently..

Regards,
Sankasu
may be you may want to try this:
# store everything is hash will make your work a lot easier

# Are you are parsing through or reading line by line?
$line = "Debug eqweb/dbcrm/jsp [Feb 04 05:58:32:209]--><829> <59174E9FF10A03527514777B22CBA902> <TAYRIC> <COMPONENT> Request Parameters: request=page&page=PA100_MyAcc"

@arr = split( /\s+/, $line );
# to count hits
$arr[4] =~ /\<\(S+)\>/;
if( defined($count{$1}) ) {
$count{$1}++;
} else {
$count{$1}=1;
}

# last item will be request=page&page=PA100_MyAcc assuming there's no space for this data item
$arr[$#arr] =~ /request=(\S+)&page=(\S+)/;
$req{$1} = $2; #this stores the request name to it's real name

Don't know whether this helps.

~* Dniz *~

Last edited by dniz; 06-11-2009 at 12:38 AM..

Previous Thread | Next Thread
Test Your Knowledge in Computers #122
Difficulty: Easy
Linux has been certified as UNIX (2019).
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

If I ran perl script again,old logs should move with today date and new logs should generate.

Appreciate help for the below issue. Im using below code.....I dont want to attach the logs when I ran the perl twice...I just want to take backup with today date and generate new logs...What I need to do for the below scirpt.............. 1)if logs exist it should move the logs with extention... (1 Reply)
Discussion started by: Sanjeev G
1 Replies

2. UNIX for Dummies Questions & Answers

Search specific string logfile specific date range

Hi, I have logfile like this.. === 2014-02-09 15:46:59,936 INFO RequestContext - URL: '/eyisp/sc/skins/EY/images/pickers/comboBoxPicker_Over.png', User-Agent: 'Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko': Unsupported with Accept-Encoding header === 2015-02-09... (8 Replies)
Discussion started by: kishk
8 Replies

3. UNIX for Dummies Questions & Answers

Search for a specific String in a log file for a specific date range

Hi, I have log file which rolls out every second which is as this. HttpGenRequest - -<!--OXi dbPublish--> <created="2014-03-24 23:45:37" lastMsgId="" requestTime="0.0333"> <response request="getOutcomeDetails" code="114" message="Request found no matching data" debug="" provider="undefined"/>... (3 Replies)
Discussion started by: karthikprakash
3 Replies

4. Shell Programming and Scripting

Search for logs traced between specific date and time from log file

HI, I want to search for a logs which are trace between specific date and time from logs file. My logs are generated like this :- Tue Jun 18 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process. Thu Jun 20 05:00:02 EEST 2013 | file_check.sh| Message:script has files to... (5 Replies)
Discussion started by: ketanraut
5 Replies

5. Shell Programming and Scripting

Search for a specific data in a file based on a date range

Hi, Currently I am working on a script to automate the process of converting the log file from binary into text format. To achieve this, partly I am depending on my application’s utility for this conversion and the rest I am relying on shell commands to search for directory, locate the file and... (5 Replies)
Discussion started by: svajhala
5 Replies

6. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

7. UNIX for Dummies Questions & Answers

Coping Files for a specific date range

Hi, we have file name appended by date in yymmdd format .. ex: abc090101.dat I need to copy all the files between abc090101 to abc090331.. could you plz help me.. Thanks. (1 Reply)
Discussion started by: kolariya4u
1 Replies

8. Shell Programming and Scripting

Perl code to extract data from the range of date

Hi All, I'm still a newbie in perl programming. I have a data below say in test.tmp The output in test.tmp will be the same data as above sample in test.tmp . So after i get all the 4th column data within the range of month and year i need, then i will use the foreach () code to execute... (1 Reply)
Discussion started by: miskin
1 Replies

9. Shell Programming and Scripting

Read Write byte range/chunk of data from specific location in file

I am new to Unix so will really appreciate if someone can guide me on this. What I want to do is: Step1: Read binary file - pick first 2 bytes, convert from hex to decimal. Read the next 3 bytes as well. 2 bytes will specify the number of bytes 'n' that I want to read and write... (1 Reply)
Discussion started by: Kbenipel
1 Replies

10. Shell Programming and Scripting

Extract data from log file from or after the specific date

Hi , I am having a script which will start a process and appends the process related logs to a log file. The log file writes logs with every line starting with date in the format of: date +"%Y %b %d %H:%M:%S". So, in the script, before I start the process, I am storing the date as DATE=`date +"%Y... (5 Replies)
Discussion started by: chiru_h
5 Replies

Featured Tech Videos