grep a log file to filter previous dates


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting grep a log file to filter previous dates
# 1  
Old 07-31-2007
Question grep a log file to filter previous dates

Hi,

I have problem of filtering a log file from my perl script.

#cat /data/pinpe.csv_20070731 | nawk -v FS=, '{print $1','$18','$22','$26}' | grep -w 100 | grep -w 1 | nawk '{print $4}'

Below is the output:

2009-06-16
2009-01-29
2009-06-02
2008-03-05
2007-08-05
2007-09-24
2007-07-10
2007-10-27
2007-10-14
2008-08-02
2007-11-10
2007-10-16
2008-09-15
2008-01-30
2008-12-08
2007-09-26
2007-11-29
2008-04-01
2007-11-21
2007-12-18
2007-10-04
2007-06-29
2008-01-13
2007-10-28
2007-10-20
2007-09-13
2008-11-05
2008-08-05
2008-04-10
2007-09-10
2008-10-12
2007-11-29
2008-04-02
2009-03-05
2009-07-18
2007-11-16
2007-10-13

Now my dilemma is I want to filter this ouput by my preferred dates. I wanted it to output the date from yesterday and backwards. In other words I want to filter all the dates less than today's date (i.e. 2007-07-30, 2007-07-29, 2007-07-28, etc. and below). And finally count the number of lines for the final output.

Tnx in advance.

Br, Pete
# 2  
Old 07-31-2007
change your last nawk to be
nawk '{ if ( $4 < DATE ) {print $4}}' DATE=$(date '+%Y-%m-%d') -

or you could use a -v parameter to feed in the date

nawk -v DATE=$(date '+%Y-%m-%d') '{ <same code> }'
# 3  
Old 08-01-2007
Code:
#!/bin/ksh
typeset -i mCnt=0
mYYYYMMDD=`date +"%Y-%m-%d"`
echo 'mYYYYMMDD = '$mYYYYMMDD
while read mLine
do
  if [[ "${mLine}" < "${mYYYYMMDD}" ]]; then
    echo 'Found = '${mLine}
    mCnt=${mCnt}+1
  fi
done < input_file
echo 'Total lines = '${mCnt}

# 4  
Old 08-01-2007
Hi perl Gurus,

Here is the code in tags. I'd made it simple code for not to look more confusing.


Code:
use Net::Telnet;

@DATE=$telnet->cmd("date '+%Y-%m-%d'");
chomp($DATE[0]);

@Discon=$telnet->cmd("cat /data/pinpe.csv_20070731 | nawk '{print \$4}' | perl -ne 'print if \$_ lt @DATE[0]' | wc -l");
chomp($Discon[0]); 

print "@Discon[0]\n";

and this is the output error:
Code:
Illegal octal digit '8' at -e line 1, at end of line

Was it perl thought that I was dealing with an octal number and when he came along with 8 or 9 (becasue of 2007-08-01) which stopped it from making sense, so perl quite rightly complained? Pls help. Tnx.

Br,
Pete

Last edited by pinpe; 08-02-2007 at 08:37 AM..
# 5  
Old 08-02-2007
Hi perl Gurus,

I just wanna make this up. Pls nedd some hand. Tnx in advance.

Br, Pete
# 6  
Old 08-03-2007
Hi,

Someone solved my problem. A "Monk" from PerlMonks - The Monastery Gates. Smilie

I just want to share it with you guys. Below is the right syntax for my code.

Code:
use Net::Telnet;
use IPadd;

$ipadd=IPadd->new();
$ipadd->ipadd1();

@Date1=$telnet->cmd("date '+%Y%m%d'");
chomp($Date1[0]);
@Date2=$telnet->cmd("date '+%Y-%m-%d'");
chomp($Date2[0]);

@Discon=$telnet->cmd("cat /data/pinpe.csv_@Date1[0] | nawk -v FS=, '{print \$1','\$18','\$22','\$26}' | grep -w $array1[$count-1] | grep -w 1 | nawk '{print \$4}' | perl -ne 'print if \$_ lt \"@Date2[0]\"' | wc -l");

The key to my problem is 'interpolation'. The code that the Perl interpreter sees is:

Code:
print 2007-08-05;

which is some arithmetic (2007 - 08 - 05) containing two octal constants and a decimal constant. The proper syntax is I need to quote the interpolated date so that the interpreter sees a string instead:

Code:
\"@Date2[0]\"

Cheers! Smilie

Br, Pete
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. What is on Your Mind?

Grep file containing dates

How to grep a file containing dates to only last 30 days then move to another folder (7 Replies)
Discussion started by: kmarcus
7 Replies

2. Shell Programming and Scripting

Display previous days dates in ksh

---------- Post updated at 03:42 AM ---------- Previous update was at 03:38 AM ---------- Sorry for a duplicate post, my post at the first place could not appear due to some net issue on my machine. Here is what i posted earlier: Hi, i am using ksh in Solaris, i wanted to assign today's,... (5 Replies)
Discussion started by: pr5439
5 Replies

3. Linux

Get all the files from a FTP location with previous week's dates in the file names using Linux

I have a weird requirement where I have to get the files from a FTP(Lets say FTP1) location and place it on my current FTP(Lets say FTP2) location. The issue is, these are daily files (in a pattern Sales_YYYYMMDD_report.csv) and are placed every day on FTP1 and my process usually runs on Monday(eg.... (2 Replies)
Discussion started by: dhruuv369
2 Replies

4. Shell Programming and Scripting

grep lines from the previous day's file

I have files called printfile.log.1121, printfile.log.1122, etc where 1121 and 1122 are the date of the file creation. The file contents are like: ================================================================================ User = d_prod Env = d_prod Dir =... (3 Replies)
Discussion started by: Daniel Gate
3 Replies

5. Shell Programming and Scripting

To filter a log file

Hi All, I have a log File and i have to make a report in desired format.can anybody help me......... log file ------ <<<<< BESI14 >>>>> <RLGAP:CELL=ALL; CELL CHANNEL GROUP ALLOCATION DATA CELL CHGR SAS ODPDCHLIMIT BUNYM18 0 MULTI 100 -------- desired format... (1 Reply)
Discussion started by: dattatraya
1 Replies

6. Shell Programming and Scripting

To get previous or future dates based on input value

Hi, I need something like, if the input date is 24/Aug/2008 and the inputvalue is +8 then the result should be 1/Sep/2008 (8 days after the input date) if the input date is 24/Aug/2008 and the inputvalue is -8 then the result should be 16/Aug/2008 (8 days before the input date) is there any... (5 Replies)
Discussion started by: Sharmila_P
5 Replies

7. UNIX for Dummies Questions & Answers

using 'date' to get previous days' dates

I am familiar with using the 'date' command to get the current date but I have a situation where I need to get the previous day's date as well as the date two days prior. Theoretically I could use 'expr' to compute these values but I need it to work in instances where the previous month's dates... (2 Replies)
Discussion started by: slant-40
2 Replies

8. Shell Programming and Scripting

grep a log file between 2 dates

Hi Currently I can grep a log file with the following command: $results = `grep -A 2 '^$date.$time.*' $log`; and the following arguments: $date = 2007/04/25 $time = 16:07 Log example: 2007/04/25 16:07:12.145701 2007/05/25 14:07:12.145701 2007/05/25 17:07:12.145701 2007/06/25... (37 Replies)
Discussion started by: Epiphone
37 Replies

9. UNIX for Dummies Questions & Answers

Dates of previous years

Is there any way to use date with previous dates such as "2 23 2000" in order to see what day of the week it was? I tried changing the current date to "date 022300452000" but then it told me that I could not do this because I was "Not the owner". Any other ways of getting the day result? (2 Replies)
Discussion started by: terms5
2 Replies

10. Shell Programming and Scripting

help searching log file with dates

Im tyring to create a script that will show me any lines in a file with todays date and yesterdays, the date format in the file is as follows ----- amqxfdcx.c : 728 -------------------------------------------------------- 07/12/05 09:53:20 AMQ6109: An internal WebSphere MQ error has... (3 Replies)
Discussion started by: csaunders
3 Replies
Login or Register to Ask a Question