Delete lines in a file by date on the line


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Delete lines in a file by date on the line
# 1  
Old 01-18-2011
Delete lines in a file by date on the line

Hi

I have a file with lines ending with a date in format dd/mm/yyyy see example below:
Code:
a|b|c|08/01/2011
d|a|e|31/11/2010
e|d|f|20/11/2010
f|s|r|18/01/2011

What I would like to do is delete all lines with a date older than 30 days.

With above example I should be left with a file like below
Code:
a|b|c|08/01/2011
f|s|r|18/01/2011

Smilie

Your help would be appreciated.

Thanks in advance.

Last edited by Scott; 01-18-2011 at 02:00 PM.. Reason: Code tags
# 2  
Old 01-18-2011
Code:
awk -F"|" -vD=$(date -d '-30 days' '+%d/%m/%Y') '{if($NF<D) print $0}'  infile


Last edited by posner; 01-18-2011 at 01:58 PM..
This User Gave Thanks to posner For This Post:
# 3  
Old 01-18-2011
Hi Posner thanks for trying but I get the following errors
Code:
date: illegal option -- d
date: illegal option -- 3
date: illegal option -- 0
date: illegal option -- 
date: illegal option -- d
date: invalid arguement -- ys
usage: date [-u] mmddHHMM[ [cc]yy] [.SS]
          date [-u] [+format]
          date -a [-]sss[.fff]

I dont know if this help but I am on an aix box using korn shell.

Thanks.

---------- Post updated at 08:14 PM ---------- Previous update was at 07:58 PM ----------

Anyone out there please help.

Last edited by Scott; 01-18-2011 at 04:47 PM.. Reason: Code tags
# 4  
Old 01-19-2011
If your date command don't support -d option, then use perl to calculate the passed date.
Code:
$ cat days_ago
days_ago()
{
  perl -e  '
        # take 86400 * # of days from right now in epoch seconds
                 $yestertime = time - (86400 * $ARGV[0]);
                 $month = (localtime $yestertime)[4] + 1;
        # day of the month
                 $day = (localtime $yestertime)[3];
        # year
                 $year = (localtime $yestertime)[5] + 1900;
        # US format mm dd yyyy
                 printf( "%4d%02d%02d", $year, $month, $day);  '  $1
}

days_ago $1

Code:
awk -F "[|\/]" -vD=$(./days_ago 30) '$NF$(NF-1)$(NF-2)>D'  infile

This User Gave Thanks to rdcwayx For This Post:
# 5  
Old 01-21-2011
thanks rdcwayx I was getting an error every time I tried your script.
Code:
awk: syntax error near line 1
awk: bailing out near line 1

Anyway since then I have found something else that helps. Not to sure about rules on this site but I got this from adifferent forum so thanks to p5wizard who wrote it. Hope its ok for me to detail it below.
Code:
#################################
#!/bin/ksh
# parameters
# days in year
yd=365
# days in leapyear
ld=366
# elapsed days before month x in non-leap year
mo[1]=0   mo[2]=31  mo[3]=59  mo[4]=90   mo[5]=120  mo[6]=151
mo[7]=181 mo[8]=212 mo[9]=243 mo[10]=273 mo[11]=304 mo[12]=334
# elapsed days before month x in leap year
ml[1]=0   ml[2]=31  ml[3]=60  ml[4]=91   ml[5]=121  ml[6]=152
ml[7]=182 ml[8]=213 ml[9]=244 ml[10]=274 ml[11]=305 ml[12]=335

# variables
date1=$1
date2=$2
#date1=01/01/2012
#date2=29/02/2012

leapyear()
{
# is year $1 a leapyear?
yr=$1
ly=0
if (( !(yr%4) ))
then
 if (( !(yr%100) ))
 then
  if (( !(yr%400) ))
  then
   ly=1
  fi
 else
  ly=1
 fi
fi
echo ${ly}
}
julian()
{
# convert month, day to julian: day of the year
l=$1
m=$2
d=$3
if (( l==0 ))
then
 ((ju=mo[m]+d))
else
 ((ju=ml[m]+d))
fi
echo ${ju}
}

echo $date1|tr '/' ' '|read d1 m1 y1
l1=$(leapyear $y1)
j1=$(julian $l1 $m1 $d1)
echo $date2|tr '/' ' '|read d2 m2 y2
l2=$(leapyear $y2)
j2=$(julian $l2 $m2 $d2)
((diff=0))
if ((y1==y2))
then
 ((diff=j2-j1))
else
 if ((y1<y2))
 then
  factor=1
 else
  # switch dates, negative diff
  echo ${y1} ${m1} ${d1} ${l1} ${j1} ${y2} ${m2} ${d2} ${l2} ${j2}|read y2 m2 d2 l2 j2 y1 m1 d1 l1 j1
  factor=-1
 fi
 # days until end of year of begin date
 if ((l1==1))
 then
  ((diff=ld-j1))
 else
  ((diff=yd-j1))
 fi
 ((y1+=1))
 # keep adding yd or ld for every year or leapyear passed
 while ((y1<y2))
 do
  l1=$(leapyear $y1)
  if ((l1==1))
  then
   ((diff+=ld))
  else
   ((diff+=yd))
  fi
  ((y1+=1))
 done
 # add julian date of end date
 ((diff+=j2))
 # positive or negative difference?
 ((diff*=factor))
fi
 
#echo "difference between ${date1} and ${date2} is ${diff} days"
diffdays=${diff}
#################################

So what I am doing to resolve my issue is to get the dates from my file and do a loop using the above as a function and only echoing out the number of days after which doing a simple "if statement" to see if the number is greater than 30 if so ignore line if it isnt the print line to a new file.

Moderator's Comments:
Mod Comment Please use code tags

Last edited by Scott; 01-21-2011 at 08:42 AM.. Reason: Code tags
# 6  
Old 01-21-2011
Quote:
Originally Posted by fas1
thanks rdcwayx I was getting an error every time I tried your script.
Code:
awk: syntax error near line 1
awk: bailing out near line 1

For Solaris, replace awk by /usr/bin/nawk or /usr/xpg4/bin/awk
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Log file - Delete duplicate line & keep last date

Hello All ! I need your help on this case, I have a csv file with this: ITEM105;ARI FSR;2016-02-01 08:02;243 ITEM101;ARI FSR;2016-02-01 06:02;240 ITEM032;RNO TLE;2016-02-01 11:03;320 ITEM032;RNO TLE;2016-02-02 05:43;320 ITEM032;RNO TLE;2016-02-01 02:03;320 ITEM032;RNO... (2 Replies)
Discussion started by: vadim-bzh
2 Replies

2. UNIX for Advanced & Expert Users

How to find a string in a line in UNIX file and delete that line and previous 3 lines ?

Hi , i have a file with data as below.This is same file. But actual file contains to many rows. i want to search for a string "Field 039 00" and delete that line and previous 3 lines in that file.. Can some body suggested me how can i do using either sed or awk command ? Field 004... (7 Replies)
Discussion started by: vadlamudy
7 Replies

3. UNIX for Dummies Questions & Answers

How to delete specific lines (2n+3 line, n=0,1,2...296) in a file?

Dear everyone, I have a file with 900 lines (there is only numbers in one line, no string), I only need the lines 2+3n (n=0,1...296), i.e line 2, 5, 8, 11...888. I tried google but only the results such as how to delete all the odd lines or all the even lines with 'awk' command. Thanks in... (4 Replies)
Discussion started by: phamnu
4 Replies

4. Shell Programming and Scripting

Grep a string from input file and delete next three lines including the line contains string in xml

Hi, 1_strings file contains $ cat 1_strings /home/$USER/Src /home/Valid /home/Review$ cat myxml <projected value="some string" path="/home/$USER/Src"> <input 1/> <estimate value/> <somestring/> </projected> <few more lines > <projected value="some string" path="/home/$USER/check">... (4 Replies)
Discussion started by: greet_sed
4 Replies

5. Shell Programming and Scripting

How to delete several lines from file by line number?

Hi I am using the following command to delete a line from the file by line number: line_number=14 sed "${line_number}d" inputfilename > newfilename Is there a way to modify this command to specify the range of lines to be deleted, lets say from line 14 till line 5 ? I tried using the... (5 Replies)
Discussion started by: aoussenko
5 Replies

6. UNIX for Dummies Questions & Answers

Delete a row from a file if one column containing a date is greater than the current system date

Hello gurus, I am hoping someone can help me with the required code/script to make this work. I have the following file with records starting at line 4: NETW~US60~000000000013220694~002~~IT~USD~2.24~20110201~99991231~01~01~20101104~... (4 Replies)
Discussion started by: chumsky
4 Replies

7. Shell Programming and Scripting

delete lines ! like date format

Vars $b=03, $a=Feb,$c=2011 & d=$b/$a/$c 2].task=AVPNetwork 200 4].workCategory=Req%20Loc 200 - 5].workCategory=Req%20Loc 30/Jan/2011 -- ... 31/Jan/2011 -- ... 01/Feb/2011 -- ... 02/Feb/2011 -- ... 03/Feb/2011 -- ... I'm trying to remove the garbage 2 lines at... (2 Replies)
Discussion started by: dba_frog
2 Replies

8. Shell Programming and Scripting

Delete lines line by match data 2 file.

i need to delete the lines is match from file data 1 & data 2 please help? data 1 4825307 4825308 4825248 4825309 4825310 4825311 4825336 data 2 4825248 0100362210 Time4Meal 39.00 41.73 MO & MT MT SMS 4825305 0100367565... (2 Replies)
Discussion started by: ooilinlove
2 Replies

9. UNIX for Dummies Questions & Answers

Delete lines with duplicate strings based on date

Hey all, a relative bash/script newbie trying solve a problem. I've got a text file with lots of lines that I've been able to clean up and format with awk/sed/cut, but now I'd like to remove the lines with duplicate usernames based on time stamp. Here's what the data looks like 2007-11-03... (3 Replies)
Discussion started by: mattv
3 Replies

10. Shell Programming and Scripting

Delete lines prior to a specific date in a log file.

Hi all. I have a database log file in which log data get appended to it daily. I want to do a automatic maintainence of this log by going through the log and deleting lines belonging to a certain date. How should i do it? Please help. Thanks. Example. To delete all lines prior to Jun... (4 Replies)
Discussion started by: ahSher
4 Replies
Login or Register to Ask a Question