Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

shell script to remove all lines which exceeds a particular date & time

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting shell script to remove all lines which exceeds a particular date & time
# 1  
Old 03-08-2011
shell script to remove all lines which exceeds a particular date & time
I have a text file which got 6th coloumn as date and 7th coloumn as time. The text contains data for last one week. I need to remove all the data whose date & time is after 03/08/2011 06:00:00 and save it on another file

TEXT FILE
========
Code:
6 dbclstr-b IXT_Web Memphis_Prod_SQL_Diff Memphis-Prod-SQL-Inc-Application-Backup 03/02/11 18:39:14
6 dbclstr-b Logs Memphis_Prod_SQL_Diff Memphis-Prod-SQL-Inc-Application-Backup 03/02/11 04:10:15
13 pbnawldb001-b TennCare Airpark_Prod_SQL_Full Airpark-Prod-SQL-Full-Application-Backup 03/02/11 02:12:03
150 pbnaaedb004-b Scratch Airpark_Prod_SQL_Diff_1 Airpark-Prod-SQL-Inc-Application-Backup 03/01/11 23:06:57
150 ixtarchive-b IXT_WarehouseStaging Memphis_Prod_SQL_Full Memphis-Prod-SQL-Full-Application-Backup 03/01/11 22:40:33
150 ixtarchive-b IXT_Warehouse Memphis_Prod_SQL_Full Memphis-Prod-SQL-Full-Application-Backup 03/01/11 22:40:32
150 ixtarchive-b IXT Memphis_Prod_SQL_Full Memphis-Prod-SQL-Full-Application-Backup 03/01/11 22:40:34
13 pbnaaedb004-b IFP Airpark_Prod_SQL_Diff_1 Airpark-Prod-SQL-Inc-Application-Backup 03/01/11 20:03:49
13 pbnaaedb004-b IFP Airpark_Prod_SQL_Diff_1 Airpark-Prod-SQL-Inc-Application-Backup 03/01/11 20:03:52
6 cmemcfdb003-b HipNY Memphis_Prod_SQL_Diff Memphis-Prod-SQL-Inc-Application-Backup 03/08/11 01:52:13
6 dbclstr-b IXT_Web Memphis_Prod_SQL_Diff Memphis-Prod-SQL-Inc-Application-Backup 03/08/11 05:37:18
6 ixtarchive-b IXT Memphis_Prod_SQL_Diff Memphis-Prod-SQL-Inc-Application-Backup 03/08/11 06:32:58
13 pbnawldb001-b TennCare Airpark_Prod_SQL_Full Airpark-Prod-SQL-Full-Application-Backup 03/08/11 02:12:03
150 pbnaaedb004-b Scratch Airpark_Prod_SQL_Diff_1 Airpark-Prod-SQL-Inc-Application-Backup 03/08/11 11:06:57


DESIRED OUTPUT (only those lines, that are after 03/08/11 06:00:00)
=============
Code:
6 ixtarchive-b IXT Memphis_Prod_SQL_Diff Memphis-Prod-SQL-Inc-Application-Backup 03/08/11 06:32:58
150 pbnaaedb004-b Scratch Airpark_Prod_SQL_Diff_1 Airpark-Prod-SQL-Inc-Application-Backup 03/08/11 11:06:57

---------- Post updated at 08:14 AM ---------- Previous update was at 08:11 AM ----------

i wrote one script, but its throwing error

SCRIPT
======
Code:
TODAY=`date +"%m/%d/%Y"`
echo $TODAY
sort -k6 -k7  file | while read STATUS CLIENT DB POLICY SCHEDULE DATE TIME
do
echo $DATE
if [ $DATE -ge $TODAY ]
then
echo $STATUS $CLIENT $DB $POLICY $SCHEDULE $DATE $TIME >> SQL_fail.$TODAY
else
echo $STATUS $CLIENT $DB $POLICY $SCHEDULE $DATE $TIME >> SQL_fail.sort
fi
done




ERROR
======
Code:
++ '[' 02/28/11 -ge 03/08/2011 ']'
: line 10: [: 02/28/11: integer expression expected

Last edited by Franklin52; 03-08-2011 at 08:15 AM.. Reason: Please use code tags, thank you
# 2  
Old 03-08-2011
Code:
awk -F"[ :]" '$6~"03/08/11"&&$7>=6' file

# 3  
Old 03-08-2011
no output

Code:
# awk -F"[ :]" '$6~"03/08/11"&&$7>=6' file
#

Moderator's Comments:
Mod Comment
Please use code tags when posting data and code samples!
Last edited by vgersh99; 03-08-2011 at 08:38 AM.. Reason: code tags, please!
# 4  
Old 03-08-2011
For your sample file, I got the output as below.

Code:
 
$ awk -F"[ :]" '$6~"03/08/11"&&$7>=6' sample_file
6 ixtarchive-b IXT Memphis_Prod_SQL_Diff Memphis-Prod-SQL-Inc-Application-Backup 03/08/11 06:32:58
150 pbnaaedb004-b Scratch Airpark_Prod_SQL_Diff_1 Airpark-Prod-SQL-Inc-Application-Backup 03/08/11 11:06:57

# 5  
Old 03-08-2011
yea its working, thanks

---------- Post updated at 10:10 AM ---------- Previous update was at 10:06 AM ----------

another concern.
If i added some more lines on the same file

6 ixtarchive-b IXT Memphis_Prod_SQL_Diff Memphis-Prod-SQL-Inc-Application-Backup 03/09/11 01:52:13
6 ixtarchive-b IXT Memphis_Prod_SQL_Diff Memphis-Prod-SQL-Inc-Application-Backup 03/09/11 06:32:58

I am supposed to get these lines also in the output (because these are the date after 03/08/11 06:00:00). But its not coming
Last edited by ajiwww; 03-08-2011 at 01:29 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Please help, need to create script to remove lines by date in file

Please Help (novice to PERL and SHELL scripting)…. Need to create a script which removes all lines in $filename = "cycle_calendar_ftp_out" older than current date – a variable which will be a number of days passed to script. For Ex it will look at the end date which is the last field (4) and... (2 Replies)
Discussion started by: m3pwr
2 Replies

2. Shell Programming and Scripting

Date / Time difference in shell script

Request ID GMDCOMTM GMDRRSTIME GMDRESTIME <36812986> : : :I want to display the date -time difference in other fields. Above I have given for only 1 record. I want to calculate for all the records. (GMCOMTM - GMDRRSTM) ,(GMDRRSTM-GMDRESTM) and... (5 Replies)
Discussion started by: ghosh_tanmoy
5 Replies

3. Shell Programming and Scripting

Date / Time difference in shell script

================================================================================ Request ID GMDCOM TIME GMDRRS TIME COM-RRS ================================================================================ <36812974> Tue Oct 1 13:32:40 2013 Tue Oct 1 20:36:42 2013... (1 Reply)
Discussion started by: ghosh_tanmoy
1 Replies

4. Shell Programming and Scripting

Date and Time comparison using shell script

Hi, I'm having two fields in the file F1|F2 20111220|102000 F1 ->YYYYMMDD F2 ->HHMMSS Now, I need to compare this with current date & time and need to return the difference value in hours. Already, I checked with datecalc from the forum. So, need hints from Shell Gurus. Thanks (10 Replies)
Discussion started by: buzzusa
10 Replies

5. Shell Programming and Scripting

How to remove a file in shell script if its size exceeds limit?

How can i remove a file using shell script when its size exceeds 10MB. Given that file is located in different location to the shell script where it is running? (4 Replies)
Discussion started by: vel4ever
4 Replies

6. Shell Programming and Scripting

shell script data & time validation

How to validate a date and optionly a time in shell scripting when i get the date and time as pararmeters that sent out with the call of the file? (in my case sh union.sh `first parameter ,second parameter...` (4 Replies)
Discussion started by: tal
4 Replies

7. Shell Programming and Scripting

Shell script: Remove blank lines

Hi gurus, I have this file with blank lines in it. How do i remove them in shell? I tried these commands but not working: sed '/^ *$/d' or sed '/^$/d' Anybody has a better idea pls? Also there are lines which starts with a single space, how do we remove the space in those lines?... (3 Replies)
Discussion started by: gholdbhurg
3 Replies

8. Shell Programming and Scripting

Convert Epoch Time to Standard Date and Time & Vice Versa

Hi guys, I know that this topic has been discuss numerous times, and I have search the net and this forum for it. However, non able to address the problem I faced so far. I am on Solaris Platform and unable to install additional packages like the GNU date and gawk to make use of their... (5 Replies)
Discussion started by: DrivesMeCrazy
5 Replies

9. Shell Programming and Scripting

Remove lines, Sorted with Time based columns using AWK & SORT

Hi having a file as follows MediaErr.log 84 Server1 Policy1 Schedule1 master1 05/08/2008 02:12:16 84 Server1 Policy1 Schedule1 master1 05/08/2008 02:22:47 84 Server1 Policy1 Schedule1 master1 05/08/2008 03:41:26 84 Server1 Policy1 ... (1 Reply)
Discussion started by: karthikn7974
1 Replies

10. Shell Programming and Scripting

How do i collect Date & Time from Different Directories in a script

How do i collect Date & Time from Different Directories in a script The script iam using for a single directory is : ls -l | grep awk '{print $8}' (2 Replies)
Discussion started by: laknar
2 Replies
Login or Register to Ask a Question