Compare and eliminate


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare and eliminate
# 1  
Old 01-13-2011
Compare and eliminate

Could any one help me to compare the date value say at 10th column with sysdate (i.e current date) and if diffrence is more than 50 days then filter them out from the file. The file contain 10000 records.

head file

Code:
 
00971502657744      A671FAHP2EW8BG1369172011HRWS   contact information                  2010-12-30 2010-12-30          
8500411188          S561FAHP2EW8BG1400762011QIJW   contact information                  2010-12-30 2011-01-01          
0505801003          S871FAHP2EW8BG1401572011MSNL   contact information                  2010-12-29 2010-12-30          
821198238127        K981FAHP2EW9BG1171772011FORD   Secret data                          2010-12-31 2010-12-31          
821094025539        K341FAHP2EW9BG1189922011QAQO   Secret data                          2011-01-02 2011-01-02          
0551088849          S341FAHP2EW9BG1401182011NIAS   contact information                  2010-12-30 2011-01-01

# 2  
Old 01-13-2011
which one is column 10?
# 3  
Old 01-13-2011
Last but one is 10th column.

2010-12-30
2010-12-30
2010-12-29
2010-12-31
2011-01-02
2010-12-30
# 4  
Old 01-13-2011
GNU awk
Code:
awk 'BEGIN{
  numdays =  86400 * 5
  now=systime()
}
{
  m=split($(NF-1),a,"-")
  yr=a[1]
  mth=a[2]
  day=a[3]
  t1=mktime(yr" "mth" "day" "00" "00" "00)
  if ( (now - t1) > numdays) {
   print
  }
}' file

# 5  
Old 01-13-2011
If Perl is ok:
Code:
#!/usr/bin/perl

use strict;
use Time::Local;

sub diffT {
   my ($segIni,$minIni,$hourIni,$dayMonthIni,$monthIni,$yearIni)=@_;
   $monthIni-=1;

   my $begin = timelocal($segIni,$minIni,$hourIni,$dayMonthIni,$monthIni,$yearIni);
   my ($seg, $min, $hours, $dayMonth, $month, $year, $dayWeek, $dayYear, $save) = localtime();
   $year+=1900;

   my $final = timelocal($seg,$min,$hours,$dayMonth,$month,$year);
   my $diffSeg = $final - $begin;

   $diffSeg;
   }
 
open(DAT,shift) or die "Unable to open file \n";

while (<DAT>) {
   chomp;
   /.+(\d\d\d\d)-(\d\d)-(\d\d)\s*$/;
   print $_ . "\n" if ( int(&diffT(0,0,0,$3,$2,$1)/(24*60*60)) > 50 );
   
   }
close (DAT);

Usage:
Code:
script.pl infile

# 6  
Old 01-13-2011
Thanks for your update. Could you please explain bit more on this cmd? While executing above cmd am getting ^ syntax error message.
# 7  
Old 01-14-2011
may be this may help

$5 is the column to compared from ur file

Code:
awk -vD=$(date -d '-50 days' '+%Y-%m-%d') '{if($5>D) print $0}' infile

This gives the lines with date from last 50 days
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Eliminate comma on Field

Hi All, Seeking for your assistance on my problem below: What i did was extract the records from database and change the extension name to .csv. My problem is there is a records contains comma between records if i opened it in excel it will generate the comma as new field(please see below... (6 Replies)
Discussion started by: znesotomayor
6 Replies

2. Shell Programming and Scripting

Need help to eliminate the records

Hi All, Please help me how to remove the records from the file if it is having more number of fields than the required one, before loading into stage Here is the sample records. File is space delimited one chandu 1121324 CC ( 2 spaces) chandu balu 434657 DD (3 spaces) -- failing due to... (10 Replies)
Discussion started by: bbc17484
10 Replies

3. Shell Programming and Scripting

To eliminate the created filename

Hi All, I have two files in the directory file1.txt file2.txt I gave ls -m > output.txt command and I got the output as file1.txt, file2.txt, output.txt But my output should be like file1.txt, file2.txt Thanks in advance. (6 Replies)
Discussion started by: dubuku_01
6 Replies

4. UNIX for Dummies Questions & Answers

[Solved] eliminate characters in vi

hey guys. what command in vi editor should I write to make this M83RDK-078A -1 1 1024 1 1024 OK X | M83RDK-055 -1 1 627 1 627 OK X | become like this M83RDK-078A 1 1024 M83RDK-055 1... (5 Replies)
Discussion started by: muhnandap
5 Replies

5. Shell Programming and Scripting

How to eliminate ^L

Hi, I am trying to create a text file from data retrieved from a query.The data retrieved is having this character '^L' at regular intervals of the data. How can i eliminate this, Please find below the sample data. I tried sed -e "s/\^L//g" to convert it, but with no luck ^LCODE*SERIAL... (11 Replies)
Discussion started by: ramkiran77
11 Replies

6. Shell Programming and Scripting

eliminate pathname from the file name in the o/p

Hi, Im finding some files form a specific path and den writing those files to another file as: find /SYS/admin/data/xml -name '*.xml' -type f ! -newer file1 -print >>out.xml and when im doing cat out.xml im getting like dis: ... (2 Replies)
Discussion started by: ss_ss
2 Replies

7. Shell Programming and Scripting

Eliminate unwanted data

Hi, I am stuck on writing a script that reads a file, retains wanted data but discards unwanted data from a CSV file. This is a sample of my table {not the actual data) ID Color tel_num Name color2 color3 abcdef green 5551212 jj88 red blue acbdfe yellow... (5 Replies)
Discussion started by: t524ube
5 Replies

8. Shell Programming and Scripting

How to eliminate inf value in AWK

Hi, I have the calculations which return me infinity (inf), -inf, other very larger number when I printed them out. I did try to insert some control condition not to print these out if the above condition is met. The code I implemented is something like:- for (i=0;i<=1000;i++){ ... (3 Replies)
Discussion started by: ahjiefreak
3 Replies

9. Shell Programming and Scripting

How to eliminate extra char?

I was trying to grep a variable with one space in my file: questionlabel=Q1 more job.tex Q1 pear Q1A applie grep -i "$questionlabel\ " job.tex > tmp But I keep getting : Q1 and Q1A both in my tmp Thanks!!!! (4 Replies)
Discussion started by: whatisthis
4 Replies

10. UNIX for Dummies Questions & Answers

boot messages - how to eliminate them

Anyone would know how to get rid off a message that comes up during a boot-up on Solaris? It says that the clusters for this software are not installed. I deleted the software because it was crashing some other stuff but it did not get rid off all the stuff. Yes, I am coming from windows and... (2 Replies)
Discussion started by: softarch
2 Replies
Login or Register to Ask a Question