Filering records based on date listed in the file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Filering records based on date listed in the file
# 1  
Old 04-15-2009
Filering records based on date listed in the file

Hi,

I have to filter out the records which are more than 6 month old and save the records to a new file which are less than 6 months old. The file format is text1:date(mm/dd/yyyy):text2. For example:

abcd:12/01/2009:sdfsdf
qwwqwra:11/03/2008:efrdas

How can I achieve it?
# 2  
Old 04-15-2009
i don't understand "save the records to a new file which are less than 6 months old". how can a new file be less than 6 months old
# 3  
Old 04-15-2009
I wrote that incorrectly. Please read that line as:

save the records (which are less than 6 months old) to a new file.

I hope that clarifies your doubt.
# 4  
Old 04-15-2009
For example:

If a file FILE1 has following data:

abcd:02/01/2009:sdfsdf
qwwqwra:11/03/2008:efrdas
adfsd:05/01/2008:edfrweio
qwee:01/03/2009:erweiuer
asdfsda:04/14/2009:xg8xzcv

Then I need the new file FILE2 to have:

abcd:02/01/2009:sdfsdf
qwee:01/03/2009:erweiuer
asdfsda:04/14/2009:xg8xzcv

Two records are removed as they have a date which is less than [Current date (2009-04-15) - 6 months].
# 5  
Old 04-15-2009
if you know Perl and you can use it as well as download modules, a recent thread here similar to what you have. just tweak it a little.
# 6  
Old 04-15-2009
Bug Try this Code

Quote:
Originally Posted by abb
For example:

If a file FILE1 has following data:

abcd:02/01/2009:sdfsdf
qwwqwra:11/03/2008:efrdas
adfsd:05/01/2008:edfrweio
qwee:01/03/2009:erweiuer
asdfsda:04/14/2009:xg8xzcv

Then I need the new file FILE2 to have:

abcd:02/01/2009:sdfsdf
qwee:01/03/2009:erweiuer
asdfsda:04/14/2009:xg8xzcv

Two records are removed as they have a date which is less than [Current date (2009-04-15) - 6 months].
Code:
#!/bin/bash
SysMth=`date +%m`
prior6month=`echo -e "07\n08\n09\n10\n11\n12\n01\n02\n03\n04\n05\n06\n07\n08\n09\n10\n11\n12\n" | head -$SysMth | tail -6 | head -1`
totrec=`wc -l records.txt | awk '{print $1;exit}'`
i=1
rm old_records.txt
rm records_new.txt
cp records.txt records_copy.txt
touch old_records.txt
while [ $i -le $totrec ];do
curyear=`date +%Y`
rec=`head -n$i records_copy.txt | tail -1`
recdate=`head -n$i records_copy.txt | tail -1 |  awk '{print $2;exit}'`
recyear=`echo $recdate | awk '{print substr($0,7,4);exit}'`
recmonth=`echo $recdate | awk '{print substr($0,1,2);exit}'`
recday=`echo $recdate | awk '{print substr($0,4,2);exit}'`
if [ $SysMth -gt 06 ]; then
 if [ $prior6month -lt $recmonth ]; then
 echo $rec >> old_records.txt
 echo "Record older than 6 months"
 else
 echo "Record not older than 6 months"
 fi
else 
 if [ $prior6month -gt $recmonth ]; then
  if [ $recyear -lt $curyear ]; then
   echo `head -n$i records_copy.txt | tail -1` >> old_records.txt
   echo "Record older than 6 months"
  else   
   echo "Record not older than 6 months"
  fi 
 else
 echo `head -n$i records_copy.txt | tail -1` >> old_records.txt
 echo "Record older than 6 months"
 fi
fi
i=$(($i+1)) 
done
comm -3 records_copy.txt old_records.txt > records_new.txt
cat old_records.txt
echo
echo
cat records_new.txt

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replace variable value in first file based on records in second

Hello , I have below files a) File A <?xml version="1.0" encoding="UTF-8" standalone="no"?> <root xmlns="http://aaa/1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema" version="2.0"> <project name="source"> <mapping name="m_Source"> <parameter... (3 Replies)
Discussion started by: Pratik4891
3 Replies

2. Shell Programming and Scripting

Script to determine Date,TotalFile,total size of file based on date

I have file listed like below -rw-r--r--+ 1 test test 17M Nov 26 14:43 test1.gz -rw-r--r--+ 1 test test 0 Nov 26 14:44 test2.gz -rw-r--r--+ 1 test test 0 Nov 27 10:41 test3.gz -rw-r--r--+ 1 test test 244K Nov 27 10:41 test4.gz -rw-r--r--+ 1 test test 17M Nov 27 10:41 test5.gz I... (5 Replies)
Discussion started by: krish2014
5 Replies

3. Shell Programming and Scripting

Filter records based on 2nd file

Hello, I want to filter records of a file if they fall in range associated with a second file. First the chr number (2nd col of 1st file and 1st col of 2nd file) needs to be matched. Then if the 3rd col of the first file falls within any of the ranges specified by the 2nd and 3rd cols , then... (4 Replies)
Discussion started by: ritakadm
4 Replies

4. Shell Programming and Scripting

Split file based on records

I have to split a file based on number of lines and the below command works fine: split -l 2 Inputfile -d OutputfileMy input file contains header, detail and trailor info as below: H D D D D TMy split files for the above command contains: First File: H DSecond File: ... (11 Replies)
Discussion started by: Ajay Venkatesan
11 Replies

5. UNIX for Dummies Questions & Answers

Delete records from a big file based on some condition

Hi, To load a big file in a table,I have a make sure that all rows in the file has same number of the columns . So in my file if I am getting any rows which have columns not equal to 6 , I need to delete it . Delimiter is space and columns are optionally enclosed by "". This can be ... (1 Reply)
Discussion started by: hemantraijain
1 Replies

6. UNIX for Dummies Questions & Answers

Filtering records from 1 file based on some manipulation doen on second file

Hi, I am looking for an awk script which should help me to meet the following requirement: File1 has records in following format INF: FAILEd RECORD AB1234 INF: FAILEd RECORD PQ1145 INF: FAILEd RECORD AB3215 INF: FAILEd RECORD AB6114 ............................ (2 Replies)
Discussion started by: mintu41
2 Replies

7. Shell Programming and Scripting

parallel while loop based on the file records

Hi, I need to execute parallel with while loop. Input File(source_file.csv) contains filenames the below source_file.csv file contains Customer1.txt Product1.txt Sales.txt Emp.txt Dept.txt Based on the number of rows that file I want to run the script ‘n' times. while... (2 Replies)
Discussion started by: onesuri
2 Replies

8. Shell Programming and Scripting

Extract file records based on some field conditions

Hello Friends, I have a file(InputFile.csv) with the following columns(the columns are pipe-delimited): ColA|ColB|ColC|ColD|ColE|ColF Now for this file, I have to get those records which fulfil the following condition: If "ColB" is NOT NULL and "ColD" has values one of the following... (9 Replies)
Discussion started by: mehimadri
9 Replies

9. Shell Programming and Scripting

Based on num of records in file1 need to check records in file2 to set some condns

Hi All, I have two files say file1 and file2. I want to check the number of records in file1 and if its atleast 2 (i.e., 2 or greater than 2 ) then I have to check records in file2 .If records in file2 is atleast 1 (i.e. if its not empty ) i have to set some conditions . Could you pls... (3 Replies)
Discussion started by: mavesum
3 Replies

10. UNIX for Dummies Questions & Answers

Filtering records of a file based on a value of a column

Hi all, I would like to extract records of a file based on a condition. The file contains 47 fields, and I would like to extract only those records that match a certain value in one of the columns, e.g. COL1 COL2 COL3 ............... COL47 1 XX 45 ... (4 Replies)
Discussion started by: risk_sly
4 Replies
Login or Register to Ask a Question