Ksh: Read line parse characters into variable and remove the line if the date is older than 50 days


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Ksh: Read line parse characters into variable and remove the line if the date is older than 50 days
# 1  
Old 07-05-2016
Ksh: Read line parse characters into variable and remove the line if the date is older than 50 days

I have a test file with the following format, It contains the username_date when the user was locked from the database.

$ cat lockedusers.txt
Code:
TEST1_21062016          
TEST2_02122015  
TEST3_01032016  
TEST4_01042016

I'm writing a ksh script and faced with this difficult scenario for my level of scripting. What I would like to do is:

Read the line from text file,
If the date value of this line is older than 50 days,
then read the line till before the underscore character e.g. TEST1 into a variable,
and then remove this line.
The variable will be used for removing the user from database. Any help would be appreciated.

Last edited by jim mcnamara; 07-05-2016 at 09:24 PM..
# 2  
Old 07-06-2016
Save as beyondfifty.pl
Code:
#!/usr/bin/perl
use strict;
use warnings;
use Time::Local;
use File::Copy qw(move);

my $locked = "lockedusers.txt";
my $keep = "$locked.tmp";

my $fifty_days_ago = 86400 * 50;
my $now = time;

open my $in, '<', $locked  or die;
open my $out, '>', $keep or die;
while(<$in>) {
    my ($user, $day, $month, $year) = /(\w+)_(\d{2})(\d{2})(\d{4})/;
    my $time_ago = timelocal(0, 0, 0, $day, $month-1, $year-1900);
    ($now - $time_ago) > $fifty_days_ago ? print "$user\n" : print $out $_;
}
close $in;
close $out;
move $locked, $locked . ".bk" and move $keep, $locked;

Run as:
Code:
perl beyondfifty.pl | while read user; do echo "db command to delete $user"; done

Output:
Code:
db command to delete TEST2
db command to delete TEST3
db command to delete TEST4

Substitute the echo "db command to delete" for your own database command. It removes the matched entries. It creates a backup lockedusers.txt.bk

Last edited by Aia; 07-06-2016 at 01:22 AM..
This User Gave Thanks to Aia For This Post:
# 3  
Old 07-06-2016
Not sure what you mean with "then remove this line", but with this, you'll have the desired LUSER variable if the date portion is longer than 50 days back:
Code:
while IFS="_ " read LUSER LDATE
  do    [ $(( $(date +%s) - $(date +%s -d"${LDATE:2:2}/${LDATE:0:2}/${LDATE: -4}") - 4320000 )) -ge 0 ] && echo $LUSER, $LDATE
  done < file

This is not ksh tested. You might be able to use ksh's printf "%(fmt)T" builtin should your system's date not allow for the -d option...
# 4  
Old 07-07-2016
It worked

@RudicC

This is great, it worked like a charm after a little modification. Where are we setting the 50 days? in case I need to adjust the days.


Thanks a lot
# 5  
Old 07-07-2016
50 * 86400 sec = 4320000 sec
This User Gave Thanks to RudiC For This Post:
# 6  
Old 07-07-2016
and 86400 sec = 60 * 60 * 24 sec = 1 day
A - B > 0 is equal to A > B so you can do
Code:
while IFS="_ " read LUSER LDATE
do
  if [ $(( $(date +%s) - $(date +%s -d"${LDATE:2:2}/${LDATE:0:2}/${LDATE: -4}") )) -ge 4320000 ]
  then
    echo "$LUSER, $LDATE"
  fi
done < lockedusers.txt

This User Gave Thanks to MadeInGermany For This Post:
# 7  
Old 07-11-2016
Hi guys, for some reason I kept getting the following error when I run the test script.

Code:
#!/usr/bin/ksh
while IFS="_ " read LUSER LDATE
do
  if [ $(( $(date +%s) - $(date +%s -d"${LDATE:2:2}/${LDATE:0:2}/${LDATE: -4}") )) -ge 4320000 ]
  then
    echo "$LUSER" >> userRemovalList
	chmod 755 userRemovalList
    grep -v "$LUSER" test> test.new && mv test.new test
	echo "UserID: "${LUSER}         "Date Removed: "${TODAYS_DATE} >> RemovedUserList
  fi
done < test


./remove.ksh[8]: : bad substitution
TEST1, 03012016
./remove.ksh[8]: : bad substitution
TEST2, 11072016
./remove.ksh[8]: : bad substitution
TEST3, 05052016
./remove.ksh[8]: : bad substitution
TEST4, 02012016
./remove.ksh[8]: : bad substitution
TEST5, 20022016

And when I run this same script without #!/usr/bin/ksh, it runs fine

Code:
TEST1, 03012016
TEST3, 05052016
TEST4, 02012016
TEST5, 20022016

Code:
oracle@11204_home>which ksh
/usr/bin/ksh

Please provide suggestion




Moderator's Comments:
Mod Comment Please use code tags as required by forum rules!

Last edited by RudiC; 07-11-2016 at 02:01 PM.. Reason: Added code tags
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to read the output of a command line by line and pass it as a variable?

Hi, I have some 2000 names in a table like below. Java Oracle/SQL ANSI SQL SQL,DWH,DB DB&Java And by using for loop in my code i am able to get a single word but if there is any special character or space then it is considering as a next line. I have to execute the below queries in... (10 Replies)
Discussion started by: Samah
10 Replies

2. UNIX for Beginners Questions & Answers

Subscribers with Date 90 days older than current date

I have to display only those subscribers which are in "unconnected state" and the date is 90 days older than today's date. Below command is used for this purpose: cat vfsubscriber_20170817.csv | sed -e 's/^"//' -e '1d' | nawk -F '",' '{if ( (substr($11,2,4) == 2017) && ( substr($11,2,8) -lt... (1 Reply)
Discussion started by: dia
1 Replies

3. Shell Programming and Scripting

Remove lines older than 30 days

Hi Experts/Gurus, Is there a way to remove lines in a file that are older than x days (i.e. 30 days) based on the date stamp in the first column? Example. $ date Sat Jan 11 14:12:06 EDT 2014 $cat sample.txt 10-10-2013 09:00:01 AM|Line test 1234567 16-10-2013 08:30:00 AM|Line test... (6 Replies)
Discussion started by: brichigo
6 Replies

4. Shell Programming and Scripting

How to read a two files, line by line in UNIX script and how to assign shell variable to awk ..?

Input are file and file1 file contains store.bal product.bal category.bal admin.bal file1 contains flip.store.bal ::FFFF:BADC:CD28,::FFFF:558E:11C5,6,8,2,1,::FFFF:81C8:CA8B,::FFFF:BADC:CD28,1,0,0,0,::FFFF:81C8:11C5,2,1,0,0,::FFFF:81DC:3111,1,0,1,0 store.bal.... (2 Replies)
Discussion started by: veeruasu
2 Replies

5. Shell Programming and Scripting

Remove files older than 2 days.

Hi All, I am new to the scripting and using solaris 10 OS. Please suggest me from the below script which modifications need to be done to delete the files more that 2days older. Current script is deleting existing file. # Remove old explorer runs if needed DIR=`dirname ${EXP_TARGET}` if ... (2 Replies)
Discussion started by: Navkreddy
2 Replies

6. UNIX for Dummies Questions & Answers

Parsing file, reading each line to variable, evaluating date/time stamp of each line

So, the beginning of my script will cat & grep a file with the output directed to a new file. The data I have in this file needs to be parsed, read and evaluated. Basically, I need to identify the latest date/time stamp and then calculate whether or not it is within 15 minutes of the current... (1 Reply)
Discussion started by: hynesward
1 Replies

7. Shell Programming and Scripting

How to read a file line by line and store it in a variable to execute a program ?

Hello, I am quite new in shell scripting and I would like to write a little scritp to run a program on some parameters files. all my parameters files are in the same directory, so pick them up with ls *.para >>dirafter that I have a dir file like that: param1.para param2.para etc... I... (2 Replies)
Discussion started by: shadok
2 Replies

8. Shell Programming and Scripting

read a file line by line in ksh

Hi, In ksh we use 'while read line' statement to read a file line by line. In my input file I have 5 spaces appended at the end of each line. When I use while read line statement it chops off the spaces at the end of each line Inp.txt aaaa<five spaces> bbbb<five spaces> cccc<five spaces> ... (3 Replies)
Discussion started by: chella
3 Replies

9. Shell Programming and Scripting

sed remove last 10 characters of a line start from 3rd line

hello experts, I need a sed command that remove last 10 characters of a line start from 3rd line. any suggestions? Thanks you (7 Replies)
Discussion started by: minifish
7 Replies

10. Shell Programming and Scripting

SED help (remove line::parse again::add line)

Aloha! I have just over 1k of users that have permissions that they shouldn't under our system. I need to parse a provided list of usernames, check their permissions file, and strip the permissions that they are not allowed to have. If upon the permissions strip they are left with no permissions,... (6 Replies)
Discussion started by: Malumake
6 Replies
Login or Register to Ask a Question